https://www.netdata.cloud/tags/cardinality/Recent content in Cardinality on NetdataHugoen-usSat, 22 Aug 2026 05:09:03 +0300https://www.netdata.cloud/blog/high-cardinality-metrics-observability-scale/Sat, 30 May 2026 00:00:00 +0000https://www.netdata.cloud/blog/high-cardinality-metrics-observability-scale/<p>The &ldquo;high cardinality is expensive&rdquo; sentence has become observability&rsquo;s version of &ldquo;in this economy&rdquo;: said so often that nobody questions whether it&rsquo;s true. Every vendor pricing page invokes it. Every glossary article repeats it. Every architecture diagram shows aggregation buffers placed <em>before</em> the storage layer.</p> <!--truncate--> <p>&ldquo;High cardinality is expensive&rdquo; is not a fact about the universe; it&rsquo;s a fact about one architectural choice: centralizing time-series storage and querying it through an index that scales with unique series. Once you accept that choice, everything follows. You pay per metric, you drop labels you wish you&rsquo;d kept, you pre-aggregate before storage, and you discover that the bug you were debugging only existed at the full resolution you already threw away.</p>https://www.netdata.cloud/features/dataplatform/metrics-management/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/dataplatform/metrics-management/Collect unlimited metrics at per-second granularity with zero configuration. Netdata&rsquo;s edge-native architecture eliminates cardinality cost traps while delivering ML-based anomaly detection on every metric.https://www.netdata.cloud/features/architecture/infinite-scalability/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/architecture/infinite-scalability/Experience observability that grows naturally with your infrastructure - no rewrites, no bottlenecks, no compromise. Monitor everything, everywhere, at per-second resolution with predictable costs that scale with nodes, not data volume.https://www.netdata.cloud/features/architecture/extreme-cardinality/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/architecture/extreme-cardinality/Automated multi-layer protection handles extreme cardinality at the edge, enabling unlimited observability without manual tuning or cost explosions.https://www.netdata.cloud/solutions/use-cases/microservices-observability/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/microservices-observability/Netdata delivers comprehensive microservices observability with per-second metrics, ML-based anomaly detection, and AI-powered troubleshooting - all without the complexity, cost overruns, or tool sprawl that plague traditional solutions.https://www.netdata.cloud/features/architecture/real-time-at-scale/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/architecture/real-time-at-scale/Netdata delivers true real-time monitoring at planetary scale through distributed edge intelligence - maintaining per-second granularity and sub-2-second latency whether you&rsquo;re monitoring 10 nodes or 100,000, with 90% cost reduction and 80% MTTR improvement.https://www.netdata.cloud/solutions/built-for/platform-engineers/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/built-for/platform-engineers/Empower platform engineering teams with Netdata&rsquo;s distributed observability platform. Get per-second visibility, automated dashboards, ML anomaly detection, and predictable costs - all without query languages or complex pipelines.https://www.netdata.cloud/blog/monitor-everything-is-an-anti-pattern/Mon, 01 Dec 2025 00:00:00 +0000https://www.netdata.cloud/blog/monitor-everything-is-an-anti-pattern/<p><strong>Bullshit and nonsense.</strong></p> <p>But let&rsquo;s take it from the beginning.</p> <p>The industry&rsquo;s story goes something like this:</p> <blockquote> <p><em>&ldquo;Monitor everything is universally recognized as an anti-pattern.&rdquo;</em><br/> <em>&ldquo;You&rsquo;ll drown in metrics, burn out your engineers, and blow your budget.&rdquo;</em><br/> <em>&ldquo;Just focus on 3–10 signals — the Four Golden Signals, RED, USE — and ignore everything else.&rdquo;</em><br/> <em>&ldquo;Trust us, you don&rsquo;t want that much telemetry.&rdquo;</em><br/> <br/> (<a href="https://www.netdata.cloud/resources/research/monitor-everything-anti-pattern/">true, read the whole story here</a>)</p> </blockquote> <p>Then, in the same breath:</p>https://www.netdata.cloud/resources/research/monitor-everything-anti-pattern/Mon, 01 Dec 2025 00:00:00 +0000https://www.netdata.cloud/resources/research/monitor-everything-anti-pattern/<blockquote> <p><strong>Research Notice</strong>: This document was compiled through online research conducted on December 1, 2025. It serves as reference material for our blog post: <a href="https://www.netdata.cloud/blog/monitor-everything-is-an-anti-pattern/">Monitor Everything is an Anti-Pattern!</a>. Sources are cited inline and summarized at the end.</p> </blockquote> <h2 id="tldr">TL;DR</h2> <p>&ldquo;Monitor everything&rdquo; is universally recognized as an anti-pattern by SRE experts, observability leaders, and major tech companies. The core reasons are:</p> <ol> <li><strong>Metric Fatigue</strong>: Teams become overwhelmed by excessive data, unable to identify critical signals</li> <li><strong>Alert Fatigue</strong>: 63% of organizations face 1,000+ daily alerts with 72-99% false positives, costing $300,000+/hour in missed incidents</li> <li><strong>Lack of Actionability</strong>: 97% of alerts are non-actionable noise rather than signals requiring response</li> <li><strong>High Costs</strong>: Organizations spend 20-40% of cloud budgets on observability (vs. optimal 10-15%), with cardinality explosions creating exponential cost increases</li> <li><strong>Employee Burnout</strong>: Costs $4,000-$21,000 per employee annually, totaling $5M+ for 1,000-person companies</li> <li><strong>System Complexity</strong>: Monitoring systems themselves become fragile, requiring constant maintenance</li> <li><strong>Monitoring Tools Creating Problems</strong>: Monitoring agents can cause the latency outliers they&rsquo;re meant to detect</li> </ol> <p><strong>Expert Consensus</strong>: Focus on 3-10 key metrics (Google&rsquo;s Four Golden Signals, RED Method, USE Method) that indicate symptoms rather than attempting comprehensive monitoring of all possible metrics.</p>https://www.netdata.cloud/academy/cardinality-estimation-in-postgres/Mon, 01 Sep 2025 00:00:00 +0000https://www.netdata.cloud/academy/cardinality-estimation-in-postgres/<p>You&rsquo;ve seen it before: a query that runs in milliseconds on your staging server takes minutes to execute in production. Or a seemingly simple <code>JOIN</code> causes the <code>query_planner</code> to choose a disastrously slow Nested Loop over a much faster Hash Join. In almost every case, the root cause of these performance mysteries is not a bug in PostgreSQL, but a flaw in its understanding of your data. This is the challenge of <code>cardinality_estimation</code>. The planner is only as smart as the statistics it&rsquo;s given, and when its <code>row_estimation</code> is wrong, the consequences are severe.</p>https://www.netdata.cloud/academy/metric-cardinality-in-observability/Sun, 10 Aug 2025 00:00:00 +0000https://www.netdata.cloud/academy/metric-cardinality-in-observability/<p>It’s a story familiar to any SRE or DevOps engineer. You add a seemingly innocuous label to a key metric—<code>user_id</code>, <code>request_id</code>, <code>container_id</code>—to gain deeper insight. Suddenly, your monitoring bill skyrockets, your <code>Prometheus_TSDB</code> instance starts gasping for memory, and dashboards slow to a crawl. You have just triggered a <code>label_explosion</code>, the single biggest challenge in modern metrics-based observability: <code>metrics_cardinality</code>.</p> <p>High cardinality isn&rsquo;t an edge case; it&rsquo;s the new normal in a world of microservices, containers, and complex user interactions. As the number of unique time series grows into the millions or even billions, it places immense pressure on monitoring systems, impacting <code>storage_costs</code>, query performance, and the fundamental ability to scale.</p>https://www.netdata.cloud/academy/what-is-cardinality-in-databases-a-comprehensive-guide/Mon, 29 Jan 2024 00:00:00 +0000https://www.netdata.cloud/academy/what-is-cardinality-in-databases-a-comprehensive-guide/<h2 id="an-introduction-to-cardinality">An Introduction To Cardinality</h2> <p>Cardinality is a fundamental concept in databases that plays a crucial role in designing efficient databases and optimizing query performance. For those new to databases, understanding what is cardinality is essential for effective data management. This comprehensive guide will explain the meaning of cardinality, its types, and its <a href="https://www.netdata.cloud/monitoring-101/mongodb-monitoring/">impact on database performance</a>, making it accessible for both beginners and intermediate users.</p> <h2 id="what-is-cardinality-in-databases">What Is Cardinality In Databases?</h2> <p>Cardinality in databases refers to the uniqueness of data values contained in a column. It essentially measures how many distinct values exist in a column compared to the total number of rows in a table.</p>https://www.netdata.cloud/guides/elasticsearch/elasticsearch-limit-of-total-fields-exceeded/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/elasticsearch/elasticsearch-limit-of-total-fields-exceeded/<p>Every write suddenly returns <code>illegal_argument_exception: Limit of total fields [1000] in index [X] has been exceeded</code>. Indexing stops. The temptation is to raise <code>index.mapping.total_fields.limit</code> and move on. Do not. This is a mapping explosion: dynamic mapping creates a new field for every unique key in your documents. The 1000-field limit is a guardrail, not the root cause. Runaway mappings bloat the cluster state, inflate heap on every node, and eventually destabilize the master. Diagnose the source, relieve pressure safely, and fix the data shape so it does not recur.</p>https://www.netdata.cloud/guides/postgres/postgres-explain-analyze-reading/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/guides/postgres/postgres-explain-analyze-reading/<p>When <code>pg_stat_statements</code> flags a query as a top consumer, <code>EXPLAIN ANALYZE</code> is the operator&rsquo;s ground truth. It shows what the executor did, node by node, buffer by buffer. Misreading the output leads to useless indexes and production changes that make performance worse.</p> <p>This guide covers the mechanics that matter in production: how actual time accumulates through the node tree, why estimated rows diverge from reality, when buffer counts reveal cache misses versus disk reads, and how artifacts like the loops multiplier hide expensive nodes. It is a field manual for deciding, in the next five minutes, whether the problem is a missing index, stale statistics, a bad plan choice, or something deeper.</p>