https://www.netdata.cloud/tags/machine-learning/Recent content in Machine Learning on NetdataHugoen-usTue, 23 Jun 2026 00:00:00 +0000https://www.netdata.cloud/features/network/network-device-monitoring/Tue, 23 Jun 2026 00:00:00 +0000https://www.netdata.cloud/features/network/network-device-monitoring/Real-time SNMP device monitoring with auto-matched vendor profiles, ML anomaly detection, and zero-config discovery.https://www.netdata.cloud/resources/best-ai-powered-observability-platforms/Sat, 30 May 2026 00:00:00 +0000https://www.netdata.cloud/resources/best-ai-powered-observability-platforms/https://www.netdata.cloud/blog/nvidia-dcgm-monitoring/Mon, 04 May 2026 00:00:00 +0000https://www.netdata.cloud/blog/nvidia-dcgm-monitoring/<p><img src="../images/dcgm-collector.svg" alt="NVIDIA DCGM Collector: Deep GPU Monitoring for Data Center and AI Infrastructure"></p> <p>GPU infrastructure is expensive and increasingly central to production workloads. Whether you&rsquo;re running ML training jobs, inference serving, video transcoding, or HPC workloads, understanding what your GPUs are actually doing, and what&rsquo;s going wrong when performance degrades, is not optional. The problem is that NVIDIA&rsquo;s Data Center GPU Manager (DCGM) exposes an enormous amount of telemetry, but getting that data into a monitoring system in a useful, organized way has traditionally required significant setup and custom dashboarding work.</p>https://www.netdata.cloud/features/aiml/ai-co-engineer/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/ai-co-engineer/Netdata&rsquo;s AI Co-Engineer combines edge-native machine learning with flexible AI integration, providing instant expert-level insights while keeping your data sovereign and secure.https://www.netdata.cloud/features/aiml/aiops/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/aiops/Enterprise AIOps intelligence without enterprise complexity. Edge-native ML, automated insights, and transparent pricing deliver operational excellence from day one.https://www.netdata.cloud/features/architecture/algorithmic-dashboards/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/architecture/algorithmic-dashboards/Netdata&rsquo;s algorithmic dashboards transform observability from a months-long configuration project into instant, comprehensive visibility. Every metric visualized automatically, every relationship revealed through intelligent point-and-click analysis, every engineer productive from day one.https://www.netdata.cloud/features/aiml/anomaly-advisor/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/anomaly-advisor/Netdata Anomaly Advisor uses edge-native machine learning with 18-model consensus to eliminate 99% of false positives while surfacing root causes in the top 30-50 metrics from thousands collected. Get sub-2-second correlation analysis at any scale without configuration, training delays, or specialist expertise.https://www.netdata.cloud/features/aiml/anomaly-detection/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/anomaly-detection/Production-ready ML anomaly detection from installation. 18 consensus models per metric achieve 99% false positive reduction in anomaly detection while catching issues competitors miss. Zero configuration. Zero false promises.https://www.netdata.cloud/features/aiml/machine-learning/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/machine-learning/Netdata trains 18 independent ML models per metric at the edge, achieving 99% false positive reduction in anomaly detection through unanimous consensus - all included at no additional cost.https://www.netdata.cloud/solutions/use-cases/application-performance/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/application-performance/Netdata revolutionizes application performance monitoring with distributed edge intelligence, zero-configuration deployment, and AI-powered troubleshooting that delivers enterprise-grade observability at a fraction of traditional APM costs.https://www.netdata.cloud/solutions/use-cases/datacenters/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/datacenters/Transform data center operations with Netdata&rsquo;s real-time monitoring platform. Per-second metrics, automated ML anomaly detection, and AI-powered troubleshooting deliver 80% faster MTTR at 90% lower cost than traditional solutions.https://www.netdata.cloud/solutions/built-for/devops/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/built-for/devops/Netdata delivers complete observability for DevOps teams with per-second metrics, zero-configuration deployment, and AI-powered root cause analysis. Monitor everything from bare metal to Kubernetes with a single platform that replaces 7 tools while reducing costs by 90%.https://www.netdata.cloud/features/architecture/extreme-cardinality/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/architecture/extreme-cardinality/Automated multi-layer protection handles extreme cardinality at the edge, enabling unlimited observability without manual tuning or cost explosions.https://www.netdata.cloud/solutions/use-cases/hybrid-cloud-observability/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/hybrid-cloud-observability/Transform hybrid cloud monitoring with Netdata&rsquo;s distributed edge-native architecture. Get per-second visibility, ML-powered insights, and 90% cost savings without pipelines, sampling, or vendor lock-in.https://www.netdata.cloud/solutions/use-cases/microservices-observability/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/microservices-observability/Netdata delivers comprehensive microservices observability with per-second metrics, ML-based anomaly detection, and AI-powered troubleshooting - all without the complexity, cost overruns, or tool sprawl that plague traditional solutions.https://www.netdata.cloud/product/netdata-agents/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/product/netdata-agents/Discover how Netdata Agents revolutionize infrastructure monitoring with edge-native intelligence, per-second granularity, and zero-configuration deployment. Complete observability in 60 seconds.https://www.netdata.cloud/comparisons/cloudwatch/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/cloudwatch/Netdata provides 90% cost reduction vs CloudWatch with superior real-time monitoring (1-second vs 1-60 minutes), comprehensive ML on all metrics, and zero-configuration simplicity. Deploy in 60 seconds and eliminate CloudWatch&rsquo;s unpredictable costs while gaining better visibility.https://www.netdata.cloud/comparisons/betterstack/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/betterstack/Netdata and BetterStack serve different observability needs. Netdata excels at deep infrastructure monitoring with per-second metrics and ML intelligence, while BetterStack focuses on incident coordination and uptime monitoring. Learn which solution fits your technical requirements.https://www.netdata.cloud/comparisons/cardinalhq/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/cardinalhq/Comprehensive comparison of Netdata&rsquo;s edge-native monitoring platform versus CardinalHQ&rsquo;s AI-powered observability optimization layer. Learn which solution fits your infrastructure needs.https://www.netdata.cloud/comparisons/guance/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/guance/https://www.netdata.cloud/comparisons/icinga/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/icinga/Comprehensive comparison of Netdata and Icinga monitoring platforms, highlighting real-time performance analysis, automated deployment, ML-based anomaly detection, and modern observability capabilities.https://www.netdata.cloud/comparisons/monit/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/monit/Compare Netdata&rsquo;s real-time observability platform with Monit&rsquo;s lightweight process supervision. Learn how Netdata provides per-second monitoring, ML anomaly detection, AI troubleshooting, and unlimited scalability versus Monit&rsquo;s basic service checks and 1,000-host ceiling.https://www.netdata.cloud/comparisons/newrelic/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/newrelic/https://www.netdata.cloud/comparisons/prometheus/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/prometheus/Netdata vs Prometheus comparison: Real-time monitoring with zero configuration, built-in ML, and 90% lower costs. See how Netdata solves Prometheus pain points while maintaining enterprise-grade capabilities.https://www.netdata.cloud/comparisons/zabbix/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/comparisons/zabbix/https://www.netdata.cloud/open-source/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/open-source/The world&rsquo;s most popular open source monitoring platform. Deploy complete infrastructure observability in 60 seconds with zero configuration. Trusted by millions of engineers worldwide.https://www.netdata.cloud/solutions/industries/ai/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/industries/ai/Monitor AI training clusters, inference APIs, and ML workloads with Netdata&rsquo;s edge-native platform. Per-second visibility, ML-powered insights, predictable pricing.https://www.netdata.cloud/solutions/built-for/platform-engineers/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/built-for/platform-engineers/Empower platform engineering teams with Netdata&rsquo;s distributed observability platform. Get per-second visibility, automated dashboards, ML anomaly detection, and predictable costs - all without query languages or complex pipelines.https://www.netdata.cloud/solutions/built-for/sre/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/built-for/sre/Transform SRE operations with Netdata&rsquo;s edge-native observability platform. Get per-second visibility, ML anomaly detection on every metric, and AI-powered troubleshooting at 90% lower cost than traditional solutions.https://www.netdata.cloud/features/visualization/troubleshooting/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/visualization/troubleshooting/Interactive debugging with per-second precision and full historical context. Netdata transforms troubleshooting through edge-native ML, automated correlation, and AI-powered analysis.https://www.netdata.cloud/solutions/technologies/redhat-monitoring/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/technologies/redhat-monitoring/Netdata delivers real-time, AI-powered observability for Red Hat OpenShift with 90% cost savings, per-second metrics, ML anomaly detection on every metric, and zero-configuration deployment. Eliminate Prometheus memory exhaustion, Loki query timeouts, and RHACM complexity.https://www.netdata.cloud/features/aiml/root-cause-analysis/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/features/aiml/root-cause-analysis/Netdata&rsquo;s edge-native ML detects anomalies as they happen, automated correlation surfaces root causes in the top 30-50 results, and AI explains incidents in plain English - achieving 80% MTTR reduction at 90% lower cost.https://www.netdata.cloud/solutions/use-cases/windows-event-logs/Thu, 18 Dec 2025 00:00:00 +0000https://www.netdata.cloud/solutions/use-cases/windows-event-logs/Netdata revolutionizes Windows Event Log monitoring by combining infrastructure metrics with native log access, ML anomaly detection, and AI troubleshooting - delivering comprehensive observability at a fraction of traditional SIEM costs.https://www.netdata.cloud/blog/our-first-ml-based-anomaly-alert/Wed, 13 Sep 2023 00:00:00 +0000https://www.netdata.cloud/blog/our-first-ml-based-anomaly-alert/<p>Over the last few years we have slowly and methodically been building out the <a href="https://learn.netdata.cloud/docs/ml-and-troubleshooting/">ML based capabilities</a> of the Netdata agent, dogfooding and iterating as we go. To date, these features have mostly been somewhat reactive and tools to aid once you are already troubleshooting.</p> <p>Now we feel we are ready to take a first gentle step into some more proactive use cases, starting with a <a href="https://github.com/netdata/netdata/pull/14687">simple node level anomaly rate alert</a>.</p>https://www.netdata.cloud/blog/anomaly-rate-by-type/Wed, 30 Aug 2023 00:00:00 +0000https://www.netdata.cloud/blog/anomaly-rate-by-type/<p>We have <a href="https://github.com/netdata/netdata/pull/15856">recently added</a> a more detailed anomaly rate chart to Netdata that breaks out the overall <a href="https://learn.netdata.cloud/docs/ml-and-troubleshooting/machine-learning-ml-powered-anomaly-detection#node-anomaly-rate">node anomaly rate</a> by type, this lets you more easily see what parts of your infrastructure might be experiencing an uptick in anomalies when you see the overall node anomaly rate increase.</p> <h2 id="what-is-type">What is <code>type</code>?</h2> <p><code>type</code> is generally the prefix of the chart id in Netdata and controls where charts live within the menu on the overview page, for example the <code>mem.available</code> chart has a type of <code>mem</code> which in part controls why it lives under the &ldquo;Memory&rdquo; section of the menu.</p>https://www.netdata.cloud/blog/ml-demo-ansible-configuration-management/Fri, 07 Jul 2023 00:00:00 +0000https://www.netdata.cloud/blog/ml-demo-ansible-configuration-management/<p>We are always trying to lower the barrier to entry when it comes to monitoring and observability and one place we have consistently witnessed some pain from users is around adopting and approaching <a href="https://www.atlassian.com/microservices/microservices-architecture/configuration-management">configuration management</a> tools and practices as your infrastructure grows and becomes more complex.</p> <p>To that end, we have begun recently publishing our own <a href="https://github.com/netdata/community/tree/main/configuration-management/ansible-ml-demo">little example ansible project</a> used to maintain and manage the servers used in our public <a href="https://app.netdata.cloud/spaces/netdata-demo/rooms/machine-learning/overview">Machine Learning Demo room</a>.</p>https://www.netdata.cloud/blog/how-netdatas-ml-based-anomaly-detection-works/Tue, 23 May 2023 00:00:00 +0000https://www.netdata.cloud/blog/how-netdatas-ml-based-anomaly-detection-works/<p><img src="../2023-05-23-how-netdatas-ml-based-anomaly-detection-works/img/img.png" alt="title image"></p> <p>How does Netdata&rsquo;s <a href="https://learn.netdata.cloud/docs/troubleshooting-and-machine-learning/machine-learning-ml-powered-anomaly-detection">machine learning (ML) based anomaly detection</a> actually work? Read on to find out!</p> <!--truncate--> <h2 id="design-considerations">Design considerations</h2> <p>Lets first start with some of the key design considerations and principles of Netdata&rsquo;s anomaly detection (<em>and some comments in parenthesis along the way</em>):</p> <ol> <li>We don&rsquo;t have any labels or examples of previous anomalies. This means we are in an <a href="https://en.wikipedia.org/wiki/Unsupervised_learning">unsupervised setting</a> (<em>best we can try to do is learn what &ldquo;normal&rdquo; data looks like assuming the collected data is &ldquo;mostly&rdquo; normal</em>).</li> <li>Needs to be lightweight and run on the agent (<em>or a parent</em>). <ul> <li>Need to be very careful of impact on CPU overhead when training and scoring (<em>lots of cheap models are better than a few expensive and heavy ones</em>).</li> <li>Models themselves need to be small so as to not drastically increase the agents memory footprint (<em>model objects need to be small for storage</em>).</li> <li>This has implications for the ML formulation (<em>sorry - no deep learning models yet) and its implementation (we need to be surgical and optimized</em>).</li> </ul> </li> <li>Needs to scale for thousands of metrics and score in realtime every second as metrics are collected. <ul> <li>Typical Netdata nodes have thousands of metrics and we want to be able to score every metric every second with minimal latency overhead (<em>we need to use sensible approaches to training like spreading the training cost over a wide training window</em>).</li> </ul> </li> <li>Needs to be able to handle a wide variety of metrics. <ul> <li>There is no single perfect model or approach for all types of metrics so we need a good all rounder that can work well enough across any and all different types of time seres metrics (<em>for any given metric of course you could handcraft a better model but thats not feasible here, we need something like a &ldquo;weak learners&rdquo; approach of lots of generally useful models adding up to &ldquo;more than the sum of their parts&rdquo;</em>).</li> </ul> </li> <li>Needs to be written in C or C++ as that is the language of the Netdata agent (<em>We are using <a href="https://github.com/davisking/dlib">dlib</a> for the current implementation</em>).</li> <li>We Need to be very careful about taking big or complex dependencies if using third party libraries. <ul> <li>We want to be able to easily build and deploy Netdata on any Linux system without having to worry about installing or managing complex dependencies (<em>we need to be careful of more complex algorithms that would have larger dependencies and potentially limit where Netdata can run</em>).</li> </ul> </li> </ol> <p>The above considerations are important and useful to keep in mind as we explore the system in more detail.</p>https://www.netdata.cloud/blog/transform-monitoring-ml-first-approach/Thu, 11 May 2023 00:00:00 +0000https://www.netdata.cloud/blog/transform-monitoring-ml-first-approach/<p>Unlocking the full potential of monitoring through ML integration, anomaly detection, and innovative scoring engines.</p> <!--truncate--> <p>Machine Learning has been making waves in various industries, but its adoption in the monitoring and observability space has been slower than expected. Many “ML” features remain gimmicky and do not provide actual real world value to users that encourages their further use.</p> <p>At Netdata, we firmly believe that ML is crucial for monitoring, and we&rsquo;ve taken an ML-first approach to provide users with powerful tools and insights. In this blog post, we&rsquo;ll discuss the reasons behind our belief in ML, how we&rsquo;ve integrated ML into our charts and visualizations, our query engines, the scoring engine we&rsquo;ve built, and how these innovations enable metrics correlations and anomaly advisor.</p>https://www.netdata.cloud/blog/netdata-ai-insights-rapid-diagnostics/Wed, 19 Apr 2023 00:00:00 +0000https://www.netdata.cloud/blog/netdata-ai-insights-rapid-diagnostics/<p>Introduction to Netdata&rsquo;s new visualisation providing AI Insights, supporting Rapid Diagnostics. <img src="https://user-images.githubusercontent.com/96257330/233125254-f93c9520-0a3f-4844-8d43-1f3202a5e411.png" alt="logo"></p> <!--truncate--> <h2 id="a-new-era-in-monitoring-systems-dashboards">A New Era in Monitoring Systems Dashboards</h2> <p>We&rsquo;re thrilled to share an important upgrade to Netdata: <strong>AI Insights &amp; Rapid Diagnostics</strong>, a technology aiming to redefine what we expect from a monitoring system.</p> <h2 id="challenges-with-traditional-monitoring-dashboards">Challenges with Traditional Monitoring Dashboards</h2> <p>Traditional monitoring systems rely on a query language to help engineers create dashboards and alerts. While these languages offer power and flexibility, they come with several challenges that make monitoring and troubleshooting more complex and time-consuming:</p>https://www.netdata.cloud/blog/anomaly-rates-in-the-menu/Wed, 29 Mar 2023 00:00:00 +0000https://www.netdata.cloud/blog/anomaly-rates-in-the-menu/<p>The menu (on the <a href="https://learn.netdata.cloud/docs/getting-started/monitor-your-infrastructure/home-overview-and-single-node-view#overview-and-single-node-view">overview or single node tab</a>) now has an <a href="https://learn.netdata.cloud/docs/troubleshooting-and-machine-learning/machine-learning-ml-powered-anomaly-detection#anomaly-rate">anomaly rate</a> button built into it that, for the entire visible window or a highlighted time range, shows the maximum chart anomaly rate within each section.</p> <p>Read on to learn more about this new feature!</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/PgVh_MFHMb0?si=F2Mq6wIxHJWaHykZ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> <h2 id="wait-what-is-an-anomaly-rate">Wait, what is an anomaly rate?</h2> <p>Netdata is the only monitoring agent that natively (for every metric, with zero config and sane defaults) produces anomaly rates in addition to just collecting raw metrics.</p>https://www.netdata.cloud/blog/anomaly-detection-on-prometheus-metrics/Wed, 01 Mar 2023 00:00:00 +0000https://www.netdata.cloud/blog/anomaly-detection-on-prometheus-metrics/<p><img src="../2023-03-01-anomaly-detection-on-prometheus-metrics/img/img.png" alt="img"></p> <p>We have recently extended the native machine learning (ML) based anomaly detection <a href="https://learn.netdata.cloud/guides/monitor/anomaly-detection">capabilities</a> of Netdata to <a href="https://github.com/netdata/netdata/issues/14218">support all metrics</a>, regardless on their collection frequency (<code>update every</code>).</p> <p>Previously only metrics collected every second were supported, but now Netdata can run anomaly detection out of the box with zero config on metrics with any collection frequency.</p> <p>This post will illustrate an example of what this means using <a href="https://prometheus.io/">Prometheus</a> metrics (via the <a href="https://learn.netdata.cloud/docs/agent/collectors/go.d.plugin/modules/prometheus#gsc.tab=0">Netdata Prometheus collector</a>) since they typically have a default collection frequency of 10 seconds.</p>https://www.netdata.cloud/blog/extending-anomaly-detection-training-window/Thu, 02 Feb 2023 00:00:00 +0000https://www.netdata.cloud/blog/extending-anomaly-detection-training-window/<p>We have been busy at work under the hood of the Netdata agent to introduce new capabilities that let you extend the &ldquo;training window&rdquo; used by Netdata&rsquo;s <a href="https://learn.netdata.cloud/docs/nightly/setup/configure-machine-learning-ml-powered-anomaly-detection">native anomaly detection capabilities</a>.</p> <p>This blog post will discuss one of these improvements to help you reduce &ldquo;<a href="https://en.wikipedia.org/wiki/False_positives_and_false_negatives#False_positive_error">false positives</a>&rdquo; by essentially extending the training window by using the new (beautifully named) <code>number of models per dimension</code> configuration parameter.</p> <h2 id="background">Background</h2> <p>One of the most important considerations of our native anomaly detection capabilities is the overhead of running the training and scoring computations required to train thousands of models (one per metric) and produce <a href="https://learn.netdata.cloud/docs/nightly/setup/configure-machine-learning-ml-powered-anomaly-detection#anomaly-bit">anomaly bits</a> every second based on those trained models.</p>https://www.netdata.cloud/blog/data-collection-strategies/Tue, 06 Sep 2022 00:00:00 +0000https://www.netdata.cloud/blog/data-collection-strategies/<!--truncate--> <p>Monitoring and troubleshooting; unfortunately, these terms are still used interchangeably, which can lead to misunderstandings about data collection strategies.</p> <p>In this article we aim to clarify some important definitions, processes, and common data collection strategies for monitoring solutions. We will specify the limitations of the described strategies, as well as key benefits which can potentially be also used for troubleshooting needs.</p> <p><strong>IT infrastructure monitoring</strong> is a business process of collecting and analyzing data over a period of time to improve business results.</p>https://www.netdata.cloud/blog/how-netdatas-machine-learning-works/Thu, 01 Sep 2022 00:00:00 +0000https://www.netdata.cloud/blog/how-netdatas-machine-learning-works/<p>Following on from the <a href="https://www.netdata.cloud/blog/introducing-anomaly-advisor-unsupervised-anomaly-detection-in-netdata" target="_blank" rel="noopener">recent launch</a> of our <a href="https://learn.netdata.cloud/docs/cloud/insights/anomaly-advisor" target="_blank" rel="noopener">Anomaly Advisor</a> feature, and in keeping with <a href="https://www.netdata.cloud/blog/our-approach-to-machine-learning/" target="_blank" rel="noopener">our approach to machine learning</a>, <a href="https://github.com/netdata/netdata/blob/master/ml/notebooks/netdata_anomaly_detection_deepdive.ipynb" target="_blank" rel="noopener">here</a> is a detailed Python notebook outlining exactly how the machine learning powering the Anomaly Advisor actually works under the hood.</p> <!--truncate--> <p>Or if you&rsquo;d rather watch a video walkthrough of the notebook then check out below.</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/L1xleckyuDQ?si=rptYzWE-eLlhSL9x" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> <p>Try it for yourself, <a href="https://learn.netdata.cloud/docs/cloud/get-started" target="_blank" rel="noopener">get started</a> by <a href="https://app.netdata.cloud/?utm_source=blog&amp;utm_content=how_netdata_ml_works" target="_blank" rel="noopener">signing in to Netdata</a> and connecting a node. Once initial models have been trained (usually after the agent has about one hour of data, zero configuration needed), you&rsquo;ll be able to start exploring in the <a href="https://learn.netdata.cloud/docs/cloud/insights/anomaly-advisor" target="_blank" rel="noopener">Anomaly Advisor</a> tab of Netdata.</p>https://www.netdata.cloud/blog/anomaly-rate-in-every-chart/Thu, 23 Jun 2022 00:00:00 +0000https://www.netdata.cloud/blog/anomaly-rate-in-every-chart/<p>A month ago, we introduced unsupervised ML &amp; Anomaly Detection in Netdata, the <a href="https://www.netdata.cloud/blog/introducing-anomaly-advisor-unsupervised-anomaly-detection-in-netdata/">Anomaly Advisor</a>. Today, we’re happy to announce that we’re bringing anomaly rates to every chart in Netdata Cloud. Anomaly information is no longer limited to the Anomalies tab and will be accessible to you from the Overview and Single Node View tabs as well. This will make your troubleshooting journey easier, as you will have the anomaly rates for any metric available with a single click. Whichever metric or chart you&rsquo;re exploring will be instant.</p>https://www.netdata.cloud/blog/metric-correlations-on-the-agent/Wed, 15 Jun 2022 00:00:00 +0000https://www.netdata.cloud/blog/metric-correlations-on-the-agent/<p>As of <a href="https://github.com/netdata/netdata/releases/tag/v1.35.0" target="_blank" rel="noopener"><code>v1.35.0</code></a> the Netdata Agent can now run <a href="https://learn.netdata.cloud/docs/cloud/insights/metric-correlations" target="_blank" rel="noopener">Metric Correlations</a> (MC) itself. This means that, for nodes with MC enabled, the Metric Correlations feature just got a whole lot faster!</p> <!--truncate--> <p>The Netdata Metric Correlations feature uses a <a href="https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test#Two-sample_Kolmogorov%E2%80%93Smirnov_test" target="_blank" rel="noopener">Two Sample Kolmogorov-Smirnov test</a> to look for which metrics have a significant distributional change around a highlighted window of interest. This can be useful when you are interested in short term &ldquo;<a href="https://en.wikipedia.org/wiki/Change_detection" target="_blank" rel="noopener">change detection</a>&rdquo; and want to try answer the question &ldquo;what else changed around this time?&rdquo;.</p>https://www.netdata.cloud/blog/introducing-anomaly-advisor-unsupervised-anomaly-detection-in-netdata/Thu, 26 May 2022 00:00:00 +0000https://www.netdata.cloud/blog/introducing-anomaly-advisor-unsupervised-anomaly-detection-in-netdata/<p>Today we are excited to launch one of our flagship ML assisted troubleshooting features in Netdata – the Anomaly Advisor.</p> <p>The Anomaly Advisor builds on earlier work to introduce unsupervised <a href="https://github.com/netdata/netdata/blob/master/ml/README.md">anomaly detection</a> capabilities into the <a href="https://www.netdata.cloud/agent/">Netdata Agent</a> from <a href="https://github.com/netdata/netdata/releases/tag/v1.32.0">v1.32.0</a> onwards.</p> <h2 id="getting-started">Getting Started</h2> <p>Once you <a href="https://learn.netdata.cloud/docs/configure/machine-learning#configuration">enable ML</a> on your nodes, each node will begin producing an &ldquo;<a href="https://learn.netdata.cloud/docs/configure/machine-learning#anomaly-bit---100--anomalous-0--normal">Anomaly Bit</a>&rdquo; every second in addition to raw metric values. This anomaly bit will be 1 when the trained ML models consider recent raw data for a metric to look anomalous or 0 when things look &rsquo;normal&rsquo;. The Anomaly Advisor leverages this information to enable seamless space or room level anomaly detection out of the box with minimal configuration.</p>https://www.netdata.cloud/blog/cncf-live-power-up-your-machine-learning-automated-anomaly-detection/Wed, 27 Apr 2022 00:00:00 +0000https://www.netdata.cloud/blog/cncf-live-power-up-your-machine-learning-automated-anomaly-detection/<h2 id="join-us-live-to-talk-ml">Join Us Live to Talk ML</h2> <p>Join ML Lead Andrew Maguire and Product Manager Shyam Sreevalsan on the 23rd of June at 5pm UTC for the Netdata Machine Learning Meetup, which will be livestreamed on Youtube. In this session, we will demo and preview new and future features, along with a Q&amp;A, and discuss the following topics:</p> <ul> <li>The role of Machine Learning in DevOps Infrastructure Monitoring &amp; Troubleshooting</li> <li>The Netdata way of approaching Machine Learning</li> <li>Challenges of building Macvhine learning solutions that are useful and user friendly.</li> </ul> Feel free to ask questions or share ideas on <a href="https://discord.gg/ZyeDHQTdaW">our Community Discord</a>, or <a href="https://(https://www.meetup.com/netdata-infrastructure-monitoring-meetup-group/events/286243158/">RSVP to the event</a>. We look forward to seeing you then! <h2 id="cncf-live-power-up-your-machine-learning---automated-anomaly-detection">CNCF Live: Power up your machine learning - Automated anomaly detection</h2> <p>Our Analytics &amp; ML lead Andrew Maguire recently had a chance to share our new <a href="https://community.netdata.cloud/t/anomaly-advisor-beta-launch/2717">Anomaly Advisor</a> feature with the wider CNCF community. In his demonstration he did some light chaos engineering (using <a href="https://www.gremlin.com/">Gremlin</a> and <a href="https://wiki.ubuntu.com/Kernel/Reference/stress-ng">stress-ng</a>) to generate some real anomalies on his infrastructure and watch how it all played out in the Anomaly Advisor in Netdata Cloud.</p>https://www.netdata.cloud/blog/our-approach-to-machine-learning/Fri, 25 Mar 2022 00:00:00 +0000https://www.netdata.cloud/blog/our-approach-to-machine-learning/<p>There is a lot of buzz in the world of machine learning (ML) and as a layperson it can be hard to keep up with it all. Therefore, we decided to write down some of our thoughts and musings on how <b>we</b> are approaching ML at Netdata.</p> <!--truncate--> <h2 id="our-approach-to-machine-learning-ml">Our Approach to Machine Learning (ML)</h2> <p>We’ll touch on the current state of applied ML in industry in general, and zoom in on ML in the monitoring industry. We’ll discuss how we can leverage “good honest ML” to punch above our weight and add some useful and novel features for our users over the next few years.</p>https://www.netdata.cloud/blog/root-cause-analysis-using-metric-correlations/Fri, 03 Sep 2021 00:00:00 +0000https://www.netdata.cloud/blog/root-cause-analysis-using-metric-correlations/<!--truncate--> <figure class="wp-block-image size-large"><img src="../wp-archive/uploads/2022/03/Screen-Shot-2021-09-03-at-1.43.32-PM-1-1200x608.png" alt="" class="wp-image-16297"/></figure> <p>As complexity of systems and applications continue to evolve and change, the number of metrics that need to be monitored grows in parallel. Whether you’re on a DevOps team, an SRE, or a developer building the code yourself, many of these components may be fragmented across your infrastructure, making it increasingly difficult to identify the root cause when experiencing downtime or abnormal behavior. To help solve this challenge, we built the <a href="https://learn.netdata.cloud/docs/cloud/insights/metric-correlations">Metric Correlations</a> feature – an automated analysis tool that evaluates all your metrics to identify which have changed the most within a given period of interest.</p>https://www.netdata.cloud/blog/netdata-cloud-metric-correlations/Wed, 16 Sep 2020 00:00:00 +0000https://www.netdata.cloud/blog/netdata-cloud-metric-correlations/<!--truncate--> <img class="alignnone size-large wp-image-16623" src="../wp-archive/uploads/2022/03/Cloud-Correlations@2x-1200x826.png" alt="" width="1200" height="826" /> <p>Today, we are excited to launch our first Netdata Cloud Insights feature, Metric Correlations, developed for discovering underlying issues more quickly and identifying the root cause more efficiently. Read on to learn more about our approach to developing this new feature, how it works, and the many benefits you’ll find incorporating this into your team’s troubleshooting workflow.</p> <h2>Some background</h2> Let’s start with a bit of a disclaimer. It seems machine learning (ML) (or “Artificial Intelligence,” if you are looking for more LinkedIn likes) has gone mainstream in the last few years, and we are probably by now somewhere near the “Peak of Inflated Expectations” on the <a title="hype cycle" href="https://en.wikipedia.org/wiki/Hype_cycle" target="_blank" rel="noopener noreferrer">hype cycle</a>. It is in this context that we want to be clear about what our goals are in this space and our approach to releasing data-driven features that draw on techniques from statistics and ML. In short, we want to be clear, open, realistic, and avoid buzzwords at all costs! <p>Over the next 12 months, we are hoping to begin building a layer of intelligence^{<a href="https://staging-www.netdata.cloud/blog/netdata-cloud-metric-correlations/#1">1</a>} throughout Netdata (both Cloud and Agent) to assist with “<a title="human in the loop" href="https://hai.stanford.edu/blog/humans-loop-design-interactive-ai-systems" target="_blank" rel="noopener noreferrer">human in the loop</a>” troubleshooting, mainly to help users more easily surface slowdowns, anomalies, or other issues and lower your <a title="cognitive load" href="https://en.wikipedia.org/wiki/Cognitive_load" target="_blank" rel="noopener noreferrer">cognitive load</a>^{<a href="https://staging-www.netdata.cloud/blog/netdata-cloud-metric-correlations/#2">2</a>} as you troubleshoot using Netdata. Simply put, we’re working to streamline your mean time to resolution (MTTR).</p>https://www.netdata.cloud/blog/contribute-machine-learning/Mon, 16 Mar 2020 00:00:00 +0000https://www.netdata.cloud/blog/contribute-machine-learning/<!--truncate--> <img class="alignnone size-full wp-image-16783" src="../wp-archive/uploads/2022/03/contribute-machine-learning.png" alt="" width="991" height="1072" /> <p>Netdata contributors have greatly influenced the growth of our company and are essential to our success. The time and expertise that contributors volunteer are fundamental to our goal of helping you build extraordinary infrastructures. We highly value end-user feedback during product development, which is why we’re looking to involve you in progressing our machine learning (ML) efforts! <span id="more-2975"></span>As we are continually looking for ways to improve and enhance Netdata, we are starting to explore how we can leverage machine learning to introduce new product features. Our main focus at the moment is around automated <a href="https://en.wikipedia.org/wiki/Anomaly_detection">anomaly detection</a>. This is a really interesting and challenging problem (high volume, high dimensional data, lack of ground truth labels, and so on), but we should be able to use some of the metrics monitored by Netdata to deliver new, awesome product features and user experiences (AI is the <a href="https://www.gsb.stanford.edu/insights/andrew-ng-why-ai-new-electricity">new electricity</a>, after all 😃). However, developing ML-driven product features is quite different than traditional software development (see steps 1 to 7 in the picture above). Mainly, this is because you never really know what specific data transformations, problem formulation, and sets of algorithms will work best in advance. (<a href="https://www.kdnuggets.com/2019/09/no-free-lunch-data-science.html">Here</a> is a good article explaining things, and if you really want to go down a rabbit hole, check out this <a href="https://ai.stackexchange.com/questions/15650/what-are-the-implications-of-the-no-free-lunch-theorem-for-machine-learning">Stack Overflow question</a> and this <a href="https://www.quora.com/What-does-the-No-Free-Lunch-theorem-mean-for-machine-learning-In-what-ways-do-popular-ML-algorithms-overcome-the-limitations-set-by-this-theorem">Quora thread</a>). Ideally, you first need to prototype your solution “in the lab” on some data you have already collected and do a few iterations of data → problem formulation → prototype. This process gives you a level of confidence in what you are doing (and some data to back it up) to move on to the even-more-complicated step of going from prototype to production. At Netdata, we are currently trying to get to step 4, where we can first prototype some solutions on real-world data and come up with ways to measure progress.</p>https://www.netdata.cloud/comparisons/chronosphere/Mon, 01 Jan 0001 00:00:00 +0000https://www.netdata.cloud/comparisons/chronosphere/Netdata provides edge-native observability with per-second granularity, automatic ML anomaly detection, and transparent pricing—eliminating PromQL learning curves and SaaS-only limitations that challenge Chronosphere users.