Introducing Anomaly Advisor – Unsupervised Anomaly Detection in Netdata
Machine Learning May 26, 2022
Today we are excited to launch one of our flagship ML assisted troubleshooting features in Netdata – the Anomaly Advisor.
Once you enable ML on your nodes, each node will begin producing an “Anomaly Bit” every second in addition to raw metric values. This anomaly bit will be 1 when the trained ML models consider recent raw data for a metric to look anomalous or 0 when things look ‘normal’. The Anomaly Advisor leverages this information to enable seamless space or room level anomaly detection out of the box with minimal configuration.
To enable ML on your nodes and get started with the Anomaly Advisor, check out the documentation or the walkthrough video provided in this section. It’s as simple as setting
enabled = yes in the
[ml] section of your
netdata.conf. Once the models have trained, you should be able to start using the Anomaly Advisor in Netdata.
Monitoring Agents Don’t Need To Be Dumb!
This feature is part of a broader philosophy we have at Netdata when it comes to how we can leverage ML-based solutions to help augment and assist traditional troubleshooting flows, without having to centralize all your data. You can read more about this approach in our recent blog post which goes into more detail.
The new Anomalies tab quickly lets you find periods of time with elevated anomaly rates across all your nodes. Once you highlight a period of interest, Netdata will return a ranked list of the most anomalous metrics across all selected nodes in the highlighted timeframe. The goal here is to quickly let you find periods of strange activity in your infrastructure and surface what metrics Netdata considered most anomalous during that time.
Or if you just want to see it in action in a more realistic live demo, check out the below video where we run a K6 load test against our demo servers and watch it all play out in the Anomaly Advisor tab.
We love feedback!
We’d love to hear any and all feedback you have about this feature. This is very much an initial iteration and we are hoping to continually improve the ML under the hood in the agent and the overall UX experience as users share their thoughts with us.
🚧 Note: This functionality is still under active development. We dogfood it internally and among early adopters within the Netdata community to build the feature. If you would like to get involved and help us with some feedback, email us at email@example.com, comment on the beta launch post in the Netdata community, or come join us in the 🤖-ml-powered-monitoring channel of the Netdata discord, or open a discussion in GitHub if that’s more your thing.
If you’d like to dive deeper and learn a little more about exactly how it all works, please feel free to check out some of the resources below.
- Anomaly Advisor documentation.
- Netdata agent ML reference documentation.
- CNCF Live session with recording on YouTube (deck).
- Anomaly Advisor playlist on the Netdata YouTube channel.
- The code itself in the Netdata Agent GitHub repo, along with a Google Collab ready notebook based Python implementation to help those interested understand how it all works under the hood, the main concepts and some illustrated explanations.
- Yet another presentation that tries to explain the main concepts and moving parts.