Monitoring Ubuntu 20.04 and Activating ML with Netdata
How to, Machine Learning Jun 7, 2022
Sometimes a hat is just a hat, the truth is just the truth, and the clearly most popular example of a category is plain to see. In this case, Ubuntu is the most popular Linux distribution currently available. With the operating system’s superior popularity also comes an amazing amount of community support. Some other reasons behind Ubuntu’s popularity are how it’s the first choice in machine learning, has robust cross platform support, offers real-time distribution through snaps, and all of that capped off with a wide range of hardware support including coming pre-installed on some machines. The operating system comes with a ton of tools right out of the box but unfortunately does not offer much to empower users to monitor the system’s efficiency. Like most Linux distros, Ubuntu only comes with the `top` command from the terminal to show system processes from a monochromatic terminal screen controlled with keyboard commands. It’s ugly, difficult to use, and Netdata was born out of frustration with the lack of tools for this purpose. Netdata’s free and open-source software immediately offers 2,000+ auto-detected metrics, and 100+ pre-configured alerts—per node. Let’s go install it and get to monitoring in seconds right now!
We’re going to assume that you already have Ubuntu downloaded, installed, and running. If not, then take a moment to head over to their guide for getting started through whichever variety you prefer! It won’t make a difference so long as we are all using the Ubuntu 20.04 LTS together. To ensure you have the correct version, just type `lsb_release a`. As soon as you are ready and your Ubuntu is running, then open the terminal to get started! Be sure that you’re logged in as a user with sudo privileges or belonging to the sudo group for all of our commands.
From your terminal, we should begin by making sure everything is up to date in this OS. First, type `sudo apt-get update` to retrieve the latest versions. Then, run `sudo apt-get upgrade` to execute all the updates that were found. After everything is updated, then restart your system to ensure that the updates all take without conflict by clicking restart or using a terminal command with sudo privileges.
Once your system is back online, then take a moment to type in `top` and enter in the command line. This is our out-of-box monitoring utility – gross! Let’s do better with free, open-source Netdata!
Start by opening your browser and navigating to the Netdata website. We’re looking for the “Get Netdata” in the top right corner or just “sign in” if you’ve already registered. If you haven’t already registered, then it only takes a few moments with single click sign on options from Google as well as GitHub. You can also create a new account with an email address.
After registering, you will immediately be greeted by our new user tooltip guide and your “Home” page view. At this point, the dashboard will look really boring because we don’t have any metrics streaming to it – let’s fix that! Look for the “Connect Nodes” link in the top left-hand corner of the sidebar. After clicking, another menu will appear with several options for installing Netdata as it works across Linux distributions, Docker containers, MacOS, and even Kubernetes clusters! Of course, we’re using Ubuntu which is Linux so copy the script that pops up in your space. It should look something like:
`wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh --claim-token YOUR_CLAIM_TOKEN --claim-url https://app.netdata.cloud`
The variable `YOUR_CLAIM_TOKEN` will be unique to your registration and the Netdata Cloud space you’re working within. Take the script you copied, paste it into your terminal window, and hit enter. You may need to enter a password and answer “yes” a few times, but the total installation time should come in at under 60 seconds, depending on system resources. After the installation completes, then return to your browser and the Netdata Cloud dashboard. Within moments, the browser application should come alive with streaming metrics about your system!
Now that Netdata is installed and streaming metrics to the Cloud dashboard, let’s take a moment to explore what kind of data we’re getting. The Cloud is meant to manage *many* nodes within your distributed network, but we can manage a single node from here as well. We start in our default space, but can create more spaces if needed for our teams. Inside of each space, we can create as many war rooms as we want to organize our nodes and team members.
While still in our browser and in our first “General” war room, we can see six tabs across the top of the window – Home, Overview, Nodes, Dashboards, Alerts, and Anomalies. To summarize:
- Home summarizes your entire system architecture being monitored
- Overview gives a composite view of metrics with the ability to display average, sum, min, and max of values on charts, plus more
- Nodes is a detailed list of connected nodes offering streaming metrics – online or not. Clicking one of these nodes will open a new tab with specifics about that one system.
- Dashboards is where we can create custom monitoring dashboards in a new tab with specific charts, text notes for teammates, and the ability to pin items for better organization
- Alerts is where we can see any issues currently triggered by the monitoring software
- Anomalies is an aggregated view of Machine Learning collected metrics
All of these features work immediately after installation *except* the machine learning Anomaly Advisor which must be activated in the configuration file. Let’s go take care of that immediately!
We need to return to our command line to make changes to our Netdata configuration. Once we’re back in the terminal window, go ahead and type `systemctl status netdata` to ensure that we’re on the right machine with Netdata installed, active, and running. After that, let’s navigate to the directory where the application was installed by entering `cd /etc/netdata` and hitting enter. If we type `ls` and hit enter, then we should see a printout of all the files in the directory, including the files that we are interested in – `edit-config` and `netdata.conf`. Just to see all the configuration options Netdata offers, go ahead and type `./edit-config` to see a *very* large print out of all our options! Wow!
That probably filled up your terminal window… get rid of it by typing `clear` and hitting enter. Now, let’s make our configuration change! Enter `sudo ./edit-config netdata.conf` to open the main configuration file we need to make changes to. On our very first open it will look very empty, but there are instructions to download the latest version of the configuration file at the top. Use ctlr + x to exit and return to the command line. Now enter `sudo wget -O /etc/netdata/netdata.conf http://localhost:19999/netdata.conf` to overwrite file we just opened with an up-to-date version. Use your arrow keys to find the command or simply type `sudo ./edit-config netdata.conf` to re-open the configuration file. It should now be an expansive document full of options! Much better!
Inside our `netdata.conf` file, scroll down to the page until you see [ml]. By default, most of this document is commented out and/or configurations are turned off. In the case of machine learning, both are true, so let’s fix both! First, go to the line that reads “enabled”. Next, uncomment the line by removing the hashtag in front of it and ensuring the text is properly aligned. Then, delete “no”, type “yes”, save, and exit. Finally, restart Netdata with the `sudo systemctl restart netdata` command. Return to the Netdata Cloud application, browse to the bottom of the right-hand sidebar, and you should see “Anomaly Detection” that wasn’t there before… super cool! The “Anomalies” tab will be empty until it’s collected enough data to come to some conclusions, but you can speed this up by making some more configuration changes.
By now, you should have the most up-to-date version of Ubuntu 20.04, Netdata installed, and machine learning activated. Good job! Now you’re ready to set up a collector and then export that collection to an external time-series database of your choice.
Remember that Netdata is the only monitoring solution that comes with every tool, collector, and exporter you could think of included in the original installation. It’s just up to you to find out what your infrastructure monitoring needs are! Be sure to check out the Netdata docs and guides for careful instructions to help you succeed. Don’t be afraid to drop by our official Discord channel or our forums if you need some help along the way! We will be there to help you out!