Web Servers Log monitoring with Netdata

Why monitor Web servers?

Web servers are among the most important components in modern IT infrastructures. They host the websites, web services, and web applications that we use on a daily basis. Social networking, media streaming, software as a service (SaaS), and other activities wouldn’t be possible without the use of web servers. And with the advent of cloud computing and the movement of more services online, web servers and their monitoring are only becoming more important. Given the extensive usage of Web servers, Sysadmins and SREs should monitor web servers as a key aspect for performance.

“Working of Web Servers”

What to monitor on Web Servers?

There are dozens of web servers, but the two most popular are Apache and NGINX. And irrespective of the web server in use, apart from the host metrics on which the web server is hosted like - Uptime, CPU Usage, Memory Usage, Cache, Threads etc - you will need to monitor metrics related to the Web server connections themselves and some of the more important ones are:

Web Server Log Collector in Netdata

Both Apache and NGINX report internal metrics via an HTML page. Apache has the mod_status module, and NGINX has the ngx_http_stub_status_module module. The Netdata plugins for NGINX Monitoring and Apache Monitoring generate metrics and alerts from these pages. But if you can provide Netdata access to their log files, you can get even more insights via the generic Web Server Log Monitoring collector.

Once the collector is set up, it comes with default summary charts (which can be modified on the custom dashboards in the future) and gives a high level indication of how the web server(s) are performing.

“Web Log”

Request Rate: Now, going into the individual metrics let us look at the most important ones and the rate of “requests” being handled by the Web servers determines the load and an indication of whether sufficient computing resources are allocated to the service if the cpu, memory, etc. are showing signs of exhaustion.


If you are monitoring multiple Web servers, you can further group by “instance” to see the requests being handled by each of your web servers. Now, the dimension names look funny. That’s because our front-end does some funky things to keep the length of each dimension within a certain width. When you hover over the chart, you see the fully expanded dimension names.

“requests 2”

Response rate / Error Codes: The next important metric to monitor is the “response” rate to compare if the request and response rates are in order. And in addition the error codes on responses will provide a good idea of any issues in serving the requests.

“Response Rate”

Response time and Response size: In addition, monitoring the “request processing time” (response time) and “bandwidth” (response size) gives a realistic view of the web server’s capability of serving multiple requests and can point to any latencies on specific web server instances.



Type of Requests: In the world of web servers, it is also crucial to understand the types of requests and clients to allocate the necessary resources/networking.

“client” “http method” “ip proto”

The Web server log collector also comes with default alerts which can be further customized to your infrastructure’s requirements. You can view all the active alerts and configured alerts on the Alerts tab.

“web servers alerts”

For more information on configuring the Web Server Log Collector using Netdata and setting up custom alerts look into the enclosed links.

Get Netdata

Sign up for free

Want to see a demonstration of Netdata for multiple use cases?

Go to Live Demo