Monitor Docker Logs with ELK – 2

Posted by in IT, Tutorial

This second post in the series provides a starting point for visualising the log data. It makes some assumptions that logs are collected in a way described by the first part, so I would suggest to read through it if you haven’t already.

Verify

First, make sure you have some data logged in:

  • You should have at least two indices:

  • The Discover tab in Kibana should display some records. If it doesn’t please adjust the Time Range (top right at the time of writing).

Tip: I set up a large window (e.g. “last 7 days”) and then click-drag the relevant period of time:

Once you have the logs, open up Timelion and select the relevant time window (again, by setting up the Time Range).

Visualise

A starting point is to split the logs on INFO, WARNING, ERROR. The simplest Timelion config I could come up with is:

(remove the “\” and new-line as the string needs to be on the same line if you want to just paste it).

I ran into two issues:

  1. The number of DEBUG messages was overwhelming
  2. A lot of 0 values popped up

After removing the DEBUG filter, I got:

Unfortunately, most entries would have 0 values for the three log levels left. Therefore, I came up with a (convoluted) expression:

The result is much better now:

I’ve stopped here for the time being because I have a heterogeneous structure with NGINX, Apache, Django/Gunicorn and NPM. Following steps are to create logstash filters for the different types of logs…

A final note: One of the apps I left had debug messaging on (hence 2.5M entries). you might want to rotate the logs/indices.

HTH,


A little experiment: If you find this post and ad below useful, please check the ad out :-)