Laur IVAN

Monitor ethOS Distro with Elastic Search – 3

Posted by in IT, Software, Tutorial

Once we’ve set up the data sending from out ethOS machine, we can check that ElasticSearch collects data. With GET /ethos/_count, we get something like:

If the count is greather than 0, we can start analysing the data. Metrics I am currently measuring only machine. Therefore, I’ve set up some simple measurements: Hash rate Temperature Network Fan speed Hash rate I am measuring the average sum of all GPUs’ hash rates and the trend: I’ve built this with Timelion and the code is:

As you can see, there…read more

Monitor Docker Logs with ELK – 2

Posted by in IT, Tutorial

This second post in the series provides a starting point for visualising the log data. It makes some assumptions that logs are collected in a way described by the first part, so I would suggest to read through it if you haven’t already. Verify First, make sure you have some data logged in: You should have at least two indices:

The Discover tab in Kibana should display some records. If it doesn’t please adjust the Time Range (top right at the time of writing). Tip: I set up a…read more

Monitor Docker Logs with ELK – 1

Posted by in IT, Tutorial

Following my attempts to monitor ethOS data via ElasticSearch and Kibana, I’ve thought it would be a good idea to start monitoring the logs produced by my various docker instances. This first part deals with configuring ELK and logspout. Prerequisites I assume the following: ELK is installed. You can do it either as a set of Docker images (oh, the meta!) or just plain package install. The receiver machine is configured. E.g., if you’re on windows, create a firewall rule opening TCP port 35000 (corresponding to the config). Once this…read more

Monitor ethOS Distro with Elastic Search – 2

Posted by in IT, Software, Tutorial

Following the previous post, we are now in position to install the monitoring code on the monitoried machine, of course. The easiest way is to download the code on the machine in /home/ethos. Following steps will assume you have the unpacked code in

Once you have downloaded the code, you need to add the location of the ElasticSearch server to the local.conf:

Dissecting the scripts The scripts are written in PHP, like ethOS‘s own scripts. I did that for two reasons: Take advantage of the already written code…read more

Don’t Symlink Indices in ElasticSearch!

Posted by in IT

The other day I’ve been trying to improve the downtime when updating an index in elasticsearch (ES) and I’ve decided the best approach would be to create a new index and do a symlink to the active one, keeping the valid index. so, my structure would be:

Note: this is likely not a best practice :) This would allow me to build an index with other means, and only do a stop/change link/restart cycle which would be much quicker :) This is simplified, of course, but you get the…read more

Search exact string in a file on Windows

Posted by in IT

The other day I found a file with a peculiar format and I needed to find it again. There are two problems I’ve encountered: Size: It’s one of 20,000,000 XML files Search: I need to find a file containing a string “xocs:doi”, which is not interpreted as a single word (due to the colon character, of course) My case is search through XML files, but I’ve figured the method is generic enough for any text files. TL;DR: Use the power of findstr, Luke! First Option – Windows Explorer Searching through…read more