Monitoring systems with Elasticsearch and MetricBeat

Beats is a companion product of Elasticsearch. It is defined as single purpose, lightweight data shippers. Each type of beat is designed to load a very specific data type into Elasticsearch. The Beats family includes Filebeat, which I covered in more detail in this post, Metricbeat, Packetbeat and Winlogbeat. Those are the official beats from Elasticsearch. But this is an open architecture and there are plenty of other beat created by the community that cover many other types of system. Those can be seen here.

This post will focus on Metricbeat, which is designed to collect metrics from various systems and visualize them as Kibana dashboards to make it easier to monitor those systems. Metricbeat was evolved from the Topbeat which was able to visualize the results of the top Linux/Unix command, but Metricbeat can do much more thanks to its modular design. It can collect metric data from a variety of sources and push them to Elasticsearch. There are modules for system data, Apache, Docker, Kafka, MySQL, MongoDB, PostgreSQL, Zookeeper and many more. Later this data is visualized as Kibana dashboards.

Lets start by installing Elasticsearch and Kibana. You can use this post for guidance and this one if you want to create an Elasticsearch cluster.

Assuming you already have the elasticsearch yum repository set up from previous step, run:

yum install metricbeat

Now edit the metricbeat configuration file at /etc/metricbeat/metricbeat.yml

The file has two main sections: The modules section that specifes which modules will be active and what data Metricbeat will gather, and the output sections that specifies where Metricbeat should send its findings – Elasticsearch or Logstash.

The configuration file I used for this post can be found here (I only used system and MongoDB modules). You should, of course, change the host names according to your setup. This is a minimal file that contains only what I needed, but there are many more options.

In the same directory where the configuration file is, you can find a file named metricbeat.full.yml that contains all the possible modules and output options, so you can see in it examples for other modules, for logging options and for Logstash output.

You can find more information on configuring MetricBeat here.

This handles the data Metricbeat gathers and where it sends it. Now we want to take care of its visualization in Kibana. Metricbeat comes with pre-built example templates that we should upload to Kibana before we can see any dashboards. There are two options for doing it.

Automatic upload

Metricbeat is shipped with a template file that is /etc/metricbeat/metricbeat.template.json

It will automatically load the template file at startup if those options are defined in the metricbeat.yml config file in the Elasticsearch output section:

template.enabled: true "metricbeat"
template.path: "/etc/metricbeat/metricbeat.template.json"
# Overwrite if template already exist
template.overwrite: true

Manual upload

You can also manually upload the latest template directly from Elasticsearch servers using the import_dashboards script that is located under the Metricbeat binary directory.  The “es” parameter is the url of Elasticsearch where the templates should be loaded:

/usr/share/metricbeat/scripts/import_dashboards -es

Now let’s go and see how it looks like in Kibana at http://<hostname>:5601

If this is the first time you enter Kibana, it will ask you to specify a default index pattern. you can enter “metricbeat-*”. If you already have a different default index pattern then you can find “metricbeat-*” under management -> index patterns:

View full size image

If you click on it you can see a very long list of fields, that corresponds to the collected metrics. You can later add those metrics to your custom dashboards:

View full size image

Metricbeats comes with basic dashboards that are good only for demonstration. If you want to see its true power you should customize those dashboards.

Just for example, we will test the system module which is the default and MongoDB module, but there are many more available.

In order to access the Metricbeat dashboards, go to dashboards -> open and you will see a list of available dashboards to the left:

When you click one of the above links, the dashboard opens. For  example, this is the CPU dashboard:

View full size image

Now let’s try MongoDB. My MongoDB is quite idle so we need to create some action in order to see something happening in the dashboard. I used this small bash script to create a 30 Million lines csv file:



while [ "$COUNTER" -le "$MAXLINES" ]
  echo $COUNTER,$NUMBER,"Line number" $COUNTER >> sampledata.csv
  let "COUNTER+=1"

Then I loaded it into MongoDB using mongoimport:

mongoimport --host mongo1:27000 --db test --collection sampledata --fields line,value,message --type csv --file /root/sampledata.csv

Let’s look at the Mongo dashboard:

View full size image

View full size image

You can see the operation counters rising and the WiredTiger cache filling as the database imports the data. However this is not very useful and many important parameters are missing (for example replication lag). You should edit the dashboard or create your own to add those extra metrics to make it really useful.

By default, auto refresh is off. To activate it you have to choose auto-refresh from the upper menu of the dashboard and set the desired interval:

View full size image

To view historical data, choose the men item just next to auto refresh and choose the time frame you want to see:

View full size image


Metricbeat is a versatile framework that can collect metric data from many sources, and the list of sources continuously grows thank to independent, Third party modules. It is also easy to configure and operate and with some Kibana skills you can create great looking dashboards.

It can do some monitoring but I do not think it can compete with specialized monitoring systems like Nagios or Ganglia. A better use case will be as a tool for convenient gathering of metric data from various machine sources and storing them in one place, in order to later run analytics on this data.


This entry was posted in Beats, ElasticSearch and tagged , , , . Bookmark the permalink.

Leave a Reply