Therefore I added this to the volumes part of elasticsearch config. This enables you to keep the data after a restart. One of the things I like is adding a volume to a local folder for the Elasticsearch data folder. I tried to stick as much as possible to the notations of the original author. One of the things with Docker and especially with docker-compose scripts is that every script you open seems to be different. Other resources are the documentation pages of Elastic: Of course, it was not completely what I needed, but with some small changes, I could easily work with it.

This gave me a jump start to configure the complete ELK stack. The most interesting resource for this experiment was this GitHub repository: “ “. The sub-goal for the experiment is gaining experience with Elastic X-Pack features that are available through Kibana concerning pipeline monitoring. Using the most recent version of Logstash (7.6 at the time of writing), I want to implement an example using multiple pipelines, one for the input of UDP messages, one for the output to elasticsearch using multiple indexes, and more than one for the filter part of different paths. The main goal for the experiment is to make a more robust Logstash configuration for a project with a lot of steps in the filter part and some performance issues on the UDP input. In this blog post, you will learn about some of my findings to make this experiment work. To make it run on my laptop I decided to have a look at a docker-compose configuration. I started a small experiment to investigate using pipelines, and persistent queues. Before diving into the configuration, I wanted to have a look at a few options that Logstash provides these days. Like always, there were some things to improve. The past week I was going over an ELK configuration for log processing at a customer.

Search Create a Robust Logstash configuration

Make my business app more robust