Checking and fixing lohstash configuration file

This is a short post that may save you some headache.

Sometimes you change the logstash configuration file and after that logstash won’t start and you can see errors like this in the log:

{:timestamp=>"2016-08-31T13:10:40.251000+0300", :message=>"fetched an invalid config", :config=>"input {\n  kafka {\n\tconsumer_threads => 1\n\ttopic_id => \"elastic\"\n\tzk_connect => \"\"\n  }\n}\noutput {\nelasticsearch {\naction => \"index\"\nhosts => \"\"\nindex => \"warandpeace\"\ndocument_type => \"line\"\nworkers => 1\n\n}\n \n}\n\n\n\n", :reason=>"Something is wrong with your configuration.", :level=>:error}

The parameters themselves will, of course, vary according to your configuration settings. However, this is not very informative. It says that something is wrong in the configuration but does not say what.

To find out, go to the directory where logstash binaries are (in my case it’s /opt/logstash/bin) and run (replace the configuration file path with your own):

logstash --configtest --config /etc/logstash/conf.d/logstash.conf

The output will tell you exactly which configuration parameters are wrong:

Unknown setting 'topic_id' for kafka {:level=>:error}
Unknown setting 'zk_connect' for kafka {:level=>:error}
The given configuration is invalid. Reason: Something is wrong with your configuration. {:level=>:fatal}

Now you can fix the bad parameters and try again.

This entry was posted in Logstash and tagged . Bookmark the permalink.

Leave a Reply