First of all, I'm a total newbie to logstash. Despite the fact that I've managed to achieve some basic logging (trying to parse an apache log file without the build in COMBINEDAPACHELOG). However, I ran stuck on the following error that gets spammed at my terminal as soon as /var/log/auth.log receives an update (lines get appended). Logstash version 1.4.2. OS Ubuntu server 14.04
Go figure what's happening here, the only thing I know is that it locks the logging process of any of the other log files.
Any ideas on what I'm doing wrong or causes this. Besides this, I have a few more questions, these are at the bottom.
additional info:
the config file
grok patterns are the defaults + the custom ones below. The groks where created using
https://grokdebug.herokuapp.com/
http://grokconstructor.appspot.com/do/match
error groks aren't perfect. grok parse errors on run so they aren't useful for visualization in kibana. See the demo logs underneath the patterns for examples to which they should be matched.
Demo logs
If additional info is required, please tell.
The other questions at hand:
how to purge the history? => force a full re-parse of all the log files
suggestions on best practices and improvements of the groks
Thanks in advance