On RedHat and CentOS 6 and 7, anacron generates syslog messages that are mangled when they are forwarded by rsyslog. I found the cause and a solution in a comment by Tomas Heinrich on this old Fedora bug. Sadly, that bug was closed instead of getting fixed, probably because Fedora is oriented toward desktop users who are not generally forwarding syslog messages! Syslog uses the following default template for messages:
"%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
The syslogtag field is truncated to 32 characters, but cronie-anacron includes a lot of data in its tag. The tag gets truncated to 32 characters, resulting in invalid syslog. If you try to send them to Logstash, you’ll get errors like this:
{"message":"Nov 4 19:01:01 my-web-03 run-parts(/etc/cron.hourly)[1858 starting 0anacron","@version":"1","@timestamp":"2016-11-05T00:01:01.146Z","port":55456,"type":"syslog","tags":["_grokparsefailure"],"host_ip":"192.168.17.23"}
Unfortunately, the problem is with the sending host, not Logstash, so you have to update the rsyslog config on every host that generates anacron messages. I prefer to do this in /etc/rsyslog.d to keep the configs more manageable. Add these two lines, which add a template that has no restriction on the length of the tag field, and activate the template when forwarding:
$Template fwd,"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%"
$ActionForwardDefaultTemplate fwd
I placed these lines in /etc/rsyslog.d/logstash.conf along with my other Logstash-specific rsyslog configs.