I have a number of CoreOS servers on Amazon AWS and would like to collect events or logs from them and forward them onto my ELK stack provider logz.io.
Being a little new to the ELK stack, I'm a little lost on how to best get the data out. It was suggested to me to use filebeat. But filebeat works with ordinary log files and most of the data is currently being fed into journald.
All my services are running on docker and I see that docker has the idea of logging drivers. Could I plug into that in some way?
There is a project called journalbeat which looks promising, but incomplete and currently forwards everything. I just want specific units and actually, specific events. So I may have to work on that project to make it do what I need. But, before I go further there, is there perhaps another way?
e.g. another lightweight service that I could run on the machine that might accept logging events and forward them on. So long as it's not logstash because that will consume too many resources on my already memory and cpu constrained aws instances.
I'm surely not the only person to run into this type of issue.