Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [elk]

a.k.a. Elasticsearch, Logstash, Kibana ; refer to elastic.co

59 questions
8
votes
1 answer

How to forward Docker container logs to ELK?

I would like to know what is the easiest way to forward my docker container logs to an ELK server, so far the solutions I have tried after having searched the internet didn't work at all. Basically I have a docker image that I run using…
ndarkness
  • 193
  • 1
  • 7
8
votes
1 answer

ELK Stack (Logstash, Elasticsearch and Kibana) with concurrent remote syslog server?

I'm building a log analyser service to start monitoring mainly our pfSense Firewalls, XenServer Hypervisors, FreeBSD/Linux servers and Windows servers. There's a lot of documentation on the internet about the ELK stack and how to make it work…
Vinícius Ferrão
  • 5,400
  • 10
  • 52
  • 91
4
votes
1 answer

Externally visible Kubernetes Service on Azure

I am deploying an ELK stack on a Kubernetes(v1.4.5) cluster on Azure. This is the configuration that creates the Kibana Service and Deployment. # deployment.yml --- apiVersion: v1 kind: Namespace metadata: name: logging --- # elasticsearch…
Georgi Tenev
  • 43
  • 4
3
votes
1 answer

Monitoring and alerting tools for small businesses in 2016

This is a kind-of-recurring question, yet the closest one I could find was asked 7 years ago, which was pretty much a different time. I run a small business and we host multiple small-to-medium client websites (nothing that ever required more than a…
Xowap
  • 151
  • 3
3
votes
1 answer

elk stack error "unable to fetch mapping do you have indices matching the pattern"

i am trying to setup ELK stack with collectd on Ubuntu 16.04 LTS (so pretty much latest version of stack available) kibana is behind nginx proxy (followed this guide…
uberrebu
  • 493
  • 5
  • 15
  • 32
3
votes
2 answers

How to add AWS ELB access logs to logstash with S3 input?

I'm using an ELK stack for server monitoring. My application's access logs which are from AWS ELB are stored in AWS S3. I am trying to add them to logstash with the following input: input { s3 { access_key_id => "my_id" secret_access_key…
apanagiotou
  • 43
  • 1
  • 4
3
votes
1 answer

Logstash event @timestamp adjustment

I have standard Windows IIS log files with event date/time stamp information and timetaken (in milliseconds). I would like to be able to adjust the event time (@timestamp) by subtracting the "timetaken" to be able to record when the event started…
Guy
  • 2,658
  • 2
  • 20
  • 24
2
votes
0 answers

Logstash to aggregate logs into ElasticSearch

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS. filebeat is already running in EKS to aggregate Kubernetes container logs. I have configured rsyslog client with…
roy
  • 119
  • 1
  • 2
  • 12
2
votes
0 answers

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", "connection": "keep-alive", …
vkpro
  • 21
  • 1
  • 2
2
votes
1 answer

Filebeat can't connect to logstash on another server

Filebeat (11.11.11.11) can't connect to logstash (22.22.22.22) on another server (connection reset by peer). But filebeat services from other servers can do it. Also I can connect from this server(11.11.11.11) using telnet to this port (telnet…
Dmitry
  • 179
  • 1
  • 2
  • 8
2
votes
0 answers

filebeat makes a lot of I/O

We have filebeat on few servers that is writeing to elasticsearch. We can see that it is doing a lot of writes: PID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 353 be/3 root 0.00 B/s 4.52 K/s 0.00 % 0.55 %…
usterk
  • 121
  • 1
  • 4
2
votes
1 answer

ELK logstash and core grok patterns

I'm evaluating the ELK stack with filebeat & logstash across a diverse range of applications/ servers. I understand the power of customising my own grok patterns for each application/log, but to get running initially it seems very inefficient to…
Dan Poltawski
  • 141
  • 1
  • 3
2
votes
1 answer

Trouble bringing in CloudWatch data to Logstash

Just getting started with a new ELK setup (never used it before, just trying to learn it). I have Logstash 2.2.4 running on ubuntu 14.04 LTS. After putting a yaml file down with my monitor user's AWS credentials (policy configured as per the…
Paul
  • 968
  • 1
  • 11
  • 19
2
votes
0 answers

Is there any way to get log events from crashlytics / fabric into ELK or a SaaS metrics platform like datadog?

If you use a stack like ELK or datadog for collecting server-side logs and events, how do you integrate mobile-side metrics? Is there any way to get these out of crashlytics directly, or does this log aggregation need to be implemented separately?
fields
  • 690
  • 1
  • 10
  • 21
2
votes
0 answers

How do I update Logstash data to Redshift real-time?

I am looking for a way to build a pipeline for storing the logs data (I use the ELK stack) to AWS Redshift, for analytics. I have gone through the Output plugins for ElasticSearch, and found one for writing data to AWS S3. But, it writes data in…
Dawny33
  • 133
  • 1
  • 7
1
2 3 4