Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine. Architected from the ground up for use in distributed environments where reliability and scalability are must haves, Elasticsearch gives you the ability to move easily beyond simple full-text search. Through its robust set of APIs and query DSLs, plus clients for the most popular programming languages, Elasticsearch delivers on the near limitless promises of search technology.

http://www.elasticsearch.org

373 questions
3
votes
3 answers

Simple full-text search server

I have been looking at search solutions like Sphinx, Solr and Elasticsearch but they are all way too complex for what I need. I'm basically looking for a server software, best distributed, that allows me to just throw in chunks of text associated…
thwd
  • 177
  • 1
  • 5
2
votes
0 answers

Logstash to aggregate logs into ElasticSearch

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS. filebeat is already running in EKS to aggregate Kubernetes container logs. I have configured rsyslog client with…
roy
  • 119
  • 1
  • 2
  • 12
2
votes
2 answers

Elasticsearch: how to "rescue" the documents that could not be parsed by mapping?

We're using ElasticSearch to store and inspect logs from our infrastructure. Some of those logs are required by law, and we can't afford to lose any. We've been parsing logs for quite some time without any mapping. That makes them mostly unusable…
aspyct
  • 340
  • 6
  • 19
2
votes
0 answers

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", "connection": "keep-alive", …
vkpro
  • 21
  • 1
  • 2
2
votes
0 answers

AWS ElasticSearch on private subnet - safe to have no signed url/access policy?

I have an ES cluster setup within our VPC and attached to one of our private subnets. It's also attached to a security group allowing access from particular instances. I have the following policy attached: { "Version": "2012-10-17", "Statement":…
Harrison Lucas
  • 155
  • 6
2
votes
1 answer

vm.max_map_count problems on GKE ElasticSearch StatefulSet

A problem appeared on working ElasticSearch cluster on GKE. Nodes with "data" roles began to crash unexpectedly with an error: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] bootstrap checks failed Of…
Victor Bessonov
  • 113
  • 1
  • 2
  • 9
2
votes
1 answer

Can't see pod logs in Stackdriver UI for cluster deployed on GKE

I deployed an Elasticsearch cluster on Google Kubernetes Engine. I am able to view logs of specific pods with the 'kubectl logs' command from my PC. >kubectl logs es-data-0 ... [2018-06-18T07:35:11,220][INFO ][o.e.m.j.JvmGcMonitorService]…
syst0m
  • 131
  • 1
  • 4
2
votes
1 answer

Filebeat can't connect to logstash on another server

Filebeat (11.11.11.11) can't connect to logstash (22.22.22.22) on another server (connection reset by peer). But filebeat services from other servers can do it. Also I can connect from this server(11.11.11.11) using telnet to this port (telnet…
Dmitry
  • 179
  • 1
  • 2
  • 8
2
votes
1 answer

How to check if network traffic on a specific port is secured/encrypted

We are running Elasticsearch 5.5.1 on Windows 2008 R2. We have installed Searchguard SSL plugin to enable SSL on Elasticsearch's transport port(9300) and are using jks certificate provided by our admin. How do I verify that the communication on this…
askids
  • 121
  • 1
  • 5
2
votes
1 answer

How to set max file descriptors for a container?

I want to run elasticsearch in a container (namely nspawn) and it requires the number of max file descriptors to be higher than the default 1024. This is done, in the container, in the systemd service file appropriate for elasticsearch. This request…
WoJ
  • 3,365
  • 8
  • 46
  • 75
2
votes
1 answer

elastichsearch node health check for haproxy

I have place haproxy in front of a three node ES(elasticsearch) cluster. So far the way i check for each node in haproxy is by using httpcheck. Bellow is a snippet of my config: backend elastic_nodes balance roundrobin option forwardfor option…
giomanda
  • 1,644
  • 4
  • 20
  • 30
2
votes
1 answer

How do I feed Elasticsearch my Snort alert log?

I started off yesterday with an ELK howto and got ELK up and running rather easily. Next thing I wanted to do was plug my Snort alert log into it. I configured Logstash (shown below) with a filter and an absolutely nasty Grok regex to split up all…
Server Fault
  • 3,454
  • 7
  • 48
  • 88
2
votes
1 answer

Log file in Topbeat

I am new to Elasticsearch and Topbeat tools. When I start the topbeat service and look through my browser by typing localhost:9200/topbeat-*/_search?pretty, I get data about my system. But I want to know where the complete information file is being…
Aniket Pandey
  • 21
  • 1
2
votes
1 answer

ElasticSearch container on CoreOS with persistent storage

I'm trying to run an ElasticSearch cluster on Docker and CoreOS [stable] on DigitalOcean. To get persistent storage working, I tried pretty much every option and running out of ideas. Pretty much every --volume option (-v /data/elasticsearch, -v…
Patrick
  • 351
  • 1
  • 3
  • 11
2
votes
1 answer

How to format log data before forwarding them as JSON to elasticsearch?

I log all events on a system to a JSON file via syslog-ng: destination d_json { file("/var/log/all_syslog_in_json.log" perm(0666) template("{\"@timestamp\": \"$ISODATE\", \"facility\": \"$FACILITY\", \"priority\": \"$PRIORITY\", \"level\":…
WoJ
  • 3,365
  • 8
  • 46
  • 75
1 2 3
…
24 25