Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine. Architected from the ground up for use in distributed environments where reliability and scalability are must haves, Elasticsearch gives you the ability to move easily beyond simple full-text search. Through its robust set of APIs and query DSLs, plus clients for the most popular programming languages, Elasticsearch delivers on the near limitless promises of search technology.

http://www.elasticsearch.org

373 questions
1
vote
1 answer

Field/value extraction with ELK

I have an industrial system producing log files where some of the lines look like this: component1 v1 component2 v2 component3 v3 ... Where vx is a numerical value (eg. 3.14159). I'm running a super basic ELK stack and I would like to extract these…
Cedric H.
  • 159
  • 1
  • 8
1
vote
1 answer

Loading field information in Graylog 2.1.x failed with status 500 (Internal Server Error)

I'm working on setting up a Graylog and I'm getting the following error: Loading field information failed with status: cannot GET http://192.168.2.108:12900/system/fields (500) I am on this…
Jason
  • 3,821
  • 17
  • 65
  • 106
1
vote
1 answer

elastic's snapshot and restore module repository_exception

I'm using elk-docker and trying to follow Snapshot And Restore | Elasticsearch Reference [2.4] | Elastic and getting following error: # curl --request PUT --data '{ "type": "fs", "settings": {"compress": true, "location":…
alexus
  • 12,342
  • 27
  • 115
  • 173
1
vote
1 answer

rsyslog and elasticsearch: How to configure multiple servers?

We are currently setting some hosts to forward their logs via rsyslog and omelasticsearch to an elasticsearch cluster. The manual for omelasticsearch seems to allow only one server name of the ES cluster to be configured, which would be a single…
Martin Schröder
  • 315
  • 1
  • 5
  • 24
1
vote
0 answers

Delayed synchronization between two AWS ElasticSearch Service instances

I need to synchronize the indexes of two AWS ElasticSearch Service instances once a day. It's really important that the synchronization is NOT real time. The first index (let's call it ES-A) is used internally by our team while the second index…
Quentin Hayot
  • 523
  • 3
  • 7
1
vote
3 answers

Elasticsearch high RSS memory consumption

I am seeing too high memory consumption in my elasticsearch instance. I set ES_HEAP_SIZE=4g. The starting command for ES starts with: /usr/lib/jvm/java-8-oracle/jre/bin/java -Xms4g -Xmx4g So far so good. But I am seeing more than 7GB RSS memory…
usamec
  • 69
  • 2
1
vote
1 answer

Signing/Generating SSL Certificates for ElasticSearch (search-guard)

I am trying to build my own ElasticSearch cluster and I decided to use an open source plugin called Search Guard. It requires generation of client and host/node certificates. In its demo there is a script which generates a root ca, client and node…
iboware
  • 111
  • 3
1
vote
2 answers

How to balance elasticsearch (java) on a very powerful machine?

On a 16 core XEON class, 128GB RAM RHEL server I want to deploy elasticsearch. What is preferable performance-wise? Have a huge elasticsearch process to use all resources on the native host? Break the host to e.g. 4 equal virtual machines (KVM)…
yannisf
  • 577
  • 2
  • 5
  • 15
1
vote
0 answers

object mapping for [doc.awsRegion] tried to parse field [awsRegion] as object, but found a concrete value

I am using Logstash Dynamodb PLugin to index data to be able to query it. Everything was running pretty smoothly, until one of my teammates started messing with the schema, and broke it. We can still index the data fine, however new data no longer…
user3521621
  • 265
  • 1
  • 4
  • 11
1
vote
1 answer

Sending docker logs to logstash

I have a number of CoreOS servers on Amazon AWS and would like to collect events or logs from them and forward them onto my ELK stack provider logz.io. Being a little new to the ELK stack, I'm a little lost on how to best get the data out. It was…
hookenz
  • 14,132
  • 22
  • 86
  • 142
1
vote
1 answer

Occasional "failed to respond" from AWS hosted Elasticsearch

We have an Elasticsearch cluster hosted on Amazon Elasticsearch Service (AWS). We are using the Jest Java HTTP Rest client for ElasticSearch. Every now and then (perhaps 1 in 10,000 requests), it appears to close the connection without a…
Rich
  • 626
  • 11
  • 28
1
vote
0 answers

Redis creates many events in logstash

I'm doing some testing for a POC with logstash. When using redis to buffer messages it appears they stay in redis and continually get added to elasticsearch as new events. For example, if I restart a service I'll see the same event many times when…
tweeks200
  • 351
  • 3
  • 11
1
vote
0 answers

Visualize multiline ruby exceptions in kibana 4

I have setup the latest version of Kibana4 ElasticSearch stack. The logs are being pooled from remote app sources which are running on Ruby. I want to search for Multi Line exceptions created by ruby. Is there a way in kibana where we can search…
Swapnil jaiswal
  • 111
  • 5
1
vote
2 answers

Nginx reverse proxy for elasticsearch - inserting filters

I've got an elasticsearch instance, with an Nginx reverse proxy sat in front of it, implementing a URL level access control mechanism - approximately as described in this article This is giving me per-index granularity, which is handy. What I'm…
Sobrique
  • 3,697
  • 2
  • 14
  • 34
1
vote
0 answers

Filebeat and downstream availability

I read here and there that a broker (like Redis) might not be required in the log pipeline (typically ELK) when Filebeat is used. From Filebeat's official page: [Filebeat] is intelligent enough to deal with [...] the temporary unavailability of…
Maxim Gueivandov
  • 83
  • 1
  • 9
1 2 3
…
24 25