Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine. Architected from the ground up for use in distributed environments where reliability and scalability are must haves, Elasticsearch gives you the ability to move easily beyond simple full-text search. Through its robust set of APIs and query DSLs, plus clients for the most popular programming languages, Elasticsearch delivers on the near limitless promises of search technology.

http://www.elasticsearch.org

373 questions
6
votes
2 answers

Sending Subversion logs to Logstash

My requirement is to send subversion logs(i.e username,revision number...) to logstash for parsing(then store it in elastic search and finally displayed it via kibana).Since subversion use its own file-based database(FSFS) and not plain text file,I…
Prashant Lakhera
  • 683
  • 1
  • 9
  • 25
6
votes
0 answers

Is there disk based file system with dedicated memory caching for Linux?

We have two (or more sometimes) applications battling for the OS page cache. In particular ElasticSearch is very dependent on the OS page cache. To the point that cache misses can make or break performance in low latency cases. ES is the only low…
Andrew White
  • 228
  • 2
  • 7
6
votes
1 answer

ElasticSearch Server Randomly Stops Working

I have 2 ES servers that are being fed by 1 logstash server and viewing the logs in Kibana. This is a POC to work out any issues before going into production. The system has ran for ~1 month and every few days, Kibana will stop showing logs at some…
Eric
  • 1,373
  • 3
  • 17
  • 33
6
votes
1 answer

How well does elasticsearch compress data?

I am looking to scope the servers required for an elasticsearch proof-of-concept. Ultimately, my question is this: Given 1GB of json text indexed by elasticsearch, how much can I expect to be occupied on disk by elasticsearch? Obviously there are…
cmonkey
  • 163
  • 1
  • 5
5
votes
1 answer

What does "GC (Allocation Failure)" mean in my ElasticSearch 5.6 logs?

I see these constantly in my Elasticsearch 5.6.3 logs. Is this a signal that I should scale up and add more RAM? Or is this just normal operations for ElasticSearch? [GC (Allocation Failure) [ParNew Desired survivor size 11141120 bytes, new…
Antonius Bloch
  • 4,480
  • 6
  • 28
  • 41
5
votes
2 answers

"Index Patterns: Please specify a default index pattern" in Kibana

I'm trying to create a simple hello world for ELK and be able to see kibana reports via the internet. I've installed kibana, logstash, nginx and elastic search. Here's my /etc/logstash/conf.d/10-syslog.conf: input { file { path => […
Dett
  • 53
  • 1
  • 4
5
votes
1 answer

Why would a node not have shards?

I am setting up a small elasticsearch cluster and one of the nodes does not get any shards. It works, is part of the cluster - just empty (eu2 in the screenshot of head below): This node is a worker, the only difference in the configuration vs. the…
WoJ
  • 3,365
  • 8
  • 46
  • 75
5
votes
1 answer

Elasticsearch Debugging

Our elasticsearch is a mess. The cluster health is always in red and ive decided to look into it and salvage it if possible. But I have no idea where to begin with. Here is some info regarding our cluster: { "cluster_name" : "elasticsearch", …
Beginner
  • 211
  • 1
  • 4
  • 6
5
votes
3 answers

Ubuntu 15.04: Elasticsearch service doesn't start

I followed the first part from the tutorial here to install Elasticsearch. I didn't install java because I already have that installed with other java programs running. sudo service elasticsearch restart Doesn't output anything. sudo service…
MadeOfAir
  • 201
  • 2
  • 3
  • 7
5
votes
2 answers

What is the purpose of elasticsearch logs? How to manage them?

As I understand indexes(or data) are being stored in /var/lib/elasticsearch by default, this folder contains nodes with 0 and 1 folders and overall size of these folders is 376M. The logs are being stored in /var/log/elasticsearch -rw-r--r-- 1…
Rustam A. Gasanov
  • 225
  • 2
  • 9
5
votes
2 answers

How to index MySQL from ElasticSearch

Im trying to add some full text searching to my LAMP web application. Ive come across ElasticSearch and I like what it has to offer. I have read the guides and understand how to index a single item by using curl -XPUT http://localhost:9200/. But how…
CraigH
  • 195
  • 2
  • 7
5
votes
2 answers

How to secure an Internet-facing Elastic Search implementation in a shared hosting environment?

(Originally asked on StackOverflow, and recommended that I move it here) I've been going over the documentation for Elastic Search and I'm a big fan and I'd like to use it to handle the search for my ASP.NET MVC app. That introduces a few…
casperOne
  • 151
  • 6
4
votes
1 answer

elasticsearch server is unreachable every 2 hours

This question is related to this one. We now know that the errors come from elasticsearch. The problems are still not resolved despite the modifications and optimizations made on the es instance. Every 2 hours the es server becomes unreachable: we…
COil
  • 207
  • 3
  • 12
4
votes
3 answers

Logstash / Elasticsearch - trasform fields to lowercase in output conf

I have a standard ELK stack currently storing numerous log outputs. I'm trying to separate my indices to be source-specific. As part of my FileBeats config, some standard fields are always generated as part of every message (and are…
m8r-3wo9bu
  • 41
  • 1
  • 2
4
votes
0 answers

elasticsearch using bulk insert to post JSON file

I have the following JSON file I have used awk to get rid of empty spaces, trailing, next line awk -v ORS= -v OFS= '{$1=$1}1' data.json I have added a create request at the top of my data.json followed by \n and the rest of my data. {"create":…
user1007727
  • 421
  • 5
  • 20
1
2
3
…
24 25