Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine. Architected from the ground up for use in distributed environments where reliability and scalability are must haves, Elasticsearch gives you the ability to move easily beyond simple full-text search. Through its robust set of APIs and query DSLs, plus clients for the most popular programming languages, Elasticsearch delivers on the near limitless promises of search technology.

http://www.elasticsearch.org

373 questions
2
votes
0 answers

How do I update Logstash data to Redshift real-time?

I am looking for a way to build a pipeline for storing the logs data (I use the ELK stack) to AWS Redshift, for analytics. I have gone through the Output plugins for ElasticSearch, and found one for writing data to AWS S3. But, it writes data in…
Dawny33
  • 133
  • 1
  • 7
2
votes
2 answers

Kibana 4, Logstash dashboard: how do I require Nginx authentication when saving but allow anonymous views?

I would like to require auth_basic nginx authentication to save all kibana 4 dashboards but allow anyone to view dashboards without authentication. I recently installed an ELK (Elasticsearch 1.4.5, Logstash 1:1.5.2-1, and Kibana 4.1.1) stack on…
Peter M
  • 953
  • 2
  • 14
  • 27
2
votes
2 answers

Elasticsearch service often goes down or gets killed

My webserver is running ubuntu 14.10 with elasticsearch 1.5.0 and java 1.7u55 For some reason, the elasticsearch service often goes down, resulting in my website not being available to my users anymore (using FOSElasticaBundle with symfony). I am…
Sébastien
  • 233
  • 3
  • 8
2
votes
0 answers

Benchmarking for DISK I/O Writes Per Second - ElasticSearch and MySQL

During working on our project we needed to decide should we use "ElasticSearch" or "MySQL" to record the audit/trail logs of the system. Searching speed is not an issue here, we needed to see the disk I/O performance for both platforms. I had no…
Arfeen
  • 121
  • 4
2
votes
0 answers

Using database for storing text files and images

I need to build a image/static file cache which would host maybe few hundred up to thousand static files from 1kb to ~200kb. I'd need super fast reads (I'm using redis, but it's one threaded and I'm not sure how it's clustering works since it's in…
Jason
  • 21
  • 2
2
votes
1 answer

ES fills up heap slowly over time and hangs at 14GB, while biggest index is 164MB?

I have an issue with Elasticsearch, at some times, it tries to run GC continously since this one is unable to free because heap size set to 14GB (min and max) is said to be completely allocated: (...) [2014-09-18 13:43:45,984][INFO ][monitor.jvm …
Alex F
  • 819
  • 1
  • 10
  • 17
2
votes
0 answers

ElasticSearch major slowdown upon big aggregations

My goal is to use ElasticSearch v1.3.2 for analyzing product cross-sales, so I need to filter for the receipts of interest (having an alcoholic product for example) and finding top selling products of each category. New data would be indexed monthly…
NikoNyrh
  • 161
  • 6
2
votes
3 answers

sudden peak in cpu usage

We are running a 4 node/machine elastic search cluster on 12 core, 96gb RAM, 4 spinning disk machines. under normal operation most cpu usage is user and around 5-10%. Every few days, one of the machine's cpu usage gets pegged at 80-100% and is all…
slushi
  • 71
  • 1
  • 5
2
votes
1 answer

iptables to allow input and output traffic to and from web server only

I have an Elastic Search server which seems to have been exploited (it's being used for a DDoS attack having had NO firewall for about a month). As a temporary measure while I create a new one I was hoping to block all traffic to and from the server…
Caedmon
  • 123
  • 6
2
votes
2 answers

PHP - does curl run asynchroniously on my system?

I'm building a web app. I have a database of books indexed in ElasticSearch and REST API written in PHP. In the app, there's a search box, where I type a name of the book and JS script calls the search request which than runs a curl request with…
Michal Artazov
  • 175
  • 2
  • 6
2
votes
2 answers

elasticsearch with kibana displays empty dashboard

I have the following is showing up when I pull up my kibana dashboard: http://192.168.10.25/#/dashboard {{dashboard.current.title}} When I tail my logs I see the following 2014/05/14 13:31:45 [error] 17152#0: *7 open()…
rahrahruby
  • 577
  • 5
  • 12
  • 20
2
votes
2 answers

Can I run elasticsearch on a single server?

I use elasticsearch as part of a Logstash stack, in which all of the components of the stack are installed on the same server. The purpose of this is to expose application logs to developers for debug purposes. I don't need to keep the indices…
Garreth McDaid
  • 3,399
  • 26
  • 41
2
votes
1 answer

Elasticsearch on EC2, webserver on Heroku // EC2 ports only accessible from Heroku

I have an instance of Elasticsearch facing the internet on an EC2. On the other hand I have my webservice as a Heroku app. I want to secure the Elasticsearch instance and allow inbound traffic only from my Heroku app. I have found this Elasticsearch…
Diolor
  • 121
  • 4
2
votes
1 answer

Logstash Groking Syslog Events

I have a logstash server sending events to an elasticsearch server that is displaying the results in Kibana. Everything is working pretty good, other than Kibana is showing 2 names in the host field. I am using the following grok filter in…
Eric
  • 1,373
  • 3
  • 17
  • 33
2
votes
1 answer

Elasticsearch standalone mode with logstash reject data after five days

I use Logstash and Elasticsearch for our squid log store and analyze.The size of the logs increase at the speed of 40Gb per day in our freeBSD ZFS storage system. the Elasticsearch gets into failure every five days and no logs can be further…
Kindule
  • 156
  • 8
1 2 3
…
24 25