Why we need to go for elasticsearch cluster with kubernetes?

Question

I have installed kubernetes on my DC and all is working fine. I have tested several services like mongodb and web services. But my goal is to find a good solution for Elasticsearch big data. From internet most of the techies are talking about Big data with kubernetes, elasticsearch cluster with kubernetes, etc. But Elasticsearch itself have simple options to extend the cluster by very easy configurations. Also I am able to use shared storage's ( NFS, Ceph ..etc ). So what is the point or advantages about Elasticsearch with kubernetes ?

Answer 1

I would say that Kubernetes provides very simple and easy ways to configure Elasticsearch in a very short time. This happens using already prepared solutions that you can always extend to your needs. You can deploy your cluster by simply applying a few commands or even use Helm Charts for Elasticsearch and Kibana to provide a standard, opinionated way to install, configure, upgrade, and run these applications on Kubernetes. This provides users with a set of best practices and templates for deploying Elasticsearch and Kibana along with access to basic free features, such as monitoring, Kibana Canvas and spaces. It’s also easy to upgrade to paid features such as security and machine learning.

The Github repository kubernetes-elasticsearch-cluster contains detailed documentation and configuration for using docker-elasticsearch with Kubernetes. As mentioned in kubernetes-elasticsearch-cluster overview:

One of the main advantages of running Elasticsearch on top of Kubernetes is how resilient the cluster becomes, particularly during node restarts. However if all data pods are scheduled onto the same node(s), this advantage decreases significantly and may even result in no data pods being available.

Also, it's very easy to control the amount of nodes of your cluster that will be used per time.

Couple of articles/urls I found useful:

Orchestrating Stateful Apps with Kubernetes StatefulSets

Elastic launches Helm Charts (alpha) for faster deployment of Elasticsearch and Kibana to Kubernetes

Github Elasticsearch Helm Chart

Deploy Elasticsearch with Kubernetes on AWS in 10 Steps