1. Who Am I

2. •
•
•
•

5. •
•
•
•
•
•
•
•
•

6. •
•
•
•
•
•
•
•
•
•
•
•
•

7. •
•
•
•
•
•
•
•
•
•
RAM, CPU type/cores, DISK, Networks matter a lot, But
the cluster design, data structures of documents and
queries has a huge impact on the Elasticsearch Clusters
and your search experiences.
I have experienced it a lot. You can check out some of the
scenarios here : https://www.found.no/foundation/crash-
elasticsearch/

11. •
•
•
•

13. •
•
•
•
•

14. •
•
•
•
•
•
•

15. •
•
•
•
•
•
•
•
•
•
•
•
•

16. •
•
•
•
•
•
•
•
•
•
•
•

19. Centralize
Analyze Monitor
Share

20. • •
•

21. •
•
•
•
•
•

22. #!/bin/bash
sudo apt-get purge openjdk-*
wget --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz
tar -xvf jdk-7u79-linux-x64.tar.gz
sudo mkdir -p /usr/lib/jvm
sudo mv ./jdk1.7.0_79 /usr/lib/jvm/
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0_79/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0_79/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0_79/bin/javaws" 1
sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chmod a+x /usr/bin/javaws
sudo chown -R root:root /usr/lib/jvm/jdk1.7.0_79
sudo update-alternatives --config java
##########Skip above if you have java already available.
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.4.deb
sudo dpkg -i elasticsearch-1.4.4.deb
sudo update-rc.d elasticsearch defaults 95 10

23. script.disable_dynamic: false
cluster.name: give_your_cluster_name
node.name:"es-master-3"
node.master: true
node.data: false
node.max_local_storage_nodes: 1
index.number_of_shards: 4
index.number_of_replicas: 1
bootstrap.mlockall: true
transport.tcp.port: 9300
http.enabled:false
discovery.zen.minimum_master_nodes:2
discovery.zen.ping.timeout:10s
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:["es-master-1:9300","es-master-2:9300", "es-master-3:9300"]
action.disable_delete_all_indices: false
action.destructive_requires_name: true
marvel.agent.exporter.es.hosts: ['es-monitor-1:6200']
/etc/elasticsearch/elasticsearch.yml
Configuration for Master Node.
Give half of total available RAM to ES:
vim /etc/init.d/elasticsearch
ES_HEAP_SIZE=2g ( on a 4 GB RAM, the more heap
size the better performance, But need to keep sweet
spot of 32 GB in mind)

24. …

29. Shipper
Logstash
Shipper
Logstash
Shipper
Logstash
Broker
Redis/RabbitMQ
Indexer
Logstash Elasticsearch
Using Logstash Shippers and Redis or RabiitMQ
as a broker

30. img source: https://deviantony.wordpress.com/2014/05/19/centralized-logging-with-an-elk-stack-elasticsearch-logback-kibana/
Using Logstash Forwarder : Lumberjack Protocol

31. img source: http://blog.greg.lu/
Or a combination of both:

32. Don’t want to use any agent/shipper for system events logging??
Use Rsyslog:
1. Send logs directly to Elasticsearch: omelasticsearch http://www.rsyslog.com/doc/v8-
stable/configuration/modules/omelasticsearch.html
2. Use base logs sending : Create and edit the file /etc/rsyslog.d/logstash.conf with
*.* @logserver.example.com:5544 and parse the logs on centralized logstash server.

33. Rivers are finally removed:
https://github.com/elastic/elasticsearch/pull/11568#event-332821650
Want to get a deep dive into Elasticsearch??
Join us @ 3rd Delhi Elasticsearch Meetup
http://www.meetup.com/Delhi-Elasticsearch-Meetup/events/223470631/