RAM, CPU type/cores, DISK, Networks matter a lot, But
the cluster design, data structures of documents and
queries has a huge impact on the Elasticsearch Clusters
and your search experiences.
I have experienced it a lot. You can check out some of the
scenarios here : https://www.found.no/foundation/crash-
Configuration for Master Node.
Give half of total available RAM to ES:
ES_HEAP_SIZE=2g ( on a 4 GB RAM, the more heap
size the better performance, But need to keep sweet
spot of 32 GB in mind)
Using Logstash Shippers and Redis or RabiitMQ
as a broker
img source: https://deviantony.wordpress.com/2014/05/19/centralized-logging-with-an-elk-stack-elasticsearch-logback-kibana/
Using Logstash Forwarder : Lumberjack Protocol
img source: http://blog.greg.lu/
Or a combination of both:
Don’t want to use any agent/shipper for system events logging??
1. Send logs directly to Elasticsearch: omelasticsearch http://www.rsyslog.com/doc/v8-
2. Use base logs sending : Create and edit the file /etc/rsyslog.d/logstash.conf with
*.* @logserver.example.com:5544 and parse the logs on centralized logstash server.
Rivers are finally removed:
Want to get a deep dive into Elasticsearch??
Join us @ 3rd Delhi Elasticsearch Meetup