A starter's guide to ELK/EFK stack for OpenStack.

1. OPENSTACK
&
LOGGING

2. ABOUT ME
Md Safiyat Reza
Fresh out of college!
Open-source enthusiast
An EMACS and KDE user.
Software Engineer at Snapdeal.com
safiyat
@reza_safiyat
reza.safiyat@acm.org

3. Logs are boring!

4. nova-compute
nova-network
nova-manage
nova-conductor
nova-scheduler
nova-api
nova-cert
nova-console
nova-consoleauth
nova-dhcpbridge
apache logs
cinder-api
cinder-scheduler
cinder-volume
syslog
keystone
glance-api
glance-registry
dhcp-agent
l3-agent
metadata-agent
openvswitch-agent
server.log
openvswitch-server
Lots of logs!

6. What logs contain...
TIMESTAMP
PID
NAME
REQUESTID
USERID
TENANTID
[INSTANCE INSTANCEID]
MESSAGE

7. All that is fine,
How to use them?

8. When shit happens...
DEBUG INFO AUDIT WARNING ERROR CRITICAL TRACE

9. Use logs for a good cause.

11. Log Collection, Aggregation & Visualization

12. Log Collectors and Aggregators

13. The flow of logs

14. • Open source
• Centralized logging
• Collection & Transport
• High availability
• Written in Ruby
• Requires Ruby (alt. td-agent)
• XML-styled configuration
• Plugins available
• Owned by Treasure Data
• Written in JRuby
• Requires JVM
• JSON-styled configuration
• A lot and lot of plugins
• Owned by Elastic

15. <source>
path /var/log/syslog
type syslog
format grok
grok_pattern %{SYSLOGLINE}
tag system
</source>
<filter>
type record_transformer
enable_ruby
<record>
timestamp ${timestamp.gsub!(" ", "T");
timestamp.gsub(/.d*/, "+05:30"}
</record>
</filter>
<match *>
type forest
subtype elasticsearch
<template>
host elasticsearchhost
index_name someindexname
type_name sometypename
</template>
</match>
Fluentd: syslog to elasticsearch

16. input {
file {
path => "/var/log/syslog"
start_position => beginning
type => syslog
sincedb_path => "/dev/null"
}
}
filter {
Grok {
match => [ "message", "%{SYSLOGLINE:log}" ]
}
}
output {
file {
path => "/home/safiyat/kafkaop"
}
elasticsearch {
protocol => "http"
}
}
Logstash: syslog to elasticsearch

17. Kibana

18. Kibana, again.

19. Thank you for bearing this!