The document summarizes the author's journey from using on-premise automation with shell scripting to using configuration management tools Chef and SaltStack. It outlines some of the challenges faced with shell scripting and Chef related to idempotence, cross-platform recipes, and lack of orchestration. The author found SaltStack improved on these areas with its use of states, pillars, grains, and modules which allowed for smarter formulas and automation of dynamic infrastructures on cloud providers. Examples are provided of using grains, pillars, peer communication, and Salt Mine to configure applications and services across minions in a declarative way.

1. from Chef
to SaltStack
@walterdalmut

2. Journey to automation
(and cloud computing)

3. 2008
On-premise distributed system
DRBD + Pacemaker
All things directly scripted in ssh sessions - no automation

4. 2010
AWS change my vision
Scalable infrastructures, automation, self-
provisioning
AWS was launched in 2006 and in 2007 180.000 developers had
signed up to use it

5. My nodes are dynamically
added/removed

6. Shell Scripting - A World of
pain

7. Satisfied
but
Maintenance problems...

8. Chef-solo
as provisioner
Imperative like shell scripting but awsome

9. Finally moved to SaltStack
Declarative and Imperative

10. My mistakes with
Chef automation

11. Idempotence came first

12. Application deployment with
Artifacts or Docker Containers
and not flow based
git clone ...
composer install
app/console ca:cl -
e=prod
...

13. Automate only what you know

14. Ruby is not my primary
language

15. But we can always learn it!

16. Crossplatform recipes

17. Time consuming
Hard to achieve
Difficult to test
Need the perfect recipe?
https://github.com/PUGTorino/application_zf

18. Lack of Orchestration

19. SaltStack
Communications using 0MQ
Only the master expose ports: 4505-4506

20. Salt foundations
States
Our system state
Pillars
Info from master to
minions
Grains
Info from minions to
master
Mines, Modules, Reactors, etc..

21. TOP.sls - What we have to do
base:
'*':
­ tools
'proxy­eu­aws­*­prod':
­ nginx
'web­eu­aws­*­prod':
­ webserver
­ webapp
'cache­eu­aws­*­prod':
­ memcached
We need the top.slsfor states and pillars

22. Use grains instead of names
base:
'*':
­ tools
­ firewall
­ firewall.munin
­ munin.node
'roles:manager':
­ match: grain
­ firewall.manager
­ redis.tools
'role:proxy':
­ match: grain
­ haproxy
­ firewall.haproxy

23. A recipe formula example
haproxy:
pkg:
­ installed
service:
­ running
watch:
­ file: /etc/haproxy/haproxy.cfg
­ file: /etc/default/haproxy
­ pkg: haproxy
haproxy_config:
file.managed:
­ name: /etc/haproxy/haproxy.cfg
­ source: salt://haproxy/haproxy.cfg
­ template: jinja
haproxy_default:
file.managed:
­ name: /etc/default/haproxy
­ source: salt://haproxy/haproxy.default
A simple file haproxy/init.sls

24. Here is a pillar:
Jinja templates for your state files
Jinja templates for managed files
Pillars on the stage
app/init.sls
app:
path: /opt/app
branch: develop
remote: https://github.com/wdalmut/app.git
hostname: www.an­hostname.tld
checkout_app:
git.latest:
­ name: {{ pillar['app']['remote'] }}
­ rev: {{ pillar['app']['branch'] }}
­ target: {{ pillar['app']['path'] }}
<VirtualHost *:80>
ServerName {{ pillar['app']['hostname'] }}
</VirtualHost>

25. Grains in your formulas
Super-smart recipes formulas
­d
­m {{ grains['mem_total'] * 3 / 4 }}
Memcached uses ¾ of available RAM
in your memcached config file memcached.conf

26. Pillars and grains
[mysqld]
{% if salt['grains.get']('rdb­master', none) == True %}
log_bin=/var/log/mysql/mysql­bin.log
{% endif %}
server­id={{ grains['rdb­id'] }}
bind­address={{ pillar['db']['bind'] }}
max_allowed_packet=64M
{% if salt['grains.get']('rdb­slave', none) == True %}
replicate­do­db={{ pillar['app']['db']['dbname'] }}
{% endif %}
Handle mysql replication configs

27. Peer communication
Master configuration
peer:
.*:
­ .*
upstream app {
{% for srv,ip in salt['publish.publish']('web.*', 'network.interfaces').items() %}
server {{ ip.eth0.inet[0].address }}:80;
{% endfor %}
}

28. The Salt Mine
Master configuration
mine_functions:
network.ip_addrs: [eth0]
{% for srv, addrs in salt['mine.get'](
'role:web', 'network.ip_addrs', expr_form='grain').items() %}
server {{ srv }} {{ addrs[0] }}:80 check
{% endfor %}

29. Commands

30. salt'*'test.ping
salt 'proxy-eu-aws-[1-3]-prod' test.ping

31. salt'*'state.highstate
salt 'proxy-eu-aws-[1-3]-prod' state.highstate

32. salt'*'cmd.run'du-hs/tmp'

33. salt'*'state.sls'app.rback'pillar="{rev:'1.4.2'}"

34. Prepare your modules
salt'web-*'app.stop
"""
Execution module for my app.
"""
def stop():
cmd = 'app/console maintenance:lock on'
out = __salt__['cmd.run'](cmd, __pillars__.get('app.path'))
return True if out else False
salt/_modules/app.py

35. Salt-Cloud
Orchestrate on Cloud Providers
An easy way to have your minions configured

36. Define providers
my­digitalocean­config:
provider: digital_ocean
personal_access_token: xxx
ssh_key_file: /path/to/ssh/key/file
ssh_key_names: my­key­name,my­key­name­2
location: New York 1
cloud.providers.d/do.conf

37. Define profiles
do­micro­ubuntu:
provider: my­digitalocean­config
image: 14.04 x64
size: 512MB
location: New York 1
private_networking: True
backups_enabled: True
ipv6: True
cloud.profiles.d/do.conf

38. __________ __ _______________ _____
salt-cloud -p do-micro-ubuntu web-1

39. salt-cloud -Pp do-micro-ubuntu web-1 web-2 web-3

40. Manually scale-out
salt-cloud -Pp do-micro-ubuntu web-4 web-5 web-6
salt web-[4-6] state.highstate
salt proxy-* state.highstate
Thanks to mines we can add more resources and update proxies
adding more web servers
{% for srv, addrs in salt['mine.get'](
'role:web', 'network.ip_addrs', expr_form='grain').items() %}
server {{ srv }} {{ addrs[0] }}:80 check
{% endfor %}

41. salt-cloud-dweb-1

42. salt-cloud-Pmdr.yml
% for i in xrange(10):
web:
­ web­${i}:
minion:
grains:
­ roles:
­ web
endfor %
% for i in xrange(3):
proxy:
­ proxy­${i}:
minion:
grains:
­ roles:
­ proxy
endfor %
...
web-1 web-2 web-3 ... web-10
proxy-1 proxy-2 proxy-3

43. Thanks for listening
Twitter: @walterdalmut