Mcollective is an open source framework for server orchestration and parallel job execution. It provides asynchronous and event-driven communication between nodes using a message broker like RabbitMQ. Nodes can be targeted based on facts, classes, or other criteria. Plugins allow mcollective to manage configurations, run puppet, install packages, manage firewall rules, and more across large server fleets. It provides a scalable and decentralized alternative to SSH loops for orchestrating infrastructure changes and operations.

1. Infrastructure Coders
Mcollective: An introduction to the marionette.
Melbourne 06-06-2012

2. Javier Turégano
#devops
@setoide
#opensource
#it-management

3. Disclaimer
Not mcollective Ninja

4. Why do we need orchestration?

6. Example architecture
Balancer
Webapp Webapp Webapp
Bdd

7. Use case: Add one webapp server
Deploy
Provision Apply config
Last version
new node as webserver
Of App
Add to
Grant access Start
LoadBalance
to DB webserver
r
Activate
In Prod!
monitoring

8. Use case: Deploy new version
Update Provision 1
Db schema New app Send traffic
(addings) server
Update
Replace
Test Db schema
App servers
(removals)
In Prod!

10. Some options available
Func
Fabric
Capistrano
mcollective

11. Marionete Colective
mcollective is a framework to build server
orchestration or parallel job execution systems
Created by R.I. Pienaar

12. Is it better than an SSH loop?
Asynchronous/Event driven
Scalable
RIP Hostnames – use facts, classes.
Modular (security, middleware, agents)
Auto discovery
De-centralized inventory

13. Architecture

14. Install middleware
# apt-get install rabbitmq
# rabbitmqctl add_user mcollective marionette
# rabbitmqctl set_user_tags mcollective
administrator
# rabbitmqctl set_permissions -p / mcollective
".*" ".*" ".*"

15. Install mcollective
In your servers:
# sudo apt-get install mcollective
In your admnistrator's machines:
# sudo apt-get install mcollective-client

16. Configure mcollective
topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/share/mcollective/plugins
logfile = /var/log/mcollective.log
loglevel = info
daemonize = 0
# Plugins
securityprovider = psk
plugin.psk = unset
connector = stomp
plugin.stomp.host= master
plugin.stomp.port= 6163
plugin.stomp.user= mcollective
plugin.stomp.password= marionette
# Facts
factsource = facter

17. Advanced config
Some interesting options:
loglevel=debug
#factsource = facter
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml
classesfile = /var/lib/puppet/state/classes.txt

18. Puppet modules
Puppet modules available.
For RHEL:
puppetlabs/rabbitmq
puppetlabs/mcollective

19. DEMO TIME!

20. Finding nodes
Targeting nodes using facts and classes:
# mc-ping
# mc-find-hosts -F virtual=virtualbox
# mc-find-hosts -F “memoryfree>100M”
# mc-find-hosts -C mcollective -F operatingsystem="Ubuntu"

21. Inventory
Powerful distributed inventory system:
# mc-facts virtual
# mc-facts processorcount
# mc-inventory box01

22. Plugins
You can get plugins from:
http://projects.puppetlabs.com/projects/mcollective-plugins/wiki
Facts-facter
Filemgr
Iptables
Package
Process
Puppetd
Service

23. Using the filemgr plugin
# mco rpc filemgr status file=/tmp/filemgr

24. Using the iptables plugin
# mco iptables block 192.168.0.0 -C base -v
Node01 # iptables -L -n -v

25. Using the package plugin
# mco package status vim-puppet -C base -v
# mco package uninstall vim-puppet -C base -v

26. Executing puppet
# mco puppetd status -C base
Edit puppet base class
# mco puppetd runonce 1 -v -C base
Check output in /var/log/syslog

27. RIP
Hosnames
QUESTIONS?

28. Images
http://www.flickr.com/photos/ag2r/3783369226/
http://www.flickr.com/photos/31348155@N03/5584078173/
http://www.flickr.com/photos/61927538@N00/205992050/
http://www.flickr.com/photos/safari_vacation/5842069535