This document discusses Docker and Puppet and how they can be used together. It suggests using Puppet to install and configure Docker on the host system, and then using Dockerfiles to build container images in a deterministic way. While Puppet could theoretically be used to build containers, the document argues it is better to use Dockerfiles for image builds and to separate operational concerns like logging and monitoring into separate containers for better portability and flexibility.
2. Jérôme Petazzoni
(@jpetazzo)
● Grumpy French DevOps
– Go away or I will replace you
with a very small shell script
● Operated and scaled dotCloud
– PAAS on EC2, with LXC, Puppet,
Python, Shell, ØMQ...
3. Jérôme Petazzoni
(@jpetazzo)
● Runs everything in containers
– VPN, firewalls
– KVM, Xorg
– Docker
– …
● Helps others to do the same
– CONTAINERIZE
ALL THE THINGS!!!
10. Containers
● Software delivery mechanism
(a bit like a package!)
● Put your application in a container,
run it anywhere
● A bit like a VM, but ...
11. I have four words for you
● CONTAINERS boot faster
(than VMs)
● CONTAINERS have less overhead
(more consolidation)
● CONTAINERS bring native performance
(on bare metal)
● CONTAINERS are cloud-compatible
(can run in VMs)
15. CONTAINERS
are cloud-compatible
Docker runs on …
● Bare metal
– packages, binary, CoreOS, Project Atomic, b2d...
● Desktop VM
– boot2docker
● Cloud VM (Xen, ESX, KVM, HyperV...)
– ready-to-run images on most public clouds
16. Docker Engine recap
● Approximation:
it's an hypervisor to run containers
● Approximation:
containers are like VMs, but lighter
● Docker makes containers available to everybody
(not just veterans from the last emacs/vim war)
21. Docker Hub
● Services operated by Docker Inc.
● Library of ready-to-use container images
● Registry for your container images
(public or private)
● Automated builds
(triggered by pushes to GitHub/Bitbucket)
● Free for public/open source code, $$ otherwise
23. Dockerfile
FROM ubuntu:14.04
MAINTAINER Docker Team <education@docker.com>
RUN apt-get update
RUN apt-get install -y nginx
RUN echo 'Hi, I am in your container'
>/usr/share/nginx/html/index.html
CMD [ "nginx", "-g", "daemon off;" ]
EXPOSE 80
24.
25. FROM ubuntu
RUN apt-get -y update
RUN apt-get install -y g++
RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe
...
RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...
RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-
RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0" >
/usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101
CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .
29. Run from scratch every time
● Pros:
– no side-effect, 100% repeatability
● Cons:
– create machine each time
– provision all the things, install tons of packages...
– takes forever
– you will eventually get bored and give up
30. Run iteratively over and over
● Pros:
– much faster
● Cons:
– have to deal with leftovers of previous run
– have to make sure everything is idempotent
– quickly gets tedious
– you will eventually reinvent CM
32. Best of both worlds
● Build from scratch everytime
(re-apply each command on top of clean build)
● Build fast
(by re-using snapshots of previous runs)
● Win!
34. Configuration Management:
the Good
● Deals with low-level stuff
● Abstracts some details (distro, sometimes OS)
● Ensures convergence to a known state
● Library of reusable, composable templates
35. Configuration Management:
the Bad
● Steep learning curve
● Generally requires an agent
(or something to trigger e.g. « puppet apply »)
● Resource-intensive
(it's OK to run the agent on a 64 GB server,
it's less OK to run 100 agents on said server)
36. Configuration Management
● Reusability is just as good as modules are
(i.e. YMMV)
● Not as deterministic as you think
● Rollbacks are harder than you think
{ 'openssl' : ensure => present }
{ 'openssl' : ensure => '1.2.3-no-poodle-pls' }
38. Dockerfile
● Doesn't have to deal with « low-level stuff »
(hardware, drivers... handled by the host)
● Doesn't need all the goodness of CM
(because it doesn't have to converge)
● Partial rebuilds are fast
(layered caching rebuilds only what is needed)
● Allows inheritance and composition
(FROM <mycustombase>; see also: ONBUILD)
● Easy learning curve
(if you know Shell, you already know Dockerfile)
39. But...
● Doesn't deal with « low-level stuff »
(hardware, drivers...)
● Doesn't define resource dependencies
(no before/after)
● Doesn't define what runs where
41. Before/After
● Use Puppet to
setup hardware
(or virtual hardware),
install packages,
deploy code,
run services.
● Use Puppet to
setup hardware
(or virtual hardware),
install Docker,
run containers.
● Use Dockerfiles
to install packages,
deploy code,
run services.
54. My other VM is a container
● write a Dockerfile to install Puppet
● start tons of containers
● run Puppet in them (agent, or one-shot apply)
Good if you want a mix of containers/VM/metal
But slower to deploy, and uses more resources
55. Sample Dockerfile
FROM ubuntu:12.04
RUN apt-get install -qy wget
RUN mkdir /puppet
WORKDIR /puppet
RUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.deb
RUN dpkg -i puppetlabs-release-precise.deb
RUN apt-get update -q
RUN apt-get install -qy puppet-common
CMD puppet agent --no-daemonize --verbose
56. Lightweight, portable VMs
● Start containers instead of VMs
– I can start 10 containers on this puny laptop!
– You can start those 10 containers too!
(Even though you have a totally different laptop!)
– We can start those containers in the Cloud!
● Deploy sshd, syslogd, crond, etc.
– You can... But do you have to?
57. The revolution will be containerized
● write a Dockerfile to install Puppet
● … and run Puppet as part of build process
● deploy fully baked, « golden » images
Faster to deploy
Easier to rollback
58. Sample Dockerfile
FROM ubuntu:12.04
RUN apt-get install -qy wget
RUN mkdir /puppet
WORKDIR /puppet
RUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.deb
RUN dpkg -i puppetlabs-release-precise.deb
RUN apt-get update -q
RUN apt-get install -qy puppet-common
ENV FACTER_HOSTNAME database42
ADD ./site.pp /puppet/site.pp
RUN puppet apply site.pp
63. What does that mean?
● Don't rebuild your app to change logging,
remote access, and other unrelated things
● Have different policies in prod/dev/QA/etc
● Ship lighter containers
64. Virtual Machine deployment
● Linux base system
● Libraries
● Application
● Logging
● Backups
● Metrics
● ...
65. With configuration management
node www {
include common
include web
include logstash
include backup
include graphite
}
66. Problems
● Conflicts between two components
– e.g. logging and metrics use different Java versions
● Software certified for different distro
– e.g. something wants RHEL 6.4 but you run Ubuntu
● Migration from one component to another
– example: from syslog to splunk
67. Container deployment
● Linux base system
● Docker
● Application container
● Logging container
● Backups container
● Metrics container
● ...
68. More about that
http://blog.docker.com/2014/06/why-you-dont-need-
to-run-sshd-in-docker/
http://www.slideshare.net/jpetazzo/containerization
-new-virtualization-docker-separation-operational-concerns
73. Would You Like To Know More?
● Now: ask me questions!
● Next hour: ask me more questions!
● Tomorrow: Docker mini-training (11am)
● Run a containers BoF at LISA?
● Later: www.docker.com, #docker, docker-user...