Story of an investigation to improve cloud The sad VirtualMachine story Containers and non-containers DEMO - Drupal Docker Drupal Testbots story in a Glance Docker as a testing automation factor DEMO - Docker Tesbot Integration path

1. Docker Containers:
Testing automation and development in Drupal.
Amsterdam - DevOps meetup, September 2014
A new strategy for automating Drupal testing
@ricardoamaro

2. About me
Free/Opensource software lover
Senior Cloud Engineer @Acquia
Drupal.org infrastructure/devops
Drupalist & Linux enthusiast
Father, artist, community facilitator
@ricardoamaro
https://drupal.org/user/666176

3. today’s meetup
Story of an investigation to improve cloud
1. The sad VirtualMachine story
2. Containers and non-containers
3. DEMO - Drupal Docker
4. Drupal Testbots story in a Glance
5. Docker as a testing automation factor
6. DEMO - Docker Tesbot
7. Integration path

4. What is virtualization?
Hardware virtualization or platform
virtualization refers to the creation of a
virtual machine that acts like a real
computer with an operating system.
Software executed on these virtual
machines is separated from the underlying
hardware resources.

5. Why should i care?
Increase
+ efficiency
+ availability
+ security
Reduce
- costs
- hardware
- energy
Cloud infrastructure providers like Amazon Web Service sell virtual
machines. EC2 revenue is expected to surpass $1B in revenue this year.
That's a lot of VMs…

6. Virtual Machine platforms

7. The sad Virtual Machine story...
➢ We are also paying for lot of
avoidable overhead.
➢ The Virtual Machine is a full-blown
operating system image.
➢ This is a heavyweight solution to
run applications in the cloud.

8. What is the solution?

9. A new concept, a new hope
CCoonnttaaiinneerrss uusseedd ttoo bbee tteerrrriibbllee,, bbuutt nnoott aannyymmoorree

10. Because Docker has arrived!

11. On any recent Linux Kernel near you!

12. Virtual Machines vs Containers
Virtualization and
paravirtualization
require a full
operating system
image for each
instance.
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

13. Virtual Machines vs Containers
Containers can
share a single
Linux Kernel and,
optionally, other
binary and library
resources.
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

14. Virtual Machines vs Containers
Virtualization and paravirtualization
require a full operating system image
for each instance.
Containers can share a single operating
system and, optionally, other binary
and library resources.
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

15. The time to provision
Source : http://www.linuxjournal.com/content/containers%E2%80%94not-virtual-machines%E2%80%94are-future-cloud

16. From the simple concept of “chroot”
mount /dev/sda /target
chroot /target
source: http://openvz.org
but that had no resource and security isolation goals
for multi-tenant designs...

17. Cpu
Devices
Processes
Memory
Disk space
Network
What if you could control...

18. Docker
Need
control
over
specific
host
resources
cgroups
Control Groups provide a mechanism for aggregating/partitioning sets
of tasks, and all their future children, into hierarchical groups with
specialized behaviour.
~$ ls /sys/fs/cgroup
blkio
cpu
cpuacct
cpuset
devices
freezer
hugetlb
memory
perf_event
Containers & Cgroups
https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

19. use Docker

20. Docker Who??

21. this Docker
and ship them as containers

22. Ship containers? Build Once, Run Anywhere

23. Build and ship Drupal the Docker way:
https://github.com/ricardoamaro/docker-drupal
https://github.com/ricardoamaro/docker-drupal-nginx

25. The docker is awesome!
the Commands:
attach Attach to a running container
commit Create a new image from a container's changes
diff Inspect changes on a container's filesystem
export Stream the contents of a container as a tar archive
history Show the history of an image
images List images
import Create a new filesystem image from the contents of a tarball
info Display system-wide information
inspect Return low-level information on a container
kill Kill a running container
login Register or Login to the docker registry server
logs Fetch the logs of a container
port Lookup the public-facing port which is NAT-ed to PRIVATE_PORT
ps List containers
pull Pull an image or a repository to the docker registry server
push Push an image or a repository to the docker registry server
restart Restart a running container
rm Remove a container
rmi Remove an image
run Run a command in a new container
start Start a stopped container
stop Stop a running container
tag Tag an image into a repository
version Show the docker version information
wait Block until a container stops, then print its exit code
the Api
http://docs.docker.io/en/latest/api/registry_index_spec/
the Builder
http://docs.docker.com/reference/builder/
the Registry
http://docs.docker.io/en/latest/api/index_api/

26. Docker on Docker > v0.6

27. Continuous Deployments, Development & Testing...
Keep only the good apples
Changes to the container can be committed
to the central index or rolled back

28. TESTBOTS
How does this picture help
the Drupal testing and the
jobrunner?

29. Testbot is `Sad`
Actual running testbots were created in
2007 and revamped in 2009, need
modernizing in order to overcome
several pending issues that make them
difficult to debug, go faster and be less
prone to error.
Since they are full virtual servers, they consume
much cpu, memory and IO. A part from that
developers have ever increasing demands.
Multiple environments and databases need to
be tested and contained.

30. Current Drupal testing workflow (simplified):
Testbots
(jobrunner)
Runs the actual
test
drupal.org
Issue queue
starts process
and displays
results
qa.drupal.org
keeps track of
jobs and detailed
info

33. Integration
round table:

34. New proposed Drupal testing workflow:
Testbot
1
drupal.org
Issue queue
starts process
and displays
results
results
drupal.org
keeps track of
jobs and detailed
info
Jenkins
Dispatcher
Testbot
2
Testbot
3
Testbot
4
Docker Container types:
- PHP5.3 / PHP5.4 / PHP5.5 / PHP5.6
- MariaDB
- MySQL
- Sqlite
- Postgres
and others...
Testbot
n
API

35. Docker Containers:
Testing automation
and development in Drupal.
Amsterdam - DevOps meetup, September 2014
@ricardoamaro

36. Thank you!
@ricardoamaro