In this video from the 2017 HPC Advisory Council Stanford Conference, Christian Kniep from Gaikai presents: Best Practices: State of Linux Containers. "Linux Containers gain more and more momentum in all IT ecosystems. This talk provides an overview about what happened in the container landscape (in particular Docker) during the course of the last year and how it impacts datacenter operations, HPC and High-Performance Big Data. Furthermore Christian will give an update/extend on the ‘things to explore’ list he presented in the last Lugano workshop, applying what he learned and came across during the year 2016." Watch the video: http://wp.me/p3RLHQ-glP Learn more: http://qnib.org and http://www.hpcadvisorycouncil.com/events/2017/stanford-workshop/ Sign up for our insideHPC Newsletter: http:/insidehpc.com/newsletter

1. ssThe State of Linux Containers
Christian Kniep, 2017-02-07

2. Containers in a Nutshell

3. Userland (OS) Userland (OS)Userland (OS)
Linux Containers
3
SERVER
HOST KERNEL
HYPERVISORSERVICES
Userland (OS)
KERNEL KERNEL
Userland (OS) Userland (OS)
SERVICES SERVICES
SERVER
HOST KERNEL
App1 App2
Traditional Virtualisation Containerisation
Containers do not spin up a distinct kernel
all containers & the host share the same
user-lands are independent
they are separated by Kernel Namespaces
SERVICES

4. 4
Interface View
Userland
Container
HostHost
Kernel
Userland
Hypervisor
Application Application
Traditional Virtualisation Containerisation
lightweight abstraction
w/o IO overhead or startup
latency
101
functions
102
functions
emulated dev /
hyper-calls
Syscall
Interface
Library
Abstraction

5. HOST
Containers are ‘grouped processes’
isolated by Kernel Namespaces
container1
5
Kernel Namespaces
bash
ls -l
container2
apache
container3
mysqld
PIDNamespaces:
container4
orted
slurmd

6. 1. Linux containers => forks in distinct Namespaces
6
Just Process Groups

7. HOST
Containers are ‘grouped processes’
isolated by Kernel Namespaces
resource restrictions applicable through CGroups (disk/netIO)
container1
7
Kernel Namespaces
bash
ls -l
container2
apache
container3
mysqld
PIDNamespaces: Network Mount IPC UTS
container4
orted
slurmd
User

8. 1. runC (Docker Inc.)
low-level runtime, reference implementation of OCI
2. systemd-nspawn
spawn services as containerised systemd unit
3. rkt (CoreOS)
Runtime to download,verify and start App Containers
4. Singularity
Runtime aimed to integrate well in existing workflows
8
Container Runtime

9. 1. runC
containerd -> docker-engine
2. systemd-nspawn
systemd config
3. rkt (CoreOS)
containerd -> docker-engine
4. Singularity
bash
9
Runtime -> Orchestration

10. Spinning up production-like environment is great
MongoDB, PostreSQL, memcached as separate containers
python2.7, python3.4
10
Container in Software Development
Like python’s virtualenv on steroids,
iteration speedup through reproducibility

11. 1. runC
containerd -> docker-engine -> SwarmKit
2. systemd-nspawn
systemd config
3. rkt (CoreOS)
Kubernetes || Nomad || containerd-*->SwarmKit
4. Singularity
integrate in your current cluster management (Slurm)
11
Orchestration -> Cluster

12. 12
Survey 2016
https://clusterhq.com/press/releases/2016-container-adoption-report/

13. Docker 2016

14. 1.11
runC - Independent Runtime
1.12
containerd
14
Docker 2016

15. 15
containerd

16. 1. consolidate the core
16
containerd [cont]

17. 1.11
runC - Independent Runtime
1.12
containerd - runtime environment
SwarmKit - orchestration framework
ingress load-balancing
17
Docker 2016

18. 1. Manager (leader/follower) model
18
SwarmKit

19. 1. Manager (leader/follower) model
2. ingress load-balancer
19
SwarmKit [cont]

20. 1.11
runC - Independent Runtime
1.12
containerd - runtime environment
SwarmKit - orchestration framework
ingress load-balancing
1.13
host network-mode
secrets
20
Docker 2016

21. Orchestration

22. 22
Service/DC Orchestration
1. Mesos (DC/OS)
“A distributed systems kernel”
Marathon to schedule containers
Running SWARM/Kubernetes on-top
2. Kubernetes
matured as (one of) the go-to orchestrators of services
3. SWARM
on the rise as orchestrator?

23. 23
Survey 2016
https://clusterhq.com/press/releases/2016-container-adoption-report/

24. 24
Status Quo Friendly
1. Shifter (NERSC)
Extracts Docker Image into squash-FS and runs chroot
2. Singularity
starting point is the user’s privilege level, not a root-daemon

25. 25
Cloud
1. AWS (re:invent 2016)
new instances c5: 72vCPU, 144GB RAM
Elastic Network Adapter (25Gb/s)
elasticGPU
snow(flake|mobile)
AWS batch

26. Split input iteration / development from operation
non-distributed stays vanilla
transition to HPC cluster using tech to foster operation
26
Separation of Concerns?
Input/Dev

27. Spinning up production-like environment is…
…not that easy
focus more on engineer/scientist, not the software developer
1. For development it might work
close to non-HPC software dev
2. But is that the iteration-focus?
rather job settings / input data?
27
Docker in HPC development

28. 28
DevOps Momentum
Software Dev
DatacenterOps
Docker focused on Dev(Ops)
Kubernetes on Ops(Dev)
SWARM targets DevOps
slurm scales Ops
singularity makes DevOps
status quo compatible
Disclaimer: subjective exaggeration

29. 1. …becomes container ready!
fork MPI process in namespaces
even better: use container runtime to be compliant
2. Ideally the container engine understands MPI
AWS ECS + Snowflake + AWS Batch + AWS Lambda
Docker-Engine (and thus SWARM)
29
What if MPI…
#getRichFast #freeStartUpIdea

30. Q&A
http://qnib.org
Container Workshop Proposed

31. Almost explored

32. 1. Where to base images on?
Ubuntu/Fedora: ~200MB
Debian: ~100MB
Alpine Linux: 5MB (musl-libc)
2. Definitely small!
pid-sharing allows for monitoring hooks to reach in
docker-engine provides HEALTHCHECK
debug-container with different permissions
32
Small vs. Big

33. 1. One Application
Ideally even only one process
At least only one application (nginx needs master/workers)
2. Benefits
context of the processes within is limited, easier to understand
clean metrics and logs provide easy to grasp
information for Dev and Ops
clean metrics and logs are easy for AI, auto-mediation
33
One vs. Many Processes Apps

34. Running OpenFOAM on small scale is cumbersome
manually install OpenFOAM on a workstation
be confident that the installation works correctly
A containerised OpenFOAM installation tackles both
34
Reproducibility / Downscaling
http://qnib.org/immutablehttp://qnib.org/immutable-paper

35. Allow Dev to run CI based
around working on single App
locally to test their work
spin up auxiliary services
fast, rather simple check to speed up dev-cycle
35
Continuous Integration

36. If containers are immutable within pipeline
continuous evaluation of stacks
testing/deployment should be automated
developers should have a production replica
pipeline to enable fast release/deploy cycle
36
Continuous Delivery

37. To Be Explored

38. 1. Since the environments are rather dynamic…
how does the containers discover services?
external registry as part of the framework?
discovery service as part of the container stacks?
38
Service Discovery

39. 1. Containers should be controlled via ENV or flags
External access/change of a running container is discouraged
docker secrets
2. Configuration management
Downgraded to bootstrap a host?
39
Immutable vs. Config Mgmt

40. 1. modularisation of basic components
push from outside to stabilise the core
make sure Docker Inc. has a foothold in the standardisation (?)
40
containerd / runC