Linux Container Technology inside Docker with RHEL7 discusses Docker containers and how they utilize Linux container technologies like namespaces and control groups. It provides an overview of how Docker images work and how processes are isolated in containers using process and filesystem namespaces. It also describes how networks are isolated using network namespaces and bridged to the host system. Finally, it briefly introduces Kubernetes and how it can manage Docker containers across multiple nodes.

1. Linux Container Technology
inside Docker with RHEL7
Etsuji Nakai
Senior Solution Architect
and Cloud Evangelist
Red Hat K.K
v1.1 2015/08/28

2. 2
Linux Container Technology inside Docker with RHEL7
$ who am i
 中井悦司 / Etsuji Nakai
– Twitter @enakai00
– Senior Solution Architect and
Cloud Evangelist at Red Hat.
– The author of Linux and OpenStack books.

3. 3
Linux Container Technology inside Docker with RHEL7
Contents
 What is Docker?
 Container Technology inside Docker
 Architecture of Kubernetes
 References

4. What is Docker?

5. 5
Linux Container Technology inside Docker with RHEL7
Quick Demo!

6. 6
Linux Container Technology inside Docker with RHEL7
Dockerfile
① Auto-build Docker images
OS Image
Application
Library / Framework
Application Binary
Describe steps
to build an image
Docker
image
Everything you need to run application
is included in the image
② Upload and publish images
③ Download and run
What you can do with Docker

7. Container Technology
inside Docker

8. 8
Linux Container Technology inside Docker with RHEL7
 "Linux Container" is a Linux kernel feature to contain a group of processes in
an independent execution environment.
 Linux kernel provides an independent application execution environment for
each container including:
– Independent filesystem.
– Independent network interface and IP address.
– Usage limit for memory and CPU time.
Linux Kernel
UserProcess
・・・
Physical Host / VM
Physical Host / VM
OS
ContainerNo Container
UserProcess
UserProcess
User Space
Linux Kernel
UserProcess
UserProcess
User Space
UserProcess
UserProcess
User Space
・・・
What is container technology?
Container

9. 9
Linux Container Technology inside Docker with RHEL7
 Container supports separation of various resources. They are internally
realized with different technologies called "namespace."
– Filesystem separation → Mount namespace (kernel 2.4.19)
– Hostname separation → UTS namespace (kernel 2.6.19)
– IPC separation → IPC namespace (kernel 2.6.19)
– User (UID/GID) separation → User namespace (kernel 2.6.23〜kernel 3.8)
– Processtable separation → PID namespace (kernel 2.6.24)
– Network separation → Network Namespace (kernel 2.6.24)
– Usage limit of CPU/Memory → Control groups
 Linux container is realized with integrating these namespace features. There
are multiple container management tools such as lxctools, libvirt and docker.
They may use different parts of these features.
Under the hood

10. 10
Linux Container Technology inside Docker with RHEL7
Filesystem
 A specific directory on the host is bind mounted as a root directory of the container.
Inside container, that directory is seen as a root directory, very similar mechanism to
the "chroot jail."
 When using traditional container management tools such as lxctools or libvirt, you need
to prepare the directory contents by hand.
– You can put minimum contents for a specific application such as application binaries
and shared libraries in the directory.
– It's also possible to copy a whole root filesystem of a specific linux distribution to
the directory.
– If necessary, special filesystems such as /dev, /proc and /sys are mounted in the
container by the management tool.
Mount namespace
/
|--etc
|--bin
|--sbin
...
/export/container01/rootfs/
|--etc
|--bin
|--sbin
...
bind mount

11. 11
Linux Container Technology inside Docker with RHEL7
Filesystem
Container
Application
Directory Tree
Mounted on the host
Assign as / filesystem
 With Docker, you don't need to prepare
the directory tree by hand.
 Docker image is mounted on the host and
used as root filesystem of the container.
Docker Image

12. 12
Linux Container Technology inside Docker with RHEL7
 Processes in all containers are executed on the same Linux kernel. But, inside a
container, you can see processes only in the container.
– This is because each container has its own process table. On host linux, which is outside
containers, you can see all processes including ones in containers.
Process table
# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:49 ? 00:00:00 /bin/sh /usr/local/bin/init.sh
root 47 1 0 09:49 ? 00:00:00 /usr/sbin/httpd
apache 49 47 0 09:49 ? 00:00:00 /usr/sbin/httpd
apache 50 47 0 09:49 ? 00:00:00 /usr/sbin/httpd
...
apache 56 47 0 09:49 ? 00:00:00 /usr/sbin/httpd
root 57 1 0 09:49 ? 00:00:00 /bin/bash
# ps -ef
UID PID PPID C STIME TTY TIME CMD
...
root 802 1 0 18:10 ? 00:01:20 /usr/bin/docker -d --selinux-enabled -H fd://
...
root 3687 802 0 18:49 pts/2 00:00:00 /bin/sh /usr/local/bin/init.sh
root 3748 3687 0 18:49 ? 00:00:00 /usr/sbin/httpd
48 3750 3748 0 18:49 ? 00:00:00 /usr/sbin/httpd
...
48 3757 3748 0 18:49 ? 00:00:00 /usr/sbin/httpd
root 3758 3687 0 18:49 pts/2 00:00:00 /bin/bash
Processes seen inside container
Processes seen outside container

13. 13
Linux Container Technology inside Docker with RHEL7
Process table
fork/exec
PID namespace
 In the example of previous page, docker daemon fork/exec-ed the initial
process "init.sh" and put it in a new "PID namespace." After that, all processes
fork/exec-ed from init.sh are put in the same namespace.
 Inside container, the initial process has PID=1 independently from the host.
Likewise, child processes of it have independent PID's.
PID=1
bash
/bin/sh /usr/local/bin/init.sh
httpd
httpd
・・・
#!/bin/sh
service httpd start
while [[ true ]]; do
/bin/bash
done
init.sh
docker daemon

14. 14
Linux Container Technology inside Docker with RHEL7
Network namespace
Network
 Container uses Linux's "veth" device for network communication.
– veth is a pair of logical NIC devices connected through a (virtual) crossover cable.
 One side of the veth pair is placed in a container's network namespace so that it can be
seen only inside the container. The other side is connected to a Linux bridge on the host.
– A device name in the container is renamed such as "eth0." By means of the namespace, network
settings such as IP address, routing table and iptables are independently configured in the
container.
– The connection between the bridge and a physical network is up to the host configuration.
Host Linux
vethXX
eth0
docker0
eth0
Physical network
 Docker creates a bridge "docker0" as a connection point
of container's network.
– Packets from containers are forwarded with IP masquerade.
– Packets from the physical network targeted to specified
ports are forwarded to the container using the port
forwarding feature of iptables.
172.17.42.1# docker run -it -p 8000:80 ...
Accessing to
the external IP
of the host
TCP 8000
TCP 80
Port
forwarding

15. 15
Linux Container Technology inside Docker with RHEL7
Network
 Example container network for 3-tier application running on the same host.
Accessing to the external IP
of the host
Container：Web Server
REST_PORT_5555_TCP_ADDR
eth0
DB_PORT_3306_TCP_ADDR
Container：App Server
eth0
Container：Database
eth0
Linux bridge（docker0）
External IP
Port 80 Port 5555 Port 3306
Port 80

16. 16
Linux Container Technology inside Docker with RHEL7
Network
 Example container network for 3-tier application running on different hosts.
REST_PORT_5555_TCP_ADDR
eth0
External IP
REST_PORT_5555_TCP_ADDR
eth0
External IP
eth0
External IP
Container：Web Server Container：App Server Container：Database
Port 80 Port 5555 Port 3306

17. Architecture of Kubernetes

18. 18
Linux Container Technology inside Docker with RHEL7
Server configuration
etcd
・・・
Backend Database（KVS）
Kubernetes Master
Kubernetes Node
・・・
Scale-out cluster
Docker Docker Docker
Add more nodes
if necessary.
Docker Registry
 Kubernetes manages multiple nodes from a single master.
– Clustering of multiple masters is not available now. You may use active-standby
configuration with standard HA tools for high availability.
– etcd (KVS) is used as a backend database. It can be configured as a scale-out cluster.

19. 19
Linux Container Technology inside Docker with RHEL7
Network configuration
etcd Kubernetes
Master
Docker
Registry
Configured as
an overlay network.
・・・
 Physical network is simple. Kubernetes works just by connecting all servers to a single
service network.
 However, you need to create an internal network for container communication using an
overlay network.
– You may use Flannel, Open vSwitch, etc. as an overlay technology.
Service network
192.168.122.0/24
Node
docker0
Node
docker0
Internal network
10.1.0.0/16

20. 20
Linux Container Technology inside Docker with RHEL7
External access
etcd Kubernetes
Master
Node
Docker
Registry
Node
API requests Image upload
・・・
Service access
 There are following cases for the external access.
– API requests are sent to the master.
– Services running on containers are accessed from nodes' external IPs via proxy
mechanism.
– Docker registry is an independent component from Kubernetes. You may use a
registry server running on a container.
Service network
Internal network

21. 21
Linux Container Technology inside Docker with RHEL7
Service
 You need to define a service so that you can access the containers inside pods. An
private and (optionally public) IP is assigned to each service.
– You define a single service which aggregates the multiple pods running the same
image. Access to the "IP + port" associated to a service is transferred to the
backend pods with the round-robin manner.
 When defining a service, you need to explicitly specify a port number. A "private IP" is
automatically assigned. The private IP is used for accessing from other pods (not
external uses.)
– Access to the private IP is received by the proxy daemon running on the local node,
and transferred to the backend pods.
– When launching a new pod, the private IPs and ports of existing services are set in
the environment variables inside new containers.
Poｄ
ProxyThe local proxy daemon
receives the packets to
the private IP.
Poｄ
Proxy
Round-robin access via
the internal network.
Poｄ
Proxy
Node Node Node

22. 22
Linux Container Technology inside Docker with RHEL7
Node
External access to services
Service access
 You can specify multiple public IPs for each service.
– By that, external users can access the service via multiple nodes so that a specific
node does not become a SPOF.
– External mechanism to select/load balance multiple nodes is required. Typically, you
can use the DNS load balancing.
Poｄ
Proxy
The proxy daemon receives
packets to service ports.
Accessing to the
nodes' public IPs.
Node
Poｄ
Proxy
Round-robin access via
the internal network.
 When defining a service, you need to specify
"Public IPs" if you need to make it accessible
from external users.
– Public IPs' correspond to nodes' IP addresses
from which external uses can access the
service.
– The packets to the corresponding nodes (for
the service port) are received by the proxy
daemon, and transferred to the backend pods.

23. 23
Linux Container Technology inside Docker with RHEL7
Baremetal / VM ・・・
Docker
Baremetal / VM
Docker
Kubernetes
Platform as a Service
・・・
Execution Resource
Container
Management
Container
Orchestration
UI, Monitoring,
Image build workflow,
etc.
RHEL
Atomic Host
OpenShift 3.0
Beyond Kubernetes: OpenShift v3
Container
Container
Container
Container
・・・ ・・・

24. References

25. 25
Linux Container Technology inside Docker with RHEL7
References
 Architecture Overview: Kubernetes with Red Hat Enterprise Linux 7.1
– http://www.slideshare.net/enakai/architecture-overview-rubbernecks-with-red-
hat-enterprise-linux-71
 Inside Docker for Fedora20/RHEL7
– http://www.slideshare.net/enakai/docker-technology-v18e
 OpenShift 3 Technical Architecture
– https://docs.google.com/presentation/d/1Isp5UeQZTo3gh6e59FMYmMs_V9QIQeBel
mbyHIJ1H_g/pub
 OpenShift v3 Internal networking details
– http://www.slideshare.net/enakai/openshift-45465283

26. EMPOWER PEOPLE,
EMPOWER ENTERPRISE,
OPEN INNOVATION.