Scaling Docker with Kubernetes
•
86 likes•14,778 views
DevoxxFR 2015 Talk http://cfp.devoxx.fr/2015/talk/WXY-1157/Scaling_Docker_with_Kubernetes Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple Docker hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Microsoft, RedHat, IBM and Docker Inc amongst others. Once you are using Docker containers the next question is how to scale and start containers across multiple Docker hosts, balancing the containers across them. Kubernetes also adds a higher level API to define how containers are logically grouped, allowing to define pools of containers, load balancing and affinity.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Scaling Docker with Kubernetes
Similar to Scaling Docker with Kubernetes (20)
More from Carlos Sanchez
More from Carlos Sanchez (20)
Recently uploaded
Recently uploaded (20)
Scaling Docker with Kubernetes
- 1. @csanchez#scalingdocker Scaling Docker with Kubernetes Carlos Sanchez @csanchez http://csanchez.org
- 2. Containers & micro services
- 3. @csanchez#scalingdocker Scale high availability distributed apps Google starts 20billion/week
- 4. @csanchez#scalingdocker But it is not trivial
- 5. @csanchez#scalingdocker But it is not trivial
- 6. @csanchez#scalingdocker But it is not trivial
- 7. @csanchez#scalingdocker Service oriented think about services, not machines
- 8. Unix processes - lightweight VMs
- 9. @YourTwitterHandle#DVXFR14{session hashtag} @csanchez#scalingdocker Docker Linux containers Union File System File System Users Processes Network
- 10. @csanchez#scalingdocker Docker Linux required but Boot2Docker to the rescue OS X Windows
- 11. @csanchez#scalingdocker Docker Build once, run anywhere (kind of) Bare metal Virtual Machines Cloud Docker
- 13. @csanchez#scalingdocker developer oriented Dependency hell installation nightmares “it ran on my machine”
- 14. @csanchez#scalingdocker ops oriented no need to know internals of apps focus on OPs problems (scale, monitoring,…) clearer deliverables from dev
- 15. @YourTwitterHandle#DVXFR14{session hashtag} @csanchez#scalingdocker Docker delivery Docker image Docker infrastructure App System
- 20. @csanchez#scalingdocker Docker docker run -t -i ubuntu /bin/bash docker ps -a docker build -t csanchez/test .
- 21. @csanchez#scalingdocker Rocket App Container Image App Container Runtime App Container Discovery
- 22. Container Optimized Operating Systems
- 23. @csanchez#scalingdocker Container OS Minimal OS Container optimized No package manager Automatic atomic updates Easy rollbacks
- 24. Kubernetes
- 25. @csanchez#scalingdocker How would you design your infrastructure if you couldn't login? Ever. Kelsey Hightower CoreOS
- 26. @csanchez#scalingdocker Kubernetes Container cluster orchestration Docker containers across multiple hosts (nodes or minions) Higher level API Enforced state Monitoring of endpoints
- 27. @csanchez#scalingdocker Master Kubernetes API Server scheduling and synchronization etcd Kubernetes Controller Manager Server implements replication algorithm watching etcd
- 28. @csanchez#scalingdocker Node Docker Kubelet ensures state of Pods Kubernetes Proxy simple network proxy etcd SkyDNS ElasticSearch + Kibana
- 33. @csanchez#scalingdocker gcloud preview container --project my-project clusters create cluster-1 --zone us-central1-a --machine-type n1-standard-1 --no-scopes --num-nodes 2 --network default Google Container Engine
- 35. @csanchez#scalingdocker Node { "id": "192.168.12.100", "kind": "Minion", "apiVersion": "v1beta1", "resources": { "capacity": { "cpu": 1000, "memory": 1073741824 } }, "labels": { "name": "node0", "role": "database" } }
- 36. @csanchez#scalingdocker Pod Group of colocated containers Same network namespace/IP Environment variables Shared volumes host mounted empty volumes GCE data disks
- 38. @csanchez#scalingdocker Pod { "id": "jenkins", "kind": "Pod", "apiVersion": "v1beta1", "desiredState": { "manifest": { "version": "v1beta1", "id": "jenkins", "containers": [ { "name": "jenkins", "image": "csanchez/jenkins-kubernetes", "ports": [ { "containerPort": 8080, "hostPort": 8080 }, { "containerPort": 50000, "hostPort": 50000 } ], "volumeMounts": [ { "name": "jenkins-data", "mountPath": "/var/jenkins_home" } ] } ],
- 39. @csanchez#scalingdocker Pod "volumes": [ { "name": "jenkins-data", "source": { "persistentDisk": { "pdName": "k8-jenkins", "fsType": "ext4" } } } ] } }, "labels": { "name": “jenkins”, "environment": “dev” } }
- 40. @csanchez#scalingdocker Replication controller Ensure a number of pods are running Pod templates Rolling update podpod pod container 1 container 2 container 3
- 41. @YourTwitterHandle#DVXFR14{session hashtag} @csanchez#scalingdocker Replication controllers Node client podA Master Node podA Node podB podA podB
- 43. @csanchez#scalingdocker Replication controller { "id": "jenkins-slave", "apiVersion": "v1beta1", "kind": "ReplicationController", "desiredState": { "replicas": 1, "replicaSelector": { "name": "jenkins-slave" }, "podTemplate": { "desiredState": { "manifest": { "version": "v1beta1", "id": "jenkins-slave", "containers": [ {"name": "jenkins-slave", "image":"csanchez/jenkins-swarm-slave:1.20", "command": [ “sh", "-c", "/usr/local/bin/jenkins-slave.sh -master http://$JENKINS_SERVICE_HOST:$JENKINS_SERVICE_PORT - username jenkins -password jenkins -executors 1" ] }] } }, "labels": { "name": "jenkins-slave" } } }, "labels": { "name": "jenkins-slave" } }
- 44. @csanchez#scalingdocker Services Pod discovery IP per service Route to pods selected with labels Can create a load balancer in GCE
- 45. @YourTwitterHandle#DVXFR14{session hashtag} @csanchez#scalingdocker Services client pod Service pod pod
- 46. @csanchez#scalingdocker Services { "id": "jenkins", "kind": "Service", "apiVersion": "v1beta1", "port": 8888, "containerPort": 8080, "selector": { "name": "jenkins" }, "createExternalLoadBalancer": true } { "id": "jenkins-slave", "kind": "Service", "apiVersion": "v1beta1", "port": 50000, "containerPort": 50000, "selector": { "name": "jenkins" } }
- 47. @csanchez#scalingdocker Networking all containers can communicate with all other containers without NAT all nodes can communicate with all containers (and vice-versa) without NAT the IP that a container sees itself as is the same IP that others see it as Containers in a Pod can talk using localhost
- 48. @csanchez#scalingdocker Networking Every machine in the cluster is assigned a full subnet ie. node A 10.0.1.0/24 and node B 10.0.2.0/24 Simpler port mapping Only supported by GCE CoreOS flannel Creates an overlay network in other providers
- 49. Weave
- 50. Related projects
- 51. @csanchez#scalingdocker Docker Machine Provision Docker engines Amazon EC2 Microsoft Azure Google Compute Engine OpenStack Rackspace VirtualBox VMware …
- 52. @csanchez#scalingdocker Docker Swarm Clustering for Docker containers Using the same API Integrates with Mesos / Mesosphere And planned Amazon EC2 Container Service (ECS) Google Kubernetes IBM Bluemix Container Service Joyent Smart Data Center Microsoft Azure
- 53. @csanchez#scalingdocker Docker Compose Orchestration of multi-container apps Based on Fig Defined by: containers configuration links volumes
- 54. @csanchez#scalingdocker Apache Mesos A distributed systems kernel Docker Containerizer Marathon & Chronos
- 55. @csanchez#scalingdocker Flocker data volume manager multi-host Docker cluster management tool ZFS volumes https://github.com/ClusterHQ/flocker
- 56. Examples
- 58. @csanchez#scalingdocker Kubernetes and Jenkins As a plugin a different approach https://github.com/jenkinsci/kubernetes-plugin
- 61. @csanchez#scalingdocker Example code and slides Available at http://slideshare.csanchez.org https://github.com/carlossg/kubernetes-jenkins http://blog.csanchez.org