1.
| PageJul 2016

2.
| Page
Scaling Docker with Kubernetes
Jul 2016
Liran Cohen
Cloud platform & DevOps TL , Liveperson

3.
| Page
Agenda
● Docker ?
● Kubernetes introduction.
● Kubernetes Addons.
● Basic Troubleshooting
● GB Demo
● Q&A
demo files - https://github.com/sliranc/k8s_workshop

4.
| Page
Docker?
Going back in time , most applications were deployed directly on physical hardware.
● Single userspace.
● Shared runtime env between applications.
● Hardware resources generally underutilized.

5.
| Page
To overcome the limitation of shared runtime env ,
underutilized resources and more. The IT industry
adopted virtualization with hypervisor such as KVM
, ESX and more.
Docker?

6.
| Page
Moving from VM => “Virtual OS” :
● We removed the hypervisor layer to reduce
complexity.
● The containers approach is to package each application
with all dependencies runtime environment.
● We have different application running on the same
host and isolated using the containers technology.
Docker?

7.
| Page
Virtual Machines VS Containers
VS

8.
| Page
Docker: Application centric.
● A clean, safe, portable runtime environment for your app.
● No more worries about missing dependencies, packages
and other pain points during deployments.
● Run each app in its own isolated container (fs , cgroup ,
pid etc ….)
● Easy to pack into a box and super portable.
Build once... (finally) run anywhere*
Docker?

9.
| Page
Docker’s architecture
● Docker uses client server architecture.
● server: running the Docker daemon.
● Client: communicate with the server via sockets or RESTful API .
● Docker registry: public or private stores from which the server upload or download images
● The client can run on any host.

10.
| Page
Docker?
DEMOFROM nginx
MAINTAINER Liran Cohen <sliranc@gmail.com>
COPY index.html /usr/share/nginx/html/index.html
CMD ["nginx", "-g", "daemon off;" ],
Dockerfile
1. Build docker image
docker build -t sliranc/hello_docker:latest .
2. Run docker container.
docker run -p 32769:80 -d --name hello_docker sliranc/hello_docker:latest
curl http://192.168.99.100:32769
3. Push to remote registry.
docker push sliranc/hello_docker

11.
| Page
The name Kubernetes originates from Greek, meaning
“helmsman” or “pilot””(WiKi).
A helmsman or helm is a person who steers a ship, sailboat, submarine...
Kubernetes - κυβερνήτης

12.
| Page
More facts:
● Originated at Google (Borg).
● Supports multiple cloud and bare-metal environments
● Supports multiple container runtimes (Docker , rkt)
● 100% Open source, written in Go
● k8s is an abbreviation derived by replacing the 8 letters “ubernete” with 8.
Manage containerized applications , not machines.
Kubernetes ?
Kubernetes is a container cluster manager. It aims to provide a
"platform for automating deployment, scaling, and operations of
application containers across clusters of machines.

13.
| Page
Deploy single tier single container APP is “easy”
Deploying a complex multi tier APP is more difficult
● One or more containers.
● replication of containers.
● Persistent storage.
Deploying lots of complex APPs (microservices) can be a challenge.
More Info...
Why kubernetes ?

14.
| Page
Control Plane
Node Controller
Replication Controller
Endpoints Controller
Service Account
Token Controllers
And more...`
architecture

15.
| Page
A node is a physical or virtual machine
running Kubernetes, onto which pods
can be scheduled.
Node
operating system
kubelet
kube-proxy
k8s

16.
| Page
Kubectl - get (Display one or many resources)
1. List kubernetes nodes.
kubectl get nodes
kubectl get nodes --context=kube-aws
DEMO

17.
| Page
Pod is a Small group of co-located containers with optionally shared
volume between the containers.
Pods are the basic deployment unit in Kubernetes.
● Shared namespace
○ Share IP address , localhost
○ Every pod gets a unique IP
● Managed Lifecycle
○ Bound to a node , in place restart
○ Cannot move between nodes
Pod(po)

18.
| Page
Pod(po) - yaml manifest
apiVersion: v1
kind: Pod
metadata:
labels:
phase: prod
role: frontend
name: myfirstpod
name: myfirstpod
spec:
containers:
- name: filepuller
image: sliranc/filepuller:latest
volumeMounts:
- mountPath: /usr/share/nginx/html
name: static-vol
- name: webserver
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html
name: static-vol
volumes:
- name: static-vol
emptyDir: {}
Spec: is the specification of the
desired state of an object.
kind: System object resource
Examples: Pod, RC, Service etc...

19.
| Page
kubectl
1. Create pod myfirstpod by filename.
kubectl create -f myfirst-pod.yaml
kubectl create -f myfirst-pod.yaml --context kube-aws
2. List pods.
kubectl get pods
kubectl get pods --context=kube-aws
DEMO

20.
| Page
Label are key / value pairs - object metadata
● Label are attached to pods , services , rc or almost any other objects in k8s
● Can be used to organize or select subset of object.
● queryable by selectors
labels:
app: rcweb
phase: production
role: frontend
http://kubernetes.io/docs/user-guide/labels/
Labels

21.
| Page
Label selector - query object using labels
● Can identify a set of objects
● Group a set of objects
● Used in svc and rc to select the monitored watched objects
replication controller selector example:
selector:
app: rcweb
phase: production
Selectors

22.
| Page
direct traffic to pods
Defines a logical set of pods and a policy by which
to access them.
● are abstraction on top of the pods (LB)
● use selector to create the logical set of pods.
● Gets a stable virtual IP and Port.
● Cluster IP are only available inside k8s
services (svc)
Can define:
● What the 'internal' IP should be.(ClusterIP)
● What the 'external' IP should be. (NodePort , LoadBalancer)
● What port the service should listen on.

23.
| Page
services LoadBalancer
NodePort
ClusterIP

24.
| Page
services (svc) - yaml manifest
apiVersion: v1
kind: Service
metadata:
name: myweb
spec:
ports:
- port: 80 # the port that this service should serve on.
# (e.g. 'www') or a number (e.g. 80)
targetPort: 80
protocol: TCP
# just like the selector in the replication controller,
# but this time it identifies the set of pods to load balance
traffic to.
selector:
name: myfirstpod
System object resource
Examples: Pod, RC, Service etc...
Spec is the specification of the
desired state of an object.
labels:
phase: prod
role: frontend
name: myfirstpod

25.
| Page
kubectl
1. Create service myweb by filename:
kubectl create -f myfirst-svc.yaml
kubectl create -f myfirst-svc.yaml --context=kube-aws
2. NodePort:
kubectl describe -f myfirst-svc.yaml
http://ctor-knb001:<node_port>/
3. LoadBalancer:
kubectl describe -f myfirst-svc.yaml --context=kube-aws
Change static content in github - Edit index.html
DEMO

26.
| Page
Service Iptables mode:
Node X
Pod
Client A
Cluster IP(VIP)
Client B
Kube-Proxy
Kubelet
NodePort NodePort
Kube-Proxy
Pod 1 Pod 2 Pod 3
Iptables

27.
| Page
Replication controller == pods supervisor
Ensures that a specified number of pod "replicas"
are running at any given time:
● Too many pods will trigger pods termination.
● Too few pods will trigger new pods creation.
● Main Goal = Replicas: x current / x desired.
replication controller will monitor all the pods
defined in the label selector.
replication controller (rc) ReplicaSet (rs)
ReplicationController
replica: 4
name: rcweb
selector:
app: rcweb
phase: production

28.
| Page
apiVersion: v1
kind: ReplicationController
metadata:
name: rcweb
labels:
name: rcweb
spec:
replicas: 2
# selector identifies the set of pods that this replication controller is responsible for managing
selector:
app: rcweb
phase: production
# template defines the 'cookie cutter' used for creating new pods when necessary
template:
metadata:
labels:
app: rcweb
role: frontend
phase: production
name: rcwebpod
spec:
containers:
- name: staticweb
image: sliranc/rcweb:latest
Pod
Template
replication controller (rc) - yaml manifest

29.
| Page
replication controller (rc) ReplicaSet
master
Node 1 Node 2
Replication controller
replica: 3
name: rcweb
selector:
app: rcweb
Phase: production
Pod 2
label:
app:rcweb
phase:production
Pod 1
label:
app:rcweb
phase:production
Pod 3
label:
app:rcweb
phase:production
Pod X
label:
app:my-app
phase:alpha

30.
| Page
kubectl
1. create all resources in a directory.
kubectl create -f rcweb
kubectl get pods -l app=rcweb
kubectl describe svc/rcweb
kubectl rolling-update rcweb --image=sliranc/rcweb:v2 --update-period="10s"
DEMO

31.
| Page
http://12factor.net/
III. Config
Store config in the environment

32.
| Page
ConfigMap & Secrets
● ConfigMap is a resource available in kubernetes for managing
application configuration. The goal is to decouple the app
configuration from the image content in order to keep the
container portable and k8s agnostic.
● ConfigMap are key value pairs of configuration data.
http://kubernetes.io/docs/user-guide/configmap/
kind: ConfigMap
apiVersion: v1
metadata:
name: default-app
data:
db-host: MYDB
apiVersion: v1
kind: Pod
metadata:
name: test-default-app
spec:
containers:
- name: test-defaultapp
image:sliranc/rcweb
env:
- name: DB_HOST
valueFrom:
configMapKeyRef:
name: default-app
key: db-host

33.
| Page
St rage

34.
| Page
● emptyDir
● hostPath
● gcePersistentDisk
● awsElasticBlockStore
● nfs
● iscsi
● flocker
● glusterfs
Volume types
● rbd
● gitRepo
● secret
● persistentVolumeClaim
● downwardAPI
● azureFileVolume
● vsphereVirtualDisk

35.
| Page
emptyDir
emptyDir is a temporary directory that
shares a pod's lifetime.
● Storage provider = Local host
● Files will be erased on pod deletion.
● Mounted by containers inside the pod.
emptyDir path on node:
/var/lib/kubelet/pods/<id>/volumes/kubernetes.io~empty-dir/<volume_name>
volumes:
- name: static-vol
emptyDir: {}

36.
| Page
hostPath
hostPath is a bare host directory volume.
● Acts as data volume in Docker.
● Containers can RW files on localhost.
● There is no control on quota.
Volumes:
- name: static-vol
hostPath:
path: /target/path/on/host

37.
| Page
PersistentVolume
Kubernetes provides
abstraction for volumes using
PersistentVolume (PV).
User - Claim PV using (PVC) persistentVolumeClaim
(pvc001 , pvc002)
PersistentVolume(PV)
nfs awsElasticBlockStore
rbdgcePersistentDisk
PersistentVolumeClaim(PVC)
Pod1 Pod2 Pod3
Admin - Creates pool of PVs (pv0001 , pv0002)
volumes:
- name: my-vol
persistentVolumeClaim:
claimName: "pvc001"

38.
| Page
Multi tenancy in kubernetes.
● A single cluster should be able to satisfy the needs of multiple users or
groups of users.
Each user community has its own:
1. resources (pods, services, replication controllers, etc.)
2. policies (who can or cannot perform actions in their community)
3. constraints (this community is allowed this much quota, etc.)
Kubernetes starts with two initial namespaces:
default - The default namespace for objects with no other namespace.
kube-system - The namespace for objects created by the Kubernetes system
Namespaces

39.
| Page
ResourceQuota
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota
spec:
hard:
cpu: "20"
memory: 1Gi
pods: "10"
replicationcontrollers: "20"
resourcequotas: "1"
services: "5"
Namespaces

40.
| Page
Managing your app
Loggos video

41.
| Page
Addons

42.
| Page
DNS
● The DNS add-on allows your services to have a DNS name
in addition to an IP address. This is helpful for simplified
service discovery between applications.
*As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager cluster add-on

43.
| Page
Monitoring - Heapster
● Heapster enables monitoring and performance analysis in Kubernetes
Clusters. Heapster collects signals from kubelets(cadvisor) and the api
server, processes them, and exports them...
Heapster
sink
Data
push
Query
Master (API)Node (Kubelet)

44.
| Page
Kubernetes Dashboard
http://kubernetes.io/docs/user-guide/ui/

45.
| Page
Centralized logging
Node X
Pod - App1
Pod - App2
Pod , k8s plugin
Pod - App3
/var/lib/docker/containers/idTail
Stdout , stderr stream
Stdout , stderr stream
Stdout , stderr stream
Stdout , stderr stream
JS
O
N

46.
| Page

47.
| Page
Basic Troubleshooting

48.
| Page
DEMO
kubectl - describe
(Show details of a specific resource or group of resources)
1. describe a pod (po)
kubectl describe pod/myfirstpod
2. describe a service (svc) - check Endpoints
kubectl describe svc/myweb
3. describe a node
kubectl describe node/ctor-knb002

49.
| Page
kubectl - logs (Print the logs for a container in a pod)
1. create logme pod from url.
kubectl create -f https://raw.githubusercontent.
com/sliranc/k8s_workshop/master/cli_demo/logme-pod.
yaml
2. Print the logs of a pod with one container
kubectl logs logme
3. stream the logs
kubectl logs -f logme
4. print the logs of a container filepuller in pod myfirstpod
kubectl logs myfirstpod -c filepuller
DEMO

50.
| Page
kubectl - exec
(Execute a command in a container)
1. Inject bash to a single pod container
kubectl exec -it logme bash
ps -auxwww
exit
2. Inject bash to a multi container pod
kubectl exec -it myfirstpod -c webserver bash
ps -auxwww
exit
DEMO

51.
| Page
kubectl
(Edit a resource from the default editor)
1. Edit ReplicationController
kubectl edit rc/rcweb
Restart pods
2. kubectl port-forward - forwards connections to a port on a pod
kubectl port-forward myfirstpod 8888:80
curl http://localhost:8888
DEMO

52.
| Page
RedisSlave
Guestbook - DEMO
Deploy multi tier web_app - Guestbook
frontend - Pod
SVC - frontend
frontend - Pod
Pod
RedisMaster
SVC
RedisMaster
SVC
RedisSlave
Pod
RedisSlave
https://github.com/kubernetes/kubernetes/tree/master/examples/guestbook

53.
| Page
Guestbook - DEMO
1. Create the replication controller for frontend
kubectl create -f gb-frontend-rc.yaml
2. Create the service for frontend
kubectl create -f gb-frontend-svc.yaml
3. Create the redis-master replication controller
kubectl create -f redis-master-rc.yaml
4. Create the service for redis-master
kubectl create -f redis-master-svc.yaml
5. Create the redis-slave replication controller
kubectl create -f redis-slave-rc.yaml
6. Create the service for redis-master
kubectl create -f redis-slave-svc.yaml
7. Get the external url for svc and browse to the website.
kubectl describe svc frontend
8. Delete all pods
Kubectl delete ...
9. Scale your rc
kubectl scale ...
10. Delete all resources svc , rc
DEMO

54.
| Page
Pod Health checks
Liveliness Readiness
On failure Kill container Stop sending traffic to pod
Check types Http , exec , tcpSocket Http , exec , tcpSocket
Declaration example
(Pod.yaml)
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
readinessProbe:
httpGet:
path: /status
port: 8080

55.
| Page
Kubernetes - Proxies
1. Api-proxy
kubectl cluster-info
<kubernetes_master_address>/api/v1/proxy/namespaces/<namespace_name>/services/<service_name>[:port_name]
http://qtvr-kma01:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
1. kubectl proxy - proxies from a localhost address to the apiserver
kubectl proxy
http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
2. Kube-proxy - proxies UDP and TCP - provides load balancing.

56.
| Page
https://github.com/kubernetes/minikube
Minikube
easily run Kubernetes locally
The project goal is to build an easy-to-use, high-fidelity
Kubernetes distribution that can be run locally on Mac, Linux
and Windows with a single command.

57.
| Page
prometheus
https://prometheus.io/
Inspired by google’s Borgmon monitoring system.

58.
| Page
K8S on Openstack OpenStack on Kubernetes
Murano Stackanetes , kolla
Magnum https://www.youtube.com/watch?
v=DPYJxYulxO4
Configuration management tools (Puppet ,
Ansible , salt etc…)
Openstack cloudprovider driver for kubernetes.
(BlockStorage , LBaaS)
Kubelet
Kubernetes <=> Openstack

59.
| Page
Q&A

60.
| Page
THANK YOU!
We are hiring

61.
| Page
Backup Slides

62.
| Page
CLI Basics - kubectl

63.
| Page
kubectl - kubernetes client.
1. Looking for Help...
kubectl help
Use "kubectl help [command]" for more information about a command
2. Version - get the server and client version.
kubectl version
3. Cluster info
kubectl cluster-info
*Display urls of the master and services(*api proxy) with label kubernetes.io/cluster-service=true

64.
| Page
Kubectl - Available commands

65.
| Page
kubectl - create
1. Create a resource by filename.
kubectl create -f myfirst-pod.yaml
2. create all resources in a directory.
kubectl create -f rcweb
3. create a resource from stdin.
cat myfirst-svc.yaml | kubectl create -f -
4. create a resource from url.
kubectl create -f https://raw.githubusercontent.
com/sliranc/k8s_workshop/master/cli_demo/logme-pod.yaml

66.
| Page
kubectl - get (Display one or many resources)
1. List all types of resource to get.
kubectl get
2. List all pods
kubectl get pods
3. List all pods in wide format
kubectl get pods -o wide
4. Display specific pod in yaml format
kubectl get pod/myfirstpod -o yaml

67.
| Page
kubectl - get (Display one or many resources)
1. Query for pod with specific label
kubectl get pod -l name=rcwebpod
2. List all pods and services
kubectl get pods,svc
3. List all nodes (no)
kubectl get nodes

68.
| Page
kubectl - describe
(Show details of a specific resource or group of resources)
1. List all types of resource to describe.
kubectl describe
2. describe a pod (po)
kubectl describe pod/myfirstpod
3. describe a service (svc)
kubectl describe svc/myweb
browse to LoadBalancer Ingress of the service http://ingress...
4. describe a node
kubectl describe node/<use one from the get nodes output>

69.
| Page
kubectl - logs (Print the logs for a container in a pod)
1. Print the logs for pod with one container
kubectl logs logme
2. stream the logs
kubectl logs -f logme
3. print the logs of a container filepuller in pod myfirstpod
kubectl logs myfirstpod -c filepuller
HW : check out the -p flag for logs.

70.
| Page
kubectl - scale
(Set a new size for a Replication Controller)
1. Scale rc rcweb to 3 replicas
kubectl get pods -l name=rcwebpod
kubectl scale --replicas=3 rc rcweb
kubectl get pods -l name=rcwebpod
2. Scale only if the current replication is X
kubectl scale --current-replicas=3 --replicas=2 rc rcweb
kubectl get pods -l name=rcwebpod

71.
| Page
kubectl - exec
(Execute a command in a container)
1. Inject bash to a single pod container
kubectl exec -it logme bash
ps -auxwww
exit
2. Inject bash to a multi container pod
kubectl exec -it myfirstpod -c webserver bash
ps -auxwww
exit

72.
| Page
kubectl - rolling-update
(Set a new size for a Replication Controller)
1. Get the service ingress and browse to http://ingress……..
kubectl describe svc/rcweb
2. upgrade your rc to new rc with 10s delay between pod.
kubectl rolling-update rcweb --image=sliranc/rcweb:v2 --update-
period="10s"
Refresh your browser.
kubectl get pods -l name=rcwebpod

73.
| Page
kubectl edit
(Edit a resource from the default editor)
1. Edit the number of replicas
kubectl edit rc rcweb
kubectl get pods -l name=rcwebpod

74.
| Page
kubectl - delete
(Delete a resource by filename, stdin, resource and name, or by resources and label selector)
1. Delete resource by file
kubectl delete -f myfirst-pod.yaml
2. Delete resource by name
kubectl delete svc myweb rcweb
3. Delete all pods
kubectl delete pods --all
4. Delete rc by name
kubectl delete rc rcweb

75.
| Page
kubectl - delete
(Delete a resource by filename, stdin, resource and name, or by resources and label selector)
1. Delete resource by file
kubectl delete -f myfirst-pod.yaml
2. Delete resource by name
kubectl delete svc myweb rcweb
3. Delete all pods
kubectl delete pods --all
4. Delete rc by name
kubectl delete rc rcweb

76.
| Page
Docker images
● Docker images are the basis of
containers.
● Docker images are read-only
templates we use for creating
containers.
● Docker images are multilayer.
● docker images are highly portable
and can be shared.

77.
| Page
External source such as DB (headless service)
Web - Pod
SVC - Web
Web - Pod
App - Pod
SVC - App
App - Pod
SVC - DB
External DATA store

78.
| Page

79.
| Page
Userspace