This document provides an introduction to overlay networking concepts in Kubernetes, including VXLAN as an example overlay technology. It discusses Linux namespaces and how they enable network isolation in containers. It then compares the networking models of Kubernetes and Docker and describes Kubernetes' service abstraction and how inter-pod communication works with overlay networks. The document uses Flannel as an example overlay implementation in Kubernetes and provides links for further reading.

1. Kubernetes networking
Introduction to overlay networks, communication models and implementation
[April 26 2016]
[ Murat Mukhtarov ]
Zendesk

2. Contents
2
● Overlay networking introduction
○ Overlay concept
○ Overlay network example: VXLAN
● Linux namespaces
● Kubernetes networking
○ Kubernetes and Docker networking comparison
○ Service abstraction
○ Inter POD communication
○ Flannel example
● Q&A and links

3. Overlay networks: it is not a new paradigm
3
Stacking nature of networking allows
encapsulation of different protocol stack at each
network layer:
- Datalink could be encapsulated in datalink, IP,
transport (e.g. UDP, IPSEC)
- IP could be encapsulated in IP, MPLS, Datalink
- MPLS allows simple stacking for complex
service-oriented topologies

4. Overlay networking concept
4
Overlay networks - encapsulation of the full tcp/ip
stack including layer 2 inside transport network
(UDP datagrams)

5. Overlay example: VXLAN
5
VXLAN overlay networking technology that allows to send Ethernet traffic encapsulated into UDP datagrams
over IP/GRE networks. Detailed description of VXLAN networking could be found in RFC7348
24 bit VNI field is VXLAN address field that could be
compared with 802.1q tag for Ethernet frames or MPLS
label.
Bare in mind MTU value when using VXLAN

6. Linux network namespaces
6
Network namespaces is a part of containerization
technology that used by Linux kernel
Network namespaces allows:
○ To create linux container network isolation
instances (namespaces)
○ With own routing table, virtual interfaces, L2
isolation
● The tool that is used to operate with network ns:
iproute2
● Network namespaces are stored in
○ /var/run/netns
● There two types of network namespaces:
○ Root namespace [ ip link ]
○ Non-root namespace [ ip netns .. ip link ]

7. Comparison network models:
Kubernetes and Docker
7
Docker host example Kubernetes node example

8. Kubernetes: service abstraction
8
Service-oriented model:
- POD represents group of containers in the
same namespace
- Service represents load-balancing group of
PODs
- Service could be mapped to overlay
network number, e.g. VXLAN number
Kubernetes networking model provides
flexibility in terms of:
- IP addressing: routable address per pod
- Organizing networks: Flannel, OVS and etc.
- Customising plugins: e.g. CNI
- Docker0 serves as L3 (IP) gateway for PODs
behind it

9. Inter POD communication with overlay:
general principle
9
Communication can be
implemented as L2 and L3, it
totally depends on chosen
network model.
This particular example assumes
that DHCP server resides
somewhere in the network
where overlay subnets are
terminated.
Further flexibility could be
achieved with CNI.

10. Inter POD communication: other examples
10
Flannel OVS
https://github.com/coreos/flannel#flannel http://kubernetes.io/docs/admin/ovs-networking/

11. Flannel example explanation
11
● Flannel creates overlay network on your choice:
○ UDP or VXLAN encapsulation
● Flannel creates interface
○ flannel.VNI
● where VNI is number that you specified in json
payload.
● Flannel interface is being assigned with ip
address 10.1.X.0/16 address, where X - is
random subnet number.
● Docker0 interface is assigned with address 10.1.
x.1/24 all hosts behind docker0 are assigned
with 10.1.x.2 - 254 addresses with 24 bit mask.
● Nodes are talking each other on switched
192.168.1.0/24 network
Apparently scaling can be questionable if we have
more than 150-200 nodes in the network

12. Q&A
mmukhtarov@zendesk.com
Links:
Kubernetes network design
https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/networking.md
Kubernetes with OVS
http://kubernetes.io/docs/admin/ovs-networking/
Kubernetes with Flannel
https://github.com/coreos/flannel#flannel
Calico BGP project
https://github.com/projectcalico/calico-containers
BaGPipe BGP CNI plugin:
https://github.com/murat1985/bagpipe-cni