OpenStack Summit Barcelona 2016

1. Copyright © 2016 Mirantis, Inc. All rights reserved
www.mirantis.com
Kubernetes SDN
Performance and
Architecture
Jakub Pavlik
Marek Celoud

2. Copyright © 2016 Mirantis, Inc. All rights reserved
Presentation Agenda
1. Overlay vs Non-Overlay
2. Calico
3. OpenContrail
4. Connection/comparison
5. Q&A

3. Copyright © 2016 Mirantis, Inc. All rights reserved
About us
Marek Celoud
mceloud@mirantis.com
@MCeloud
Jakub Pavlík
jpavlik@mirantis.com
@JakubPav

4. Copyright © 2016 Mirantis, Inc. All rights reserved
Networking in Kubernetes
● Networking in containers used to be an issue
● Kubernetes solved the biggest problems of port mapping
● Different approaches for different use cases
● Overlay vs. Non-overlay
● Multitenancy and security
● Performance and scaling
● Multiple plugins similar like OpenStack Neutron

5. Copyright © 2016 Mirantis, Inc. All rights reserved
Network solutions in Kubernetes
SDNs:
● Calico
● OpenContrail
● Romana
● Weave
● Contiv
● OpenVSwitch
● ...

6. Copyright © 2016 Mirantis, Inc. All rights reserved
Overlay vs. Non-overlay
Common Overlay concerns:
● Loose benefit of simplicity
● Loose performance
● Difficult to maintain and
troubleshoot
Overlay benefits:
● Multitenancy, Security,
Micro-segmentation
● L2, L3, EVPN, L3VPN
capability
● Analytics
From performance perspective not using an overlay, it is still
necessary to use an internal bridge to demux the container
virtual-ethernet interface pairs.
“The key aspect to consider is operational complexity!”
Pedro Marques

7. Copyright © 2016 Mirantis, Inc. All rights reserved
Test environment
● Run various functional and performance tests
● Calico bare metal
● OpenContrail bare metal
● OpenContrail running on Kubernetes with Calico
● OpenContrail and Kubernetes next together
● Calico in OpenStack with OpenContrail
● OpenContrail Kubernetes in OpenStack with OpenContrail
● 100 nodes with 32GB RAM with 8 CPUs and 2x 10Gb links

8. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico

9. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico Overview
● CNI network plugin
● BIRD routing daemon
● Etcd
● Confd
● Felix
● Pure L3

10. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico

11. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico
Calico
Pros:
● No overhead
● Reduce Complexity
● Using standard
protocols
Cons:
● Underlay depended
● No L2

12. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico with k8s
● Using CNI
● Calico 0.22.0 version with kubernetes 1.4
● Kubernetes Policy for security

13. Copyright © 2016 Mirantis, Inc. All rights reserved
Production consideration for Calico
● Use separate etcd cluster for Calico
● Use at least etcd v3
● Disable BGP full mesh peering
● Do not run Calico in k8s manifests, but as separated
systemd/docker

14. Copyright © 2016 Mirantis, Inc. All rights reserved
OpenContrail

15. Copyright © 2016 Mirantis, Inc. All rights reserved
OpenContrail Overview
● Overlay SDN
● Control, config, analytics, database, agent
● Multiple encapsulations (MPLSoverGRE/UDP, VXLAN)
● Uses (usually) physical gateways

16. Copyright © 2016 Mirantis, Inc. All rights reserved
OpenContrail overview

17. Copyright © 2016 Mirantis, Inc. All rights reserved
OpenContrail overview
OpenContrail
Pros:
● Underlay agnostic
● Advanced networking
features
● Uses physical
gateways
Cons:
● Overhead
● Complex

18. Copyright © 2016 Mirantis, Inc. All rights reserved
OpenContrail with s8s
● Network manager which provides bridge between Contrail
and k8s
● Using ECMP instead of kube-proxy (iptables) balancing
● Networks created based on labels in manifests
● Security and Multi-tenancy done by policy
● Contrail 3.0.3 supports Kubernetes 1.4

19. Copyright © 2016 Mirantis, Inc. All rights reserved
Production consideration for OpenContrail
● Separate Cassandra cluster for analytics
● Use physical routers as gateways

20. Copyright © 2016 Mirantis, Inc. All rights reserved
Comparison

21. Copyright © 2016 Mirantis, Inc. All rights reserved
Performance

22. Copyright © 2016 Mirantis, Inc. All rights reserved
Why not both?

23. Copyright © 2016 Mirantis, Inc. All rights reserved
Multi-cloud examples
● Connection
Baremetal, VMs,
container
● Run k8s on top of
OpenStack with
same Contrail (VM
sub-interfaces)

24. Copyright © 2016 Mirantis, Inc. All rights reserved
Kubernetes production findings
● build own binaries (Mirantis Downstream) instead of
reusing existing docker containers with unknown origin
● use single or high available cluster setup
● run ETCD control services in systemd not only in
manifests and docker
● cleanup from mixing bash, salt, and unrelated features for
production
● manage native SSL cert by Salt or external cert entity
● pull images from private docker registry with
authentication

25. Copyright © 2016 Mirantis, Inc. All rights reserved
Calico vs OpenContrail comparison

26. Copyright © 2016 Mirantis, Inc. All rights reserved
MCP

27. Copyright © 2016 Mirantis, Inc. All rights reserved
Q&A
Thank you for your time

28. Copyright © 2016 Mirantis, Inc. All rights reserved
Backup Slides