Video: https://youtu.be/C_u4_l84ED8 Karl Isenberg reviews the history of distributed computing, clarifies terminology for layers in the container stack, and does a head to head comparison of several tools in the space, including Kubernetes, Marathon, and Docker Swarm. Learn which features and qualities are critical for container orchestration and how you can apply this knowledge when evaluating platforms.

1. © 2016 Mesosphere, Inc. All Rights Reserved. 1
CONTAINER
ORCHESTRATION
WARS

2. © 2016 Mesosphere, Inc. All Rights Reserved. 2
KARL WHO?
Where?
- Mesosphere
- Pivotal
- Williams-Sonoma
What?
- DC/OS
- Kubernetes
- CloudFoundry
- BOSH
- Spring Ecommerce Platform
- github.com/karlkfi
- twitter.com/karlkfi
- linkedin.com/in/karlkfi
- karl.isenberg.us

3. © 2016 Mesosphere, Inc. All Rights Reserved. 3
Revisionist History of Distributed Computing
Container Orchestration
Container Platforms
Platform as a Service
Distributed Operating Systems
Competitive Landscape
AGENDA

4. © 2016 Mesosphere, Inc. All Rights Reserved. 4
IN THE
BEGINNING...

5. © 2016 Mesosphere, Inc. All Rights Reserved.
Hardware
Operating System
Application
5
COMPUTERS

6. © 2016 Mesosphere, Inc. All Rights Reserved.
Hardware
Operating System
Web Application
6
INTERNET

7. © 2016 Mesosphere, Inc. All Rights Reserved.
Web App
Hardware
Operating
System
7
SCALE &
HIGH
AVAILABILITY
Operating
System
Operating
System
Hardware Hardware
Web App Web App

8. © 2016 Mesosphere, Inc. All Rights Reserved.
Service Service Service
Web App Web App Web App
Hardware
Operating
System
8
SERVICE-
ORIENTED
ARCHITECTURE
Operating
System
Operating
System
Hardware Hardware

9. © 2016 Mesosphere, Inc. All Rights Reserved.
Service Service Service
Web App Web App Web App
Machine
Operating
System
9
HARDWARE
VIRTUALIZATION
Operating
System
Operating
System
Infrastructure
Machine Machine

10. © 2016 Mesosphere, Inc. All Rights Reserved.
Operating
System
Operating
System
Operating
System
ServiceApp ServiceServiceAppApp
10
MICROSERVICES
Machine
Infrastructure
Machine Machine
ServiceService ServiceServiceServiceService

11. © 2016 Mesosphere, Inc. All Rights Reserved.
ServiceApp ServiceServiceAppApp
OS
11
CONTAINERIZED
MICROSERVICES
OS OS
Machine
Infrastructure
Machine Machine
Container Runtime Container Runtime Container Runtime
ServiceService ServiceServiceServiceService

12. © 2016 Mesosphere, Inc. All Rights Reserved.
Service
Container Runtime Container Runtime Container Runtime
OS
12
CONTAINER
ORCHESTRATION
OS OS
Container Orchestration
Machine
Infrastructure
Machine Machine
App ServiceServiceAppApp

13. © 2016 Mesosphere, Inc. All Rights Reserved. 13
“The automated arrangement,
coordination, and management of
complex computer systems,
middleware and services.”
- Wikipedia
ORCA-WHAT-NOW?

14. © 2016 Mesosphere, Inc. All Rights Reserved. 14
“Continuous automated
scheduling, coordination, and
management of complex systems
of containerized components and
the resources they consume.”
- Karl
CONTAINER ORCHESTRATION

15. © 2016 Mesosphere, Inc. All Rights Reserved.
Orchestration
15
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
CONTAINER
ORCHESTRATION
Machine & OS Machine & OS
Container Runtime Container Runtime
Revised

16. © 2016 Mesosphere, Inc. All Rights Reserved. 16

17. © 2016 Mesosphere, Inc. All Rights Reserved. 17
SCHEDULING
- Placement
- Replication/Scaling
- Resurrection
- Rescheduling
- Rolling Deployment
- Upgrades
- Downgrades
- Collocation
RESOURCE
MANAGEMENT
- Memory
- CPU
- GPU
- Volumes
- Ports
- IPs
SERVICE
MANAGEMENT
- Labels
- Groups/Namespaces
- Dependencies
- Load Balancing
- Readiness Checking
FUNCTIONAL CAPABILITIES

18. © 2016 Mesosphere, Inc. All Rights Reserved. 18
SCALABILITY
Performance, Responsiveness, Efficiency
AVAILABILITY
Fault Tolerance, Robustness, Reliability, Resilience,
Disaster Recovery
FLEXIBILITY
Format Support, Portability, Interoperability,
Extensibility
USABILITY
Familiarity, Maintainability, Compatibility,
Debuggability
PORTABILITY
Container Runtimes, Host OS, Hosted, Cloud, Bare-
Metal
SECURITY
Auditability, Secrets Management, Encryption,
Isolation
NON-FUNCTIONAL QUALITIES

19. © 2016 Mesosphere, Inc. All Rights Reserved. 19
“A platform is any base of
technologies on which other
technologies or processes are
built.”
- Some Guy on the Internet
CLOUD PLATFORM?

20. © 2016 Mesosphere, Inc. All Rights Reserved. 20
“Platform as a service (PaaS) is a
category of cloud computing
services that provides a platform
allowing customers to develop,
run, and manage applications
without the complexity of building
and maintaining the infrastructure
typically associated with
developing and launching an app.”
- Wikipedia
PAAS?

21. © 2016 Mesosphere, Inc. All Rights Reserved. 21
“A distributed system on which
users develop, run, and manage
containerized applications and
services.”
- Karl
CONTAINER PLATFORM

22. © 2016 Mesosphere, Inc. All Rights Reserved.
APPLICATION DEVELOPMENT LIFECYCLE
Source Repo, Continuous Integration, Artifact Repo
CONTAINER ORCHESTRATION
Scheduling, Resource Management, Service
Management
DAY 2 OPERATIONS
Debugging, Maintenance
22
CONTAINER OPERATIONS

23. © 2016 Mesosphere, Inc. All Rights Reserved. 23
APPLICATION DEVELOPMENT LIFECYCLE
CONTAINER OPERATIONS
Code
Build, Test,
Package
Publish Deploy

24. © 2016 Mesosphere, Inc. All Rights Reserved.
Applications & Services
24
APPLICATION DEVELOPMENT LIFECYCLE
CONTAINER OPERATIONS
Source Control
Continuous
Integration
Artifact
Repository
Orchestration

25. © 2016 Mesosphere, Inc. All Rights Reserved. 25
APPLICATION DEVELOPMENT LIFECYCLE
CONTAINER OPERATIONS
Applications & Services
Source Control
Continuous
Integration
Artifact
Repository
Orchestration

26. © 2016 Mesosphere, Inc. All Rights Reserved. 26
DEBUGGING
- Logging & Metrics
- Emitting
- Aggregation
- Search
- Reporting
- Alerting
- Dashboards
- Shell Access
MAINTENANCE
- Package Management
- Upgrading
- Cluster Resizing
- Application Autoscaling
- Capacity Planning
- Software Defined Networking
- Backups
- Disaster Recovery
DAY 2 OPERATIONS
CONTAINER OPERATIONS

27. © 2016 Mesosphere, Inc. All Rights Reserved.
System Space
Orchestration
User Space
Source Control
Continuous
Integration
Artifact
Repository
27
CONTAINER OPERATIONS
Load Balancer
Package
Manager
Security
Networking
Logging &
Metrics
Storage

28. © 2016 Mesosphere, Inc. All Rights Reserved.
Kernel
28
OPERATING
SYSTEM
System Space
Hardware
User Space

29. © 2016 Mesosphere, Inc. All Rights Reserved. 29
DISTRIBUTED
OPERATING
SYSTEM
Hardware Hardware Hardware
User Space
Circa 1992
Microkernel
System Space
Microkernel Microkernel

30. © 2016 Mesosphere, Inc. All Rights Reserved. 30
DYSEAC &
MOBIDIC
Circa 1954

31. © 2016 Mesosphere, Inc. All Rights Reserved. 31
PLAN 9
Circa 1992

32. © 2016 Mesosphere, Inc. All Rights Reserved. 32
THEN
- Rewrite your Applications
- Concurrent Programming Language
- Virtual Machine Environment
- UNIX-like, POSIX-like
- Custom Kernel
- Process-level Fault Tolerance
NOW
- BYO Applications
- BYO Programming Language
- BYO Container Operating System
- BYO Container Runtime
- BYO Machine Operating System
- System-level Fault Tolerance
DISTRIBUTED OPERATING SYSTEM

33. © 2016 Mesosphere, Inc. All Rights Reserved. 33
Machine Infrastructure
Applications & Services
Orchestration
Container Runtime
System Services
DISTRIBUTED
OPERATING
SYSTEM
Container Runtime Container Runtime
Revised
OS OS OS
Machine Machine Machine
DistributedOS

34. © 2016 Mesosphere, Inc. All Rights Reserved. 34

35. © 2016 Mesosphere, Inc. All Rights Reserved. 35
SECURITY
CONCERNS - End-To-End Encryption
- Secrets
- Certificate Management
- Public Key Infrastructure
- Service Accounts
- User Accounts
- User Groups
- Directory Service Integration
- Multitenancy
- Permissions
- Security Policies
- License Management

36. © 2016 Mesosphere, Inc. All Rights Reserved. 36
SERVICE
CATALOG - Distributed Storage
- Databases
- Message Bus
- Stream Processing
- Machine Learning
- API Brokers
- Autoscalers
- Log Search
- Version Control
- Continuous Integration
- Container Repositories

37. © 2016 Mesosphere, Inc. All Rights Reserved. 37

38. © 2016 Mesosphere, Inc. All Rights Reserved. 38
WHO ARE
TODAY’S
CONTESTANTS?

39. © 2016 Mesosphere, Inc. All Rights Reserved. 39

40. © 2016 Mesosphere, Inc. All Rights Reserved. 40
TODAY’S
CONTESTANTS
- Docker Swarm
- ECS
- Kontena
- Nomad
- Kubernetes
- Rancher
- OpenShift
- DC/OS
Non-Exhaustive List

41. © 2016 Mesosphere, Inc. All Rights Reserved. 41
CONTAINER ORCHESTRATION
- Scheduling
- Resource Management
- Service Discovery
CONTAINER PLATFORM
- Container Lifecycle Management
PLATFORM AS A SERVICE
- Application Lifecycle Management
DISTRIBUTED OPERATING SYSTEM
- Container Operations
- Distributed System Services
- Services Catalog & Application Ecosystem
What’s
What?

42. © 2016 Mesosphere, Inc. All Rights Reserved. 42
Docker Swarm

43. © 2016 Mesosphere, Inc. All Rights Reserved. 43
Docker Swarm
CONTAINER
ORCHESTRATION

44. © 2016 Mesosphere, Inc. All Rights Reserved. 44
Amazon EC2 Container Service (ECS)

45. © 2016 Mesosphere, Inc. All Rights Reserved. 45
Amazon EC2 Container Service (ECS)
CONTAINER
ORCHESTRATION

46. © 2016 Mesosphere, Inc. All Rights Reserved. 46
Kontena

47. © 2016 Mesosphere, Inc. All Rights Reserved. 47
Kontena
CONTAINER
ORCHESTRATION

48. © 2016 Mesosphere, Inc. All Rights Reserved. 48
Nomad by HashiCorp

49. © 2016 Mesosphere, Inc. All Rights Reserved. 49
Nomad by HashiCorp
ORCHESTRATION

50. © 2016 Mesosphere, Inc. All Rights Reserved. 50
Kubernetes

51. © 2016 Mesosphere, Inc. All Rights Reserved. 51
Kubernetes
CONTAINER
PLATFORM

52. © 2016 Mesosphere, Inc. All Rights Reserved. 52
Rancher

53. © 2016 Mesosphere, Inc. All Rights Reserved. 53
Rancher
CONTAINER
PLATFORM

54. © 2016 Mesosphere, Inc. All Rights Reserved. 54
OpenShift

55. © 2016 Mesosphere, Inc. All Rights Reserved. 55
OpenShift
PaaS

56. © 2016 Mesosphere, Inc. All Rights Reserved. 56
DC/OS

57. © 2016 Mesosphere, Inc. All Rights Reserved.
DISTRIBUTED
OPERATING
SYSTEM 57
DC/OS

58. © 2016 Mesosphere, Inc. All Rights Reserved. 58

59. © 2016 Mesosphere, Inc. All Rights Reserved. 59
ORCHESTRATION
FUNCTIONAL
COMPARISON

60. © 2016 Mesosphere, Inc. All Rights Reserved. 60
SCHEDULING
ORCHESTRATION FUNCTIONAL COMPARISON (06/2016)
Placement
Kubernetes Mesos/Marathon ECS Swarm Nomad Cattle Kontena
Replication/Scaling
Readiness Checking
Resurrection
Rescheduling
Rolling Deployment
Collocation
Included External/Partial

61. © 2016 Mesosphere, Inc. All Rights Reserved. 61
RESOURCE MANAGEMENT
ORCHESTRATION FUNCTIONAL COMPARISON (06/2016)
Memory
Kubernetes Mesos/Marathon ECS Swarm Nomad Cattle Kontena
CPU
GPU
Disk Space
Persistent Volumes
Ports
IPs
Included External/Partial
Volumes

62. © 2016 Mesosphere, Inc. All Rights Reserved. 62
SERVICE MANAGEMENT
ORCHESTRATION FUNCTIONAL COMPARISON (06/2016)
Labels
Kubernetes Mesos/Marathon ECS Swarm Nomad Cattle Kontena
Groups/Namespaces
Dependencies
Load Balancing
Readiness Checking
Included External/Partial

63. © 2016 Mesosphere, Inc. All Rights Reserved. 63
"Let's face it, the universe is messy. It is nonlinear,
turbulent, and chaotic. It is dynamic. It spends its time in
transient behavior on its way to somewhere else, not in
mathematically neat equilibria. It self-organizes and
evolves. It creates diversity, not uniformity. That's what
makes the world interesting, that's what makes it
beautiful, and that's what makes it work."
Donella H. Meadows
(Thinking in Systems: A Primer)