Mesos allows clusters of nodes to be treated as one big computer by sharing computational resources like cores, memory, and disks. It is a top-level Apache project that acts as a cluster resource negotiator, scaling to 10,000s of nodes in a fault-tolerant manner. Mesos supports running Docker containers at scale through frameworks like Marathon and Chronos, which provide interfaces for launching long-running and batch/one-off services respectively. Twitter uses Mesos as the "cornerstone" of their elastic compute infrastructure to build new services and scale to millions of daily users.

1. Docker containers at scale with
Apache Mesos
Distributed Systems Engineer
Apache Mesos PMC
Timothy Chen
@tnachen
tim@mesosphere.io

2. Mesoswhat? Marathon
Chronos
Demo!

3. Status quo is static partitioning
and use of virtual machines

4. Add some virtual machines

5. Provision Hadoop

6. Provision a web service

7. Moar data, moar Hadoop

8. Mesos let us treat a cluster of
nodes...

9. As one big computer

10. Not as individual machines
Not as VMs

11. But as computational resources
like cores, memory, disks, etc.

12. 240 million monthly active
users
500 million tweets per day
Up to 150k tweets per second
More than 100TB per day of
compressed data

13. “Mesos is the cornerstone of our elastic
compute infrastructure — it's how we build all
our new services and is critical for Twitter's
continued success at scale. It's one of the
primary keys to our data center efficiency.” —
Chris Fry, SVP of Engineering at Twitter

18.
Containerization in Mesos, a brief history

19. Mesos is...
 A top-level Apache project
 A cluster resource negotiator
 Scalable to 10,000s of nodes
 Fault-tolerant, battle-tested
 An SDK for distributed apps

21. Mesos for all the things

22. Marathon Mesoswhat?
Chronos
Demo!

23. a self-serve interface to your cluster

24. distributed "init" for long-running services

25. a private fault-tolerant PaaS

26. Marathon does it!
Start, stop, scale, update apps
Nice web interface, API
Highly available, no SPoF
Native Docker support
Fully featured REST API
Pluggable event bus
Rolling deploy / restart
Application health checks
Artifact staging

27. Chronos
Mesoswhat?
Marathon
Demo!

28. Introducing Chronos
a scheduler for batch and one-off jobs

29. Distribute a graph of jobs
Dependency graph for execution

30. Features
Distributed job scheduler
Web interface, API
Highly available, no SPoF
Native Docker support
Easy scheduling with repeating intervals

31. Chronos  REST
PUTchronos-node:8080/scheduler/job/job1
GETchronos-node:8080/scheduler/jobs
DELETEchronos-node:8080/scheduler/task/kill/job2

32.
Demo!
Mesoswhat?
Marathon
Chronos

33. Thanks!
Come and talk to me
P.S., we're hiring!