Apache Mesos provides a platform for building distributed systems. Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with API’s for resource management and scheduling across entire datacenter and cloud environments. How to use that platform and what to make of it becomes a complex task requiring not only understanding of where the system has been but also where it is going. Using schedulers like Marathon and Aurora help to get your applications scheduled and executing on Mesos. In many cases it makes sense to build a framework and integrate directly. This talk will breakdown what is involved in building a framework in Go, how to-do this with examples and why you would want to-do this. Frameworks are not only for generally available software applications (like Kafka, HDFS, Spark ,etc) but should also be used for custom internal R&D built software applications too.

1. Get started with Developing
Frameworks in Go on Apache Mesos

2. Joe Stein
CEO, Co-Founder of Elodina Inc, Software Platform focused on simplified
operations of data oriented systems by providing the tools and support
necessary to fully utilize their data center resources.
LinkedIn: http://linkedin.com/in/charmalloc
Twitter : @allthingshadoop

3. ◉ What is Mesos?
◉ Framework = (Scheduler + Executor)
◉ Framework API Examples
◉ More Mesos!
Overview

4. What is Mesos?

5. Origins
◉ A Common Substrate for Cluster Computing https://people.csail.mit.
edu/matei/papers/2009/hotcloud_nexus.pdf
◉ Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center
http://static.usenix.org/event/nsdi11/tech/full_papers/Hindman_new.pdf
◉ Google Borg - https://research.google.com/pubs/pub43438.html
◉ Google Omega: flexible, scalable schedulers for large compute clusters http:
//eurosys2013.tudos.org/wp-content/uploads/2013/paper/Schwarzkopf.pdf

10. static vs elastic

11. Data Center Kernel

12. Data Center Operating System
Mesosphere’s Data Center Operating System (DCOS) is an operating
system that spans all of the machines in a datacenter or cloud and treats
them as a single computer, providing a highly elastic and highly scalable
way of deploying applications, services, and big data infrastructure on
shared resources. DCOS is based on Apache Mesos and includes a
distributed systems kernel with enterprise-grade security. It also includes
a set of core system services, such as a native Marathon instance to
manage processes and installable services, and Mesos-DNS for service
discovery. DCOS provides a web interface and a command-line interface
(CLI) to manage the deployment and scale of applications.

16. Framework = (Scheduler + Executor)

18. Scheduler

19. Executors

20. Marathon
https://github.com/mesosphere/marathon
Cluster-wide init and control system for
services in cgroups or docker based on
Apache Mesos

21. Stack Deploy https://github.com/elodina/stack-deploy

22. Use YAML to Configure Your Deploy

23. Framework API & Examples

24. Mesos Protos
https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto
Everything is good to understand but here is a good place to start
◉ FrameworkInfo
◉ TaskInfo
◉ TaskState
◉ MasterInfo
◉ SlaveInfo

25. Mesos Framework Development Guide
http://mesos.apache.org/documentation/latest/app-framework-development-guide/
Go Bindings https://github.com/mesos/mesos-go
◉ Scheduler API
○ registered, reregistered, disconnected
○ resourceOffers, offerRescinded, statusUpdate, frameworkMessage
○ slaveLost, executorLost, error
◉ Executor API
○ registered, reregistered, disconnected
○ launchTask, killTask, frameworkMessage

26. Go Kafka Client Mesos
https://github.com/elodina/go-kafka-client-mesos

27. More About the Scheduler

28. Scheduler

29. Scheduler

30. Scheduler

31. Task Reconciliation
http://mesos.apache.org/documentation/latest/reconciliation/
It is the responsibility of Mesos (scheduler driver / Master) to
ensure that the framework is notified when a disconnection, and
subsequent (re-)registration occurs. At this point, the scheduler
should perform task state reconciliation.

32. Executor

33. Executor

35. More Schedulers to Check out
◉ https://github.com/elodina/statsd-mesos-kafka
◉ https://github.com/elodina/syslog-service
◉ https://github.com/mesos/kafka
◉ https://github.com/elodina/exhibitor-mesos-framework
◉ https://github.com/elodina/go-kafka-client-mesos
◉ https://github.com/elodina/datastax-enterprise-mesos
◉ https://github.com/elodina/zipkin-mesos-framework

36. More Mesos!

37. The Mesos system has two basic methods to describe the slaves
that comprise a cluster. One of these is managed by the Mesos
master, the other is simply passed onwards to the frameworks
using the cluster.
Attributes
The attributes are simply key value string pairs that Mesos passes along when it sends offers to frameworks.
Resources & Attributes

38. Resources
The Mesos system can manage 3 different types of resources: scalars, ranges, and sets. These are used to represent the
different resources that a Mesos slave has to offer. For example, a scalar resource type could be used to represent the
amount of memory on a slave. Each resource is identified by a key string.

39. Predefined Uses & Conventions
The Mesos master has a few resources that it pre-defines in how it handles them. At the current time, this list consist of:
●
●
●
●
In particular, a slave without and resources will never have its resources advertised to any frameworks. Also, the Master’s
user interface interprets the scalars in and in terms of . IE: the value is displayed as .

40. Examples
Here are some examples for configuring the Mesos slaves.
In this case, we have three different types of resources, scalars, a range, and a set. They are called , , , and the range type
is .
● scalar called , with the value
● scalar called , with the value
● scalar called , with the value
● range called , with values through (inclusive)
● set called , with the values , and
In the case of attributes, we end up with three attributes:
● with value
● with value
● with value

41. Roles
Total consumable resources per slave, in the form 'name(role):value;name(role):value...'. This value can be set to limit
resources per role, or to overstate the number of resources that are available to the slave.
All * roles will be detected, so you can specify only the resources that are not all roles (*).

42. Constraints
Constraints control where apps run to allow optimizing for fault tolerance or locality. Constraints are made up of three parts: a field
name, an operator, and an optional parameter. The field can be the slave hostname or any Mesos slave attribute.
Fields
Hostname field
field matches the slave hostnames, see for usage example.
field supports all operators of Marathon.
Attribute field
If the field name is none of the above, it will be treated as a Mesos slave attribute. Mesos slave attribute is a way to tag a slave node,
see to learn how to set the attributes.

43. Unique
tells Marathon to enforce uniqueness of the attribute across all of an app's tasks. For example the following constraint ensures
that there is only one app task running on each host:
via the Marathon gem:
via curl:

44. allows you to run all of your app's tasks on slaves that share a certain attribute.
You can also use this attribute to tie an application to a specific node by using the hostname property:
Cluster

45. Group By
can be used to distribute tasks evenly across racks or datacenters for high availability.
via the Marathon gem:
via curl:
Optionally, you can specify a minimum number of groups to try and achieve.

46. Like
accepts a regular expression as parameter, and allows you to run your tasks only on the slaves whose field values match the
regular expression.
via the Marathon gem:
via curl:

47. Unlike
Just like operator, but only run tasks on slaves whose field values don't match the regular expression.
via the Marathon gem:
via curl:

48. Go Mesos Constraints & More!
https://github.com/elodina/go-mesos-utils
--constraints: Constraints (hostname=like:^master$,rack=like:^1.*$).

50. Questions?
Joe Stein
http://www.elodina.net