The document provides an overview and comparison of several container orchestration platforms: Docker Swarm, Kubernetes, and Mesos/Marathon. It characterizes each based on their origins, support levels, scheduling approaches, modularity, updating processes, networking implementations, and abilities to scale and maintain high availability. While each has strengths for certain use cases, no single orchestrator is argued to be universally superior.
Dataservices - Processing Big Data The Microservice WayJosef Adersberger
We see a big data processing pattern emerging using the Microservice approach to build an integrated, flexible, and distributed system of data processing tasks. We call this the Dataservice pattern. In this presentation we'll introduce into Dataservices: their basic concepts, the technology typically in use (like Kubernetes, Kafka, Cassandra and Spring) and some architectures from real-life.
Introduction to containers, k8s, Microservices & Cloud NativeTerry Wang
Slides built to upskill and enable internal team and/or partners on foundational infra skills to work in a containerized world.
Topics covered
- Container / Containerization
- Docker
- k8s / container orchestration
- Microservices
- Service Mesh / Serverless
- Cloud Native (apps & infra)
- Relationship between Kubernetes and Runtime Fabric
Audiences: MuleSoft internal technical team, partners, Runtime Fabric users.
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...Matteo Ferroni
Internet of Things (IoT) is experiencing a huge hype these days, thanks to the increasing capabilities of embedded devices that enable their adoption in new fields of application (e.g. Wireless Sensor Networks, Connected Cars, Health Care, etc.). On the one hand, this is leading to an increasing adoption of multi-tenancy solutions for Cloud and Fog Computing, to analyze and store the data produced. On the other hand, power consumption has become a major concern for almost every digital system, from the smallest embedded circuits to the biggest computer clusters, with all the shades in between. Fine-grain control mechanisms are then needed to cap power consumption at each level of the stack, still guaranteeing Service Level Agreements (SLA) to the hosted applications. In this work, we propose DockerCap, a software-level power capping orchestrator for Docker containers that follows an Observe-Decide-Act loop structure: this allows to quickly react to changes that impact on the power consumption by managing resources of each container at run-time, to ensure the desired power cap. We show how we are able to obtain results comparable with the state of the art power capping solution provided by Intel RAPL, still being able to tune the performances of the containers and even guarantee SLA constraints.
Full paper: http://ieeexplore.ieee.org/document/7982228/
Dataservices - Processing Big Data The Microservice WayJosef Adersberger
We see a big data processing pattern emerging using the Microservice approach to build an integrated, flexible, and distributed system of data processing tasks. We call this the Dataservice pattern. In this presentation we'll introduce into Dataservices: their basic concepts, the technology typically in use (like Kubernetes, Kafka, Cassandra and Spring) and some architectures from real-life.
Introduction to containers, k8s, Microservices & Cloud NativeTerry Wang
Slides built to upskill and enable internal team and/or partners on foundational infra skills to work in a containerized world.
Topics covered
- Container / Containerization
- Docker
- k8s / container orchestration
- Microservices
- Service Mesh / Serverless
- Cloud Native (apps & infra)
- Relationship between Kubernetes and Runtime Fabric
Audiences: MuleSoft internal technical team, partners, Runtime Fabric users.
[EUC2016] DockerCap: a software-level power capping orchestrator for Docker c...Matteo Ferroni
Internet of Things (IoT) is experiencing a huge hype these days, thanks to the increasing capabilities of embedded devices that enable their adoption in new fields of application (e.g. Wireless Sensor Networks, Connected Cars, Health Care, etc.). On the one hand, this is leading to an increasing adoption of multi-tenancy solutions for Cloud and Fog Computing, to analyze and store the data produced. On the other hand, power consumption has become a major concern for almost every digital system, from the smallest embedded circuits to the biggest computer clusters, with all the shades in between. Fine-grain control mechanisms are then needed to cap power consumption at each level of the stack, still guaranteeing Service Level Agreements (SLA) to the hosted applications. In this work, we propose DockerCap, a software-level power capping orchestrator for Docker containers that follows an Observe-Decide-Act loop structure: this allows to quickly react to changes that impact on the power consumption by managing resources of each container at run-time, to ensure the desired power cap. We show how we are able to obtain results comparable with the state of the art power capping solution provided by Intel RAPL, still being able to tune the performances of the containers and even guarantee SLA constraints.
Full paper: http://ieeexplore.ieee.org/document/7982228/
Design and Implementation of Incremental Cooperative Rebalancingconfluent
Watch this talk here: https://www.confluent.io/online-talks/design-and-implementation-of-incremental-cooperative-rebalancing-on-demand
Since its initial release, the Kafka group membership protocol has offered Connect, Streams and Consumer applications an ingenious and robust way to balance resources among distributed processes. The process of rebalancing, as it’s widely known, allows Kafka APIs to define an embedded protocol for load balancing within the group membership protocol itself.
Until now, rebalancing has been working under the simple assumption that every time a new group generation is created, the members join after first releasing all of their resources, getting a whole new load assignment by the time the new group is formed. This allows Kafka APIs to provide task fault-tolerance and elasticity on top of the group membership protocol.
However, due to its side-effects on multi-tenancy and scalability this simple approach in rebalancing, also known as stop-the-world effect, is limiting larger scale deployments. Because of stop-the-world, application tasks get interrupted only for most of them to receive the same resources after rebalancing. In this technical deep dive, we’ll discuss the proposition of Incremental Cooperative Rebalancing as a way to alleviate stop-the-world and optimize rebalancing in Kafka APIs.
This talk will cover:
-The internals of Incremental Cooperative Rebalancing
-Uses cases that benefit from Incremental Cooperative Rebalancing
-Implementation in Kafka Connect
-Performance results in Kafka Connect clusters
Diego: Re-envisioning the Elastic Runtime (Cloud Foundry Summit 2014)VMware Tanzu
Keynote delivered by Onsi Fakhouri, Engineering Manager at Pivotal.
Diego is a ground-up rewrite of the DEA - a major component of the Cloud Foundry Elastic Runtime. This talk will motivate the need for Diego, the philosophy behind Diego, and present a few choice technical details to illustrate some of the more interesting ideas we've been playing with.
This session endeavors to explain high-speed reactive microservice architecture, a set of patterns for building services that can readily back mobile and web applications at scale. It uses a scale-up and -out versus a scale-out model to do more with less hardware. A scale-up model uses in-memory operational data, efficient queue handoff, and microbatch streaming, plus async calls to handle more calls on a single node. High-speed microservice architecture endeavors to get back to OOP roots, where data and logic live together in a cohesive, understandable representation of the problem domain, and away from separation of data and logic, because data lives with the service logic that operates on it.
Making Cassandra more capable, faster, and more reliable (at ApacheCon@Home 2...Scalar, Inc.
Cassandra is widely adopted in real-world applications and used by large and sometimes mission-critical applications because of its high performance, high availability and high scalability. However, there is still some room for improvement to take Cassandra to the next level. We have been contributing to Cassandra to make it more capable, faster, and more reliable by, for example, proposing a non-invasive ACID transaction library, adding GroupCommitLogService, and maintaining and conducting Jepsen testing for lightweight transactions. This talk will present the contributions we have done including the latest updates in more detail, and the reasons why we made such contributions. This talk will be one of the good starting points for discussing the next generation Cassandra.
DevoxxUK: Optimizating Application Performance on KubernetesDinakar Guniguntala
Now that you have your apps running on K8s, wondering how to get the response time that you need ? Tuning a polyglot set of microservices to get the performance that you need can be challenging in Kubernetes. The key to overcoming this is observability. Luckily there are a number of tools such as Prometheus that can provide all the metrics you need, but here is the catch, there is so much of data and metrics that is difficult make sense of it all. This is where Hyperparameter tuning can come to the rescue to help build the right models.
This talk covers best practices that will help attendees
1. To understand and avoid common performance related problems.
2. Discuss observability tools and how they can help identify perf issues.
3. Look closer into Kruize Autotune which is a Open Source Autonomous Performance Tuning Tool for Kubernetes and where it can help.
Help, my Kafka is broken! (Emma Humber, IBM) Kafka Summit SF 2019confluent
Abstract Summary: While Apache Kafka is designed to be fault-tolerant, there will be times when your Kafka environment just isn't working as expected. Whether it's a newly configured application not processing messages, or an outage in a high-load, mission-critical production environment, it's crucial to get up and running as quickly and safely as possible. IBM has hosted production Kafka environments for several years and has in-depth knowledge of how to diagnose and resolve problems rapidly and accurately to ensure minimal impact to end users. This session will discuss our experiences of how to most effectively collect and understand Kafka diagnostics. We'll talk through using these diagnostics to work out what's gone wrong, and how to recover from a system outage. Using this new-found knowledge, you will be equipped to handle any problem your cluster throws at you.
High-Speed Reactive Microservices - trials and tribulationsRick Hightower
Covers how we built a set of high-speed reactive microservices and maximized cloud/hardware costs while meeting objectives in resilience and scalability. This has more notes attached as it is based on the ppt not the PDF.
Bootstrapping Microservices with Kafka, Akka and SparkAlex Silva
Compared to a traditional analysis-centric data hub, today's data platforms need to fulfill many different use cases. The need for real-time, transport-agnostic data protocols have become a crucial feature shared across many different use cases.
During this talk, we will discuss our approach to bootstrapping and bounded context subscription, leveraging a mix of open source technologies and home-grown services aimed at providing a full end-to-end solution.
We will demonstrate and discuss our use of Kafka, Spark Streaming and Akka to orchestrate a unified data transfer protocol that frees developers from having to listen to and process events within their bounded contexts. More specifically:
- Leveraging Kafka as the source of truth
- Topic serialization formats, Avro, and retention rules
- Using Kafka's distributed commit logs to produce durable datasets
- Log compaction and its role in service bootstrapping
- Ingestion at scale: consuming data in different formats across different teams at different latencies
- Using Spark Streaming and Akka to perform near real-time replication protocols
Christian Kniep presented this deck at the 2016 HPC Advisory Council Switzerland Conference.
"With Docker v1.9 a new networking system was introduced, which allows multi-host network- ing to work out-of-the-box in any Docker environment. This talk provides an introduction on what Docker networking provides, followed by a demo that spins up a full SLURM cluster across multiple machines. The demo is based on QNIBTerminal, a Consul backed set of Docker Images to spin up a broad set of software stacks."
Watch the video presentation:
http://wp.me/p3RLHQ-f7G
See more talks in the Swiss Conference Video Gallery:
http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter:
http://insidehpc.com/newsletter
Designing a reactive data platform: Challenges, patterns, and anti-patterns Alex Silva
Presentation given at the O'Reilly Software Architecture Conference in NYC, April 2016.
Covers the key architectural decisions made behind the design of a reactive self-service data ingestion analytics platform that is able to fulfill several business use cases at massive scale, both at real-time and batch scopes, while leveraging and integrating Kafka and Spark in an efficient, easy to use way.
The presentation describes a message-driven, reactive and distributed design that leverages REST and Hypermedia protocols, and several open source frameworks and platforms, including Akka, Kafka, Hadoop and Spark.
Cloud Native Night, April 2018, Mainz: Workshop led by Jörg Schad (@joerg_schad, Technical Community Lead / Developer at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night/
PLEASE NOTE:
During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
DETAILS ABOUT THE WORKSHOP:
Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
Reactive Java: Promises and Streams with Reakt (JavaOne talk 2016)Rick Hightower
see labs at https://github.com/advantageous/j1-talks-2016
Import based on PDF. This is from our JavaOne Talk 2016 on Reakt, reactive Java programming with promises, circuit breakers, and streams. Reakt is a reactive Java lib that provides promises, streams, and a reactor to handle asynchronous call coordination. It was influenced by the design of promises in ES6. You want to async-call serviceA and then serviceB, take the results of serviceA and serviceB, and then call serviceC. Then, based on the results of call C, call D or E and then return the results to the original caller. Calls to A, B, C, D, and E are all async calls, and none should take longer than 10 seconds. If they do, then return a timeout to the original caller. The whole async call sequence should time out in 20 seconds if it does not complete and should also check for circuit breakers and provide back pressure feedback so the system does not have cascading failures. Learn more in this session.
OpenStack Collaboration made in heaven with Heat, Mistral, Neutron and more..Trinath Somanchi
Cross-project collaboration is something OpenStack community has embraced for a long time. Common libraries like Oslo reduces the time and effort to build a new service. Another way this manifests is in new OpenStack services getting built using existing services to solve an higher level use-case.
In this talk we are present how the band of projects comprising of Mistral, Tacker, Neutron, Heat, TOSCA-parser and Barbican came together to build an industry leading ETSI NFV Orchestrator that leveraged the best of these projects. Each of these projects brought in critical functionalities needed towards the final product. You will learn how, when strung together, this solution follows the classic Microservices design pattern that the industry is rapidly adopting.
Design and Implementation of Incremental Cooperative Rebalancingconfluent
Watch this talk here: https://www.confluent.io/online-talks/design-and-implementation-of-incremental-cooperative-rebalancing-on-demand
Since its initial release, the Kafka group membership protocol has offered Connect, Streams and Consumer applications an ingenious and robust way to balance resources among distributed processes. The process of rebalancing, as it’s widely known, allows Kafka APIs to define an embedded protocol for load balancing within the group membership protocol itself.
Until now, rebalancing has been working under the simple assumption that every time a new group generation is created, the members join after first releasing all of their resources, getting a whole new load assignment by the time the new group is formed. This allows Kafka APIs to provide task fault-tolerance and elasticity on top of the group membership protocol.
However, due to its side-effects on multi-tenancy and scalability this simple approach in rebalancing, also known as stop-the-world effect, is limiting larger scale deployments. Because of stop-the-world, application tasks get interrupted only for most of them to receive the same resources after rebalancing. In this technical deep dive, we’ll discuss the proposition of Incremental Cooperative Rebalancing as a way to alleviate stop-the-world and optimize rebalancing in Kafka APIs.
This talk will cover:
-The internals of Incremental Cooperative Rebalancing
-Uses cases that benefit from Incremental Cooperative Rebalancing
-Implementation in Kafka Connect
-Performance results in Kafka Connect clusters
Diego: Re-envisioning the Elastic Runtime (Cloud Foundry Summit 2014)VMware Tanzu
Keynote delivered by Onsi Fakhouri, Engineering Manager at Pivotal.
Diego is a ground-up rewrite of the DEA - a major component of the Cloud Foundry Elastic Runtime. This talk will motivate the need for Diego, the philosophy behind Diego, and present a few choice technical details to illustrate some of the more interesting ideas we've been playing with.
This session endeavors to explain high-speed reactive microservice architecture, a set of patterns for building services that can readily back mobile and web applications at scale. It uses a scale-up and -out versus a scale-out model to do more with less hardware. A scale-up model uses in-memory operational data, efficient queue handoff, and microbatch streaming, plus async calls to handle more calls on a single node. High-speed microservice architecture endeavors to get back to OOP roots, where data and logic live together in a cohesive, understandable representation of the problem domain, and away from separation of data and logic, because data lives with the service logic that operates on it.
Making Cassandra more capable, faster, and more reliable (at ApacheCon@Home 2...Scalar, Inc.
Cassandra is widely adopted in real-world applications and used by large and sometimes mission-critical applications because of its high performance, high availability and high scalability. However, there is still some room for improvement to take Cassandra to the next level. We have been contributing to Cassandra to make it more capable, faster, and more reliable by, for example, proposing a non-invasive ACID transaction library, adding GroupCommitLogService, and maintaining and conducting Jepsen testing for lightweight transactions. This talk will present the contributions we have done including the latest updates in more detail, and the reasons why we made such contributions. This talk will be one of the good starting points for discussing the next generation Cassandra.
DevoxxUK: Optimizating Application Performance on KubernetesDinakar Guniguntala
Now that you have your apps running on K8s, wondering how to get the response time that you need ? Tuning a polyglot set of microservices to get the performance that you need can be challenging in Kubernetes. The key to overcoming this is observability. Luckily there are a number of tools such as Prometheus that can provide all the metrics you need, but here is the catch, there is so much of data and metrics that is difficult make sense of it all. This is where Hyperparameter tuning can come to the rescue to help build the right models.
This talk covers best practices that will help attendees
1. To understand and avoid common performance related problems.
2. Discuss observability tools and how they can help identify perf issues.
3. Look closer into Kruize Autotune which is a Open Source Autonomous Performance Tuning Tool for Kubernetes and where it can help.
Help, my Kafka is broken! (Emma Humber, IBM) Kafka Summit SF 2019confluent
Abstract Summary: While Apache Kafka is designed to be fault-tolerant, there will be times when your Kafka environment just isn't working as expected. Whether it's a newly configured application not processing messages, or an outage in a high-load, mission-critical production environment, it's crucial to get up and running as quickly and safely as possible. IBM has hosted production Kafka environments for several years and has in-depth knowledge of how to diagnose and resolve problems rapidly and accurately to ensure minimal impact to end users. This session will discuss our experiences of how to most effectively collect and understand Kafka diagnostics. We'll talk through using these diagnostics to work out what's gone wrong, and how to recover from a system outage. Using this new-found knowledge, you will be equipped to handle any problem your cluster throws at you.
High-Speed Reactive Microservices - trials and tribulationsRick Hightower
Covers how we built a set of high-speed reactive microservices and maximized cloud/hardware costs while meeting objectives in resilience and scalability. This has more notes attached as it is based on the ppt not the PDF.
Bootstrapping Microservices with Kafka, Akka and SparkAlex Silva
Compared to a traditional analysis-centric data hub, today's data platforms need to fulfill many different use cases. The need for real-time, transport-agnostic data protocols have become a crucial feature shared across many different use cases.
During this talk, we will discuss our approach to bootstrapping and bounded context subscription, leveraging a mix of open source technologies and home-grown services aimed at providing a full end-to-end solution.
We will demonstrate and discuss our use of Kafka, Spark Streaming and Akka to orchestrate a unified data transfer protocol that frees developers from having to listen to and process events within their bounded contexts. More specifically:
- Leveraging Kafka as the source of truth
- Topic serialization formats, Avro, and retention rules
- Using Kafka's distributed commit logs to produce durable datasets
- Log compaction and its role in service bootstrapping
- Ingestion at scale: consuming data in different formats across different teams at different latencies
- Using Spark Streaming and Akka to perform near real-time replication protocols
Christian Kniep presented this deck at the 2016 HPC Advisory Council Switzerland Conference.
"With Docker v1.9 a new networking system was introduced, which allows multi-host network- ing to work out-of-the-box in any Docker environment. This talk provides an introduction on what Docker networking provides, followed by a demo that spins up a full SLURM cluster across multiple machines. The demo is based on QNIBTerminal, a Consul backed set of Docker Images to spin up a broad set of software stacks."
Watch the video presentation:
http://wp.me/p3RLHQ-f7G
See more talks in the Swiss Conference Video Gallery:
http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter:
http://insidehpc.com/newsletter
Designing a reactive data platform: Challenges, patterns, and anti-patterns Alex Silva
Presentation given at the O'Reilly Software Architecture Conference in NYC, April 2016.
Covers the key architectural decisions made behind the design of a reactive self-service data ingestion analytics platform that is able to fulfill several business use cases at massive scale, both at real-time and batch scopes, while leveraging and integrating Kafka and Spark in an efficient, easy to use way.
The presentation describes a message-driven, reactive and distributed design that leverages REST and Hypermedia protocols, and several open source frameworks and platforms, including Akka, Kafka, Hadoop and Spark.
Cloud Native Night, April 2018, Mainz: Workshop led by Jörg Schad (@joerg_schad, Technical Community Lead / Developer at Mesosphere)
Join our Meetup: https://www.meetup.com/de-DE/Cloud-Native-Night/
PLEASE NOTE:
During this workshop, Jörg showed many demos and the audience could participate on their laptops. Unfortunately, we can't provide these demos. Nevertheless, Jörg's slides give a deep dive into the topic.
DETAILS ABOUT THE WORKSHOP:
Kubernetes has been one of the topics in 2017 and will probably remain so in 2018. In this hands-on technical workshop you will learn how best to deploy, operate and scale Kubernetes clusters from one to hundreds of nodes using DC/OS. You will learn how to integrate and run Kubernetes alongside traditional applications and fast data services of your choice (e.g. Apache Cassandra, Apache Kafka, Apache Spark, TensorFlow and more) on any infrastructure.
This workshop best suits operators focussed on keeping their apps and services up and running in production and developers focussed on quickly delivering internal and customer facing apps into production.
You will learn how to:
- Introduction to Kubernetes and DC/OS (including the differences between both)
- Deploy Kubernetes on DC/OS in a secure, highly available, and fault-tolerant manner
- Solve operational challenges of running a large/multiple Kubernetes cluster
- One-click deploy big data stateful and stateless services alongside a Kubernetes cluster
Reactive Java: Promises and Streams with Reakt (JavaOne talk 2016)Rick Hightower
see labs at https://github.com/advantageous/j1-talks-2016
Import based on PDF. This is from our JavaOne Talk 2016 on Reakt, reactive Java programming with promises, circuit breakers, and streams. Reakt is a reactive Java lib that provides promises, streams, and a reactor to handle asynchronous call coordination. It was influenced by the design of promises in ES6. You want to async-call serviceA and then serviceB, take the results of serviceA and serviceB, and then call serviceC. Then, based on the results of call C, call D or E and then return the results to the original caller. Calls to A, B, C, D, and E are all async calls, and none should take longer than 10 seconds. If they do, then return a timeout to the original caller. The whole async call sequence should time out in 20 seconds if it does not complete and should also check for circuit breakers and provide back pressure feedback so the system does not have cascading failures. Learn more in this session.
OpenStack Collaboration made in heaven with Heat, Mistral, Neutron and more..Trinath Somanchi
Cross-project collaboration is something OpenStack community has embraced for a long time. Common libraries like Oslo reduces the time and effort to build a new service. Another way this manifests is in new OpenStack services getting built using existing services to solve an higher level use-case.
In this talk we are present how the band of projects comprising of Mistral, Tacker, Neutron, Heat, TOSCA-parser and Barbican came together to build an industry leading ETSI NFV Orchestrator that leveraged the best of these projects. Each of these projects brought in critical functionalities needed towards the final product. You will learn how, when strung together, this solution follows the classic Microservices design pattern that the industry is rapidly adopting.
Style-aware robust color transfer, H. Hristova, O. Le Meur, R. Cozot and K. Bouatouch, Computational Aesthetic, 2015.
Transferring features, such as light and colors, between input and reference images is the main objective of color transfer methods. Current state-of-the-art methods focus mainly on the complete transfer of the light and color distributions. However, they do not successfully grasp specific light and color variations in image styles. In this paper, we propose a local method for carrying out a transfer of style between two images. Our method partitions both images to Gaussian distributed clusters by considering their main style features. These features are automatically determined by the classification step of our algorithm. Moreover, we present several novel policies for input/reference cluster mapping, which have not been tackled so far by previous methods. To complete the style transfer, for each pair of corresponding clusters, we apply a parametric color transfer method and a local chromatic adaptation transform. Results, subjective user evaluation as well as objective evaluation show that the proposed method obtains visually pleasing and artifact-free images, respecting the reference style.
Neil Dhillon has thirty years of experience working in the political sphere, with expertise in government relations, strategic communications, and advocacy in our nation’s capital. He has held many prestigious positions in politics including senior aide to President Bill Clinton and Chief of Staff for Congressman Bob Matsui.
Prosthodontics is one of nine dental specialties recognized by the American Dental Association (ADA). Dentists in the field of prosthodontics (prosthodontists) offer the most advanced form of sequencing treatment, restorative treatment and maintenance.
Characterizing and Contrasting Kuhn-tey-ner Awr-kuh-streyt-orsSonatype
Lee Calcote, Solar Winds
Running a few containers? No problem. Running hundreds or thousands? Enter the container orchestrator. Let’s take a look at the characteristics of the four most popular container orchestrators and what makes them alike, yet unique.
Swarm
Nomad
Kubernetes
Mesos+Marathon
We’ll take a structured looked at these container orchestrators, contrasting them across these categories:
Genesis & Purpose
Support & Momentum
Host & Service Discovery
Scheduling
Modularity & Extensibility
Updates & Maintenance
Health Monitoring
Networking & Load-Balancing
High Availability & Scale
OpenStack and Kubernetes - A match made for Telco HeavenTrinath Somanchi
With the advent of Containerization of Telco Clouds for NFV and SDN based deployments, OpenStack with Kubernetes is a best chosen option to solve the challenges is a better way to build a containerized Telco cloud. This involves, "Kubernetes in OpenStack", "OpenStack in Kubernetes" and "Independent OpenStack and Kubernetes". With this complementing collaboration, in the Stadium of OpenStack's Open Infrastructure, Telecom gaints are developing cloud-native solutions to best fit the next generation networking deployments. In this Presentation, we talk about Containerization and benefits, OpenStack and Kubernetes match making and we give a brief overview on Airship and Kata Container projects.
Build cloud native solution using open source Nitesh Jadhav
Build cloud native solution using open source. I have tried to give a high level overview on How to build Cloud Native using CNCF graduated software's which are tested, proven and having many reference case studies and partner support for deployment
Container orchestration engine for automating deployment, scaling, and management of containerized applications.
What are Microservices?
What is container?
What is Containerization?
What is Docker?
This presentation covers how app deployment model evolved from bare metal servers to Kubernetes World.
In addition to theoretical information, you will find free KATACODA workshops url to perform practices to understand the details of the each topics.
An in depth overview of Kubernetes and it's various components.
NOTE: This is a fixed version of a previous presentation (a draft was uploaded with some errors)
Recent momentum around the evolution of Containers are gradually increase in last two years.Containers virtualize an OS and applications running in each container believe that they have full access to their very own copy of that OS. This is analogous to what VMs do when they virtualize at a lower level, the hardware. In the case of containers, it’s the OS that does the virtualization and maintains the illusion.
Recent past many software companies have quickly adopted container technologies, including Docker Containers, aware of the threat and advantage of the approach. For example, Linux companies have also jumped into the ground, seeing as this as an opportunity to grow the Linux market. Also Microsoft is going to add features to support containers and VMware have made efforts in integrating support for Docker into virtual machine technology.
Similar to Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors (20)
Benchmarking Service Meshes - CNCF Networking WGLee Calcote
Presented at the CNCF Networking Working Group in March 2019. A project to provide apples-to-apples comparison of performance overhead induced by different service meshes. Recording - https://www.youtube.com/watch?v=2_JwCc-kLMA
Istio: Using nginMesh as the service proxyLee Calcote
With microservices and containers becoming mainstream, container orchestrators provide much of what the cluster (nodes and containers) needs. With container orchestrators' core focus on scheduling, discovery, and health at an infrastructure level, microservices are left with unmet, service-level needs, such as:
- Traffic management, routing, and resilient and secure communication between services
- Policy enforcement, rate-limiting, circuit breaking
- Visibility and monitoring with metrics, logs, and traces
- Load balancing and rollout/canary deployment support
Service meshes provide for these needs. In this session, we will dive into Istio - its components, capabilities, and extensibility. Istio envelops and integrates with other open source projects to deliver a full-service mesh. We'll explore these integrations and Istio's extensibility in terms of choice of proxies and adapters, such as nginMesh.
CNCF, State of Serverless & Project NuclioLee Calcote
The Serverless working group within the Cloud Native Computing Foundation (CNCF) is one of many. In this talk, we’ll answer why the working group exists and how our efforts help the ecosystem. We'll also take a look at some of the current Serverless and FaaS projects and cover some of the common Serverless myths. Finally, we'll look ahead toward what we foresee as some of Serverless's biggest challenges and best-suited use cases.
Load Balancing in the Cloud using Nginx & KubernetesLee Calcote
Presented on March 16, 2017 through O'Reilly - http://www.oreilly.com/pub/e/3864
Modern day applications bring modern day infrastructure requirements. Whether you bring your own or you use your cloud provider's managed load-balancing services, even moderately sophisticated applications are likely to find their needs underserved.
Overlay/Underlay - Betting on Container NetworkingLee Calcote
Presented at Rackspace Austin (downtown) on July 27th, 2016.
An inherent to component to any distributed application, networking is one of the most complicated and expansive infrastructure technologies. Container networking needs to be developer-friendly. Application-driven and portable. With developers busily adopting container technologies, the time has come for network engineers and operators to prepare for the unique challenges brought on by cloud native applications. What container networking specifications bring to the table and how to leverage them.
A brisk introduction to container runtimes (engines) and an understanding of when container orchestrators enter and what role they play. We’ll look at what makes them alike, yet unique. Presented at ContainerizeThis 2016.
Characterizing and Contrasting Container OrchestratorsLee Calcote
Presented at OpenStack Summit Austin 2016 - Container Day.
Running a few containers? No problem. Running hundreds or thousands? Enter the container orchestrator. Let’s take a look at the characteristics of the three most popular container orchestrators and what makes them alike, yet unique.
E-commerce Application Development Company.pdfHornet Dynamics
Your business can reach new heights with our assistance as we design solutions that are specifically appropriate for your goals and vision. Our eCommerce application solutions can digitally coordinate all retail operations processes to meet the demands of the marketplace while maintaining business continuity.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
AI Genie Review: World’s First Open AI WordPress Website CreatorGoogle
AI Genie Review: World’s First Open AI WordPress Website Creator
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-genie-review
AI Genie Review: Key Features
✅Creates Limitless Real-Time Unique Content, auto-publishing Posts, Pages & Images directly from Chat GPT & Open AI on WordPress in any Niche
✅First & Only Google Bard Approved Software That Publishes 100% Original, SEO Friendly Content using Open AI
✅Publish Automated Posts and Pages using AI Genie directly on Your website
✅50 DFY Websites Included Without Adding Any Images, Content Or Doing Anything Yourself
✅Integrated Chat GPT Bot gives Instant Answers on Your Website to Visitors
✅Just Enter the title, and your Content for Pages and Posts will be ready on your website
✅Automatically insert visually appealing images into posts based on keywords and titles.
✅Choose the temperature of the content and control its randomness.
✅Control the length of the content to be generated.
✅Never Worry About Paying Huge Money Monthly To Top Content Creation Platforms
✅100% Easy-to-Use, Newbie-Friendly Technology
✅30-Days Money-Back Guarantee
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
#AIGenieApp #AIGenieBonus #AIGenieBonuses #AIGenieDemo #AIGenieDownload #AIGenieLegit #AIGenieLiveDemo #AIGenieOTO #AIGeniePreview #AIGenieReview #AIGenieReviewandBonus #AIGenieScamorLegit #AIGenieSoftware #AIGenieUpgrades #AIGenieUpsells #HowDoesAlGenie #HowtoBuyAIGenie #HowtoMakeMoneywithAIGenie #MakeMoneyOnline #MakeMoneywithAIGenie
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeAftab Hussain
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Graspan: A Big Data System for Big Code AnalysisAftab Hussain
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
5. One size does not fit all.
A strict apples-to-apples comparison is inappropriate and not
the objective, hence characterizing and contrasting.
@lcalcote
6. Let's not go here today.
Container orchestrators may be intermixed.
@lcalcote
7. Categorically Speaking
Scheduling
Genesis & Purpose
Support & Momentum
Host & Service Discovery
Modularity & Extensibility
Updates & Maintenance
Health Monitoring
Networking & Load-Balancing
High Availability & Scale@lcalcote
11. Genesis & Purpose
Swarm is simple and easy to setup.
Swarm is responsible for the clustering and
scheduling aspects of orchestration.
Originally an imperative system, now declarative
Swarm’s architecture is not complex as those of
Kubernetes and Mesos
Written in Golang, Swarm is lightweight, modular and
extensible
@lcalcote
14. Support & Momentum
Contributions:
Standalone: ~3,000 commits, 12 core maintainers (140 contributors)
Swarmkit: ~2,000 commits, 12 core maintainers (40 contributors)
~250 Docker meetups worldwide
Production-ready:
Standalone announced 8 months ago (Nov 2015)
Swarmkit announced < 1 month ago (July 2016)
@lcalcote
15. Host & Service Discovery
Host Discovery
used in the formation of clusters by the Manager to
discover for Nodes (hosts).
Service Discovery
Embedded DNS and round robin load-balancing
Services are a new concept
image: iStock@lcalcote
16. Scheduling
Swarm’s scheduler is pluggable
Swarm scheduling is a combination of strategies and
filters/constraint:
Strategies
Random
Binpack
Spread*
Plugin?
Filters
container constraints (affinity, dependency, port) are defined as
environment variables in the specification file
node constraints (health, constraint) must be specified when
starting the docker daemon and define which nodes a container
may be scheduled on.
image: pickywallpapers
Swarm Mode only supports Spread
17. Modularity & Extensibility
Ability to remove batteries is a strength for Swarm:
Pluggable scheduler
Pluggable network driver
Pluggable distributed K/V store
Docker container engine runtime-only
Pluggable authorization (in docker engine)*
image: Alan Chia
@lcalcote
18. Updates & Maintenance
Nodes
Nodes may be Active, Drained and Paused
Manual swarm manager and worker updates
Applications
Rolling updates now supported
--update-delay
--update-parallelism
--update-failure-action
image: 123RF@lcalcote
19. Health Monitoring
Nodes
Swarm monitors the availability and resource usage
of nodes within the cluster
Applications
One health check per container may be run
check container health by running a command inside the container
--interval=DURATION (default: 30s)
--timeout=DURATION (default: 30s)
--retries=N (default: 3)
@lcalcote
20. Networking & Load-
Balancing
Swarm and Docker’s multi-host networking are simpatico
provides for user-defined overlay networks that are micro-segmentable
uses a gossip protocol for quick convergence of neighbor table
facilitates container name resolution via embedded DNS server (previously via etc/hosts)
You may bring your own network driver
Load-balancing based on IPVS
expose Service's port externally
L4 load-balancer; cluster-wide port publishing
Mesh routing
send a request to any one of the nodes and it will be routed automatically
send a request to any one of the nodes and it will be internally load balanced
21. High Availability & Scale
Managers may be deployed in a highly-available
configuration
Active/Standby - only one active Leader at-a-time
Maintain odd number of managers
Rescheduling upon node failure
No rebalancing upon node addition to the cluster
Does not support multiple failure isolation regions or
federation
although, with caveats, .
federation is possible
@lcalcote
22. Scaling swarm to 1,000 AWS nodes
and 50,000 containers
@lcalcote
23. Suitable for orchestrating a combination of infrastructure containers
Has only recently added capabilities falling into the application b
Swarm is a young project
advanced features forthcoming
natural expectation of caveats in functionality
No rebalancing, autoscaling or monitoring, yet
Only schedules Docker containers, not containers using other specificat
Does not schedule VMs or non-containerized processes
Need separate load-balancer for overlapping ingress ports
While dependency and affinity filters are available, Swarm does not pro
the ability to enforce scheduling of two containers onto the same host o
at all.
Filters facilitate sidecar pattern. No “pod” concept.
Swarm works. Swarm is simple and easy to
deploy.
1.12 eliminated the need for much third-party software
Facilitates earlier stages of adoption by organizations viewing
containers as faster VMs
now with built-in functionality for applications
Swarm is easy to extend, if can already know
Docker APIs, you can customize Swarm
Highly modular:
Pluggable scheduler
Pluggable K/V store for both node and multi-
host networking
25. Genesis & Purpose
an opinionated framework for building distributed
systems
or as its tagline states "an open source system for automating
deployment, scaling, and operations of applications."
Written in Golang, Kubernetes is lightweight, modular
and extensible
considered a third generation container orchestrator
led by Google, Red Hat and others.
bakes in load-balancing, scale, volumes, deployments, secret
management and cross-cluster federated services among other features.
Declaratively, opinionated with many key features
included
27. Support & Momentum
Kubernetes is young (about two years old)
Announced as production-ready 13 months ago (July 2015)
Project currently has over 1,000 commits per month
(~34,000 total)
made by about 100 (862 total) Kubernauts (Kubernetes enthusiasts)
~5,000 commits made in the latest release - 1.3.
Under the governance of the Cloud Native Computing
Foundation
Robust set of documentation and ~90 meetups
@lcalcote
28. Host & Service Discovery
Host Discovery
by default, the node agent (kubelet) is configured to register
itself with the master (API server)
automating the joining of new hosts to the cluster
Service Discovery
Two primary modes of finding a Service
DNS
SkyDNS is deployed as a cluster add-on
environment variables
environment variables are used as a simple way of providing compatibility with Docker links-style
networking image: iStock
29. Scheduling
By default, scheduling is handled by kube-scheduler.
Pluggable
Selection criteria used by kube-scheduler to identify the best-
fit node is defined by policy:
Predicates (node resources and characteristics):
PodFitPorts , PodFitsResources, NoDiskConflict
, MatchNodeSelector, HostName , ServiceAffinit, LabelsPresence
Priorities (weighted strategies used to identify “best fit” node):
LeastRequestedPriority, BalancedResourceAllocation, ServiceSpreadingPriority,
EqualPriority
@lcalcote
30. Modularity &
Extensibility
One of Kubernetes strengths its pluggable architecture
Choice of:
database for service discovery or network driver
container runtime
users may choose to run Docker with Rocket
containers
Cluster add-ons
optional system components that implement a cluster
feature (e.g. DNS, logging, etc.)
shipped with the Kubernetes binaries and are considered
an inherent part of the Kubernetes clusters
31. Updates & Maintenance
Applications
Deployment objects automate deploying and rolling
updating applications.
Support for rolling back deployments
Kubernetes Components
Upgrading the Kubernetes components and hosts is
done via shell script
Host maintenance - mark the node as unschedulable.
existing pods are not vacated from the node
prevents new pods from being scheduled on the node
image: 123RF@lcalcote
32. Health Monitoring
Nodes
Failures - actively monitors the health of nodes within the cluster
via Node Controller
Resources - usage monitoring leverages a combination of open
source components:
cAdvisor, Heapster, InfluxDB, Grafana
Applications
three types of user-defined application health-checks and uses the
Kubelet agent as the the health check monitor
HTTP Health Checks, Container Exec, TCP Socket
Cluster-level Logging
collect logs which persist beyond the lifetime of the pod’s container
images or the lifetime of the pod or even cluster
standard output and standard error output of each container can be ingested using a
agent running on each nodeFluentd
33. Networking & Load-
Balancing
…enter the Pod
atomic unit of scheduling
flat networking with each pod receiving an IP address
no NAT required, port conflicts localized
intra-pod communication via localhost
Load-Balancing
Services provide inherent load-balancing via kube-
proxy:
runs on each node of a Kubernetes cluster
reflects services as defined in the Kubernetes API
supports simple TCP/UDP forwarding and round-robin and Docker-links-
based service IP:PORT mapping.
34. High Availability & Scale
Each master component may be deployed in a highly-
available configuration.
Active/Standby configuration
In terms of scale, v1.2 brings support for 1,000 node
clusters and a step toward fully-federated clusters
(Ubernetes)
Application-level auto-scaling is supported within
Kubernetes via Replication Controllers
@lcalcote
35. Only runs containerized applications
For those familiar with Docker-only,
Kubernetes requires understanding of new
concepts
Powerful frameworks with more moving pieces beget
complicated cluster deployment and management.
Lightweight graphical user interface
Does not provide as sophisticated techniques
for resource utilization as Mesos
Kubernetes can schedule docker or rkt
containers
Inherently opinionated with functionality built-
in.
little to no third-party software needed
builds in many application-level concepts and services
(secrets, petsets, jobsets, daemonsets, rolling updates, etc.)
advanced storage/volume management
Kubernetes arguably moving the quickest
Relatively thorough project documentation
Multi-master, cross-cluster federation, robust
logging & metrics aggregation
37. Genesis & Purpose
Mesos is a distributed systems kernel
stitches together many different machines into a logical computer
Mesos has been around the longest (launched in 2009)
and is arguably the most stable, with highest (proven) scale currently
Mesos is written in C++
with Java, Python and C++ APIs
Marathon as a Framework
Marathon is one of a number of frameworks (Chronos and Aurora other
examples) that may be run on top of Mesos
Frameworks have a scheduler and executor. Schedulers get resource offers.
Executors run tasks.
Marathon is written in Scala
39. Support & Momentum
MesosCon 2015 in Seattle had 700 attendees
up from 262 attendees in 2014
78 contributors in the last year
Under the governance of Apache Foundation
Used by Twitter, AirBnb, eBay, Apple, Cisco, Yodle
@lcalcote
40. Host &
Service Discovery
Mesos-DNS generates an SRV record for each Mesos
task
including Marathon application instances
Marathon will ensure that all dynamically assigned
service ports are unique
Mesos-DNS is particularly useful when:
apps are launched through multiple frameworks (not just Marathon)
you are using an IP-per-container solution like
you use random host port assignments in Marathon
Project Calico
image: iStock@lcalcote
41. Scheduling
Two level scheduler
First level scheduling happens at mesos master based
on allocation policy , which decides which framework
get resources
Second level scheduling happens at Framework
scheduler , which decides what tasks to execute.
Provide reservations, over-subscriptions and
preemption
@lcalcote
42. Modularity & Extensibility
Frameworks
multiple available
may run multiple frameworks
Modules
extend inner workings of Mesos by creating and using
shared libraries that are loaded on demand
many types of Modules
Replacement, Isolator, Allocator, Authentication, Hook, Anonymous
@lcalcote
43. Updates &
Maintenance
Nodes
- Mesos has maintenance mode
Applications
Marathon can be instructed to
deploy containers based on that
component using a blue/green
strategy
where old and new versions co-exist for a
time.
image: 123RF@lcalcote
44. Health Monitoring
Nodes
Master tracks a set of statistics and metrics to
monitor resource usage
Counters and Gauges
Applications
support for health checks (HTTP and TCP)
an event stream that can be integrated with load-
balancers or for analyzing metrics
45. Networking & Load-
Balancing
Networking
An IP per Container
No longer share the node's IP
Helps remove port conflicts
Enables 3rd party network drivers
isolator with
MesosContainerize
Load-Balancing
Marathon offers two TCP/HTTP proxies
A simple shell script and a more complex one called marathon-lb that
has more features.
Pluggable (e.g. Traefic for load-balancing)
Container Network Interface (CNI)
46. High Availability & Scale
A strength of Mesos’s architecture
requires masters to form a quorum using ZooKeeper (point of failure)
only one Active (Leader) master at-a-time in Mesos and Marathon
Scale is a strong suit for Mesos. Used at Twitter,
AirBnB... TBD for Marathon
Great at asynchronous jobs. High availability built-in.
Referred to as the “golden standard” by Solomon Hykes, Docker CTO.
47. Universal Containerizer
abstract away from docker, rkt, kurma?, runc, appc
Can run multiple frameworks, including Kubernetes and Swarm.
Only of the container orchestrators that supports multi-tenancy
Good for Big Data house and job-oriented or task-oriented workloads.
Good for mixed workloads and with data-locality policies
Powerful and scalable, Battle-tested
Good for multiple large things you need to do 10,000+ node cluster system
Marathon UI is young, but promising
Still needs 3rd party tools
Marathon interface could be more Docker friendly
(hard to get at volumes and registry)
May need a dedicated infrastructure IT team
an overly complex solution for small deployments@lcalcote