Building Microservices with Containers (CON308-R1) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Microservices with Containers
Arun Gupta (@arungupta)
Principal Open Source Technologist
Amazon Web Services
C O N 3 0 8 R
Sarah Wells (@sarahjwells)
Technical Director for Operations and Reliability
Financial Times

Development transformation at Amazon: 2001–2009
monolithic
application + teams
2001
Lesson learned: decompose for agility
2009
microservices
+ 2 pizza teams

Full ownership
Full accountability
“DevOps”
Focused innovation
Two-pizza teams

What changes
have to be made
in this new world?
Architectural patterns
Operational model
Software delivery

Monoliths are good,
but they have some limitations
Coordination overhead
Poor modularity - No DRY
High impact of change
Poor scalability
Long build time

When the impact of change is small,
release velocity can increase
Monolith
Does everything
Microservices
Does one thing

MICROSERVICE
API
API
MICROSERVICE
MICROSERVICE
EVENT
API
MICROSERVICE
EVENT
API
MICROSERVICE
APPLICATION
Mobile client
Client
IoT
PERSISTENCE PERSISTENCE

Manage APIs with API Gateway
Mobile apps
Websites
Services
Internet Amazon
CloudFront
Amazon CloudWatch
monitoring
API
Gateway
cache
Any other
AWS service
All publicly
accessible
endpoints
AWS
Lambda
functions
Endpoints
in your VPC
Regional API Endpoints
AWS Lambda
functions
Endpoints on
Amazon EC2
Your VPCAWS

Event-driven architectures

Decouple state from code using messaging
Queues
Simple
Fully-managed
Any volume
Pub/sub
Simple
Fully-managed
Flexible
Amazon Simple
Queue Service
Amazon Simple
Notification
Service
Messaging
Synchronization
Rapid
Fully-managed
Real-time
Amazon
CloudWatch
Events

Track status of data
and execution
Remove
redundant code
Build workflows to orchestrate everything

AWS Services for Containerized Microservices
Storage & Database
Amazon
DynamoDB
Amazon ElastiCache
Amazon Relational
Database Service
(Amazon RDS)
Amazon
Simple Storage
Service
(Amazon S3)
Logging & Monitoring
Amazon
CloudWatch
AWS
CloudTrail
AWS X-Ray
Application Integration
Amazon
MQ
Amazon
Simple
Notification
Service
(Amazon
SNS)
Amazon
Simple Queue
Service
(Amazon
SQS)
AWS Step
Functions
Developer Tools
AWS
CodeBuild
AWS
CodePipeline
AWS
Cloud9
Networking & API Proxy
Amazon API
Gateway
Elastic Load
Balancing
Amazon
Route 53
Compute
Amazon
EKS
Amazon
ECS
Amazon
ECR
Fargate
AWS Elastic
Beanstalk

Eight reasons why Amazon uses microservices
Pick the right
tool for the job
Create new
compositions easily
Improve resilience
and security
Experiment
and fail safely
Lower costs with
granular scaling
Adopt
technology faster
Optimize
team productivity
Deploy features safely
and quickly

AWS operational responsibility models
On-Premises Cloud
Less More
Compute Virtual Machine
EC2 Elastic Beanstalk AWS LambdaFargate
Databases MySQL MySQL on EC2
RDS MySQL RDS Aurora Aurora Serverless DynamoDB
Storage Storage
S3
Messaging ESBs
Amazon MQ Kinesis SQS / SNS
Analytics
Hadoop Hadoop on EC2 EMR Elasticsearch Service Athena

Comparison of operational responsibility
AWS Lambda
Serverless functions
AWS Fargate
Serverless containers
ECS/EKS
Container-management as a service
EC2
Infrastructure-as-a-Service
More opinionated
Less opinionated
AWS manages Customer manages
• Data source integrations
• Physical hardware, software, networking,
and facilities
• Provisioning
• Application code
• Container orchestration, provisioning
• Cluster scaling
• Physical hardware, host OS/kernel,
networking, and facilities
• Security config and updates, network config,
management tasks
• Container orchestration control plane
• Physical hardware software,
• Work clusters
firewall, management tasks
• Physical hardware software,
• Scaling
management tasks
• Provisioning, managing scaling and
patching of servers

Containers and Microservices
• Do one thing, really well
• Any app, any language
• Test and deploy same artifact
• Self-contained services
• Isolated execution environment
• Faster startup
• Scaling and upgrading
Container Container
Container Container

How do we monitor and control all
of these microservices?

Putting logic inside each microservice is complex
Microservice
Application Code

Easier: decouple operational logic and SDKs
Application CodeMicroservice
Proxy
Logic for:
Monitoring
Routing
Discovery
Deployment

Observability and Traffic Control
• Identify root causes and bottlenecks using monitoring and tracing
• Simplified blue/green using client-side traffic shaping
• Retries and circuit breaker using failure routing
• Work across cluster and container services
• Avoid “framework explosion” across multiple languages

Web Shopping Cart
Notifications
Inventory
Inventory

The fast
companies are 2555x
faster than
the slow
Compared to low performers, high
performers have:
46x more frequent code deployments
2555x faster lead time from commit to deploy
2604x faster mean time to recover from downtime
7.0x lower change failure rate
DevOps - Time to Value

Monolith development lifecycle
monitorreleasetestbuild
developers
delivery pipelines
services

Microservice development lifecycle
developers services
delivery pipelines

How Amazon
does DevOps
(microservices, 2 pizza teams)
(governance, templates)

Moved their large monolithic
application to a microservices-
based architecture
Their time to deploy
software changes went
from hours to minutes
AWS case studies—containerized microservices
Built their Arc publishing
platform as containerized
microservices running on AWS
It supports over 100
microservices composed
of more than 3,000 containers
Built and launched their Home
Delivery platform in less than
4 months using a
microservices architecture,
scaling to 20,000 orders per
second

Adopting microservices at the Financial Times
Sarah Wells, Technical Director for Operations & Reliability

@sarahjwells #reInvent
Step 1: monolith to
microservices

Context
First microservices case studies: March 2012
Martin Fowler/James Lewis blog post: March 2014
Sam Newman book: February 2015

2013:
2014:
2015:
2016:
2017:
2018:
September - first code committed

Conditions for change

If you’re going to do things lots of times, you need to
be able to do them quickly

Provisioning a server

Self service via FT Platform

Context
FT Platform on private cloud: early 2012
FT Platform on AWS: September 2014

Don’t start with microservices
until you can provision a server
in minutes

Automated deployment

Zero downtime deployments

Setting up new deployment pipelines has to be quick

Our old build and release process was very manual

12 releases a year

New deployment pipelines based on Jenkins and
Puppet

If you can’t deploy the smallest
possible change to production in
under an hour, fix that first

Continuous delivery will force
you to change your culture

No more change approval boards

Moving to microservices

Single responsibility

Asychronous communication via queue is essential

Polyglot persistence

What we gained

Small = easy to understand

Clear boundaries = easy to test the component

Releases are small and self contained = easy to
understand what the change is

Moved from 12 to > 2000 deployments a year

Much lower failure rate - less than 1% vs over 15%

Decreasing the time to get new
features out is THE reason to do
microservices

Challenges

Getting the right level of granularity

Be prepared to combine or split
services

Eventual consistency / data in many places

The development flow is different

Repeating yourself can be a
good idea

Be lenient in what you accept:
ignore fields you don’t need

Operational complexity

Devops is essential

‘Grey’ failure is the norm

150 services, 2 instances in each region
5 checks
Every minute

-> over 4 million checks a day

Lots of these failures are things you don’t care about

Focus on things you’d want
someone to call you about
at 2 am

Log aggregation

Tracing

Microservices can be expensive to run

Any manual change has to be done many times

Automation and abstraction will
save you

Step 2: the move to containers

Context
Docker released: March 2013
Docker 1.0: October 2014

2013:
2014:
2015:
2016:
2017:
2018:
Start looking at containers

Spend your innovation tokens
wisely
http://mcfunley.com/choose-boring-technology

In 2014, writing our own cluster orchestration was an
innovation token we were willing to spend

CoreOS on EC2 + docker + our own services

Cluster management through fleet and systemd

Routing via vulcand and etcd

Deployment via custom go ‘deployer’ microservice

2013:
2014:
2015:
2016:
2017:
2018:
Running docker in production

~40% reduction in AWS costs

Many fewer steps to start running a new service in
production

Development, staging and production are the same

Supportability of an in-house platform is a challenge

Technical debt

- routing complex and error prone

- limited rebalancing of services, limited self healing

By late 2016, tools were
maturing

Choose boring technology
http://mcfunley.com/choose-boring-technology

Using leading edge technologies
requires you to be comfortable
with change

Don’t be (too) scared about
making the wrong decision
http://uk.businessinsider.com/jeff-bezos-on-type-1-and-
type-2-decisions-2016-4

2013:
2014:
2015:
2016:
2017:
2018:
Evaluate cluster orchestration options

Step 3: migrating to kubernetes

Context
First kubernetes release: June 2014
Kubernetes 1.6 release: March 2017
Github migrate to kubernetes: August 2017

2013:
2014:
2015:
2016:
2017:
2018:
Start migrating to kubernetes

Kubernetes solved our biggest
issues

Kubernetes assigns names to every service - routing
is simpler and less error prone

Kubernetes is self healing - unhealthy instances get
restarted

Kubernetes will move services around

Deployments

We migrated from
systemd service files to
Helm charts

Integrating the service
into a templated Jenkins
pipeline

2013:
2014:
2015:
2016:
2017:
2018: Live on Kubernetes

Moving to a new platform takes
time and requires investment

Stability

Supportability

We still have a platform to maintain

Now what?

Containers are here to stay

Kubernetes vs ECS vs Elastic Beanstalk

Building a container deployment platform

Making it someone else’s problem?

Get comfortable with change

Thank you!
Arun Gupta (@arungupta)
Principal Open Source Technologist
Amazon Web Services
Sarah Wells (@sarahjwells)
Technical Director for Operations and Reliability
Financial Times

Building Microservices with Containers (CON308-R1) - AWS re:Invent 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building Microservices with Containers (CON308-R1) - AWS re:Invent 2018

Similar to Building Microservices with Containers (CON308-R1) - AWS re:Invent 2018 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Building Microservices with Containers (CON308-R1) - AWS re:Invent 2018