AWS - Managing Your Cloud Assets 2013

Journey through the cloud:
Managing your cloud assets
Ryan Shuttleworth – Technical Evangelist
@ryanAWS

Journey through the cloud

Common use cases & stepping stones into the AWS cloud
Learning from customer journeys
Best practices to bootstrap your projects

Managing cloud assets

Getting to grips with system sprawl in an unconstrained environment
Leverage management frameworks to take control
Choose systems that suite your technology delivery style

Agenda
Setting the scene
A shrink wrapped approach – Elastic Beanstalk
Fine grained control – CloudFormation
A middle way - OpsWorks

Cloud assets are growing quickly
Defined architectural patterns
More applications
More data
More services

Greater agility = faster moving
environments

aws.amazon.com/architecture

How do we maintain control over what is
deployed in the cloud?

Controlled Meticulous
chaos management

chaos management
Prototype an app
Deploy to an instance
Test and develop cycles
Go to production
Deploy more instances
Deploy RDS
Add load balancers
Add autoscaling
Audit what you have?

Quickly becomes DevOps
nightmare

chaos management
Prototype an app Prototype an app
Deploy to an instance Plan infrastructure
Test and develop cycles Build standards
Go to production Deploy environment
Deploy more instances Manage through change control
Deploy RDS Formal processes
Add load balancers Formal tools
Add autoscaling Embed configuration in
Audit what you have? chef/puppet

Quickly becomes DevOps Tightly controlled, but lots of
nightmare effort to achieve

chaos management
People
Prototype an app Prototype an app
Deploy to an instance
Processes Plan infrastructure
Build standards
Test and develop cycles Controls Deploy environment
Go to production
Deploy more instances
Deploy RDS
Tools Manage through change control
Formal processes
Add load balancers Formal tools
Add autoscaling Embed configuration in
Audit what you have? chef/puppet

Quickly becomes DevOps Tightly controlled, but lots of
nightmare effort to achieve

High level, Fine grained,
shrink declarative
wrapped control

Tools
“I just want my app to “I want to source control
run, and I only want my entire datacenter
access to machines if I deployment like I would
really need to” my apache config”

shrink declarative
wrapped control

Elastic Beanstalk Tools CloudFormation


shrink declarative
wrapped control
Middle ground
Elastic Beanstalk AWS CloudFormation
OpsWorks

Quickly deploy and manage apps in AWS…

…into a range of containers

.Net Java Ruby
PHP Python Node.js

…into a range of containers

Keep updated on new
containers:
http://aws.typepad.com

Application package is deployed into Beanstalk

Which creates the container in EC2

User Application

Application Service

HTTP Service

Language Interpreter

Operating System

Host

Beanstalk takes care of the environment…

…adding Elastic Load Balancer…

…Auto Scaling groups

…and launching instances

…all wired into the Beanstalk framework

…and published under a CNAME

…with logs and app versions held in S3

Elastic Beanstalk
…Python mod-wsgi & apache

User Application

Application Service

Apache
mod_wsgi
Python

Operating System

Host

Elastic Beanstalk
Passenger & Nginx…

User Application

Application Service

Nginx
passenger
Ruby

Operating System

Host

Rails rake

Git Beanstalk RDS

Django syncdb

How do you create an app for
AWS ElasticBeanstalk?

No dependencies on AWS inherent for Beanstalk deployment

Source
uploaded and
executed in
AWS

Source
downloaded
and executed
locally

Package up as normal…

Java .war file PHP .zip file

IDE plugins Git integration

Microsoft Web
Python .zip file
Deploy package

Console deployments and version updates

Command line tools and wizards

CheckDNSAvailability eb init
elastic-beanstalk-check-dns-availability wizard to initialize an application

CreateApplication eb start/stop
elastic-beanstalk-create-application start/stop an application

CreateApplicationVersion eb update
elastic-beanstalk-create-application-version update application version

CreateEnvironment eb status
elastic-beanstalk-create-environment get status of a running application

Eclipse integration

Visual Studio integration

Git integration

Container configuration

Full admin access on underlying resources
EC2 instances, AMIs, AutoScaling groups…

What happens under the covers of Beanstalk?

AWS
Create security group

Create auto-scaling policy

Create elastic load balancer
Elastic
Beanstalk Add instance

Install software
Consolidate logs

What happens under the covers of Beanstalk?

AWS
Create security group

Create auto-scaling policy
A series of calls into AWS
to marshal resource Create elastic load balancer

creation/configuration Add instance

Install software
Consolidate logs

CloudFormation – underlying automation framework

Compute
Scaling
Security
CDN Backup
DNS Database
Storage Load Balancing
Workflow Monitoring
Networking
Messaging

CloudFormation – underlying automation framework

Compute
CloudFormation Scaling
Security
A consistent scripting interface for CDN Backup
creating/configuring AWS DNS Database
resources Storage Load Balancing
A framework for lifecycle Workflow
management of resources Monitoring
created using scripts Networking
Messaging

CloudFormation components & terminology

CloudFormation
Template Stack

JSON formatted file Framework Configured AWS services
Parameter definition Stack creation Comprehensive service support
Resource creation Stack updates Service event aware
Configuration actions Error detection and rollback Customisable

Template examples

Snippet: Taking user defined parameters
Template {

"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}
JSON formatted file
Parameter definition
Resource creation
Configuration actions

Template examples

Snippet: Creating RDS instance
Template {
"AWSTemplateFormatVersion" : "2010-09-09",

"Description" : "Create RDS with fixed username and password",

"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "5",
"DBInstanceClass" : "db.m1.small",
JSON formatted file "Engine" : "MySQL",
"EngineVersion" : "5.5",
Parameter definition "MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
Resource creation }
Configuration actions }
}
}

Template examples

Snippet: Installing software
Template "AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql" : [],
"mysql-server" : [],
"httpd" : [],
"php" : [],
"php-mysql" : []
}
},
JSON formatted file
"sources" : {
Parameter definition "/var/www/html" : "https://s3.amazonaws.com/
cloudformation-examples/
Resource creation CloudFormationPHPSample.zip"
Configuration actions },

The benefit of templated environments

Procedural definition
Create it programmatically
Template


Template

Parameter driven
Dynamic and user-driven
templates


Template

Parameter driven Collaboration
Dynamic and user-driven Share templates with ease as
templates just files


Procedural definition Known configuration
Create it programmatically Store stack configuration in
Template source control

Parameter driven Collaboration
Dynamic and user-driven Share templates with ease as
templates just files

NASA TV & Curiosity Route 53 DNS
Multi-region & AZ
Elastic Loadbalancing
Amazon Live Streaming
Cloud Front
Cloud Formation

NASA TV & Curiosity CloudFormation 53 DNS
Route template
Multi-region & AZ
defined video caching
infrastructure
Stack
Cloud Front
Template
Cloud ELBs to front
Formation
secondary cache

~100 Nginx
secondary cache
servers

2-3 Nginx mid-tier
cache servers

NASA TV & Curiosity As demand increasedDNS
Route 53 new
Multi-region & AZ
stacks added to an availability
zone using template
Cloud Front
Cloud Formation

One-click
complex
rollout Availability Zone

Where to get more information?

Server templates
High availability configurations
RDS configurations
Open source stacks
Snippets

+ Cloud Former (prototype)

http://aws.amazon.com/cloudformation/aws-cloudformation-templates/

Comprehensive service support

Amazon EC2 Instances Amazon CloudWatch Alarms
(On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Simple Storage Service (S3) Buckets
Amazon Elastic Block Store (EBS) Volumes Amazon Simple DB Domains
Elastic Load Balancers Amazon Simple Queue Service (SQS) Queues
Elastic IP Addresses Amazon Simple Notification Service (SNS) Topics
Amazon EC2 Security Groups Amazon SNS Subscriptions
Auto Scaling Groups Amazon Route 53 DNS Records
Amazon Relational Database Service (RDS) Instances Amazon CloudFront Distributions and Multi-Origin support
Amazon RDS Security Groups AWS Identity and Access Management users and groups
Amazon ElastiCache Clusters AWS Identify and Access Management policies
Amazon ElastiCache Security Groups Amazon Virtual Private Cloud VPCs, Subnets, Gateways
Amazon ElastiCache Parameter Groups Amazon DynamoDB Tables

Comprehensive service support

Amazon EC2 Instances Amazon CloudWatch Alarms
(On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Simple Storage Service (S3) Buckets
Amazon Elastic Block Store (EBS) Volumes Amazon Simple DB Domains
Elastic Load Balancers Amazon Simple Queue Service (SQS) Queues
Elastic IP Addresses Amazon Simple Notification Service (SNS) Topics
Amazon EC2 Security Groups Amazon SNS Subscriptions
Auto Scaling Groups Amazon Route 53 DNS Records
Amazon Relational Database Service (RDS) Instances Amazon CloudFront Distributions and Multi-Origin support
Amazon RDS Security Groups AWS Identity and Access Management users and groups
Amazon ElastiCache Clusters AWS Identify and Access Management policies
Amazon ElastiCache Security Groups Amazon Virtual Private Cloud VPCs, Subnets, Gateways
Amazon ElastiCache Parameter Groups Amazon DynamoDB Tables

+
AWS Elastic Beanstalk

shrink + declarative
wrapped control

Elastic Beanstalk + CloudFormation

shrink + declarative
wrapped control

Elastic Beanstalk + CloudFormation

Developer sandboxes Comprehensive control
with production ready of all aspects of
infrastructure infrastructure

Elastic Beanstalk OpsWorks CloudFormation
Powerful management framework with Chef support

Stack Layers Apps Management

Managed Collection of Your application Management
environment resources assets services
Definition of environment Blueprint for a Resources to deploy Scaling, cloning, user
such as production or test collection of resources and run in layers access, self healing
(instances, EBS, EIPs
etc)

Simple stack creation through layers

Load balancing
Layers HA Proxy installation in an availability zone

Application container
Static, Node.js, Rails, PHP

Database layer
MySQL

Simple stack creation through layers

Layers
Ganglia

+ MemCached

Custom layers

Scaling and auto-healing

Auto healing
Replacement of an ec2
instance regardless of
scaling thresholds
i.e. always maintain 3
nodes across 3 AZs


Auto healing Time based scaling Load based scaling
Replacement of an ec2 Run an ec2 instance on a Add and remove instances
instance regardless of given schedule from a layer based upon
scaling thresholds load rules
i.e. always maintain 3 i.e. run these instances only i.e. run these instances only
nodes across 3 AZs during weekend peaks when CPU goes over 80%

Load based
Time based Scaling and auto-healing

App deployments

Customization with chef recipes

Get started fast with Beanstalk
(control the chaos)

Manage with layers and recipes with
OpsWorks
(control with chef customization)

Take fine grained control with
CloudFormation
(auditable environment specs)

Find out more…

http://aws.amazon.com/elasticbeanstalk/
http://aws.amazon.com/cloudformation/
http://aws.amazon.com/opsworks/

aws.amazon.com
get started with the free tier

AWS - Managing Your Cloud Assets 2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to AWS - Managing Your Cloud Assets 2013

Similar to AWS - Managing Your Cloud Assets 2013 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

AWS - Managing Your Cloud Assets 2013