AWS Elastic Beanstalk AWS OpsWorks AWS CloudFormation Amazon EC2

1. DEPLOYMENT AND
MANAGEMENT ON AWS:
A DEEP DIVE ON OPTIONS
AND TOOLS
Danilo Poccia – Solutions Architect

2. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

3. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

4. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

5. AWS
ELASTIC BEANSTALK
QUICKLY DEPLOY
AND MANAGE
APPLICATIONS

9. Application
Application
Version Environment
!
!Environment
Configuration
Configuration
Template

10. NEW FEATURES
SINGLE INSTANCE
WORKER ROLE

13. .ebextensions/01install.config
packages:
yum:
libmemcached: []
ruby-devel: []
gcc: []
rubygems:
chef: '0.10.2'
apt:
mysql-client: []
packages:
msi:
mysql: http://dev.mysql.com/.../mysql-connector-net-6.6.5.msi/.../

14. .ebextensions/02copy.config
files:
"/home/ec2-user/myfile" :
mode: "000777"
owner: ec2-user
group: ec2-user
source: http://foo.bar/myfile
"/home/ec2-user/myfile2" :
mode: "000777"
owner: ec2-user
group: ec2-user
content: |
# this is my file
# with content

15. .ebextensions/03run.config
commands:
test:
command: myscript.py
cwd: /home/ec2-user
env:
myvarname: myvarvalue

16. .ebextensions/04run_after.config
container_commands:
collectstatic:
command: "django-admin.py collectstatic --noinput"
01syncdb:
command: "django-admin.py syncdb --noinput"
leader_only: true
02migrate:
command: "django-admin.py migrate"
leader_only: true
99customize:
command: "scripts/customize.sh"

17. AWS Elastic Beanstalk
Resource Types Reference
AWS Resource Resource Type Identifier
Amazon CloudWatch AWS::CloudWatch::Alarm
DynamoDB Table AWS::DynamoDB::Table
Amazon ElastiCache Cache Cluster AWS::ElastiCache::CacheCluster
Amazon ElastiCache Security Group AWS::ElastiCache::SecurityGroup
Amazon ElastiCache Security Group Ingress AWS::ElastiCache::SecurityGroupIngress
Amazon SNS Subscription AWS::SNS::Subscription
Amazon SNS Topic AWS::SNS::Topic
Amazon SQS Queue AWS::SQS::Queue

18. AWS Elastic Beanstalk
Built-in Resources
Resource Name Description
AWSEBAutoScalingGroup
The name of the Auto Scaling group that Elastic Beanstalk
uses when it launches Amazon EC2 instances.
AWSEBAutoScalingLaunchConfiguration
The name for the launch configuration settings that Elastic
Beanstalk uses when it launches EC2 instances.
AWSEBEnvironmentName The name of the Elastic Beanstalk environment.
AWSEBLoadBalancer
The name of the elastic load balancer used in the Elastic
Beanstalk environment.
AWSEBRDSDatabase The name of the Amazon RDS database.
AWSEBSecurityGroup
The name for the EC2 security group that Elastic Beanstalk
uses when it launches EC2 instances.

19. Production
Environment
Test
Environment
master
branch
test
branch
Git
Repository
git commit
git aws.push
git aws.push
Swap
URLs

20. AWS OPSWORKS
MODEL AND MANAGE
THE ENTIRE APPLICATION

21. STACKS

22. LAYERS

23. INSTANCES

24. APPS

26. LIFE CYCLE EVENTS
setup configure deploy undeploy shutdown

27. CONTINUOUS CONFIGURATIONnew/stopped
online
requested
pending
booting
setup
configure
terminating
shutting
down
deploy
configure

28. AWS OPSWORKS
SAMPLE USE CASE

29. GATHER AND SHIP LOGS TO
AMAZON S3 AS SOON AS
THE CPU LOAD
IS TOO HIGH

30. AWS OPSWORKS STORES
1-MINUTE METRICS
IN CLOUDWATCH

31. EVERY INSTANCE
CREATES AN ALARM
FOR HIGH CPU LOAD

32. CLOUDWATCH
ALARM ACTION:
WRITE TO
SNS TOPIC

33. SNS PUBLISHES
TO QUEUE IN SQS

34. WATCHER
INSTANCE
POLLS
SQS FOR
NOTIFICATIONS

35. ON ALARM
NOTIFICATION,
CALL OPSWORKS
API TO …

36. EXECUTE A
SCRIPT ON THE
AFFECTED
SERVER

37. LOGS ARE
GATHERED AND
WRITTEN TO
AMAZON S3

38. CHEF 11.10 + BERKSHELF

39. AWS
CLOUDFORMATION
INFRASTRUCTURE IS CODE

40. CLOUDFORMATION
TEMPLATE

42. Stack

43. Stack Template

44. Stack Template
Conditions
Resources
References
Mappings
Outputs
Parameters

45. Conditions
Resources
References
Mappings
Outputs
Parameters
"Parameters" : {!
"InstanceType" : {!
"Type" : "String",!
"Default" : "t1.micro",!
"AllowedValues" : ["t1.micro", "m1.small", "m1.large"],!
"Description" : "Enter t1.micro, m1.small, or m1.large. Default is t1.micro."!
}!
}

46. Conditions
Resources
References
Mappings
Outputs
Parameters
"Mappings" : {!
"RegionMap" : {!
"us-east-1" : {!
"AMI" : "ami-76f0061f"!
},!
"us-west-1" : {!
"AMI" : "ami-655a0a20"!
},!
"eu-west-1" : {!
"AMI" : "ami-7fd4e10b"!
},!
"ap-southeast-1" : {!
"AMI" : "ami-72621c20"!
}!
}!
}

47. Conditions
Resources
References
Mappings
Outputs
Parameters "Parameters" : {!
"EnvType" : {!
"Description" : "Environment type.",!
"Default" : "test",!
"Type" : "String",!
"AllowedValues" : ["prod", "test"]!
}!
},!
!
"Conditions" : {!
"CreateProdInstance" : {"Fn::Equals" : [{"Ref" : "EnvType"}, "prod"]}!
}!
!
"ProductionInstance" : {!
"Type" : "AWS::EC2::Instance",!
"Condition" : "CreateProdInstance",!
"Properties" : {!
"InstanceType" : "c1.xlarge",!
"SecurityGroups" : [ { "Ref" : "ProdSecurityGroup" } ],!
"KeyName" : { "Ref" : "ProdKeyName" },!
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}!
}!
}

48. Conditions
Resources
References
Outputs
Parameters
"Resources" :{!
"MySimpleImage" : {!
"Type" : "AWS::EC2::Image",!
"Properties" : {!
"ImageId" : "myLinuxBundle-2011-12-30",!
}!
}!
}
Mappings

49. Conditions
References
Outputs
Parameters
"Parameters" : {!
"MyURL" : {!
"Type" : "String",!
"Default" : "http://aws.amazon.com"!
},!
!
...!
!
"Outputs" : {!
"URL" : {!
"Value" : { "Ref" : "MyURL" }!
}!
}
Mappings
Resources

50. Conditions
Outputs
Parameters
"Outputs" : {!
"URL" : {!
"Value" : "http://aws.amazon.com/cloudformation"!
}!
}
Mappings
Resources
References

51. Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
Intrinsic
Functions

52. Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
Resources : {!
"MyVolume" : {!
"Type" : "AWS::EC2::Volume",!
"Properties" : {!
"Size" : "4",!
"SnapshotId" : "snap234",!
"AvailabilityZone" : "us-east-1a"!
}!
}!
}
Intrinsic
Functions

53. Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources
"Outputs" {!
"MyStacksRegion" : { "Value" : { "Ref" : "AWS::Region" } }!
}
Intrinsic
Functions

54. Conditions
Outputs
Mappings
References
Pseudo
Parameters
Resource
Properties
Parameters
Resources "Outputs" : {!
"URL" : {!
"Value" : { "Fn::GetAtt" : [ "MyLoadBalancer", "DNSName" ] }!
}!
}
Intrinsic
Functions
Name
Fn::Base64
Fn::FindInMap
Fn::GetAtt
Fn::GetAZs
Fn::Join
Ref

55. Stack Template
Conditions
Resources
References
Mappings
Outputs
Parameters
Pseudo
Parameters
Resource
Properties
Intrinsic
Functions

56. {
"Description"
:
"Create
RDS
with
username
and
password",
"Resources"
:
{
!
"MyDB"
:
{
"Type"
:
"AWS::RDS::DBInstance",
"Properties"
:
{
"AllocatedStorage"
:
"500",
"DBInstanceClass"
:
"db.m1.small",
"Engine"
:
"MySQL",
"EngineVersion"
:
"5.6",
"MasterUsername"
:
"MyName",
"MasterUserPassword"
:
"MyPassword"
}
}
}
}

57. "AWS::CloudFormation::Init"
:
{
"config"
:
{
!
"packages"
:
{
"yum"
:
{
"mysql"
:
[],
"mysql-­‐server"
:
[],
"httpd"
:
[],
"php"
:
[],
"php-­‐mysql"
:
[]
}
},
"sources"
:
{
"/var/www/html"
:
"https://s3.amazonaws.com/my-­‐builds/build-­‐v4.zip"
}
}

58. {
"Parameters"
:
{
"KeyName"
:
{
"Description"
:
"Name
of
an
existing
EC2
KeyPair
to
enable
SSH
access
to
the
instance",
"Type"
:
"String"
}
},
}

60. AMAZON EC2
API / SDK / CLI

61. aws ec2 describe-instances!
--filter Name=instance-state-name,Values=running,!
Name=tag:aws:autoscaling:groupName,Values=XXX!
--query Reservations[].Instances[].PublicDnsName!
--output text

62. for ip in $(!
!
aws ec2 describe-instances!
--filter Name=instance-state-name,Values=running,!
Name=tag:aws:autoscaling:groupName,Values=XXX!
--query Reservations[].Instances[].PublicDnsName!
--output text!
!
); do ssh -t ec2-user@$ip "sudo yum update"; done!
!

63. for ip in $(!
!
aws ec2 describe-instances!
--filter Name=instance-state-name,Values=running,!
Name=tag:aws:autoscaling:groupName,Values=XXX!
--query Reservations[].Instances[].PublicDnsName!
--output text!
!
); do ssh -t ec2-user@$ip "cd /app ; git pull"; done!
!

64. for ip in $(!
!
aws ec2 describe-instances!
--filter Name=instance-state-name,Values=running,!
Name=tag:aws:autoscaling:groupName,Values=XXX!
--query Reservations[].Instances[].PublicDnsName!
--output text!
!
); do ssh -t ec2-user@$ip "/app/update.sh"; done!
!

65. AWS CLI
http://aws.amazon.com/cli/!
https://github.com/aws/aws-cli!
+!
./jq!
https://github.com/stedolan/jq

66. aws sns list-subscriptions |!
jq -c '.Subscriptions[] |!
select(.Protocol == "sqs")' |!
while read s!
do!
if [ "$(aws sqs get-queue-url —queue-name!
$(echo $s | jq -r .Endpoint |!
sed ’s/^arn:aws:sqs:[^:]*:[^:]*://')!
2>&1 >/dev/null ;!
echo $?)" != "0" ]!
then!
echo "Unsubscribing $s ..."!
aws sns unsubscribe —subscription-arn!
$(echo $s | jq -r .SubscriptionArn)!
echo "Done."!
fi;!
done

67. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

68. DEPLOYMENT AND
MANAGEMENT ON AWS:
A DEEP DIVE ON OPTIONS
AND TOOLS
Danilo Poccia – Solutions Architect