(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

November 12, 2014 | Las Vegas, NV
Becky Weiss, Principal Software Engineer, Amazon EC2 Networking

Elastic
network
interface
Subnet A
us-east-1a
10.0.1.0/24
10.0.1.100
Subnet A2
us-east-1a
10.0.2.0/24
10.0.1.101
10.0.2.50
10.0.2.51
Subnet C
us-east-1c
10.0.3.0/24
10.0.3.99
Instance
1
Instance
2
Instance
3 Instance
4

elastic
network
interface
Subnet A
us-east-1a
10.0.1.0/24
10.0.1.100
Subnet A2
us-east-1a
10.0.2.0/24
10.0.1.101
10.0.2.50
10.0.2.51
Subnet C
us-east-1c
10.0.3.0/24
10.0.3.99
Instance
1
Instance
2
Instance
3 Instance
4
Placement group

C:> aws ec2 run-instances --image-id ami-b66ed3de --instance-type c3.8xlarge --subnet-id
subnet-c03cfb99 --security-group-ids sg-72caf017 --key-name NetworkingTestSSHKey --count
2
---------------------------------------------------------------------------------
| RunInstances |
+----------------------------------------+--------------------------------------+
| OwnerId | 123456789012 |
| ReservationId | r-9f5404b5 |
+----------------------------------------+--------------------------------------+
| Instances |
|+-----------------------------------+-----------------------------------------+|
|| AmiLaunchIndex | 0 ||
|| Architecture | x86_64 ||
|| ClientToken | None ||
|| EbsOptimized | False ||
|| Hypervisor | xen ||
|| ImageId | ami-b66ed3de ||

2
---------------------------------------------------------------------------------
| RunInstances |
+----------------------------------------+--------------------------------------+
| OwnerId | 123456789012 |
+----------------------------------------+--------------------------------------+
| Instances |
|+-----------------------------------+-----------------------------------------+|
AMI: More about this
choice later…

2
---------------------------------------------------------------------------------
| RunInstances |
+----------------------------------------+--------------------------------------+
| OwnerId | 123456789012 |
+----------------------------------------+--------------------------------------+
| Instances |
|+-----------------------------------+-----------------------------------------+|
Big instance type:
c3.8xlarge

NetworkingTestPlacementGroup available cluster

2 --placement GroupName=NetworkingTestPlacementGroup
---------------------------------------------------------------------------------
| RunInstances |
+----------------------------------------+--------------------------------------+
| OwnerId | 123456789012 |
| ReservationId | r-13374839 |
+----------------------------------------+--------------------------------------+
| Instances |
|+-----------------------------------+-----------------------------------------+|

Instance 1 Instance 2
...........

Virtualization layer
eth0
eth1
Instance Virtual NICs
Physical NIC

eth0
Instance
Physical NIC
VF Driver
eth1
VF

[ec2-user@ip-10-0-3-70 ~]$
ethtool -i eth0
driver: vif
version:
firmware-version:
bus-info: vif-0
…
[ec2-user@ip-10-0-3-70 ~]$
ethtool -i eth0
driver: ixgbevf
version: 2.14.2+amzn
firmware-version: N/A
bus-info: 0000:00:03.0
…

amzn-ami-hvm-2012.03.1.x86_64-ebs
hvm

--attribute
sriovNetSupport
InstanceId i-37c5d1d9
Not yet!

[ec2-user@ip-10-0-3-125 ~]$ sudo yum update
OS update

reboot-instances
Reboot
(OS update)
(Not shown here: analogous steps for other Linux distros)

stop-instances
Stop the instance

stop-instances
--sriov-net-support
simple
Enable SRIOV
Cannot be undone

start-instances
--attribute
sriovNetSupport
InstanceId i-37c5d1d9
Value simple
We’re on

 modinfo ixgbevf
 aws ec2 register-image
--name MyEnhancedNetworkingImage
--image-location …
--sriov-net-support-simple

i2.8xlarge
Storage-optimized instance

require 'mongo‘
'randomdb'
until Time SECONDS_TO_RUN
KEY_MAX
:key
Time
if
:times_accessed
:key
else
:key :value
:times_accessed
end
Time
end
Spin in tight loop:
Read a random document
Then write it back

def add_write_statistic
:sample_count
:sum
:minimum :minimum
:maximum :maximum
end
Aggregating statistics for CloudWatch

require 'aws-sdk'
AWS CloudWatch Client
if Time
:namespace 'NetworkingTest/MongoDemo',
:metric_data => [{:metric_name => 'WriteTime',
:dimensions => [{:name => 'RunId', :value => MY_RUN_ID}],
:statistic_values => write_stats}],
:unit => 'Seconds'
Time
:sample_count :sum
end
CloudWatch PutMetricData:
Writing a custom metric

# ec2-run-instances ami-b66ed3de --instance-type c3.large --subnet subnet-c03cfb99 --
group sg-72caf017 --placement-group NetworkingTestPlacementGroup --monitor --user-data-
file my_startup_script.sh --iam-profile NetworkingTestIAMRole --instance-count 10
RESERVATION r-d13d6f37 123456789012
INSTANCE i-fb6d5352 ami-b66ed3de ip-10-0-1-113.ec2.internal
pending NetworkingTestSSHKey 0 c3.large 2014-10-30T13:26:33+0000
us-east-1a monitoring-pending 10.0.1.113
vpc-ca28afaf subnet-c03cfb99 ebs NetworkingTestPlacementGroup hvm
xen sg-72caf017 defaultfalse arn:aws:iam::123456789012:instance-
profile/NetworkingTestIAMRole
NIC eni-b560caed subnet-c03cfb99 vpc-ca28afaf 123456789012 in-use 10.0.1.113
true
NICATTACHMENT eni-attach-fb6ddf9d 0 attaching 2014-10-30T06:26:33-0800
true
GROUP sg-72caf017 default
...

true
true
...
CloudWatch detailed monitoring:
1-minute metrics

true
true
...
Startup script file

# cat startup_script.sh
Download client test script from S3
Then gogogo!

true
true
...
Security best practice:
Launch instances with IAM roles if
they need to access any AWS
resources

# aws iam list-role-policies --role-name NetworkingTestIAMRole
{
"PolicyNames": [
"NetworkingTestIAMRole-CloudWatchPolicy",
"NetworkingTestIAMRole-S3Policy"
]
}

# aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole-S3Policy
Allow retrieving objects from a particular S3 bucket

# aws iam get-role-policy --role-name NetworkingTestIAMRole --policy-name NetworkingTestIAMRole-
CloudWatchPolicy
Allow CloudWatch PutMetricData

Label WriteTime
389483.0 2014-10-29T02:30:00Z Seconds
390189.0 2014-10-29T02:33:00Z Seconds
392373.0 2014-10-29T02:34:00Z Seconds
392387.0 2014-10-29T02:32:00Z Seconds
377256.0 2014-10-29T02:31:00Z Seconds
SampleCount statistic:
How many of these WriteTime statistics
were written across all instances during
each minute?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
“WriteTime” SampleCount statistic
by number of client instances
TPS, regular TPS, enhanced

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
DiskWriteBytes 1-minute Sum statistic
by number of client instances
Regular Enhanced

Instance
VF driver

(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (8)

Similar to (SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014

Similar to (SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014