Over the last 11 years, the Amazon EC2 virtualization platform has quietly evolved to take advantage of unique hardware and silicon, an accelerated network and storage architecture, and with the launch of C5 instances, a bespoke hypervisor to deliver the maximum amount of resources and performance to instances. Come to this deep dive to get a behind-the-scenes look at how our virtualization stack has evolved, including a peak at how our latest generation platform works under the covers.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
C5 Instances and the Evolution of
Amazon EC2 Virtualization
C M P 3 3 2
N o v e m b e r 2 9 , 2 0 1 7
A n t h o n y L i g u o r i , S e n i o r P r i n c i p a l E n g i n e e r , E C 2

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introductions

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
 What is the Nitro Project and how does it relate to AWS C5?
 Background on virtualization
 The evolution of the Nitro Project
 Compatibility
 FAQ
 What’s next

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From the C5 launch:
Q. What is the new hypervisor for Amazon EC2?
The new hypervisor for Amazon EC2, introduced with the launch of C5 instances, is a component that
primarily provides CPU and memory isolation for C5 instances. VPC networking and EBS storage resources are
implemented by dedicated hardware components that are part of all current generation EC2 instance families.
It is built on core Linux Kernel-based Virtual Machine (KVM) technology, but does not include general purpose
operating system components.
What is Nitro?

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From the C5 launch:
Q. What is the new hypervisor for Amazon EC2?
The new hypervisor for Amazon EC2, introduced with the launch of C5 instances, is a component that
primarily provides CPU and memory isolation for C5 instances. VPC networking and EBS storage resources are
implemented by dedicated hardware components that are part of all current generation EC2 instance families.
It is built on core Linux Kernel-based Virtual Machine (KVM) technology, but does not include general purpose
operating system components.
The Nitro Hypervisor is the “new hypervisor,” but more than just a hypervisor
What is Nitro?

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Stepping back...

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization
ERROR

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What happened?
 Operating Systems use special instructions that are not available to applications.
 A processor is virtualizable when access to these instructions cause an error that privileged software can
intercept or trap.

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Virtualization
ERROR

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Trap & Emulate: Virtual Machine Monitor
TRAP
VMM

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Trap & Emulate
VMM
EMULATE

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Trap & Emulate
VMM

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What happened?
 The VMM is the heart of a hypervisor.
 As long as a statistical majority of instructions execute natively, we call this virtualization.
 Not all emulation can be handled by the VMM.

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f in $0xf,%al
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)
Trap & Emulate
EMULATE
VMM
TRAP
Device
Model
Device
Model
Device
Model

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What happened?
 A hypervisor consists of:
 - Virtual Machine Monitor
 - Many device models (10 to 100s)
 - Scheduler, memory manager, etc.
 This was state of the art in 1974
 Not all of the assumptions held true though...

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Early Intel processors did not trap
 The Xen project found a clever solution
 Paravirtualization modifies the OS to trap
 Hypercalls directly invoke the VMM
 EC2 launched using Xen Paravirtualization
From 1974 to 2006
<_start>:
e9 59 e1 17 00 jmpq ffff82d08037e15e
0f 1f 00 nopl (%rax)
<multiboot1_header_start>:
02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh
00 00 add %al,(%rax)
fb sti
4f 52 rex.WRXB push %r10
e4 0f HYPERCALL io_in
<multiboot1_header_end>:
0f 1f 40 00 nopl 0x0(%rax)
<multiboot2_header_start>:
d6 (bad)
50 push %rax
52 push %rdx
e8 00 00 00 00 callq ffff82d080200020
88 00 mov %al,(%rax)

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Evolution of the Nitro System

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Circa 2012
 Can we do better than the software-only hypervisor architecture?
 Device models compete for CPU and system resources, jitter is hard to avoid.
 Can we decompose the hypervisor and shuffle components around?
 Let’s begin our journey with the state of the art instance type from 2012.

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CR1 (no Nitro) Jan 2013
Amazon
RDS
IAM
Amazon
Linux
cr1.8xlarge
EBS Volumes
Hardware Software
DM
Instance Storage
DM DM DM
VPC Networking

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CR1 (no Nitro) Jan 2013
Amazon
RDS
IAM
Amazon
Linux
cr1.8xlarge
EBS Volumes
Hardware Software
DM
Instance Storage
DM DM DM
VPC Networking

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C3 (early Nitro) Nov 2013
Amazon
RDS
IAM
Amazon
Linux
c3.8xlarge
Enhanced Networking
Hardware Software
DMDM DM EBS Volumes
Instance Storage

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C3 (early Nitro) Nov 2013
Amazon
RDS
IAM
Amazon
Linux
c3.8xlarge
Enhanced Networking
Hardware Software
DMDM DM EBS Volumes
Instance Storage

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C4 Jan 2015
Amazon
RDS
IAM
Amazon
Linux
c4.8xlarge
EBS Volumes
Enhanced Networking
Hardware Software
DM EBS Volumes

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C4 Jan 2015
Amazon
RDS
IAM
Amazon
Linux
c4.8xlarge
EBS Volumes
Enhanced Networking
Hardware Software
DM EBS Volumes

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
X1 May 2016
Amazon
RDS
IAM
Amazon
Linux
x1.32xlarge
Instance Storage
Enhanced Networking
Hardware Software
DM DM EBS Volumes
Instance Storage

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
X1 May 2016
Amazon
RDS
IAM
Amazon
Linux
x1.32xlarge
Instance Storage
Enhanced Networking
Hardware Software
DM DM EBS Volumes
Instance Storage

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
I3 Feb 2017
Amazon
RDS
IAM
Amazon
Linux
i3.16xlarge
EBS Volumes
Instance Storage
Enhanced Networking
Hardware Software
DM
DM EBS Volumes

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
I3 Feb 2017
Amazon
RDS
IAM
Amazon
Linux
i3.16xlarge
EBS Volumes
Instance Storage
Enhanced Networking
Hardware Software
DM
DM EBS Volumes

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C5 Nov 2017
Amazon
RDS
IAM
c5.18xlarge
EBS Volumes
Enhanced Networking
Hardware Software
Nitro Hypervisor

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 Bare Metal Nov 2017
Amazon
RDS
IAM
i3.metal
EBS Volumes
Instance Storage
Enhanced Networking
Hardware

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VMware on AWS Aug 2017
Amazon
RDS
IAM
i3.metal
EBS Volumes
Instance Storage
Enhanced Networking
Hardware

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Nitro Hypervisor
 Lightweight hypervisor
 Nitro Card
 Storage
 Networking
 Management
 Monitoring
 Security
 Nitro Security Chip
 Integrated into the motherboard
The Nitro System

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FAQs
1) Will my existing AMIs work on Nitro-based instances?
Yes. Most ENA capable AMIs have the necessary drivers.

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FAQs
1) Will my existing AMIs work on Nitro-based instances?
Yes. Most ENA capable AMIs have the necessary drivers.
2) Will applications need to be modified?
Most of the time, no. Some applications have relied on undocumented behavior to detect they are running
within EC2 and they may require adjustment.

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FAQs
1) Will my existing AMIs work on Nitro-based instances?
Yes. Most ENA capable AMIs have the necessary drivers.
2) Will applications need to be modified?
Most of the time, no. Some applications have relied on undocumented behavior to detect they are running
within EC2 and they may require adjustment.
3) Will all new instance types be based on the Nitro System?
In the fullness of time, we expect most (if not all) new instance types to be Nitro-based. We have no plans to
convert existing instance types to Nitro and expect to continue to launch Xen based instance types where
appropriate.

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s next?

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!