The Nitro Project: Next-Generation EC2 Infrastructure - AWS Online Tech Talks

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda  What is the Nitro Project and how does it relate to AWS C5?  Background on virtualization  The evolution of the Nitro Project  Compatibility  What’s next  Q&A

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Nitro? From the C5 launch: Q. What is the new hypervisor for Amazon EC2? The new hypervisor for Amazon EC2, introduced with the launch of C5 instances, is a component that primarily provides CPU and memory isolation for C5 instances. VPC networking and EBS storage resources are implemented by dedicated hardware components that are part of all current generation EC2 instance families. It is built on core Linux Kernel-based Virtual Machine (KVM) technology, but does not include general purpose operating system components.

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Nitro? From the C5 launch: Q. What is the new hypervisor for Amazon EC2? The new hypervisor for Amazon EC2, introduced with the launch of C5 instances, is a component that primarily provides CPU and memory isolation for C5 instances. VPC networking and EBS storage resources are implemented by dedicated hardware components that are part of all current generation EC2 instance families. It is built on core Linux Kernel-based Virtual Machine (KVM) technology, but does not include general purpose operating system components. The Nitro Hypervisor is the “new hypervisor,” but more than just a hypervisor

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Virtualization <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax)

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Virtualization <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) ERROR

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What happened?  Operating Systems use special instructions that are not available to applications.  A processor is virtualizable when access to these instructions cause an error that privileged software can intercept or trap.

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Virtualization <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) ERROR

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Trap & Emulate: Virtual Machine Monitor <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) TRAP VMM

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Trap & Emulate <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) VMM EMULATE

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What happened?  The VMM is the heart of a hypervisor.  As long as a statistical majority of instructions execute natively, we call this virtualization.  Not all emulation can be handled by the VMM.

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Trap & Emulate <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) VMM

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Trap & Emulate <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f in $0xf,%al <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax) EMULATE VMM TRAP Device Model Device Model Device Model

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What happened?  A hypervisor consists of: - Virtual Machine Monitor - Many device models (10 to 100s) - Scheduler, memory manager, etc.  This was state of the art in 1974  Not all of the assumptions held true though...

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. From 1974 to 2006  Early Intel processors did not trap  The Xen project found a clever solution  Paravirtualization modifies the OS to trap  Hypercalls directly invoke the VMM  EC2 launched using Xen Paravirtualization <_start>: e9 59 e1 17 00 jmpq ffff82d08037e15e 0f 1f 00 nopl (%rax) <multiboot1_header_start>: 02 b0 ad 1b 03 00 add 0x31bad(%rax),%dh 00 00 add %al,(%rax) fb sti 4f 52 rex.WRXB push %r10 e4 0f HYPERCALL io_in <multiboot1_header_end>: 0f 1f 40 00 nopl 0x0(%rax) <multiboot2_header_start>: d6 (bad) 50 push %rax 52 push %rdx e8 00 00 00 00 callq ffff82d080200020 88 00 mov %al,(%rax)

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Circa 2012  Can we do better than the software-only hypervisor architecture?  Device models compete for CPU and system resources, jitter is hard to avoid.  Can we decompose the hypervisor and shuffle components around?  Let’s begin our journey with the state of the art instance type from 2012.

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CR1 (no Nitro) Jan 2013 Amazon RDS IAM Amazon Linux cr1.8xlarge EBS Volumes Hardware Software DM Instance Storage DM DM DM VPC Networking

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CR1 (no Nitro) Jan 2013 Amazon RDS IAM Amazon Linux cr1.8xlarge EBS Volumes Hardware Software DM Instance Storage DM DM DM VPC Networking

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. C3 (early Nitro) Nov 2013 Amazon RDS IAM Amazon Linux c3.8xlarge Enhanced Networking Hardware Software DMDM DM EBS Volumes Instance Storage

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. C3 (early Nitro) Nov 2013 Amazon RDS IAM Amazon Linux c3.8xlarge Enhanced Networking Hardware Software DMDM DM EBS Volumes Instance Storage

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. X1 May 2016 Amazon RDS IAM Amazon Linux x1.32xlarge Instance Storage Enhanced Networking Hardware Software DM DM EBS Volumes Instance Storage

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. X1 May 2016 Amazon RDS IAM Amazon Linux x1.32xlarge Instance Storage Enhanced Networking Hardware Software DM DM EBS Volumes Instance Storage

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. I3 Feb 2017 Amazon RDS IAM Amazon Linux i3.16xlarge EBS Volumes Instance Storage Enhanced Networking Hardware Software DM DM EBS Volumes

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. I3 Feb 2017 Amazon RDS IAM Amazon Linux i3.16xlarge EBS Volumes Instance Storage Enhanced Networking Hardware Software DM DM EBS Volumes

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.  Nitro Hypervisor  Lightweight hypervisor  Nitro Card  Storage  Networking  Management  Monitoring  Security  Nitro Security Chip  Integrated into the motherboard The Nitro System

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. FAQs 1) Will my existing AMIs work on Nitro-based instances? Yes. Most ENA capable AMIs have the necessary drivers. 2) Will applications need to be modified? Most of the time, no. Some applications have relied on undocumented behavior to detect they are running within EC2 and they may require adjustment.

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. FAQs 1) Will my existing AMIs work on Nitro-based instances? Yes. Most ENA capable AMIs have the necessary drivers. 2) Will applications need to be modified? Most of the time, no. Some applications have relied on undocumented behavior to detect they are running within EC2 and they may require adjustment. 3) Will all new instance types be based on the Nitro System? In the fullness of time, we expect most (if not all) new instance types to be Nitro-based. We have no plans to convert existing instance types to Nitro and expect to continue to launch Xen based instance types where appropriate.