Come spendere fino al 90% in meno con i container e le istanze spot

1. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Come spendere fino al 90% in meno con i container e le istanze spot Massimo Sporchia – AWS Solutions Architect

2. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Agenda • Spot fundamentals • What type of workloads are Spot-friendly? • How can I use Spot in K8s workloads? • Customer Spotlight: Spreaker • The simple rules of Spot

3. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Lower TCO with Amazon EC2 purchase options On-Demand Spiky workloads, to define needs Pay for compute capacity by the second with no long-term commitments Reserved Instances and Savings Plan Committed and steady-state usage Make a commitment and receive a significant discount off compute Spot Instances Fault-tolerant, flexible, stateless workloads Spare EC2 capacity at savings of up to 90% off On-Demand prices

4. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Do more for less with Spot Instances Low, predictable prices Up to 90% discount over On-Demand prices Faster results Increase throughput up to 10x while staying in budget Easy to use Launch through AWS services (e.g. ECS, Batch, EMR) or integrated third-parties Spot Instances are reclaimed with a 2-minute warning and only when On-Demand needs capacity back—no bidding!

5. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Spare capacity at scale AWS customers are using more compute capacity on Amazon EC2 Spot instances than customers in 2014 were running across all of Amazon EC2 Over 2.3 million simulation jobs on a single HPC cluster of 1 million vCPUs built using Amazon EC2 Spot Instances Time to results: 20 days à 8 hours

6. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Saved 75% per month by changing four lines of code Was able to save 74% on their K8s cluster Completed 2.5 million tasks in 8 hours by spinning up an EC2 cluster with over 1 million vCPUs Reduced grid infrastructure costs by 60% Reduced monthly compute costs by 75% while gaining more compute power Processes over 100 billion requests per day with an average response time of 90ms, saving over $3M per year Decreased the time it took to analyze 10,000 biological samples from 7 years to 7 days Saved 70% running Kubernetes workloads on AWS Uses Spot Instances and auto scaling for it’s rendering-as-a-service workload to spend less and scale more 75% reduction in compute cost with Fleet A containerized environment that would have taken more than a year to create previously only took a few months. Saved 40% over Reserved Instances and increased ETL processing time 100%

8. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential $0.27 $0.29$0.28 AZ2 AZ3AZ1 8XL $0.19 $0.16$0.214XL $0.07 $0.08$0.082XL $0.05 $0.04$0.04XL $0.01 $0.02$0.01L C4 $1.76 On- Demand $0.88 $0.44 $0.22 $0.11 Each instance family Each instance size Each availability zone In every Region Is a separate Spot pool R5 M4 M5 I3 C5R4 i3en R5a R5d Amazon EC2 Spot Instance pools explained

9. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential (Not so) New EC2 Spot Pricing Model Smooth pricing After bidding was Removed on Nov 2017 Predictable prices with slow, infrequent variations. Interruptions may only happen when EC2 needs to reclaim capacity for a pool. Spot “max_price”

11. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential To optimize Amazon EC2, combine purchase options with EC2 Auto Scaling Use RIs or Savings Plan for known, steady-state workloads On-demand, for new or stateful spiky workloads Scale using Spot for fault-tolerant, flexible, stateless workloads

12. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Web application - Spot interruption handling AZ1 AZ2 Web servers 1 EC2 needs to reclaim Spot capacity for On-Demand. Spot Interruption Notice is triggered to inform that the instance will be interrupted in 2 minutes via EC2 metadata and CloudWatch Event. A CloudWatch Events rule catches the interruption event and triggers a Lambda Function that: Calls the AutoScaling API to Detach the Instance from the AutoScaling group The Detach action tells ALB to stop sending new connections to the instance that will be interrupted and drain existing ones leveraging the 2 minutes. The Detach action also tells AutoScaling to launch a replacement instance according to the configured allocation strategy (capacity-optimized or lowest- price) 1 2 Lambda Function D B 3 2 5 3 4 4 5 App servers ASG1 Amazon CloudWatch EC2 AutoScaling API draining Event rule M5.L M4.L

13. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers + Spot = match made in heaven ü Containers are often stateless, fault-tolerant, and a great fit for Spot Instances ü Deploy containerized workloads and easily manage clusters at any scale at a fraction of the cost with Spot Instances ü Spot Instances can be used with ECS or Kubernetes to run any containerized workload Skyscanner is a travel fare aggregator website and travel metasearch engine based in Edinburgh, Scotland “We are currently tracking 74% saving over all regions.” —Paul Gillespie, Principal Architect/Tribe Lead

14. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers interruption handling example- Kubernetes Pod pod podPod Node-termination- handler Worker nodes Pod pod podPod Node-termination- handler Pod pod podPod Node-termination- handler AZ1 AZ2 Control plane OD or RI. Worker nodes on Spot instances. Node-termination-handler will identify a Spot Instance being reclaimed and will call the Kube-API in the master node to: • Taint the node and cordon it off to prevent new pods from being placed. • Drain the running pods. • Replace the evicted pods on remaining nodes. Workers will be scaled based on Cluster Autoscaler when PODs on pending-state. Control plane Kube- scheduler Kube-API ASG1 Kube- controller manager Cluster autoscaler Node-termination- handler overpr

15. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers interruption handling example- Kubernetes Node-termination- handler Worker nodes Pod pod podPod Node-termination- handler Pod pod podPod Node-termination- handler AZ1 AZ2 Control plane OD or RI. Worker nodes on Spot instances. Node-termination-handler will identify a Spot Instance being reclaimed and will call the Kube-API in the master node to: • Taint the node and cordon it off to prevent new pods from being placed. • Drain the running pods. • Replace the evicted pods on remaining nodes. Workers will be scaled based on Cluster Autoscaler when PODs on pending-state. Control plane Kube- scheduler Kube-API ASG1 Kube- controller manager Cluster autoscaler pod pod Node-termination- handler overpr

16. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers interruption handling example- Kubernetes Worker nodes Pod pod podPod Node-termination- handler Pod pod podPod Node-termination- handler AZ1 AZ2 Control plane OD or RI. Worker nodes on Spot instances. Node-termination-handler will identify a Spot Instance being reclaimed and will call the Kube-API in the master node to: • Taint the node and cordon it off to prevent new pods from being placed. • Drain the running pods. • Replace the evicted pods on remaining nodes. Workers will be scaled based on Cluster Autoscaler when PODs on pending-state. Control plane Kube- scheduler Kube-API ASG1 Kube- controller manager Cluster autoscaler Pod pod podPod Node-termination- handler

17. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers interruption handling example- Kubernetes Worker nodes Pod pod podPod Node-termination- handler Pod pod podPod Node-termination- handler AZ1 AZ2 Control plane OD or RI. Worker nodes on Spot instances. Node-termination-handler will identify a Spot Instance being reclaimed and will call the Kube-API in the master node to: • Taint the node and cordon it off to prevent new pods from being placed. • Drain the running pods. • Replace the evicted pods on remaining nodes. Workers will be scaled based on Cluster Autoscaler when PODs on pending-state. Control plane Kube- scheduler Kube-API ASG1 Kube- controller manager Cluster autoscaler Pod pod podPod Node-termination- handler Node-termination- handler overpr

19. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Spot pricing Smooth, infrequent changes no spikes, more predictable Up to 90% off Interruptions Happen when EC2 needs capacity Spot infrastructure Is same as On-Demand and RIs Diversify Choose different instance types, size and AZ in a single fleet The simple rules of Spot

20. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS Europe (Milan) Region On April, 28th AWS expanded its global footprint with the opening of the AWS Infrastructure Region in Italy. The new Region AWS Europe (Milano) brings advanced cloud technologies that enable opportunities for innovation, entrepreneurship, and digital transformation. For additional information about services and characteristics of an AWS Region, you can check the website: aws.amazon.com/local/italy/milan/