Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

India cloudsummit Bangalore - Advanced Container Use-cases on AWS Container Services

How to leverage AWS Batch, AWS ECS and AWS EKS for running batch jobs, spot instances and ML workloads.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

India cloudsummit Bangalore - Advanced Container Use-cases on AWS Container Services

  1. 1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Advanced Use-Cases On AWS Container Services For India cloud summit, Bengaluru Feb 15th 2019 Mani Chandrasekaran Solutions Architect Amazon Internet Services Pvt. Ltd. http://twitter.com/cmani http://www.linkedin.com/in/cmanikandan
  2. 2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential When the impact of change is small, release velocity can increase Monolith Does everything Microservices Do one thing
  3. 3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Mobile client Client IoT Microservices architectures API Gateway Account DB Shipping DB Inventory DB Store Front Web App Account Lookup Shipment Query Inventory Query
  4. 4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Cloud-native architectures are small pieces, loosely joined
  5. 5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers are the best on ramp towards modern applications
  6. 6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Why are enterprises adopting containers? • Accelerate software development • Build modern applications • Automate operations at web scale © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential
  7. 7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Make AWS the BEST PLACE to run ANY containerized applications © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential
  8. 8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Helping customers scale containers 450+% growth Hundreds of millions of containers started each week of millions of container instances
  9. 9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Why customers love AWS container services Containers are a first-class citizen of the AWS Cloud Deeply integrated with AWS Security and Compliance Broad selection of compute instances and IAM security, VPC networking, load balancing, and autoscaling ISO, HIPPA, PCI, SOC1, SOC2, SOC3 Infocomm Media Development Auth. DevOps Workflow Best place to build and operate a complete DevOps workflow for containers—AWS DevTools and Cloud9 DEV OPS
  10. 10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Typical use cases • Microservices: Java, Node.js, Go, Web Apps, etc. • Continuous Integration and Continuous Deployment (CICD) • Batch Processing and ETL jobs • Common PaaS Stack for Application Deployment • Legacy Application Migration to the Cloud • Hybrid Workloads • AI/ML • Scale Testing • Backend for IoT use cases
  11. 11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS container services landscape Management Deployment, Scheduling, Scaling & Management of containerized applications Hosting Where the containers run Amazon Elastic Container Service Amazon Elastic Container Service for Kubernetes Amazon EC2 AWS Fargate Image Registry Container Image Repository Amazon Elastic Container Registry
  12. 12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon EC2 Purchasing Options On-Demand Pay for compute capacity by the second with no long-term commitments Spiky workloads, to define needs Reserved Instances Make a 1- or 3-year commitment and receive a significant discount off On-Demand prices Committed, steady-state usage Spot Instances Spare EC2 capacity at savings of up to 90% off On-Demand prices Fault-tolerant, flexible, stateless workloads
  13. 13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Why Spot Instances? Inexpensive Significant price savings of up to 90% over On-Demand Instances. Perfect for stateless and fault-tolerant workloads Faster Results Easily run multiple projects simultaneously and speed up job flows to generate business results faster and innovate faster Easy Access Launch spot instances via the RunInstances function, run-instances cmd, or from AWS Console. Integrated with other AWS and 3rd party services Resource Flexibility Flexibility of ad-hoc provisioning for multiple instance types with an option to hibernate, stop or terminate instances when reclaimed.
  14. 14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Provision Amazon EC2 Spot Instances • Run Instances API • Launch configuration / Auto scaling groups API’s • EC2 Fleet and/or Spot Fleet API • RequestSpotInstances API
  15. 15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon Elastic Container Service
  16. 16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Scheduling and Orchestration Cluster Manager Placement Engine ECS
  17. 17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Containers + Spot = match made in heaven ü Containers are stateless, fault-tolerant and a great fit for Spot Instances ü Deploy containerized workloads and easily manage clusters at any scale at a fraction of the cost with Spot Instances ü Spot instances can be used with ECS or Kubernetes to run any containerized workload Skyscanner is a travel fare aggregator website and travel metasearch engine based in Edinburgh, Scotland
  18. 18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Enhanced GPU support in Amazon ECS Blog - https://aws.amazon.com/blogs/compute/scheduling-gpus-for-deep- learning-tasks-on-amazon-ecs/
  19. 19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Handson labs using ECS and Spot Instances Amazon ECS Catsndogs workshop - https://github.com/aws- samples/amazon-ecs-catsndogs-workshop
  20. 20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS Batch
  21. 21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Introducing AWS Batch Managed No software to install or servers to manage. AWS Batch provisions, manages, and scales your infrastructure Integrated with AWS Natively integrated with the AWS platform, AWS Batch jobs can easily and securely interact with services such as Amazon S3, DynamoDB, and Rekognition Cost-optimized resource provisioning AWS Batch automatically provisions compute resources tailored to the needs of your jobs using Amazon EC2 and Spot Instances
  22. 22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Introducing AWS Batch • Fully-managed batch primitives • Focus on your applications (shell scripts, Linux executables, Docker images) and their resource requirements • We take care of the rest!
  23. 23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS Batch Concepts • Jobs • Job Definitions • Job Queue • Compute Environments • Scheduler
  24. 24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Job Definitions Similar to ECS task definitions, AWS Batch job definitions specify how jobs are to be run. While each job must reference a job definition, many parameters can be overridden. Some of the attributes specified in a job definition: • IAM role associated with the job • vCPU and memory requirements • Mount points • Container properties • Environment variables • Retry strategy $ aws batch register-job-definition --job-definition-name gatk --container-properties ...
  25. 25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Jobs Jobs are the unit of work executed by AWS Batch as containerized applications running on Amazon EC2. Containerized jobs can reference a container image, command, and parameters. Or, users can fetch a .zip containing their application and run it on a Amazon Linux container. $ aws batch submit-job --job-name variant-calling --job-definition gatk --job-queue genomics
  26. 26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Easily run massively parallel jobs Today, users can submit a large number of independent “simple jobs.” AWS Batch also supports “array jobs” that run many copies of an application against an array of elements. Array jobs are an efficient way to run: • Parametric sweeps • Monte Carlo simulations • Processing a large collection of objects
  27. 27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Demo !!!
  28. 28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Demo !!! Run Monte Carlo Simulations on AWS Batch - https://github.com/aws-samples/ec2-spot-montecarlo- workshop
  29. 29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Demo !!! Using EC2 Spot fleet
  30. 30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Demo !!! Using a fully managed AWS batch
  31. 31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential More use cases for AWS Batch .. 1. Fraud detection - https://www.slideshare.net/AmazonWebServices/building-fraud- detection-systems-with-aws-batch-and-containers-dvc301-aws- reinvent-2018 2. Genome workflows - https://aws.amazon.com/blogs/compute/building-high- throughput-genomics-batch-workflows-on-aws-introduction- part-1-of-4/ 3. Data pipelines - http://tech.adroll.com/blog/data/2018/08/08/running-jobs- with-aws-batch.html 4. Much more …
  32. 32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon Elastic Container Service for Kubernetes
  33. 33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential But where you run Kubernetes matters Quality of the cloud platform Quality of the applications Your users
  34. 34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential —CNCF survey
  35. 35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential EKS is Kubernetes certified
  36. 36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon EKS now available in Mumbai !!
  37. 37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential How are customer using Amazon EKS? Microservices PaaS Platform-as-a-Service Enterprise App Migration Machine Learning
  38. 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  39. 39. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon EKS architecture Workers Workers Workers Availability zone Availability zone Availability zone etcd Master Nodes Availability zone Availability zone Availability zone etcd Master Nodes etcd Master Nodes
  40. 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  41. 41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Auto scaling with Amazon EKS • Two dimensions to scaling • EC2 instance through K8s cluster AutoScaler • Scale out Amazon EC2 Spot instances • Pods through HPA • Scale out pods
  42. 42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Auto scaling with EKS – Con’t … Amazon EC2 Autoscaling • Min • Max • Instance type • ASG group K8s Cluster Autoscaler • Min, max, instance type • K8s node groups • CA modes • Auto-discovery • Multi ASG • Single ASG • Master node • HPA
  43. 43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential EKS with Spot – sample architecture VPC Availability zone Availability zone Availability zone Auto Scaling Auto Scaling Auto Scaling m4.large Spot t2.medium Spot On-Demand Instances Instances Instances Spot Instance Spot Instance Spot Instance Spot Instance Spot Instance Spot Instance Spot Interruption Cluster Autoscaler
  44. 44. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential EKS with Spot Instance – Handson Labs https://eksworkshop.com/spot
  45. 45. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Machine Learning Frameworks on EKS https://github.com/aws-samples/machine-learning-using-k8s
  46. 46. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Kubeflow on Amazon EKS https://aws.amazon.com/blogs/opensource/kubeflow-amazon-eks/
  47. 47. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential
  48. 48. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS Container Services Region availability – as of Feb 15th 2019 Amazon ECS AWS Batch AWS Fargate Amazon EKS Singapore, Tokyo, Sydney, Seoul, Mumbai, Beijing, Ningxia, Ireland, Frankfurt, London, Paris, Stockholm, N.Virginia, Ohio, Oregon, N.California, Montreal, Sao Paolo N.Virginia, Ohio, Oregon, N.California, Montreal, Sao Paolo, Ireland, Frankfurt, London, Paris, Singapore, Tokyo, Sydney, Seoul, Mumbai Singapore, Tokyo, Sydney, Seoul, Mumbai, Ireland, Frankfurt, London, N.Virginia, Ohio, Oregon, N.California, Montreal Singapore, Tokyo, Sydney, Seoul, Mumbai, Ireland, Frankfurt, London, Paris, Stockholm, N.Virginia, Ohio, Oregon https://aws.amazon.com/about-aws/global-infrastructure/regional-product- services/
  49. 49. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential THANK YOU https://aws.amazon.com/containers Mani Chandrasekaran Solutions Architect Amazon Internet Services Pvt. Ltd. http://twitter.com/cmani http://www.linkedin.com/in/cmanikandan

×