What kind of design patterns are useful for applications adopting the cloud? How can apps achieve the scalability and availability promised by the cloud? Presentation from Interop 2011 Enterprise Cloud Summit.

1. Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus

2. Shlomo Swidler Founder, Orchestratus Strategic and technicalIT consulting Customers include: Cloud Developer Tips bloghttp://shlomoswidler.com/ Among top community-ranked contributors to Amazon Web Services discussion forums 2

3. Ten CloudDesign Patterns Shlomo SwidlerFounderOrchestratus

4. Ten Cloud ApplicationDesign Patterns Shlomo SwidlerFounderOrchestratus

5. Ten Cloud ApplicationDesign Patterns Many Shlomo SwidlerFounderOrchestratus

6. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. 6

7. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. AKA Architectural Pattern 7

8. What is a Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Goal AvailableResources Constraints 8

9. A Design Pattern A reusable recipe for building (software) systems that solve a particular problem. Meets affirmativerequirements Goal Does not violatenegative requirements AvailableResources Constraints Can be implemented 9

10. Challenges Faced by Apps in the Cloud Application Scalability Cloud promises rapid (de)provisioning of resources. How do you tap into that to create scalable systems? Application Availability Underlying resource failures happen … usually more frequently than in traditional data centers. How do you overcome that to create highly available systems? 10

11. The Scalability Challenge Scalability: Handle more (or fewer) requests It’s not Performance (handle requests faster) It’s not Availability (tolerate failures) But improving Scalability often improves Availability 11

12. The Scalability Challenge Two different components to scale: State (inputs, data store, output) Behavior (business logic) Any non-trivial application has both. Scaling one component means scaling the other, too. 12

13. App Scalability Patterns for State Data Grids Distributed Caching HTTP Caching Reverse Proxy CDN Concurrency Message-Passing Dataflow Software Transactional Memory Shared-State Partitioning CAP theorem: Data Consistency Eventually Consistent Atomic Data DB Strategies RDBMS Denormalization Sharding NOSQL Key-Value store Document store Data Structure store Graph database 13

14. App Scalability Patterns for Behavior Compute Grids Event-Driven Architecture Messaging Actors Enterprise Service Bus Domain Events Event Stream Processing Event Sourcing Command & Query Responsibility Segregation (CQRS) Load Balancing Round-robin Random Weighted Dynamic Parallel Computing Master/Worker Fork/Join MapReduce SPMD Loop Parallelism 14

15. The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery 15

16. The Availability Challenge Availability: Tolerate failures Traditional IT focuses on increasing MTTF Mean Time to Failure Cloud IT focuses on reducing MTTR Mean Time to Recovery What follows is four availability scenarios: [low, high] X [MTTF, MTTR] 16

17. Availability and MTTF, MTTR 17

18. Availability and MTTF, MTTR Uptime 53% 86% 69% 30% 18

19. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% 69% 30% 19

20. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% 20

21. Availability and MTTF, MTTR Traditional IT Uptime 53% 86% Cloud 69% 30% Cloud done wrong 21

22. Design Patterns for Availability Pattern: Replication Pattern: Fail-Over Often used together. 22

23. Availability Pattern: Fail-Over Source: Michael Nygaard 23

24. Availability Pattern: Fail-Over In practice, fail-over is not this simple Source: Michael Nygaard 24

25. Availability Pattern: Fail-Over Source: Michael Nygaard 25

26. Availability Pattern: Fail-Over with Fail-Back Source: Michael Nygaard 26

27. Availability’s Nemesis Single Points of Failure 27

28. SPOT the SPOF* *Single Point of Failure

29. Spot the SPOF: 1 29 Internet Cloud App App Instance

30. Spot the SPOF: 1b 30 Internet Cloud App App Instance

31. Spot the SPOF: 1b 31 Internet Cloud App App Instance

32. Spot the SPOF:2 32 Internet Elastic IP Address Cloud App App App Instance App Instance Fail-over

33. Spot the SPOF:2 33 Internet Might work… Until you need more App instances Or until another SPOF fails… Elastic IP Address Cloud App App App Instance App Instance Fail-over

34. Spot the SPOF: 2a 34 Internet LB Load Balancer Instance Cloud App App

35. Spot the SPOF: 2a 35 Internet LB Load Balancer Instance Cloud App App

36. Spot the SPOF: 3 36 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over

37. Spot the SPOF: 3 37 Internet Elastic IP Address LB LB Availability Zone Replicated configuration Cloud App App Fail-over

38. Spot the SPOF: 4 38 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App

39. Spot the SPOF: 4 39 Internet Elastic Load Balancer (Magic) ELB Availability Zone Cloud App App

40. Spot the SPOF: 5 40 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over

41. Spot the SPOF: 5 41 Internet Elastic IP Address LB LB Availability Zone Availability Zone Replicated configuration Region App App App App Fail-over

42. Spot the SPOF: 6 42 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App

43. Spot the SPOF: 6 43 Internet Elastic Load Balancer (Magic) ELB Availability Zone Availability Zone Region App App App App

44. Spot the SPOF: 7 44 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region Or…

45. Spot the SPOF: 7a 45 Internet LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

46. Spot the SPOF: 7/7a 46 Internet Elastic IPs are single-region only LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

47. Spot the SPOF: 7b 47 Internet ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

48. Spot the SPOF: 7b 48 Internet ELB is single-region only ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

49. Spot the SPOF: 7c 49 Internet DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

50. Spot the SPOF: 7c 50 Internet ELB Can’t Do That Multiple CNAMEs Violate RFC 2181 DNS ELB ELB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

51. Spot the SPOF: 7d 51 Internet DNS LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

52. Spot the SPOF: 7d 52 Internet DNS Cloud Provider LB LB LB LB Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App Region Region

53. Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 53

54. Spot the SPOF: 8 Internet DNS AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 54

55. Spot the SPOF: 8 Internet DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 55

56. Spot the SPOF: 8 Internet Fail-over mechanism DNS and... AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 56

57. Spot the SPOF: 8 Internet Fail-over mechanism DNS and... Ops staff and AWS LB LB LB LB LB Rackspace Availability Zone Availability Zone Availability Zone Availability Zone App App App App App App App App App App Region Region 57

58. Availability: Ensure Redundancies Physical Virtual resource (instance, disk, etc.) Availability zone Region Provider Human (ops staff) 58

59. Availability Best Practice:Chaos Monkey AKA Error Injection Testing Forcibly create fault conditions in your cloud components. Kill instances, detach disks, screw up DNS, etc. Automate recovery from the errors. The team gets really good at reducing MTTR, increasing availability! Popularized by Netflix, who run it on their live environment. 59

60. For more on Designing forAvailability, Scalability Jonas BonérScalability, Availability, Stability Patterns http://slidesha.re/cK3NJv George ReeseThe AWS Outage: The Cloud’s Shining Momenthttp://oreil.ly/eKCGG9 John Ciancutti of Netflix5 Lessons We’ve Learned Using AWShttp://bit.ly/h8rU8b 60

61. Ten Cloud ApplicationDesign Patterns Thank you! Many Shlomo SwidlerFounderOrchestratus shlomo@orchestratus.com @ShlomoSwidler