Asgard, the Grails App that Deploys Netflix to the Cloud
•Download as KEY, PDF•
42 likes•30,563 views
Overview and technical exploration of Asgard, a graphical web console created by Netflix for cloud deployments and operations. Presented at the GR8 Conference in Copenhagen, Denmark, June 8, 2012. Apple Keynote version with animations available at http://bit.ly/asgardgr8denmark
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Asgard, the Grails App that Deploys Netflix to the Cloud
Similar to Asgard, the Grails App that Deploys Netflix to the Cloud (20)
Recently uploaded
Recently uploaded (20)
Asgard, the Grails App that Deploys Netflix to the Cloud
- 1. // Feed your brain GR8Conf Europe 2012 gr8.technologies.each{ June 6th-8th yourBrain << it Copenhagen, Denmark } The Grails App that Deploys Netflix to the Cloud Joe Sondow, Netflix @joesondow #gr8conf
- 2. Slides online http://slideshare.net/joesondow @joesondow #gr8conf
- 3. Agenda
- 4. Agenda Who am I? Why am I here?
- 5. Agenda Who am I? Why am I here? Asgard, Screen shots
- 6. Agenda Who am I? Why am I here? Asgard, Screen shots Netflix, Freedom and Responsibility
- 7. Agenda Who am I? Why am I here? Asgard, Screen shots Netflix, Freedom and Responsibility Regional URLs
- 8. Agenda Who am I? Why am I here? Asgard, Screen shots Netflix, Freedom and Responsibility Regional URLs Cloud Deployment Model
- 9. Agenda Who am I? Why am I here? Asgard, Screen shots Netflix, Freedom and Responsibility Regional URLs Cloud Deployment Model Offline Development
- 10. Agenda Who am I? Why am I here? Asgard, Screen shots Netflix, Freedom and Responsibility Regional URLs Cloud Deployment Model Offline Development Why not the AWS Console?
- 11. Who am I? @joesondow #gr8conf
- 12. Who am I? @joesondow #gr8conf
- 13. Who am I? Joe Sondow @joesondow #gr8conf
- 14. Who am I? Joe Sondow Netflix since 2010 @joesondow #gr8conf
- 15. Who am I? Joe Sondow Netflix since 2010 Asgard lead @joesondow #gr8conf
- 16. Who am I? Joe Sondow Netflix since 2010 Asgard lead Grails @joesondow #gr8conf
- 17. Who am I? Joe Sondow Netflix since 2010 Asgard lead Grails jQuery @joesondow #gr8conf
- 18. Why am I here? @joesondow #gr8conf
- 19. Why am I here?
- 20. Why am I here? Sell you something
- 21. Why am I here? Sell you something Discuss business plans
- 22. Why am I here? Sell you something Discuss business plans Answer technical questions
- 23. Why am I here? Sell you something Discuss business plans Answer technical questions Be a smaller fish in AWS
- 24. Why am I here? Sell you something Discuss business plans Answer technical questions Be a smaller fish in AWS Give back to community
- 25. Why am I here? Sell you something Discuss business plans Answer technical questions Be a smaller fish in AWS Give back to community Steal your engineers
- 26. Asgard
- 27. Asgard
- 30. Asgard Auto Scaling Group list
- 31. Asgard Cluster deployment, ready for fast rollback
- 32. Asgard
- 36. Asgard Cloud deployment Cloud management Started 2010 Open source in 2012
- 37. Asgard Cloud deployment Cloud management Started 2010 Open source in 2012 Follow @NetflixOSS
- 38. Asgard Cloud deployment Cloud management Started 2010 Open source in 2012 Follow @NetflixOSS 100’s of Jira tickets
- 39. Asgard Cloud deployment Cloud management Started 2010 Open source in 2012 Follow @NetflixOSS 100’s of Jira tickets Actively developed
- 40. Joe Sondow Clay McCoy Jason Gritman @joesondow @claymccoy @jgritman Asgard Team June 2012 (The dolphin writes integration tests)
- 42. With more than 25 million streaming members in the United States, Canada, Latin America, the United Kingdom and Ireland, Netflix, Inc. (NASDAQ: NFLX) is the world's leading internet subscription service for enjoying movies and TV programs. (from http://ir.netflix.com)
- 45. Freedom and Responsibility Cloud SOA
- 46. Freedom and Responsibility Cloud SOA 100’s of services
- 47. Freedom and Responsibility Cloud SOA 100’s of services Small teams
- 48. Freedom and Responsibility Cloud SOA 100’s of services Small teams Independent releases
- 49. Freedom and Responsibility Cloud SOA 100’s of services Small teams Independent releases Controlled chaos
- 50. Regional URLs
- 51. Regional URLs Let’s see some code
- 52. Regional URLs Cloud object URLs were unique, conventional, intuitive Default Grails URL mapping is excellent
- 53. Regional URLs Later, Netflix needed to use multiple Amazon regions
- 54. Regional URLs Region is another URL dimension Add URL mapping, but don’t break existing links
- 58. Regional URLs Some controllers should never be regional RegionService detects RegionAgnostic annotation
- 60. Cloud deployment model Applications and Clusters
- 62. Cloud deployment model Auto Scaling Group
- 63. Cloud deployment model Auto Scaling Group Launch Configuration
- 64. Cloud deployment model Elastic Load Balancer Auto Scaling Group Launch Configuration
- 65. Cloud deployment model Elastic Load Balancer Auto Scaling Group Launch Configuration Amazon Machine Image
- 66. Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Group Launch Configuration Amazon Machine Image
- 67. Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
- 68. Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
- 69. Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
- 70. Cloud deployment model Elastic Load Balancer Auto Scaling Group Security Instances Group Launch Configuration Amazon Machine Image
- 72. Cloud deployment model Search
- 73. Cloud deployment model API Search
- 74. Cloud deployment model Ratings API Search
- 75. Cloud deployment model Streaming Starts Ratings API Search
- 76. Cloud deployment model Streaming Starts Ratings API Search Autocomplete
- 77. Cloud deployment model Sign Up Streaming Starts Ratings API Search Autocomplete
- 78. Cloud deployment model Sign Up Streaming Starts Ratings Application Application Application API Search Autocomplete Application Application Application
- 80. Inventing the Application Problem: Application is not an Amazon concept Solution: Create an Application domain in SimpleDB Enforce naming conventions on Amazon objects
- 81. Fast Rollback
- 82. Fast Rollback Optimism causes outages
- 83. Fast Rollback Optimism causes outages Production traffic is unique
- 84. Fast Rollback Optimism causes outages Production traffic is unique Keep old version running
- 85. Fast Rollback Optimism causes outages Production traffic is unique Keep old version running Switch traffic to new version
- 86. Fast Rollback Optimism causes outages Production traffic is unique Keep old version running Switch traffic to new version Monitor results
- 87. Fast Rollback Optimism causes outages Production traffic is unique Keep old version running Switch traffic to new version Monitor results Revert traffic quickly
- 88. Fast Rollback
- 89. Fast Rollback api-frontend api-usprod-v007
- 90. Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
- 91. Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
- 92. Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
- 93. Fast Rollback api-frontend api-usprod-v007 api-usprod-v008
- 94. Fast Rollback api-frontend api-usprod-v007
- 96. Inventing the Cluster Problem: Two ASGs with one function but different names Solution: Append version number in reserved format Parse ASG name to determine long-term “cluster”
- 97. Inventing the Cluster Instead of keeping a database in sync, use naming conventions to store the source in truth in Amazon’s API api Application api-usprod Cluster api-usprod-v007 Auto Scaling Group api-usprod-v008 Auto Scaling Group
- 99. Offline Development Makes on a plane
- 100. Offline Development Mock data Mock behavior System property switch offline=true
- 102. Mock Data Parse JSON
- 103. Mock Behavior Override Amazon Java client methods
- 104. System Property grails run-app -Doffline=true
- 105. Why not the AWS console?
- 106. Why not the AWS console? No, not to keep me busy
- 107. Why not the AWS console?
- 108. Why not the AWS console?
- 109. Why not the AWS console? Hide keys
- 110. Why not the AWS console? Hide keys Customize model
- 111. Why not the AWS console? Hide keys Customize model Enforce conventions
- 112. Why not the AWS console? Hide keys Customize model Enforce conventions Automate workflow
- 113. Why not the AWS console? Hide keys Customize model Enforce conventions Automate workflow Log changes
- 114. Why not the AWS console? Hide keys Customize model Enforce conventions Automate workflow Log changes Integrate systems
- 115. Why not the AWS console? Hide keys Customize model Enforce conventions Automate workflow Log changes Integrate systems Create REST API
- 116. @NetflixOSS
- 120. @NetflixOSS Thank you
- 121. @NetflixOSS Thank you Questions? @joesondow slideshare.net/joesondow linkedin.com/in/joesondow
Editor's Notes
- \n
- \n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- Asgard is a Grails application used by Netflix engineers to deploy code changes and manage resources in the Amazon cloud on a massive scale. This presentation will include slides and discussion about aspects of Asgard from both the user and developer points of view.\n\nThe Netflix cloud model of Applications and Clusters built on top of Amazon building blocks\nAsgard&#x2019;s process for large deployments and fast rollback\nOpen sourcing Asgard on github in 2012\nComparison with the AWS Management Console\nTour of the user interface\nPublishing a REST interface API for other tools to call Asgard\nEnhancing Grails URLs to add region-specificity\nUsing external configuration to pull Netflix-specific details out of Asgard\nStoring metadata on cloud objects instead of a database\nUsing closures to help cache massive cloud metadata for multiple Amazon regions\nTango open source icons to improve the user experience\nJavaScript filters for large list pages and drop-downs\nMocking the AWS back-end for offline automated testing\n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n