More Related Content Similar to DVC303-Technological Accelerants for Organizational Transformation (20) More from Amazon Web Services (20) DVC303-Technological Accelerants for Organizational Transformation1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
DVC303: Technological Accelerants
for Organizational Transformation
2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ant Stanley
Organizer Serverless London
User Group, co-founder JeffConf
Paul Duvall
CTO Stelligent, AWS Community
Hero
Ben Kehoe
Chief Cloud Robotics Research
Scientist, AWS Community Hero
3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“There is no such thing as a new idea.”
Mark Twain
4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ephemeralization - “more and more with less and
less until eventually you can do everything with
nothing”
Buckminster Fuller
5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Everything is getting smaller
6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Unit of Deployment
Entire Machine Operating System Virtual
Machine
Container Function
7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Time to Deploy
Years Months Days Hours Minutes Seconds
8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Average Lifespan
> 5 Years < 5 Years < 3 Years Days Seconds
Entire Machine Operating System Virtual
Machine
Container Function
9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Frequency of Deployment
Years Months Days Hours Minutes
10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scope of Change
Entire Platform Operating System Major
Version
Minor
Version
Commit
11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jidoka – Automation with a human element
Invented by Sakichi Toyoda circa 1902
12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Change Advisory Board
Network Security
Storage
Sysadmin
13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Network Security
Storage
Sysadmin
14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Teams replaced by services
Consistent standardized interface
API driven
Idempotent
16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Large array of services automated by a
small team
Focus on delivering value
Takes full responsibility for service
17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“AWS will release a better feature than your
horrific workaround”
Quinn’s Law, 2017
20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“Any organization that designs a system
(defined broadly) will produce a design whose
structure is a copy of the organization's
communication structure”
Mel Conway (Conway’s Law, 1968)
21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“Your AWS Organization structure will replicate
your company’s communication structure”
Conway’s Law for AWS (2017)
25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
New (old) challenges
27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Persephone (FSV304)
Creates accounts, sets policy, defines which services are enabled in
the account, and runs Lambda functions to ensure continuous
compliance.
Cloud Custodian
Custodian can be used to manage AWS accounts by ensuring real
time compliance to security policies (like encryption and access
requirements), tag policies, and cost management via garbage
collection of unused resources and off-hours resource management.
https://github.com/capitalone/cloud-custodian
29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DevOps Transformation Examples
P a u l D u v a l l
30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About Me
@PaulDuvall AWS Community
Hero
Continuous
Integration
@Stelligent
31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DEPLOYMENT PIPELINE
FEEDBACK LOOPDevelopers Customers
Based on http://www.slideshare.net/AmazonWebServices/dvo202-devops-at-amazon-a-look-at-our-tools-processes
≈≈≈≈≈
DevOps is About Feedback
Build Test Release
Plan Monitor
32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Organizational Cultural Process Tooling
Principle: Feedback
33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Four DevOps
Transformation Examples
1 2 3 4
Breaking Down Silos Identifying Process
Bottlenecks
Making Work Visible Fixing Problems Early
https://stelligent.com/dvc303
34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
App
Release Management
QA/Testers
Production
Support
DBAs
App Developers
UI/UX
Analyst
Build
Gate(s)
Gate(s)
Gate(s)
Change
Advisory Board
Security
Gate(s)
Problem: How to Breakdown Silos?
35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Product
Security
Tooling
Spend Management
Continuity Management
API
API
API
API
App
Developers
UI/UX
Analyst
Test/QA
Infrastructure
Build | Test | Deploy | Run | Support
Example: Organize Around
API-Enabled Services
36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: API-Enabled Services
https://github.com/stelligent/infrastructure-api
Image from https://aws.amazon.com/blogs/compute/how-to-provision-complex-on-demand-infrastructures-by-using-amazon-api-gateway-and-aws-lambda/
Amazon API
Gateway
Lambda
Function
CloudFormation
Business-Function
Template
CloudFormation
Stacks
1 2 3 4
37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Problem: How to Make Work Visible
38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Amazon CloudWatch
Dashboard
39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Amazon CloudWatch
Dashboard
CodePipeline
CloudWatch
Event Rule
Lambda
Function
CloudWatch
Metrics
CloudWatch
Dashboard
Public
Events
Target Put-Metric
Get-
Statistics
CloudWatch
Event Rule
Lambda
Function
CloudWatch
Scheduled Event
CloudWatch
Dashboard
List-Metrics Trigger
Put-
Dashoard
40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Community: Pipeline Dashboard
https://github.com/stelligent/pipeline-dashboard
41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VALUE
WASTE
5 min
15 min ½ week
2 min
2 weeks
15 min
2 weeks
2 hours
1 week
15 min
3 hrs 45 min ½ week
3 min 2 hrs 40 min
6 weeks + 4 hrs
Extra 15 minutes to fill
out request form
Only 15 minutes of 4
hours should be needed
to verify
Source: Implementing Lean Software Development: From Concept to Cash.
Boston: Addison-Wesley Professional; 2006, ISBN: 0321437381
Problem: How to Identify Process
Bottlenecks?
Request
Approve
and
Prioritize
Technical
Assessment
Code and
Test
Verify
and Fix
Depoy
Form Sent
to Queue
To
Verification
To
Operations
1%
Efficiency
Weekly review of
requests means an
average wait of ½ week
Wait an average of 2
weeks for an architect
Wait an average of 2
weeks for developers
Biweekly releases means
a wait of an average of 1
week for verification
Form Sent
to Queue
Form Sent
to Queue
42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: Value-Stream Map
as a Pipeline
43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example: CodePipeline in
CloudFormation
My SNS Tpoic
Topic
CodeBuild
Role
Lambda
Role
CodePipeline
Role
Bucket
Bucket
CodeBuild
Project
CodeBuild
Role
CodePipeline
Parameter
EventRule
Rule
CloudFormati
on
Role
CodePipeline
Pipeline
44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Community: CodePipeline Mocks
https://github.com/stelligent/mu
https://github.com/stelligent/mock-pipeline
45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Problem: How to Fix Problems Early?
46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solution: Stop the Line
47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Community: CodePipeline SNS
Notifications
https://github.com/stelligent/cfn_nag
https://github.com/stelligent/devops-essentials/tree/master/samples/ec2
48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Additional Resources
https://stelligent.com/dvc303
1 2 3 4
Breaking Down Silos Identifying Process
Bottlenecks
Making Work Visible Fixing Problems Early
49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
iRobot
B e n K e h o e
50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ben Kehoe
• Cloud Robotics Research Scientist
at iRobot
• Serverless evangelist
• AWS Community Hero
About me
@ben11kehoe
51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud Robotics:
Connecting robots to the
internet to help them do
more and better things
53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2015
54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Then
58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Then Now
59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Then Now Future
60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Then Now Future
2015
61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Event-driven
• Scalable
• Lean for device makers
• Reverse: AWS Greengrass
• Focus
IoT + serverless: a natural fit
62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS IoT + serverless: an even naturaler fit
63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Serverless itself
• Event-driven
• Scalable
• Integrates with AWS ecosystem
• Device shadows
• Integrates with your process
AWS IoT
64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Fully serverless production cloud
• 2 million connected robots by 2018
• Mostly serverless analytics
platform
• Basis for future data-powered
platform
Long story short: success!
65. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• No provider is immune to
problems
• Small effects are more common
than big outages
• More services = blips could be
encountered more frequently
• This comes with the territory
• Set expectations internally
• Architecting robustly is key
The cloud has weather
66. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• You only know what the provider tells you
• Architecture
• Security
• Operations
• How do they actually do all of the stuff
they do?
• Many known unknowns and
unknown unknowns
• Unknown unknown unknowns: what you
don’t know that they don’t know they don’t
know
Visibility
67. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Heed the warning
• Information flow is different in
serverless architecture
• Organization must change for
architecture to succeed
Conway's Law
https://commons.wikimedia.org/wiki/File:Gospers_glider_gun.gif
68. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Developers can be platform testers and
canaries
• Same environment in the platform as
production
• Orders of magnitude more churn
• Exercises the account limits
• Tests metrics to determine relevance
and meaning
• Bonus – Developer activity provides
additional visibility into how the platform is
currently behaving
• Higher volume of deployments in
many different AWS accounts means
problems found quickly
• This can alert us prior to problems
hitting prod
DevOps
69. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Servers → serverless is like
on-prem → cloud
• Easier overall and in most
respects
• But also includes new
challenges
• Outsourcing doesn’t mean you
do zero work
• Being clear about this
organizationally is critical
DiffOps
70. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Everything is getting smaller
• How DevOps and Serverless affect and
improve organizational transformation
• Try some the examples:
stelligent.com/dvc303
• Become part of the AWS Community
Summary and Next Steps
71. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Q&A