DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Using AWS to Achieve Both
Autonomy and Governance at 3M
N a t h a n S c o t t , S e n i o r C o n s u l t a n t , C l o u d A r c h i t e c t , A W S
J a m e s M a r t i n , M a n a g e r , A u t o m a t i o n E n g i n e e r i n g , 3 M
C a s e y L e e , C h i e f A r c h i t e c t , S t e l l i g e n t
AWS re:INVENT
D E V 3 3 2
N o v e m b e r 2 8 , 2 0 1 7

The problem
We have to move FAST, give us access…
Individual business and development teams needed the ability
to move fast and self-serve to capture market opportunities
Not so fast, there are rules…
The organization as a whole needs governance to ensure security
compliance and minimize risk

What to expect
We will hear firsthand from
About
• How we solved the problem and achieved balance between
autonomy and governace at 3M

What to expect
Topics
• People, process, technology
• Governance requirements
• Solution approach using:
• Continuous delivery
• Self-service model
• Monitoring of resources

5
Historical business
Our legacy 1983–2011
Helping healthcare organizations
get complete and accurate
reimbursement and mitigate
compliance risks
Streamlining and simplifying the
process of documenting the
patient’s encounter
in a hospital
Working with hospitals
to efficiently access, compile,
code, classify, report, store,
and exchange health information

6
Leading in a changing landscape
Our present course and future
Analyzing the cost, quality, and
outcomes data of both
patients and populations
over time and across the
healthcare continuum
Ensuring providers capture the
full burden of illness of their
patients to deliver effective
care management and receive
accurate and complete
payment
Measuring performance and
effectiveness among payer and
provider networks to deliver
higher quality outcomes at
lower total costs

73M Confidential.
3M HIS grouper applications
22 states (27 grouper adoptions) through 1983–2006
11 additional states (37 grouper adoptions) 2007–2010
6 additional states (33 grouper adoptions) 2011–Q3 2012
• Industry-recognized expertise
in payment methodologies and
patient classification
• 24 states have adopted APR
DRGs for payment, including
the eight largest Medicaid
programs in the country
• The APR DRG adoption by
payers typically yields over
75% downstream penetration
with providers
• Lays a foundation for further
payment products
87%
of the US
population is
covered by 3M
patient
classification
systems

8
Not moving fast enough
Lift and shift got us out of the traditional data center, but…
Lots of software is getting built with nowhere to go, so it’s time to evolve
again.

9
Development bottlenecks
Development time
Manual testing
Manual QA
Manual deployment

10
Desired bottlenecks
Development time
Manual testing
Manual QA
Manual deployment

Deployment pipeline
Feedback loop
plan monitor
build test release
Developers Customers
Based on slideshare.net/AmazonWebServices/dvo202-devops-at-amazon-a-look-at-our-tools-processes
Continuous delivery

12
The path to continuous delivery

13
Building the automation team
Automation engineering team
• Deep knowledge of AWS services
• Comfortable talking to other
development teams
• Understands the complete development
lifecycle—from commit to deploy

14
Choosing the right technology
• Focus on the problem at hand
• Don’t try to predict the future
• Use native AWS services/AWS
Lambda/software as a service
(Saas) services

15
Working with security
• Gain buy-in early
• Security from the start
• Security as consumers
• Freedom (with guard rails)
• Sensitive data

16
• Find a simple application
• Just enough to prove your pipeline
• Rinse, repeat
The right services and teams

17
The right services and teams
Find the hungry team that
• Wants the power
• Is willing to do the work
• Has a champion
• Has the business need

18
Embed with the AppDev team
• Establish success criteria
• Works closely with application team
• Participates in the team’s sprint cycle
• Helps AppDev team consume the pipeline process and tools
AppDev
team
Automation
engineering

19
Establishing a CI/CD process at scale
Problems
• Complex components
• Special snowflakes
• Limited governance

20
The pipeline factory
Goals
• Reduce barrier to entry
• Reduce snowflakes
• Reduce setup time
• Enforce security controls

Solution
Self-serviceContinuous delivery Monitor

Continuous delivery

Continuous
delivery

AWS
CodePipeline
Continuous delivery

AWS
CodeCommit
Continuous delivery

mavenJob(jobName) {
triggers {
scm('* * * * *')
}
rootPOM('pom.xml')
goals('clean')
goals('compile')
goals('pmd:pmd')
goals('findbugs:findbugs')
goals('package')
}
Continuous delivery

Jenkins
Continuous delivery

Approved?
Continuous delivery

AWS
CloudFormation
Continuous delivery

AWS
CodeDeploy
Continuous delivery

Automated testing
Continuous delivery

Blue/green switch
Continuous delivery
ELB
Old ASG New ASG

Self-service

docs.aws.amazon.com/servicecatalog/latest/adminguide/admin-overview-workflow.html
AWS Service Catalog

Demo
Self-service

ProdBuckets:
Type: Custom::CloudFormationStack
DeletionPolicy: Retain
Properties:
ServiceToken: arn:aws:...
AssumeRole: arn:aws:iam:...
TemplateURL: https://s3.amazonaws.com/.../buckets.yml
Parameters:
TeamName: !Ref TeamName
CIAccount: !Ref CIAccount
TestAccount: !Ref TestAccount
ProdAccount: !Ref ProdAccount
Self-service

Monitor—cfn_nag
https://github.com/stelligent/cfn_nag

https://github.com/capitalone/cloud-custodian
Monitor—Cloud Custodian

- name: s3-global-access
mode:
type: cloudtrail
events:
- event: PutBucketACL
resource: s3
filters:
- type: global-grants
allow_website: false
actions:
- delete-global-grants
- remove-website-hosting
- type: notify
to:
- resource-owner

- name: cost-center-absent-ec2
mode:
type: periodic
schedule: cron(30 * * * ? *)
resource: ec2
filters:
- and:
- tag:Cost Center: absent
- tag:Team: absent
actions:
- stop
- type: mark-for-op
op: terminate
days: 2

Pipeline
Invokes
Governor
stack
AWSGovernor AWSGovernor
TestTypeCatalog
Organization
Level Tests
Policy Service
1. Execute CloudFormation
2. Run AWSGovernor
3. Describe stack resources
4. Get all registered tests
5. Run organization tests
6. Run product tests
7. Report success or failure
Pipeline
Deploys App
Infra/Code
Tools Account
Security Account
Product
Level Test
1
2
3
4
5
6
7
Governor
Production Account

Monitor—pipeline dashboard

ServerlessFunc:
Type: 'AWS::Serverless::Function'
Properties:
Handler: handler.pipeline_event
Runtime: python3.6
CodeUri: ../app
Events:
PipelineEventRule:
Type: CloudWatchEvent
Properties:
Pattern:
source:
- "aws.codepipeline"
detail-type:
- "CodePipeline Pipeline Execution State Change"
- "CodePipeline Stage Execution State Change"
- "CodePipeline Action Execution State Change"

DashboardFunc:
Type: 'AWS::Serverless::Function'
Properties:
Handler: handler.dashboard_event
Runtime: python3.6
CodeUri: ../app
Events:
DashboardEventRule:
Type: Schedule
Properties:
Schedule: "cron(*/5 * * * ? *)"

What we learned
Autonomy
• Achieved through continuous delivery and self-
service
• All infrastructure defined as code
• All deployments done via a pipeline
• Pipeline is triggered via commit
• Only manual step is approve/reject
• Use ServiceCatalog to enable self service

What we learned
Governance
• Achieved through self-service and monitoring
• CloudFormation all the things
• ServiceCatalog for CloudFormation governance
• CloudFormation static analysis with cfn_nag
• Cloud Custodian to assess and enforce compliance
• Monitor pipeline metrics

Resources
https://stelligent.com/dev332

Thank you!

DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M

Similar to DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M (20)

More from Amazon Web Services

More from Amazon Web Services (20)

DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M