Cloud engineering teams at Corteva Agriscience, Agriculture Division of DowDuPont, have a challenge: how to support a global business of research scientists and software developers in building a world-class innovation organization.  Modern agriculture produces larger and more varied data types, so their approach must be not only scalable and flexible, but also commit to operational excellence while remaining adoptable. This session will walk through how Corteva Agriscience builds container-based infrastructures with CI/CD pipelines that remove undifferentiated heavy lifting and allow teams to empower developers. Members of the cloud engineering team will discuss problems they face, solutions they implement, and show an example of how they leverage AWS services (AWS CodeCommit, AWS CodePipeline, AWS CloudFormation, AWS Fargate) to deploy a novel machine learning algorithm for scoring genetic markers.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Remove Undifferentiated Heavy
Lifting from CI/CD Toolsets with
Corteva Agriscience
Randy Black
Lead Cloud Architect
Corteva Agriscience
D E V 3 4 8
Duke Takle
Principal Architect
CMR Solutions
Nathan Taber
Sr. Product Marketing Manager
AWS

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our leadership principles shape our approach
Customer obsession
Start with the customer and work backwards
Earn and keep customer trust
Invent and simplify
Always find ways to simplify
Look for new ideas everywhere
Bias for action
Speed matters in business
We value calculated risk taking

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“
”
—
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we focus on the customer
ExperimentIterate
Listen
Innovation
flywheel

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon builds for rapid experimentation
Try a lot of
experiments
Eliminate the
collateral damage
from failed experiments

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We build for rapid experimentation by …
• Simplifying infrastructure management
• Componentizing applications
• Standardizing and automating operations
• Improving application performance
• Creating a culture of innovation
• Updating applications and infrastructure quickly
• Ensuring trust
This is modern application development

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How Amazon builds and runs modern applications
Microservices
Componentization via services
Organized around business capabilities
Products not projects
Infrastructure automation
Serverless
No infrastructure to provision or manage
Auto scale by unit of consumption
Pay for value billing model
Built-in availability and fault tolerance
DevOps
Cultural philosophies
Cross-disciplinary teams
CI/CD
Automation tools

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Companies in every segment are building modern applications
EnterprisesStartups B2B B2C Verticals

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Companies in every segment are building modern applications
Agriculture

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Undifferentiated heavy lifting
“Stop spending money on undifferentiated heavy lifting.”
“All this heavy lifting is taken out of your hands”
“We thrive on feedback how to build better services.”
Dr. Werner Vogels, CTO of Amazon
https://www.cio.co.nz/article/466635/amazon_cto_stop_spending_money_undifferentiated_heavy_lifting_/

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Managed services
What makes them worthwhile
• Relatively easy to implement
• Little to zero FTE involvement to maintain
• Pay as you use

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Managed services
How to choose
• Does the service exist
• What is your expectation of the service
• What percentage of needs are met
• We use a goal of 80%

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What drivers affect our decisions

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What drivers affect our decisions
• Large and more varied data types

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What drivers affect our decisions
• Large and more varied data types
• Scalable and flexible

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What drivers affect our decisions
• Large and more varied data types
• Scalable and flexible
• Operational excellence

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What drivers affect our decisions
• Large and more varied data types
• Scalable and flexible
• Operational excellence
• Easily adoptable

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Low cost
• Zero maintenance
• Little to zero downtime provider SLA
• Faster realization – Months to days
• Trivial implementation
• Rapid change and deploy/redeploy
• No context shift – Focus for developers
Observed benefits

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Slow adoption
• Not built here
• Inexperience
• Myopic
• In-house options expensive
• Cloud is hard
Lessons learned

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuing our process

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuing our process
• More active guidance and evangelism

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuing our process
• More active guidance and evangelism
• Continued feature requests; make them known to AWS

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuing our process
• More active guidance and evangelism
• Continued feature requests; make them known to AWS
• Results are in – Other teams/projects help drive adoption

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Duke Takle
Principal Architect
CMR Solutions
Slides available at https://www.slideshare.net/RandyBlack1
Code available at https://github.com/cmrsol/example

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Problem
Implement a highly available genetic analysis API

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Implement a highly available genetic analysis API
• Proof-of-concept had been done with a Docker container
• Must avoid undifferentiated heavy lifting
• Solution must be totally defined in code
• Solution must be containerized
• Solution must be automatically built and deployed
• Developers may have control of CI/CD and visibility into
orchestrated process

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution - Genetic analysis API
This is the “secret sauce” part of the problem

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure
● Utility account for management
● Deployed from utility account
● Hub and spoke design

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure
● Utility account for management
● Deployed from utility account
● Hub and spoke design

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure
● Utility account for management
● Deployed from utility account
● Hub and spoke design

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure
● Exports live in all accounts

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS account structure
● Parametrized AWS CloudFormation templates

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Completely defined in code
• Strong bias toward AWS CloudFormation
• Layered and connected with imports/exports
• One static template for all environments
• When AWS CloudFormation falls short, boto3
• Number of libraries and utilities
• Key is extracting commonality to utilities
• Use of console outside “sandbox” is discouraged

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Containerized
• Containers are cliché for a reason
• Our images built/stored in the central account’s Amazon Elastic
Container Registry (Amazon ECR)
• Images promoted not rebuilt in our SDLC
• This solution used an Amazon Elastic Container Service (Amazon
ECS) Fargate cluster

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automated build & deploy

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Detailed example
● The rest of the time ...

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Management Console detail

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Images at: http://static.mknote.us/re/invent2018

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Images at: http://static.mknote.us/re/invent2018

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Images at: http://static.mknote.us/re/invent2018

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Images at: http://static.mknote.us/re/invent2018

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The anatomy of most of my pipelines:
• 2 … n stages
• Each stage has 1 … n actions that do the real work
• Actions are where the AWS CodeBuild things are put into play
Create the AWS CodePipeline

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What happens in all the CodeBuild projects

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Describing what happens in all the CodeBuild projects

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Slides Available at https://www.slideshare.net/RandyBlack1
Code Available at https://github.com/cmrsol/example

63. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Randy Black
randy.black@pioneer.com
Duke Takle
duke@cmrsol.com

64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.