DevOps. Microservices. Containers. These terms have a lot of buzz for their role in cloud-native application development and operations. But, if you haven't automated your tests and builds with continuous integration (CI), none of them matter. Continuous integration is the automation of building and testing new code. Development teams that use CI can catch bugs early and often; resulting in code that is always production ready. Compared to manual testing, CI eliminates a lot of toil and improves code quality. At the end of the day, it's those code defects that slip into production that slow down teams and cause apps to fall over. The journey to continuous integration maturity has some requirements. Join Pivotal's James Ma, product manager for Concourse, and Dormain Drewitz, product marketing to learn about: - How Test-Driven Development feeds the CI process - What is different about CI in a cloud-native context - How to measure progress and success in adopting CI Dormain is a Senior Director of Product and Customer Marketing with Pivotal. She has published extensively on cloud computing topics for ten years, demystifying the changing requirements of the infrastructure software stack. She’s presented at the Gartner Application Architecture, Development, and Integration Summit; Open Source Summit; Cloud Foundry Summit, and numerous software user events. James Ma is a product manager at Pivotal and is based out of their office in Toronto, Canada. As a consultant for the Pivotal Labs team, James worked with Fortune 500 companies to hone their agile software development practices and adopt a user-centered approach to product development. He has worked with companies across multiple industries including: mobile e-commerce, finance, heath and hospitality. James is currently a part of the Pivotal Cloud Foundry R&D group and is the product manager for Concourse CI, the continuous "thing do-er". Presenters : Dormain Drewitz & James Ma, Pivotal

1. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0
Cloud-Native Fundamentals: Accelerating
Development with Continuous Integration
Dormain Drewitz James Ma
@dormaindrewitz @PioverPi
May, 2018

2. Cover w/ Image
Topics
● Economic Imperative of Automated
Testing
● What is Cloud Native
● CI and Microservices
● CI and DevOps
● CI and Continuous Delivery
● CI and Containers
● Example: Concourse

3. Shifting left and automating QA are
economic imperatives of cloud-
native software development
@dormaindrewitz

4. 4
From idea to production: 6-9 months
@dormaindrewitz

5. It’s all about feedback loops
https://youtu.be/ZQGmtuG0Nx8

6. Tenet: Cost to fix bugs goes up with every handoff
Source: Crossing the Value Stream: Improving Development with Pivotal Cloud Foundry

7. Dividend Yield: Higher quality code, less process waste
Source: Crossing the Value Stream: Improving Development with Pivotal Cloud Foundry

8. Liberty Mutual: Crossing the CI/CD Chasm
https://content.pivotal.io/slides/crossing-the-ci-cd-devops-chasm
Total number of builds per month
Total number
of deploys
per month
2015: Intro of CF, Spring
Boot, and microservices

9. Cover w/ Image
Microservices: Architecture that gives
teams autonomy and independence
DevOps: Culture of shared responsibility for
well-functioning code
Continuous Delivery: Process of
automating the deployment of software at
anytime
Containers: Technology that homogenizes
infrastructure
What is Cloud-Native?

10. Cover w/ Image
Microservices: Architecture that gives
teams autonomy and independence
● How do we safely decompose code?
DevOps: Culture of shared responsibility for
well-functioning code
● Well-functioning code is tested code
Continuous Delivery: Process of
automating the deployment of software at
anytime
● Is the code ready to deploy at anytime?
Containers: Technology that homogenizes
infrastructure
● How can we use containers to test?
Cloud-Native: Testing is
key

11. Microservices
Continuous Integration

12. When you're strangling out a
microservice, what you’re really
doing is actually figuring out those
high level functional tests that you
need to maintain and make sure that
they're still working.
- Cassandra Shum, ThoughtWorks
https://softwareengineeringdaily.com/2017/05/22/microservices-transition-with-cassandra-shum/

13. “As you're strangling [a monolith],
you are writing unit tests as you're
rewriting that code into that
microservice”
- Cassandra Shum, ThoughtWorks
https://softwareengineeringdaily.com/2017/05/22/microservices-transition-with-cassandra-shum/

14. Cover w/ Image
Why TDD
Writing a test first helps:
● Gain the CONFIDENCE you need to
REFACTOR your code to keep it
CLEAN so that you can GO FAST
FOREVER.
● Tease out and think through your APIs.
● Clarify exactly what behavior you’re
trying to build.
● Know when you’re done.
● Triangulate on simple, maintainable
implementations, by making each test
pass one by one.
● Write just enough code to make each
test pass.Matthew Kane Parker, Head of Engineering, Pivotal Labs
https://builttoadapt.io/why-tdd-489fdcdda05e

15. But What About Microservices Complexity?
What happens when I have to test LOTS of microservices… together?
https://www.slideshare.net/apigee/is-microservices-soa-done-right

16. Rather than a myriad of
checkboxes, pipelines are defined
as a single declarative config file,
composing together just three core
concepts.
As your project grows, your
pipeline will grow with it, and
remain understandable.
Simple and
Scalable

17. DevOps
Continuous Integration

18. Culture of shared responsibility for well-functioning
code
https://springoneplatform.io/sessions/take-devops-to-11-and-sprinkle-cloud-on-it-with-rainbows-and-unicorns

19. It’s a lot of work, but it pays off
2017 State of Devops Report
By Puppet and DORA [1]
“High” IT performers “Low” IT Performers
Deployment
frequency
On-demand
1 week - 1 month
(quarterly or worse)
Lead time for
changes
< 1 hr 1 week - 1 month
Mean time to
recovery
< 1 hr 1 week - 1 month
Change Failure
Rate
0 - 15% 31 - 45%
[1] https://puppet.com/resources/whitepaper/state-of-devops-report

20. CI: Top Technology/Automation to Scale DevOps
Initiatives
Source: Gartner, “Build Continuous Quality Into Your DevOps Toolchains”, 13 March 2018, Gartner Figure 4

21. Embrace (and invest) in Test and Release Automation
@dormaindrewitz
https://content.pivotal.io/springone-platform-2017/take-devops-to-11-and-sprinkle-cloud-on-it-with-rainbows-and-unicorns-matt-curry

22. The point of tests is to find failures.
And then fix them.

23. Cover w/ Image
Making Defects Visible
● green means good
● red indicates a problem
● yellow indicates a step is actively
executing
● Teams quickly aligned on issues with
context

24. Continuous Delivery
Continuous Integration

25. Enterprises are
deploying faster
Scotiabank deploys over 3,000 times a month in 10
months after adopting PCF
Garmin deploys to production 1,100 times per months
The Home Depot ships to production 1,500 times a
month, and 17,000 times a month to all environments.
Liberty Mutual deploys 1,000 times a day to production
on 2,500 daily builds
But what is their secret?

26. The Deployment Equation
Micro-
services
Automated
Pipelines
Self-service
Infrastructure
Deploys

27. The Deployment Equation
Micro-
services
Automated
Pipelines
Self-service
Infrastructure
Deploys
Test-Driven Development
- Increase test coverage
- Co-locate cognitive
work
- Reduce hand-off waste
Continuous Integration
- Automate
undifferentiated heavy
lifting
- Code always ready for
deployment to
production
Visibility
- Team alignment
- Minimize interpretation
overhead

28. Delivery is hard
How do we make sure our code works in
production-like environments?
● Test the code in a production-like environment
● Use automated tests to verify correct business functionality and
integration with other services
● Apply additional checks for security, compliance, etc.
Solution: Continuous Delivery!
“Continuous Delivery
doesn't mean every
change is deployed to
production ASAP. It
means every change is
proven to be
deployable at any
time”
Carl Cacum[1]

29. Cerner: Modeling Processes into Pipelines
https://content.pivotal.io/inspiring-use-cases/concourse-in-the-real-world-a-case-study-in-ci-cd-and-devops-greg-meyer-bryan-
kelly-cerner

30. Cerner: Modeling Processes into Pipelines
https://content.pivotal.io/inspiring-use-cases/concourse-in-the-real-world-a-case-study-in-ci-cd-and-devops-greg-meyer-bryan-
kelly-cerner
“The cool thing you can do inside of Concourse is
that all these ISO process pieces and artifacts
can just become a Resource.”
- Greg Meyer, Cerner Corp.

31. Containers
Continuous Integration

32. Rather than sharing state, every
task runs in its own container,
controlling its own dependencies.

33. Dependability
Through
Containerization
> test.sh
Rather than sharing state, every
task runs in its own container,
controlling its own dependencies.

34. Maintaining the testing
infrastructure itself shouldn’t be
your job.

35. Concourse controls the inputs to
your pipeline so that the results are
repeatable every time.
Rather than sharing state, every
task runs in its own container,
controlling its own dependencies.
Dependable
Results

36. Concourse
One option for Continuous Integration

37. Pipelines are defined as a single
declarative config file composing
together just three core concepts.

38. Resources
● Track versions of
external artifacts used for
CI / CD
● Any entity that can be
checked for new versions,
pulled down at a specific
version, and/or pushed to
● Can by one of many types
of built in resources; git
repositories, Amazon S3
Buckets, Docker Images,
or a custon
implementation
Three Core Concepts
Tasks
● Allow the execution of
arbitrary scripts against a
set of resources.
● Can output directories
representing a new
resource version.
● Run in a container using a
configurable container
image.
Jobs
● Represent the plan for a
build step within a
pipeline.
● Can contain operations
against resources, or
tasks as steps.
● Builds of a job’s plan can
be triggered manually or
trigger on new versions of
resource.

39. Fly your friendly Concourse pipeline
Flexible integration
of resources
Simple modeling of
components
Pipeline status is
immediately visible
Build components are
expressed as code

40. Examples

43. Transforming How The World Builds Software
© Copyright 2017 Pivotal Software, Inc. All rights Reserved.

44. CN: Concepts, Goals, Challenges, Advantages
Microservices
(Architecture)
DevOps
(Culture)
Continuous Delivery
(Process)
Containers
(Technology)
Decompose safely
Automate
everywhere
Deployable always
Isolate workloads
Maintain
understandability
Embrace
unpredictability
Dependency
flexibility
Maintain portability
Fan-Out/Fan-In
Usable Interface
Simple extensibility
Cloud-neutral