1. C o n t i n u o u s D e l i v e r y i n P r a c t i c e
FINE-TUNING OF AGILE
DEVELOPMENT
Isa Goksu, Cengiz Han

2. ABOUT US
§ Isa Goksu, @IsaGoksu
Tech Principal
§ Cengiz Han, @hancengiz
Tech Lead
© 2014 ThoughtWorks, Inc. All rights reserved. 2

3. RECOGNIZE THIS?
§ Single line of change takes a week to deploy
§ Slow Feedback (10 weeks after delivering the
features, receiving your first feedbacks)
§ You started a new project, defect from old
project is bugging you
§ Code Freeze?
§ Sleepless nights during deployments
© 2014 ThoughtWorks, Inc. All rights reserved. 3

4. “How long would it take your
organization to deploy a change that
involves just one single line of code?”
Mary and Tom Poppendieck
© 2014 ThoughtWorks, Inc. All rights reserved. 4

5. THE CONTINUOUS DELIVERY MATURITY MODEL
© 2014 ThoughtWorks, Inc. All rights reserved. 5

6. Where to start?
© 2014 ThoughtWorks, Inc. All rights reserved. 6

7. 1
7
make it always production ready
© 2014 ThoughtWorks, Inc. All rights reserved.

8. TRUNK-BASED DEVELOPMENT
8
§ One and only one branch
§ Continuously Integrate with the team
§ All the time production ready
§ At least 1-commit per day per pair/dev
§ Quality increase

9. FEATURE TOGGLING
§ Helps you to have production-ready code
all the time
§ Gives you the flexibility of trying out
multiple features
§ QA can test individual features or group
of features
§ Business can get fast feedback thru
feature previewing
© 2014 ThoughtWorks, Inc. All rights reserved. 9

10. 10
Features developed
in an iteration
Enabled features
in production
© 2014 ThoughtWorks, Inc. All rights reserved.

11. Many more..
C# - FeatureSwitcher
Golang - flag
Python - Waffle
Ruby - Feature-Toggles
Node.js - feature-flags
11
© 2014 ThoughtWorks, Inc. All rights reserved.

12. FEATURE BRANCHING
§ Avoid it!
§ CI is a problem
§ Long-lived
§ Merging hell
§ Works on
§ Small teams
§ Short-lived
© 2014 ThoughtWorks, Inc. All rights reserved. 12

13. 2
13
build quality in
© 2014 ThoughtWorks, Inc. All rights reserved.

14. TEST STRATEGY
§ Make the QA the key participant of your
flow
§ QAs are not just testers (assuring quality)
§ Enforces you to deliver vertical slices
§ Consider using new approaches like
Specification by Example, Generative
Testing, Parametric Testing)
§ Automation is a must
© 2014 ThoughtWorks, Inc. All rights reserved. 14

15. © 2014 ThoughtWorks, Inc. All rights reserved. 15

16. © 2014 ThoughtWorks, Inc. All rights reserved. 16

17. 3
17
keep everything in source control
© 2014 ThoughtWorks, Inc. All rights reserved.

18. STORING DB CHANGES IN SCM
§ Start with a clean DB
§ Use proven tools (dbdeploy, liquidbase, etc)
§ Make sure deltas are incremental, small and
immutable
§ Have rollback scripts ready and auto rollback if
possible, or do only roll-forward
§ Fail directly if any delta fails
§ Run each delta in order
§ Stored together
© 2014 ThoughtWorks, Inc. All rights reserved. 18

19. 19
DBDeploy
Metadata
Baseline
Database
Apply Deltas
Test
Apply Deltas Apply Deltas Apply Deltas Apply Deltas
Fail
Fast
© 2014 ThoughtWorks, Inc. All rights reserved.

20. © 2014 ThoughtWorks, Inc. All rights reserved. 20

21. INFRASTRUCTURE AS CODE
§ Determinism vs. Indeterminism
§ Treat your infrastructure as you are
coding
§ Helps you to have system consistency
§ Use proven tools (puppet, chef, ansible,
bcm, cfengine, etc)
§ Have a test machine/vm to try
© 2014 ThoughtWorks, Inc. All rights reserved. 21

22. 22
System Config.
SCM Puppetmaster
WebServers AppServers WebServers AppServer s OOththeerSrSeervreversr s
© 2014 ThoughtWorks, Inc. All rights reserved.

23. 23
Puppet example
manifest for
Apache HTTP
server
© 2014 ThoughtWorks, Inc. All rights reserved.

24. TEST IT BEFORE GET IT OUT
© 2014 ThoughtWorks, Inc. All rights reserved. 24

25. 4
25
repeatable, risk-free deployments
© 2014 ThoughtWorks, Inc. All rights reserved.

26. UNIFIED WAY
§ Build one binary and use it everywhere
§ Deploy the same way to all environments
§ Unattended deployments
§ Separate the things that change from the
thing that don’t
© 2014 ThoughtWorks, Inc. All rights reserved. 26

27. VIRTUALIZATION
§ Use one
§ Containers vs Hypervisors
§ Availability, Fail-safe
§ Fast provisioning
§ You can automate it!
© 2014 ThoughtWorks, Inc. All rights reserved. 27

28. PACKAGING
28
RPM | CHOCOLATEY
sudo rpm –ivh awesome-app-1.0.rpm
Cinst awesome-app –version 1.0
© 2014 ThoughtWorks, Inc. All rights reserved.

29. 5
29
everybody is responsible
© 2014 ThoughtWorks, Inc. All rights reserved.

30. MTBF, MTRS
§ Failure is inescapable just like getting sick
§ Self-healing systems (auto-scale, retries, etc)
30
© 2014 ThoughtWorks, Inc. All rights reserved.

31. IMMUNE SYSTEMS
§ Zero access policy (phoenix or immutable
servers)
§ Monitoring (graphite, riemann, munin,
nagios, etc)
§ Logging (ELK clusters, Graylog2, syslog,
etc)
§ Application Status / Healthcheck /
Configuration Endpoints
© 2014 ThoughtWorks, Inc. All rights reserved. 31

32. 32
Status Endpoint Healthcheck Endpoint
GET http://
my.service/status
GET http://my.service/
healthcheck
© 2014 ThoughtWorks, Inc. All rights reserved.

33. SUMMARY
§ DONE means released!
§ Automate everything
§ Be always production ready
§ If anything fails, stop the delivery line
§ Keep everything in SCM
§ Build binaries only once
§ Use precisely the same mechanism to the every
environment
© 2014 ThoughtWorks, Inc. All rights reserved. 33

34. Q/A
Ping us later @IsaGoksu, @hancengiz
© 2014 ThoughtWorks, Inc. All rights reserved. 34