https://www.meetup.com/Cork-Software-Craftsmanship-Meetup/events/267961979/

1. Cork Software Crafters
Follow us on
Twitter: @CorkSwCraft
Meetup: www.meetup.com/Cork-Software-Craftsmanship-Meetup
Slack: softwarecrafters.slack.com/messages/cork
Continuous Delivery non-breaking changes exercise
25th February 2020
facilitated by Paulo Clavijo Esteban
Thanks

2. Software Crafters Communities Worldwide
softwarecrafters.org

3. follow Cork Software Crafters
@CorkSwCraft

4. We are always looking for new ideas, new talks and
mostly new hands-on sessions for our meetups.
Contact us if you want to collaborate!

5. 6:15pm - Welcome.
6:30pm - Introduction.
6:40pm - Hands-on time. We will work in groups on the exercise.
7:50pm - Retrospective.
Paulo Clavijo (@pclavijo)
Agenda
Cork Software Crafters

6. Non-breaking changes exercise
We favour a Continuous Integration/Continuous Delivery approach, integrating changes with the
trunk branch and deploying all the way to production multiple times per day. Our pull requests should
always take in consideration a blue/green deployment. But there is a multitude of "breaking changes"
that can make our life complicated :-(.
In this group exercise, we will go through different scenarios for breaking code changes (UI changes,
API changes, Database changes, Message Queue changes, Performance improvements ...) and discuss
together what could be the right steps to integrate changes in a non-breaking way.
Paulo Clavijo @pclavijo
Cork Software Crafters

7. Continous Delivery
“Continuous Delivery is a software development discipline where you build software in
such a way that the software can be released to production at any time… You achieve
continuous delivery by continuously integrating the software done by the development
team, building executables, and running automated tests on those executables to detect
problems… You push those executables into increasingly production-like environments
to ensure that the software will work in production.” - Martin Fowler
Paulo Clavijo @pclavijo
Cork Software Crafters

8. Trunk-Based Development
Paulo Clavijo @pclavijo
Cork Software Crafters
http://trunkbaseddevelopment.com

9. Blue-green deployment
Paulo Clavijo @pclavijo
Users Router
Green
environment
Blue
environment
create a new green environment and switch from Blue (old) to Green (new)
example.com
new-example.com
code v1.0
code v1.1
Cork Software Crafters

10. Practice Time

11. We are here to learn .
Cork Software Crafters Paulo Clavijo (@pclavijo)
Focus on doing it right .
We need to slow down .
Collaboration .
Coding Dojo Mindset

12. Collaboration
● We will work in groups of 4 to 5 people
● 5 min for scenario
● 3 min to share ideas and solutions
● Swap groups
Paulo Clavijo @pclavijo

13. Keep in mind
● Service is available continuously
● Deployment requires parallel versions (blue/green)
● Deploy the smallest changes you can
● We do not deploy multiple services at once (avoid coupled
changes)
Paulo Clavijo @pclavijo
Cork Software Crafters

14. UI changes

15. Scenario UI-1
Replacing and old home page by a new and redesigned one on a
SPA UI.
- The current home page view has /my-dashboard as route. This route is also the
default route in the app, users are redirected to it when they just use the base url
www.app.com.
- The new page will have multiple components and actions, it will take around 2 week
to develop.
- Users can not see the new page until it is completely finished.
Paulo Clavijo @pclavijo
Cork Software Crafters

16. Scenario UI-2
New feature to allow product reviews.
- Customers should be able to see reviews for a product on the produt detail
view.
- Logged-in customers can add their own review for a product.
- This new feature will take 1 week to develop.
- Users can not see the new feature until it is completely finished.
Paulo Clavijo @pclavijo
Cork Software Crafters

17. Feature Toggles
“A feature toggle is a technique in software development that
attempts to provide an alternative to maintaining multiple
source-code branches (known as feature branches), such that a
feature can be tested even before it is completed and ready for
release. Feature toggle is used to hide, enable or disable the
feature during run time. For example, during the development
process, a developer can enable the feature for testing and disable
it for other users.” - Wikipedia
Paulo Clavijo @pclavijo

18. API changes

19. Scenario API-1
Make the paramater category required in an existing /products
endpoint.
- The parameter allows filtering products GET /products?category=home
Paulo Clavijo @pclavijo
Cork Software Crafters

20. Scenario API-2
Rename the request query parameter pcod to productId in a
existing /products endpoint.
- The parameter is used for retrieving a single product GET /products?pcod=12345
fun getProduct(@RequestParam pcod: String): Product {
return productsService.getProduct(pcod)
}
Paulo Clavijo @pclavijo
Cork Software Crafters

21. Advice and considerations
API breaking changes:
● adding a new or modifying an existing validation to an existing resource
● requiring a parameter that wasn’t required before
● changing existing error response codes/messages
● modifying the expected payload
● changing the data type of an existing field
● changing supported filtering on existing endpoints
● renaming a field or endpoint
● changing the URL structure of an existing endpoint
…
Consider API versioning, Consumer-Driven Contracts
Paulo Clavijo @pclavijo
Cork Software Crafters

22. Parallel Change
“Parallel change, also known as expand and contract, is a pattern
to implement backward-incompatible changes to an interface in a
safe manner, by breaking the change into three distinct phases:
expand, migrate, and contract.” - Martin Fowler
Paulo Clavijo @pclavijo
Cork Software Crafters

23. Parallel Change
Paulo Clavijo @pclavijo
Cork Software Crafters

24. Parallel Change
Paulo Clavijo @pclavijo
Cork Software Crafters

25. Parallel Change
Paulo Clavijo @pclavijo
Cork Software Crafters

26. DB changes

27. Scenario DB-1
Rename the attribute information_compliance to
regulatory_requirements in the Application entity.
- We are using a table on a relational database.
- Attribute not related to other tables.
- Change in code and table column.
Paulo Clavijo @pclavijo
Cork Software Crafters

28. Scenario DB-2
Transform the attribute name to firstname and lastname in the
Customer entity.
- We are using a table on a relational database.
- Attribute not related to other tables.
- Change in code and table column.
- Data already exist in the table.
- No other service connects to this database.
Paulo Clavijo @pclavijo
Cork Software Crafters

29. Advice and considerations
Database changes that can cause downtime or errors:
● removing a column
● adding a column with a default value
● renaming a column
● changing the type of a column
● setting NOT NULL on an existing column
● renaming a table
● adding a foreign key
● backfilling data
● adding an index non-concurrently
...
Paulo Clavijo @pclavijo
Cork Software Crafters

30. Advice and considerations
● Read chapter 5 (Evolutionary Data) from Building Evolutionary Architectures book.
● Take a look at github.com/ankane/strong_migrations#dangerous-operations
Paulo Clavijo @pclavijo
Cork Software Crafters

31. Large Scale changes

32. Scenario LS-1
Change the persistence technology from Relational Database to a
NoSql store.
We will start with the Customer entity.
- Other entities only refer to Customer using its ID.
- Customer is not used in joins.
Paulo Clavijo @pclavijo
Cork Software Crafters

33. Branch by Abstraction
“Branch by Abstraction is a technique for making a large-scale
change to a software system in gradual way that allows you to
release the system regularly while the change is still in-progress.” -
Martin Fowler
Paulo Clavijo @pclavijo

34. Branch by Abstraction
“Branch by Abstraction is a technique for making a large-scale
change to a software system in gradual way that allows you to
release the system regularly while the change is still in-progress.” -
Martin Fowler
Paulo Clavijo @pclavijo

35. Message queue changes

36. Scenario Q-1
- Server A sends a message OrderPlaced with a field old_field
using a queue to Server B and Server C.
- We need to change old_field to new_field.
Server A
queue
Message
Server B
Server C
Paulo Clavijo @pclavijo
Cork Software Crafters

37. Postel’s Law and the Tolerant Reader pattern
“When communicating between systems, Postel’s Law and the
Tolerant Reader pattern state that an implementation should be
liberal when receiving data and conservative when sending data.
This makes system integration less painful because changes to data
fields ignored by the receiving system becomes seamless to the
overall integration.” - Secure By Design
Paulo Clavijo @pclavijo
Cork Software Crafters

38. Performance changes

39. Scenario P-1
We have a service that calculates daily delivery routes for
postmen. It uses a cpu and memory inefficient route calculation
logic, we want to reimplement the calculation with a new
algorithm that improves performance.
- Calculation is complex with lots of corner cases.
- We expect the same outcome from old and new logic.
- It will take around 2 week to develop.
- This service is highly important for the business.
- Paulo Clavijo @pclavijo
Cork Software Crafters

40. Advice and considerations
● Consider Canary Release
● Consider Branch Branch By Abstraction with verification (involves verifying that the
two implementations return the same results to requests)
● Take a look at Scienties github.com/github/scientist (a library for carefully
refactoring critical paths)
Paulo Clavijo @pclavijo
Cork Software Crafters

41. Verify Branch By Abstraction
“... a run-time configuration toggle delegates to a verifying
implementation that calls both components with the same inputs and fails
fast if they do not produce the same output. This reduces the feedback
loop for old and new component incompatibilities, and increases
confidence in our evolving architecture.” - Steve Smith
Paulo Clavijo @pclavijo

42. References
- Taller Parallel Changes, Eduardo Ferro. Software Crafters Barcelona 2018.
- BlueGreenDeployment, Martin Fowler
- BranchByAbstraction, Martin Fowler
- Parallel Change, Danilo Sato
- Parallel Change Refactoring, Oleksii Fedorov
- Parallel Change (Parallel Design), Jakub Holy
- Effective Design (slides 8 and 9), Kent Beck
- The Limited Red Society, Joshua Kerievsky
- StranglerFigApplication, Martin Fowler
- CanaryRelease, Danilo Sato
Paulo Clavijo (@pclavijo)Cork Software Crafters

43. Recommended Books
- Building Evolutionary Architectures: Support Constant Change by Neal Ford, Rebecca Parsons
and Patrick Kua.
- The DevOps Handbook by Gene Kim, Jez Humble, Patrick Debois, John Willis (chapter 12).
- The Mikado Method by Ola Ellnestam.
- Working Effectively with Legacy Code by Michael Feathers.
- Refactoring by Martin Fowler.
Paulo Clavijo (@pclavijo)Cork Software Crafters