Kubernetes allows a lot. After discovering its features, it’s easy to think it can magically transform your application deployment process into a painless no-event. For Hello World applications, that is the case. Unfortunately, not many of us do deploy such applications day-to-day because we need to handle state. Though it would be much easier to have stateless apps, and despite our best efforts in this direction, state is found in (at least) two places: sessions and databases. You need to think keeping the state while stopping and starting application nodes. In this talk, I’ll demo how to update a Spring Boot app deployed on a Kubernetes cluster with a non-trivial database schema change with the help of Hazelcast, while keeping the service up during the entire update process.

1. @nicolas_frankel
Zero downtime on
Kubernetes with
Hazelcast
Nicolas Fränkel

2. @nicolas_frankel
Me, myself and I
 Developer
 Developer Advocate
 We live in interesting times and I’m
curious

3. @nicolas_frankel
Hazelcast
HAZELCAST IMDG is an operational,
in-memory, distributed computing
platform that manages data using
in-memory storage and performs
execution for breakthrough
and scale.
HAZELCAST JET is the ultra
fast, application embeddable,
3rd generation stream
processing engine for low
latency batch and stream
processing.

4. @nicolas_frankel
Why zero downtime?
1. Business wants it
• Downtime has a cost
2. Users expect it
• When was the last time you saw
Google Search display “Please come
back later”?

5. @nicolas_frankel
Blue-Green deployment

6. @nicolas_frankel
Blue-Green deployment

7. @nicolas_frankel
Blue-Green deployment variant

8. @nicolas_frankel
Zero downtime’s issues relate to state
 State in memory
• User sessions
 State in the database

9. @nicolas_frankel
Kubernetes rolling updates principle
1.0 1.0
1.0
2.0 2.0 2.0

10. @nicolas_frankel
Solving state-related issues
 Sessions
• Session replication
 Database
• That’s the hard spot!

11. @nicolas_frankel
Option 1: Keep the same database
 The application needs to cope with two
versions of the schema

12. @nicolas_frankel
Rolling upgrade issue with a single database
1.0 1.0
1.0
2.0
1.0
2.0

13. @nicolas_frankel
1.0
2.0
More issues with rollback
1.0 1.0
2.0
1.0
2.0
2.0

14. @nicolas_frankel
An e-commerce use-case

15. @nicolas_frankel
New business requirement comes in!
 We want to keep a track when a cart was
created
• To send a reminder email after some
time has passed

16. @nicolas_frankel
Target schema

17. @nicolas_frankel
Handling schema-related breaking changes
 Split the breaking change into a series of
changes compatible side-by-side
 Plan for rollback (it happens!)

18. @nicolas_frankel
Steps’ decomposition
1. Create CART table
• App uses “old” data model
• Trigger inserts CART when the first CART_LINE is inserted
2. CART becomes the “source of truth”
• App uses the CART table
• Trigger updates CART_LINE with CUSTOMER_ID every time it’s inserted
3. Migration of untouched data
4. Cleanup

19. @nicolas_frankel
Issues of keeping the same database
 Requires steps’ decomposition
 Rollback a single step only
 Needs planning across the organization
(devs, DBAs, Ops)
 You will need to migrate data anyway

20. @nicolas_frankel
Option 2: Embrace data migration
 Have two different databases
 Migration implemented by:
• Change-Data-Capture
• Data streaming
 Developers are not impacted by Ops’
concerns
 It works with any deployment option e.g.
canary release

21. @nicolas_frankel
Change-Data-Capture
“In databases, Change Data Capture is a set
of software design patterns used to determine
and track the data that has changed so that
action can be taken using the changed data.
CDC is an approach to data integration that is
based on the identification, capture and
delivery of the changes made to enterprise
data sources.”
-- https://en.wikipedia.org/wiki/Change_data_capture

23. @nicolas_frankel

24. @nicolas_frankel

25. @nicolas_frankel

26. @nicolas_frankel

27. @nicolas_frankel

28. @nicolas_frankel
Implementation details
 Hazelcast for Session Replication
• Via Spring Session
 Hazelcast for CDC
• With Debezium

29. @nicolas_frankel
Hazelcast IMDG
“Use Hazelcast IMDG to store your data in
RAM, spread and replicate it across a
cluster of machines, and perform data-
local computation on it. Replication gives
you resilience to failures of cluster
nodes.”

30. @nicolas_frankel
Hazelcast Jet

31. @nicolas_frankel
Hazelcast Jet & Debezium

32. @nicolas_frankel

33. @nicolas_frankel
Takeaways
1. Zero-downtime is within your reach
2. Session replication
3. Change-Data-Capture + Data
Streaming for the database

34. @nicolas_frankel
Thanks for your attention!
 https://blog.frankel.ch/
 @nicolas_frankel
 https://bit.ly/zero-downtime
 https://slack.hazelcast.com/
 https://training.hazelcast.com/