5 Levels of High Availability: From Multi-instance to Hybrid Cloud

1. 5 Levels of High Availability
From Multi-instance to Hybrid Cloud
Rafał Leszko
@RafalLeszko
rafalleszko.com
Hazelcast

2. About me
● Cloud Software Engineer at Hazelcast
● Worked at Google and CERN
● Author of the book "Continuous Delivery
with Docker and Jenkins"
● Trainer and conference speaker
● Live in Kraków, Poland

3. About Hazelcast
● Distributed Company
● Open Source Software
● 140+ Employees
● Products:
○ Hazelcast IMDG
○ Hazelcast Jet
○ Hazelcast Cloud
@Hazelcast

4. ● Introduction
● High Availability Levels
○ Level 0: Single Instance
○ Level 1: Multi Instance
○ Level 2: Multi Zone
○ Level 3: Multi Region
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

5. Introduction

6. application service

7. micro-service
application service

8. micro?
application service

9. application service

10. application service

12. application service

13. Stateless

15. application service

16. application servicedata store

17. application servicedata store
queue

18. application servicedata store
queue
other service

19. Data is the problem!

20. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance
○ Level 1: Multi Instance
○ Level 2: Multi Zone
○ Level 3: Multi Region
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

21. Level 0: Single Instance

22. application service data store
Level 0: Single Instance
machine
request

23. Level 0: Single Instance
YOLO!

24. LATENCY EXPERIMENT

26. What does "Level 0: Single Instance" mean to You?
No high availability!
No scalability!
Super low latency:
● in-process memory
● no network
● local file system
Data consistency

27. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance
○ Level 2: Multi Zone
○ Level 3: Multi Region
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

28. Level 1: Multi Instance

29. If one machine is down,
the system is still available

30. application
data
Level 1: Multi Instance
request
data
load balancer
application
application
application
application
machine 1
machine 2
machine 3
machine 4

31. application
data
Level 1: Multi Instance
data
application
application
application
application
machine 1
machine 2
machine 3
machine 4

32. data
Level 1: Multi Instance
data
machine 3
machine 4

33. data store
Level 1: Multi Instance
machine 1
data store
machine 2
Assumptions:
● Local network
● Fast
● Reliable
For example:
● EC2 Instances in the
same availability
zone
● GCP VM instances in
the same zone
● Your on-premises
server machines
connected with LAN

35. data store
Level 1: Multi Instance
machine 1
data store
machine 2

36. Data replication

37. data store
Option 1: Active-Passive (Master-Slave) Replication
machine 1
data store
machine 2
application service
application service
application service
active
passive
SQL

38. data store
Option 1: Active-Passive (Master-Slave) Replication
machine 1
data store
machine 2
application service
application service
application service
active
passive

39. data store
Option 1: Active-Passive (Master-Slave) Replication
machine 1
data store
machine 2
application service
application service
application service
down
active

40. data store
Option 1: Active-Passive (Master-Slave) Replication
machine 1
data store
machine 2
application service
application service
application service
active
passive
SQL

41. data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
data store
[T-Z]
machine 3
NoSQL

42. data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
application service
application service
application service
data store
[T-Z]
machine 3

43. data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
application service
application service
application service
data store
[T-Z]
machine 3

44. 1
Hazelcast Member 1
data
partitions
4
7
data
partition
backups
3 5
9
3
Hazelcast Member 3
data
partitions
6
9
data
partition
backups
2 4
8
2
Hazelcast Member 2
data
partitions
5
8
data
partition
backups
1 6
7

45. data store
[A-G]
Option 2: Clustering
machine 1
data store
[H-S]
machine 2
data store
[T-Z]
machine 3
NoSQL

46. Synchronous
vs
Asynchronous

47. Synchronous (Consistency) or Asynchronous (Latency)?
data
store
machine 1
machine 2 machine 3
data
store
machine 1
data
store
machine 2
active
passive
data
store
data
store

48. LATENCY EXPERIMENT

51. Synchronous (Consistency) or Asynchronous (Latency)?
data
store
machine 1
machine 2 machine 3
data
store
machine 1
data
store
machine 2
active
passive
data
store
data
store
synchronous

52. Synchronous (Consistency) or Asynchronous (Latency)?
data
store
machine 1
machine 2 machine 3
data
store
machine 1
data
store
machine 2
active
passive
data
store
data
store
synchronous?

53. What does "Level 1: Multi Instance" mean to You?
Data consistency!
Most tools supported
Cloud-specific toolkit (e.g. AWS SQS)
Simple setup (even on-premises)
High latency if accessed multi regions

54. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance ✔
○ Level 2: Multi Zone
○ Level 3: Multi Region
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

55. Level 2: Multi Zone

56. If one availability zone is down,
the system is still available

58. application
data
Level 2: Multi Zone
request
data
load balancer
application
application
application
application
zone 1
zone 2

59. Is multi-zone deployment any
different?

60. No

61. No but… Yes

62. LATENCY EXPERIMENT

64. No but… Yes

65. Level 2: Multi Zone
zone 1
zone 2
data data
data data

66. Level 2: Multi Zone
zone 1
zone 2
data data
data data
Assumptions:
● Machines in at
least 2 AZ
● Fast and
reliable
network
For example:
● EC2 Instances
in 2 AWS
Availability
Zones
● Azure VM
instances in 2
Availability
Sets

67. 1
Hazelcast Member 1
data
partitions
4
7
data
partition
backups
3 5
9
3
Hazelcast Member 3
data
partitions
6
9
data
partition
backups
2 4
8
2
Hazelcast Member 2
data
partitions
5
8
data
partition
backups
1 6
7

68. Level 2: Multi Zone
machine 1 machine 2
Hazelcast Member 2
zone 1
zone 2
Hazelcast Member 1
machine 3 machine 4
Hazelcast Member 4Hazelcast Member 3

69. Hazelcast configuration:
hazelcast:
partition-group:
enabled: true
group-type: ZONE_AWARE
Hazelcast Zone Aware Feature

70. Hazelcast Zone Aware Feature

71. Level 2: Multi Zone
machine 1 machine 2
Hazelcast Member 2
zone 1
zone 2
Hazelcast Member 1
machine 3 machine 4
Hazelcast Member 4Hazelcast Member 3

72. No but… Yes

73. Make sure your
data store is
ZONE AWARE

74. What does "Level 2: Multi Zone" mean to You?
Currently top 1 choice!
Data consistency!
Cloud-specific toolkit (e.g. AWS SQS)
High latency if accessed multi regions
Not all tools are "zone aware"

75. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance ✔
○ Level 2: Multi Zone ✔
○ Level 3: Multi Region
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

76. Level 3: Multi Region

77. If one region is down,
the system is still available

78. application
data
Level 3: Multi Region
geo
load balancer
region 1
load balancer
load balancer
region 2
application
application
application
application
application
application
application
data
data
data
data
data
geo replication

79. data
Level 3: Multi Region
region 1
region 2
data
data
data
data
data
geo replication
Assumptions:
● Machines in at
least 2
geographical
regions
● Network may be
slow and unreliable
For example:
● EC2 Instances in
regions: eu-central-
1 and us-west-2

80. 10 000 km

81. Speed of light: 300 000 km/s
Distance: 10 000 km
RTT (Round Trip Time) = 60 ms
Level 3: Multi Region

82. Geo-replication

83. data
data
data
geo replication
Level 3: Multi Region (Geo-replication)
data
data
data

84. Geo-replication
● It's asynchronous
● Your data store must support it
● You must be prepared for data loss
● Two modes:
○ Active-Passive
○ Active-Active

85. data
data
data
geo replication
Active-Passive Geo-replication
data
data
data
active passive

86. data
data
data
geo replication
Active-Passive Geo-replication
data
data
data
active passive
● data loss possible
● (eventual) consistency

87. data
data
data
geo replication
Active-Active Geo-replication
data
data
data
active active

88. data
data
data
geo replication
Active-Active Geo-replication
data
data
data
active active
● data loss possible
● eventual consistency
● conflict resolution

89. Hazelcast WAN Replication
hazelcast:
wan-replication:
batch-publisher:
target-endpoints: 35.184.122.109

90. Do I really need to lose
consistency?

91. LATENCY EXPERIMENT

95. Strong Consistency in Multi Region
● NewSQL (Spanner, CockroachDB)
● Multi-region distributed transactions
● Consensus algorithms (Paxos, Raft)
● Always a trade-off: consistency vs latency

96. What does "Level 3: Multi Region" mean to You?
Super high available!
Low latency if accessed from multi regions
Sometimes possible to use Cloud-specific
toolkit (e.g. Google Spanner - yes, AWS
Elasticache - no)
Geo-replication (asynchronous)!
Eventual consistency (conflict resolution)

97. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance ✔
○ Level 2: Multi Zone ✔
○ Level 3: Multi Region ✔
○ Level 4: Multi Cloud
○ Level 5: Hybrid Cloud
● Summary
Agenda

98. Level 4: Multi Cloud

99. If one cloud provider is down,
the system is still available

100. app
data
Level 4: Multi Cloud
global
load balancer
load
balancer
app
app
data
data
replication
app
data
load
balancer
app
app
data
data

101. data
Level 4: Multi Cloud
data
data
replication
data
data
data
cloud provider 1
cloud provider 2
Assumptions:
● Machines in at
least 2 cloud
providers
● Network may be
slow and unreliable
● Machines may be
in different geo
regions
For example:
● EC2 Instances in
eu-central-1 and
GCP VM Instances
in us-west1-a

102. What's different from
multi-region?

103. Level 4: Multi Cloud
● No Cloud-specific tools
● No VPC Peering across Cloud providers
○ Latency
○ Security
● Cost

104. Is High Availability the only
reason for Multi-Cloud?

105. Reasons for Multi-Cloud
● High Availability / Disaster Recovery
● Avoiding vendor lock-in
● Cloud cost optimization
● Risk Mitigation
● Low latency
● Data Protection / Regulations / Compliance
● Best-Fit Technology (Cloud-specific portfolios)

106. What does "Level 4: Multi Cloud" mean to You?
No vendor lock-in!
Cloud cost negotiations
Low latency if accessed from multi-cloud
Complex setup!
No Cloud toolkit (e.g. AWS SQS)

107. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance ✔
○ Level 2: Multi Zone ✔
○ Level 3: Multi Region ✔
○ Level 4: Multi Cloud ✔
○ Level 5: Hybrid Cloud
● Summary
Agenda

108. Level 5: Hybrid Cloud

109. If all cloud providers are down,
the system is still available

111. Is it possible that
all cloud providers
are down?

112. No!

113. Reasons for Hybrid Cloud
● Data requirements / regulations
● Data security
● Moving to Cloud
● Cost reduction
● All mentioned already in Multi-Cloud

114. Level 5: Hybrid Cloud
global
load balancer
On-Premises

116. What does "Level 5: Hybrid Cloud" mean to You?
No Cloud lock-in!
Low latency if accessed from custom
networks
Super complex setup!
Usually extra layer needed (e.g.
Kubernetes, OpenShift)
Costs a fortune!

117. ● Introduction ✔
● High Availability Levels
○ Level 0: Single Instance ✔
○ Level 1: Multi Instance ✔
○ Level 2: Multi Zone ✔
○ Level 3: Multi Region ✔
○ Level 4: Multi Cloud ✔
○ Level 5: Hybrid Cloud ✔
● Summary
Agenda

118. Summary

119. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

120. distributed system
(data replication)
Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

121. distributed system
(data replication)
Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
zone aware

122. distributed system
(data replication)
Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
long distance
(geo-
replication)
zone aware

123. distributed system
(data replication)
Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
no cloud
tools
long distance
(geo-
replication)
zone aware

124. distributed system
(data replication)
Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud
no cloud
tools
own
infrastructure
long distance
(geo-
replication)
zone aware

125. Which
High Availability Level
is for me?

126. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

127. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

128. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

129. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

130. Single
Instance
Multi Instance Multi Zone Multi Region Multi Cloud Hybrid Cloud

131. Thank You!
Rafał Leszko
@RafalLeszko
rafalleszko.com