Galera replication works by synchronizing data across multiple database servers so that any server can accept writes and all servers instantly reflect the new data. It uses global transaction IDs and group communication to replicate write sets in parallel to all nodes, ensuring consistency. Any node can join the cluster as long as it knows the cluster name and can find an active member to bootstrap from.
8. Frédéric Descamps
@lefred
Working for Percona since 2011
Senior Architect
Managing MySQL since 3.23
devops believer
and I installed my first Galera Cluster in February 2010 ;-)
8
/
262
15. Cluster Membership
determined by the cluster
wsrep_cluster_address is just a pointer
any node is permitted to join that
knows the cluster name
can find a single active cluster node
15
/
262
16. Cluster Membership
determined by the cluster
wsrep_cluster_address is just a pointer
any node is permitted to join that
knows the cluster name
can find a single active cluster node
㫤㫠㫟㫜㫝㫘㫚㫙㫢㫠㫡㫜㫟㫘㫘㫛㫛㫟㫜㫠㫠㫔㫖㫔㫞㫚㫜㫚㫚㫣㫘㫘㫚㫢㫛㫗㫚㫟㫡㫗㫚㫗㫚㫕㫚㫢㫛㫗㫚㫟㫡㫗㫚㫗㫛㫕㫚㫢㫛㫗㫚㫟㫡㫗㫚㫗㫜
16
/
262
28. Replication
Delivers the writeset to all nodes in the cluster
and all nodes acknowledge the writeset
Cost is ~roundtrip latency to furthest node
28
/
262
29. Replication
Delivers the writeset to all nodes in the cluster
and all nodes acknowledge the writeset
Cost is ~roundtrip latency to furthest node
Serialized by Group Communication
29
/
262
32. GTID
Not the same as 5.6 Aynchronous GTID's
though they appear the same
32
/
262
33. GTID
Not the same as 5.6 Aynchronous GTID's
though they appear the same
939aac77-f7d1-11e3-bd5e-b211d6ab1ec6:1534285
33
/
262
34. GTID
Not the same as 5.6 Aynchronous GTID's
though they appear the same
939aac77-f7d1-11e3-bd5e-b211d6ab1ec6:1534285
GTIDs ensure cluster members are consistent with each other
34
/
262
35. GTID
Not the same as 5.6 Aynchronous GTID's
though they appear the same
939aac77-f7d1-11e3-bd5e-b211d6ab1ec6:1534285
GTIDs ensure cluster members are consistent with each other
nodes joining a cluster have their GTIDs checked
35
/
262
36. GTID
Not the same as 5.6 Aynchronous GTID's
though they appear the same
939aac77-f7d1-11e3-bd5e-b211d6ab1ec6:1534285
GTIDs ensure cluster members are consistent with each other
nodes joining a cluster have their GTIDs checked
GTIDs can be used to compare downed nodes to each other
36
/
262
37. GTID
The highest GTID is the most recently written
Generally the best practice it to bootstrap the node with the most recent
data
37
/
262
47. Global Transaction IDs : Galera vs
MySQL 5.6
In MySQL 5.6
㫢㫘㫞㫚㫚㫙㫠㫙㫖㫠㫙㫞㫢㫖㫚㫚㫜㫛㫖㫢㫘㫛㫝㫖㫙㫡㫙㫙㫛㫠㫟㫛㫙㫡㫘㫝㫣㫜㫛
㫢㫘㫞㫚㫚㫙㫠㫙㫖㫠㫙㫞㫢㫖㫚㫚㫜㫛㫖㫢㫘㫛㫝㫖㫙㫡㫙㫙㫛㫠㫟㫛㫙㫡㫘㫝㫣㫜㫜
㫢㫘㫞㫚㫚㫙㫠㫙㫖㫠㫙㫞㫢㫖㫚㫚㫜㫛㫖㫢㫘㫛㫝㫖㫙㫡㫙㫙㫛㫠㫟㫛㫙㫡㫘㫝㫣㫜㫝
㫘㫔㫛㫜㫤㫔㫚㫘㫠㫡㫜㫟㫔㫝㫟㫜㫚㫜㫡㫜㫛㫔㫘
㫜㫜㫚㫚㫝㫘㫝㫠㫖㫠㫚㫚㫘㫖㫚㫚㫜㫛㫖㫢㫜㫜㫜㫖㫚㫡㫙㫘㫘㫢㫝㫛㫢㫞㫟㫛㫣㫚
㫜㫜㫚㫚㫝㫘㫝㫠㫖㫠㫚㫚㫘㫖㫚㫚㫜㫛㫖㫢㫜㫜㫜㫖㫚㫡㫙㫘㫘㫢㫝㫛㫢㫞㫟㫛㫣㫛
㫜㫜㫚㫚㫝㫘㫝㫠㫖㫠㫚㫚㫘㫖㫚㫚㫜㫛㫖㫢㫜㫜㫜㫖㫚㫡㫙㫘㫘㫢㫝㫛㫢㫞㫟㫛㫣㫜
47
/
262
48. Global Transaction IDs : Galera vs
MySQL 5.6
In Galera
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫡
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫢
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫙
48
/
262
49. Global Transaction IDs : Galera vs
MySQL 5.6
In Galera
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫡
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫢
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫙
㫘㫔㫛㫜㫤㫔㫚㫘㫠㫡㫜㫟㫔㫝㫟㫜㫚㫜㫡㫜㫛㫔㫘
49
/
262
50. Global Transaction IDs : Galera vs
MySQL 5.6
In Galera
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫡
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫚㫢
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫙
㫘㫔㫛㫜㫤㫔㫚㫘㫠㫡㫜㫟㫔㫝㫟㫜㫚㫜㫡㫜㫛㫔㫘
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫚
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫛
㫙㫝㫙㫢㫚㫛㫜㫞㫖㫝㫞㫟㫙㫖㫚㫚㫜㫛㫖㫙㫡㫙㫙㫖㫚㫜㫜㫝㫘㫙㫙㫞㫜㫞㫠㫛㫘㫣㫚㫚㫛㫜
50
/
262
51. GTID Assignment
UUID
The UUID section, 128-bit, is generated during bootstrapping to identify
the cluster.
Generated by mixing the timer value and pseudo-random numbers
(depending primarily from the timer and PID), but, currently, there is no
use of NIC's MAC-address although in theory it may be done that way.
More info: https://gist.github.com/lefred/88a2cec88d03854d9934
51
/
262
52. GTID Assignment (3)
SEQNO
The seqno, 64-bit, is incremented only when the transaction passes
certification and is ready for commit.
For the curious, the algorithm used to derive sequence number is Totem
Single-ring Ordering protocol.
Before that, there is already communication between the nodes, group
communication is used to define a group-channel id which is a locally
maintained counter by each node in sync with the group.
52
/
262
70. Replication Roles
Within the cluster, all nodes are equal
'master/donor node'
The node a given transaction was written and committed on.
70
/
262
71. Replication Roles
Within the cluster, all nodes are equal
'master/donor node'
The node a given transaction was written and committed on.
'slave/joiner node'
71
/
262
72. Replication Roles
Within the cluster, all nodes are equal
'master/donor node'
The node a given transaction was written and committed on.
'slave/joiner node'
The node that received the given transaction via Galera
replication.
72
/
262
73. Replication Roles
Within the cluster, all nodes are equal
'master/donor node'
The node a given transaction was written and committed on.
'slave/joiner node'
The node that received the given transaction via Galera
replication.
The terms master and slave in this context are only relevant for a
given transaction
73
/
262
74. Replication Roles
Within the cluster, all nodes are equal
'master/donor node'
The node a given transaction was written and committed on.
'slave/joiner node'
The node that received the given transaction via Galera
replication.
The terms master and slave in this context are only relevant for a
given transaction
Writeset: Galera’s term for a transaction. One or more RBR row
changes
74
/
262
75. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
75
/
262
76. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
76
/
262
77. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
Donor = Node giving a copy of the datadir
77
/
262
78. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
Donor = Node giving a copy of the datadir
State Snapshot transfer
78
/
262
79. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
Donor = Node giving a copy of the datadir
State Snapshot transfer
Full backup of Donor to Joiner
79
/
262
80. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
Donor = Node giving a copy of the datadir
State Snapshot transfer
Full backup of Donor to Joiner
Incremental Snapshot transfer
80
/
262
81. State Transfer Roles
New nodes joining an existing cluster get provisioned automatically
Joiner = Mew node
Donor = Node giving a copy of the datadir
State Snapshot transfer
Full backup of Donor to Joiner
Incremental Snapshot transfer
Only changes since node left cluster
81
/
262
95. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
95
/
262
96. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
96
/
262
97. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
97
/
262
98. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
Results are not reported to the cluster
98
/
262
99. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
Results are not reported to the cluster
Pass: enter apply queue (commit success on master/donor)
99
/
262
100. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
Results are not reported to the cluster
Pass: enter apply queue (commit success on master/donor)
Fail: drop transaction (or return deadlock on master/donor)
100
/
262
101. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
Results are not reported to the cluster
Pass: enter apply queue (commit success on master/donor)
Fail: drop transaction (or return deadlock on master/donor)
Serialized by group communication sequence (and GTID will be
synchronized following the same sequence)
101
/
262
102. Certification
Can this writeset be applied?
Based on unapplied earlier transactions on master/donor
Such conflicts must come from other nodes
Happens on every node
Should be deterministic on every node
Results are not reported to the cluster
Pass: enter apply queue (commit success on master/donor)
Fail: drop transaction (or return deadlock on master/donor)
Serialized by group communication sequence (and GTID will be
synchronized following the same sequence)
Cost based on # of keys or # of rows
102
/
262
104. Apply
Apply is done on slave nodes after certification
Can be parallelized
104
/
262
105. Apply
Apply is done on slave nodes after certification
Can be parallelized
if wsrep_slave_threads > 1
105
/
262
106. Apply
Apply is done on slave nodes after certification
Can be parallelized
if wsrep_slave_threads > 1
if there are no other writesets with conflicting keys also being
applied
106
/
262
107. Apply
Apply is done on slave nodes after certification
Can be parallelized
if wsrep_slave_threads > 1
if there are no other writesets with conflicting keys also being
applied
Cost: size of transaction
107
/
262
108. Apply
Apply is done on slave nodes after certification
Can be parallelized
if wsrep_slave_threads > 1
if there are no other writesets with conflicting keys also being
applied
Cost: size of transaction
Generates brute force aborts on local node for conflicts
108
/
262
112. Commit
Final local InnoDB commit
i.e., innodb_flush_log_at_trx_commit
GTID gets generated
Done by applier threads on slaves/joiners
112
/
262
113. Commit
Final local InnoDB commit
i.e., innodb_flush_log_at_trx_commit
GTID gets generated
Done by applier threads on slaves/joiners
Done by client thread on master/donor
113
/
262
114. Commit
Final local InnoDB commit
i.e., innodb_flush_log_at_trx_commit
GTID gets generated
Done by applier threads on slaves/joiners
Done by client thread on master/donor
innodb_flush_log_at_trx_commit=1 not required generally for PXC!
114
/
262
187. Flow Control
Ability of any node in the cluster to ask the rest of the nodes to
pause writes while it catches up
187
/
262
188. Flow Control
Ability of any node in the cluster to ask the rest of the nodes to
pause writes while it catches up
Feedback mechanism for replication process
188
/
262
189. Flow Control
Ability of any node in the cluster to ask the rest of the nodes to
pause writes while it catches up
Feedback mechanism for replication process
ONLY caused by 㫤㫠㫟㫜㫝㫘㫙㫜㫚㫘㫙㫘㫟㫜㫚㫣㫘㫞㫢㫜㫢㫜 exceeding a node's
㫝㫚㫘㫙㫠㫚㫠㫡
189
/
262
190. Flow Control
Ability of any node in the cluster to ask the rest of the nodes to
pause writes while it catches up
Feedback mechanism for replication process
ONLY caused by 㫤㫠㫟㫜㫝㫘㫙㫜㫚㫘㫙㫘㫟㫜㫚㫣㫘㫞㫢㫜㫢㫜 exceeding a node's
㫝㫚㫘㫙㫠㫚㫠㫡
CAN pause the entire cluster and look like a cluster stall !
190
/
262
193. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
193
/
262
194. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
increase in replication conflicts in multi-node writings
194
/
262
195. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
increase in replication conflicts in multi-node writings
increase in apply lag on nodes
195
/
262
196. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
increase in replication conflicts in multi-node writings
increase in apply lag on nodes
increase in commit lag on nodes
196
/
262
197. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
increase in replication conflicts in multi-node writings
increase in apply lag on nodes
increase in commit lag on nodes
One node with FC issues
197
/
262
198. Tuning Flow Control
Too low:
frequent FC from any and all nodes in the cluster
Too high:
increase in replication conflicts in multi-node writings
increase in apply lag on nodes
increase in commit lag on nodes
One node with FC issues
deal with that node -bad hardware? too slow ?
198
/
262
221. State Transfer
There are two types of State Transfer in Galera:
1. SST (Snapshot State Transfer): full data copy
221
/
262
222. State Transfer
There are two types of State Transfer in Galera:
1. SST (Snapshot State Transfer): full data copy
rsync
mysqldump
xtrabackup
222
/
262
223. State Transfer
There are two types of State Transfer in Galera:
1. SST (Snapshot State Transfer): full data copy
rsync
mysqldump
xtrabackup
2. IST (Incremental State Transfer): only copy the missing events
223
/
262
224. State Transfer
There are two types of State Transfer in Galera:
1. SST (Snapshot State Transfer): full data copy
rsync
mysqldump
xtrabackup
2. IST (Incremental State Transfer): only copy the missing events
It's always better to try to avoid SST!
224
/
262
225. State Transfer
There are two types of State Transfer in Galera:
1. SST (Snapshot State Transfer): full data copy
rsync
mysqldump
xtrabackup
2. IST (Incremental State Transfer): only copy the missing events
It's always better to try to avoid SST!
㫤㫠㫟㫜㫝㫘㫠㫠㫡㫘㫛㫜㫛㫜㫟 can be used to specify the donor
225
/
262
227. State Transfer: grastate.dat
When MySQL starts it checks 㫞㫟㫘㫠㫡㫘㫡㫜㫗㫛㫘㫡 file (in datadir)
This file placeholds GTID between MySQL restarts
227
/
262
228. State Transfer: grastate.dat
When MySQL starts it checks 㫞㫟㫘㫠㫡㫘㫡㫜㫗㫛㫘㫡 file (in datadir)
This file placeholds GTID between MySQL restarts
Contains UUID and Seqno
228
/
262
229. State Transfer: grastate.dat
When MySQL starts it checks 㫞㫟㫘㫠㫡㫘㫡㫜㫗㫛㫘㫡 file (in datadir)
This file placeholds GTID between MySQL restarts
Contains UUID and Seqno
㫕㫔㫗㫖㫗㫖㫗㫖㫔㫠㫘㫣㫜㫛㫔㫠㫡㫘㫡㫜
㫣㫜㫟㫠㫠㫜㫛㫣㫔㫛㫗㫚
㫢㫢㫠㫛㫣㫔㫔㫔㫔㫛㫙㫛㫞㫘㫚㫙㫝㫖㫞㫚㫘㫛㫖㫚㫚㫜㫞㫖㫡㫙㫢㫘㫖㫜㫛㫠㫢㫘㫟㫡㫛㫞㫝㫙㫠
㫠㫜㫞㫛㫜㫣㫔㫔㫔㫚㫝㫙㫙㫚
㫚㫜㫟㫡㫘㫠㫛㫛㫜㫥㫣
229
/
262
230. State Transfer: grastate.dat
When MySQL starts it checks 㫞㫟㫘㫠㫡㫘㫡㫜㫗㫛㫘㫡 file (in datadir)
This file placeholds GTID between MySQL restarts
Contains UUID and Seqno
㫕㫔㫗㫖㫗㫖㫗㫖㫔㫠㫘㫣㫜㫛㫔㫠㫡㫘㫡㫜
㫣㫜㫟㫠㫠㫜㫛㫣㫔㫛㫗㫚
㫢㫢㫠㫛㫣㫔㫔㫔㫔㫛㫙㫛㫞㫘㫚㫙㫝㫖㫞㫚㫘㫛㫖㫚㫚㫜㫞㫖㫡㫙㫢㫘㫖㫜㫛㫠㫢㫘㫟㫡㫛㫞㫝㫙㫠
㫠㫜㫞㫛㫜㫣㫔㫔㫔㫚㫝㫙㫙㫚
㫚㫜㫟㫡㫘㫠㫛㫛㫜㫥㫣
node shutdown cleanly
230
/
262
235. State Transfer: IST
When a node starts, it knowns the UUID of the cluster it belonged and
the last sequence number it applied.
So, it sends that position to the other members of the cluster and if a
node can send the next events (ws/trx), IST will be performed, if none,
then SST will be triggered.
235
/
262
237. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
237
/
262
238. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
238
/
262
239. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
default size is 128M
239
/
262
240. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
default size is 128M
can be increase via provider option 㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜
240
/
262
241. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
default size is 128M
can be increase via provider option 㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜
㫤㫠㫟㫜㫝㫘㫝㫟㫜㫣㫠㫛㫜㫟㫘㫜㫝㫡㫠㫜㫛㫠㫔㫖㫔㫔㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜㫖㫚㫗㫔
241
/
262
242. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
default size is 128M
can be increase via provider option 㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜
㫤㫠㫟㫜㫝㫘㫝㫟㫜㫣㫠㫛㫜㫟㫘㫜㫝㫡㫠㫜㫛㫠㫔㫖㫔㫔㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜㫖㫚㫗㫔
Galare Cache is mmaped (I/O buffered to memory)
242
/
262
243. Galera Cache
Those events are stored on the 㫞㫘㫙㫜㫟㫘㫗㫚㫘㫚㫟㫜 file.
preallocated file with a specific size
used to store the writesets in circular buffer style
default size is 128M
can be increase via provider option 㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜
㫤㫠㫟㫜㫝㫘㫝㫟㫜㫣㫠㫛㫜㫟㫘㫜㫝㫡㫠㫜㫛㫠㫔㫖㫔㫔㫞㫚㫘㫚㫟㫜㫗㫠㫠㫚㫜㫖㫚㫗㫔
Galare Cache is mmaped (I/O buffered to memory)
㫤㫠㫟㫜㫝㫘㫙㫜㫚㫘㫙㫘㫚㫘㫚㫟㫜㫛㫘㫛㫜㫤㫛㫡㫜 provide the first seqno present in
the cache for that node
243
/
262