MySQL HA with Group Replication

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
MySQL High Availability
with Group Replication
Nuno Carvalho (nuno.carvalho@oracle.com)
Principal Software Engineer, MySQL Replication Service Team Lead
1

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Program Agenda

Program Agenda
Background
MySQL Group Replication
Architecture
Use cases
Performance
Conclusion
1
2
3
4
5
4
6

Background1

M S
S
S
S
M
write clients read clients
read clients
write clients
More
reads?
More
slaves!
Read scale-out
Background: What is Replication Used For?
6

C
B
A
C
B
ACrash
C
B
A
B is the
new master
Uh Oh! Whew!
Redundancy: If master crashes, promote slave to master
Background: What is Replication Used For?
7

• What is MySQL Group Replication?
“Multi-master update everywhere replication plugin for MySQL with built-in automatic
distributed recovery, conflict detection and group membership.”
• What does the MySQL Group Replication plugin do for the user?
– Removes the need for handling server fail-over.
– Provides fault tolerance.
– Enables update everywhere setups.
– Automates group reconfiguration (handling of crashes, failures, re-connects).
– Provides a highly available replicated database.
8

9
M M M M M
Replication Group
Clients

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 10
Timeline - Server GA and GR Releases
MySQL 5.6.10
MySQL 5.7.9
lifecycle interfaces
P_S tables for GR
Server side changes
GR 0.2.0 labs
Hello world!
GR 0.3.0 labs
support for corosync 2.x
GR 0.4.0 labs
version handling
GR 0.5.0 labs
auto-inc fields handling
recovery enhancements
GR 0.6.0 labs
multi-platform support
Paxos-based consensus
GR 0.7.0 labs
bug fixes
GR 0.8.0 labs (beta)
performance enhancements
replication stream compression
SSL support
IP whitelisting
read-only mode
error log enhancements
split brain handling
GR 0.9.0 labs (RC)
multi-threaded applier support
single primary mode

MySQL Group Replication2
2.1 Multi-Master

http://mysqlhighavailability.com/mysql-group-replication-transaction-life-cycle-explained/
Multi-Master update everywhere!
• Any two transactions on different servers can write to the same tuple.
• Conflicts will be detected and dealt with.
– First committer wins rule.
12
M M M M M
UPDATE t1 SET a=4 WHERE a=2UPDATE t1 SET a=3 WHERE a=1

13
M M M M M
OKOK

14
M M M M M

15
M M M M M
OK

2.1
2.2
Multi-Master
Automatic distributed server recovery

http://mysqlhighavailability.com/distributed-recovery-behind-the-scenes/
Automatic distributed server recovery!
• Server that joins the group will automatically synchronize with the others.
17
M M M M M N
I want to play with you

18
M M M M M N
ONLINE
RECOVERING

19
M M M M M N
ONLINE

• If a server leaves the group, the others will automatically be informed.
20
M M M M M M
My machine needs maintenance
or a system crash happens
Each membership configuration
is identified by a view_id
view_id: 4

• If a server leaves the group, the others will automatically be informed.
21
M M M M M

• Server that (re)joins the group will automatically synchronize with the
others.
22
M M M M M M
RECOVERING -> ONLINE
view_id: 5

2.1
2.2
2.3
Multi-Master
MySQL/InnoDB look & feel

http://mysqlhighavailability.com/mysql-group-replication-monitoring/
MySQL look & feel!
• MySQL Plugin
– Regular MySQL Plugin. Nothing new.
• MySQL InnoDB
– Use InnoDB as normally you would. Nothing new.
– Transparent optimizations in InnoDB to better support Group Replication.
• MySQL Performance Schema
– Monitor Group Replication using regular Performance Schema tables. Nothing new.
24

http://mysqlhighavailability.com/mysql-group-replication-monitoring/
MySQL look & feel!
• Outcome
– Group Replication is no alien component.
– Existing MySQL users feel right at home.
– New MySQL users only have to learn MySQL tech, nothing else.
25

Full GTID support!
• All group members share the same UUID, the group name.
26
M M M M M
INSERT y;
Will have GTID: group_name:2
INSERT x;

Full GTID support!
• Users can specify the identifier for the transaction.
27
M M M M M
INSERT y;
SET GTID_NEXT= “UUID:50”
INSERT x;

Full GTID support!
• You can even replicate from a outside server to a group, global identifiers
will be preserved.
28
M M M M M
Conflicts will be detected!

Full GTID support!
• You can also replicate from a group to a outside server, global identifiers
will be preserved.
29
M M M M M

2.1
2.2
2.3
2.4
2.5
Multi-Master
Auto-increment configuration/handling
Full GTID support

• Group is configured not to generate the same auto-increment value on all
members.
31
M M M M M
INSERT y;
y: 4
INSERT z;
z: 11
INSERT x;
x: 1

[1]: http://mysqlhighavailability.com/mysql-group-replication-auto-increment-configuration-handling/
• By default, the offset is provided by server_id and increment is 7 [1].
32
M M M M M
INSERT y;
y: 4
INSERT z;
z: 11
INSERT x;
x: 1
server_id: 1
server_id: 4

• Users can change the increment size to their needs using
GROUP_REPLICATION_AUTO_INCREMENT_INCREMENT option.
33
M M M M M
INSERT y;
y: 4
INSERT z;
z: 11
INSERT x;
x: 1
server_id: 1
server_id: 4

Plugin version access control
2.1
2.2
2.3
2.4
2.5
2.6
Multi-Master
Full GTID support

Plugin Version Access Control
• When joining, versions are crucial when determining if a member is
compatible with a group.
35
5.7.13 5.7.13 5.7.14 5.7.15 5.7.12
Member with different patch
version is allowed to join

36
8.0.0 8.0.0 8.0.0 8.0.0 5.7.15
Member with lower major version
than the major version in
the group is not allowed to join

37
5.7.14 5.7.14 5.7.15 5.7.15 8.0.0
Member with higher major
version is allowed to join but
is not allowed to do writes

Built-in communication engine
Plugin version access control
2.1
2.2
2.3
2.4
2.5
2.6
2.7
Multi-Master
Full GTID support

http://mysqlhighavailability.com/mysql-group-replication-for-mysql-5-7-9/
Built-in Communication Engine
• Feature rich new replication plugin based on proven distributed systems
algorithms (Paxos).
– Compression, multi-platform, dynamic membership, distributed agreement, quorum
based message passing, SSL, IP whitelisting.
• No third-party software required.
• No network multicast support required.
– MySQL Group Replication can operate on cloud based installations on which multicast
is not allowed.
39

Read-only mode
2.8

http://mysqlhighavailability.com/mysql-group-replication-important-behavior-and-configuration-changes/
Read-only mode
• When a member joins the group, during distributed recovery, read-only
mode is set automatically.
• On a unlikely event of a member failure, read-only mode is set
automatically to prevent inconsistency with the group and member state
changes to ERROR.
41
M M M M M

Full stack secure connections
Read-only mode
2.8
2.9

http://mysqlhighavailability.com/mysql-group-replication-securing-the-perimeter/
• Following the industry standards, Group Replication supports secure
connections along the complete stack
– Client connections
– Distributed recovery connections
– Connections between members
• IP Whitelisting
– Restrict which hosts are allowed to connect to the group
– By default it is set to the value AUTOMATIC, which allows connections from private
subnetworks active on the host
43

Parallel applier support
Read-only mode
2.8
2.9
2.10

• Group Replication now also takes full advantage of parallel binary log
applier infrastructure
– Reduces applier lag and improves replication performance considerably
– Configured in the same way as in asynchronous replication
45
--slave_parallel_workers=NUMBER
--slave_parallel_type=logical_clock
--slave_preserve_commit_order=ON

Single Primary Mode
Read-only mode
2.8
2.9
2.10
2.11

Single Primary Mode
• Configuration mode that makes a single member act as a writeable master
(PRIMARY) and the rest of the members act as hot-standbys (SECONDARY).
– The group itself coordinates automatically to figure out which is the member that will
act as the PRIMARY, through a leader election mechanism.
• Default mode
– Closer to classic asynchronous replication setups, simpler to reason about from the
beginning.
– Avoids some of the limitations of multi-master mode.
47

Single Primary Mode
• Automatic leader election mechanism
• Secundaries are automatically set to read-only
48
S S S S P S
Primary

Single Primary Mode
• Automatic leader election mechanism
49
S S S P S
Primary

Single Primary Mode
• The current primary member UUID can be known by executing the
following SQL statement
50
mysql> SELECT * FROM performance_schema.global_status WHERE
VARIABLE_NAME='group_replication_primary_member';
VARIABLE_NAME VARIABLE_VALUE
group_replication_primary_member dcd3b36b-79c5-11e6-97b8-00212844d44e

Requirements
Single Primary Mode
Read-only mode
2.8
2.9
2.10
2.11
2.12

• Concurrent DDL (on multi-master)
Requirements (by design)
• Requires InnoDB storage engine
• Primary key is required on every table
• Requires global transaction identifiers
turned on
• Requires binary log turned on
• Requires binary log row format
• Optimistic execution: transactions may
abort on COMMIT due to conflicts
with concurrent transactions on other
members
• Up to 9 servers in the group
52
• Serializable (on multi-master)
• Cascading Foreign Keys (on multi-master)
• Transaction savepoints
• Binary log events checksum
Forbidden
Warnings

Architecture
Introduction
3
3.1

MySQL Group Replication is
• Built on top of proven technology!
– Shares many pieces of MySQL Replication.
– Multi-Master approach to replication.
• Built on reusable components!
– Layered implementation approach.
– Interface driven development.
– Decoupled from the server core.
– The plugin registers as listener to server events.
– Reuses the capture procedure from regular replication.
– Provides further decoupling from the communication infrasctructure.
54

Major Building Blocks
Architecture
Introduction
3
3.1
3.2

Major Building Blocks (1)
56
M M M M M
Com. API
Replication
Plugin
API
MySQL
Server
Group Comm.
System (Corosync)
Group Com. Engine

• Server calls into the plugin through a generic
interface
– (Most of server) internals are hidden from the plugin.
– Some of the semi-sync interfaces were reused. Others
were created.
• Plugin interacts with the server through a generic
interface
– Replication plugin determines the fate of the commit
operation through a well defined server interface.
– The plugin makes use of the relay log infrastructure to
inject changes in the receiving server.
57
Com. API
Replication
Plugin
API
MySQL
Server
Group Comm.
System (Corosync)
Group Com. Engine

• The plugin is responsible for
– Maintaining distributed execution context.
– Detecting conflicts.
– Handling distributed recovery:
• Detect membership changes;
• Donate state if needed;
• Collect state if needed.
– Receiving and handling transactions from other
members.
– Deciding the fate of on-going transactions.
58
Com. API
Replication
Plugin
API
MySQL
Server
Group Comm.
System (Corosync)
Group Com. Engine

• The communication API (and bindings) is
responsible for:
– Abstracting the underlaying communication engine
from the plugin itself.
– Mapping the interface to a specific communication
engine.
59
Com. API
Replication
Plugin
API
MySQL
Server
Group Comm.
System (Corosync)
Group Com. Engine

• The communication engine:
– Variant of Paxos developed at MySQL.
– Building block to provide distributed agreement
between servers.
60
Com. API
Replication
Plugin
API
MySQL
Server
Group Comm.
System (Corosync)
Group Com. Engine

The Complete Stack
Major Building Blocks
Architecture
Introduction
3
3.1
3.2
3.3

The Complete Stack
62
API
Replication
Plugin
API
MySQL
Server
Performance Schema Tables: Monitoring
MySQL
APIs: Lifecycle / Capture / Applier
InnoDB
Replication Protocol
Group Com. API
Group Com. Engine
Network
Plugin
Capture Applier
Conflicts
Handler
Group Comm.
System (Corosync)
Group Com. Engine
Group Com. Binding
Recovery

Use cases4

Use Cases
• Elastic Replication
– Environments that require a very fluid replication infrastructure, where the number
of servers has to grow or shrink dynamically and with as little pain as possible.
64
M M M M M

Use Cases
• Highly Available Shards
– Sharding is a popular approach to achieve write scale-out. Users can use MySQL
Group Replication to implement highly available shards. Each shard can map into a
Replication Group.
65
M M M M M M M M M M M M M M M

Use Cases
• Alternative to Master-Slave replication
• Single Primary mode provides further automation on such setups
– Automatic PRIMARY/SECONDARY roles assignment
– Automatic new PRIMARY election on PRIMARY failures
– Automatic setup of read/write modes on PRIMARY and SECONDARIES
– Global consistent view of which server is the PRIMARY
66
S S S S P S
Primary

Performance5

Performance
68
5000
10000
15000
20000
25000
30000
2 3 5 7 9
THROUGHPUT(TPS)
NUMBER OF SERVERS IN THE GROUP.
Group Replication Throughput
(as perceived by the client application)
Single-master Sustained Single-master Peak Multi-master Sustained
Multi-master Peak Single-server (MySQL 5.7.14)
Replication Performance blogs at:
http://mysqlhighavailability.com/category/performance/
Servers
9 Dual Xeon E5-2660-v3
Enterprise SSD Storage
10Gbps Ethernet Network
Client
1 Dual Xeon E5-2699-v3
10Gbps Ethernet Network
Sysbench 0.5 RW workload
Peak Throughput
The number of transactions that
writers can propagate to the group
(per second).
Sustained Throughput
The number of transactions that
can be propagated to the group
without increasing the replication
lag on any member (per second).

Performance
69
• On a sustained throughput:
– Multi-master performance degrades gracefully while going from a group with 2
servers to a group with 9 servers.
– Single primary performance degrades marginally when growing the group size.
• On a peak throughput:
– Multi-master exhibits 1.8X speedup when compared to the single server.
• Read load is balanced across the servers in the group.
• Write load is lower since execution is balanced across the group, whereas in single primary mode the
primary becomes a bottleneck.
– With a single primary there is no lag on the other members.

Conclusion6

Summary
• Cloud Friendly
– Great techonology for deployments where elasticity is a requirement, such as cloud
based infrastructures.
• Integrated
– With server core through a well defined API.
– With GTIDs, row based replication, performance schema tables.
• Autonomic and Operations Friendly
– It is self-healing: no admin overhead for handling server fail-overs.
– Provides fault-tolerance, enables multi-master update everywhere and a dependable
MySQL service.
• Lab releases provide a sneak peek at what is coming - a new replication plugin which is
the core for the new MySQL InnoDB Cluster.
71

MySQL InnoDB Cluster: The End Goal
72
S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
HA
ReplicaSet1
S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
HA
ReplicaSet2
ReplicaSet3
MySQL Connector
Application
MySQL Router
S1 S2 S3 S4
M
M M
HA

Where to go from here?
• Packages
– http://labs.mysql.com
• Blogs from the Engineers (news, technical information, and much more)
– http://mysqlhighavailability.com
73

MySQL HA with Group Replication

MySQL HA with Group Replication

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to MySQL HA with Group Replication

Similar to MySQL HA with Group Replication (20)

Recently uploaded

Recently uploaded (20)

MySQL HA with Group Replication