1. MySQL Multi-Source Replication
Wagner Bianchi - wagner.bianchi@percona.com
Principal Technical Services Engineer
Max Bubenick - max.bubenick@percona.com
Technical Operations Manager

2. www.percona.com
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
This is gonna be a multi-source replication walkthrough on MariaDB and Percona Server

3. www.percona.com
•
•
● enables a replication slave to receive transactions from multiple
sources/masters simultaneously;
● aggregate data from multiple servers - Data Mart/Data Warehouse;
● merge table shards - no auto_increment conflict control;
● can be carefully used connecting many location on one geo-positioned
cluster;
● centralize all data for backup propose.

4. www.percona.com

5. www.percona.com

6. www.percona.com
•
•
● No gtid_mode variable to turn on GTID
● Binary logs has positions and GTIDs
• CHANGE MASTER ‘’ TO
• CHANGE MASTER 'foo' TO
• CHANGE MASTER TO
● MASTER_USE_GTID=CURRENT_POS|SLAVE_POS
•
● Accessed using SHOW ALL SLAVES STATUS;
● SET @@default_master_connection works well;
set @@default_master_connection=''; -- default slave connection
show status like 'Slave_running';
set @@default_master_connection='connection_name01'; -- named connection name
show status like 'Slave_running';

7. www.percona.com
•
box01
Multi-Source Slave
box02
box03
box04
MariaDB [(none)]> pager egrep "Connection|Gtid"
PAGER set to 'egrep "Connection|Seconds"'
MariaDB [(none)]> show all slaves statusG
Connection_name: box02
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
Connection_name: box03
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
Connection_name: box04
Gtid_IO_Pos: 1-1-63,3-3-1
Gtid_Slave_Pos: 1-1-63,3-3-1
3 rows in set (0.00 sec)
On the multi-source slave side:
● Use SHOW ALL SLAVES STATUSG
● Use @@default_master_connection
● Use STOP/START ALL SLAVES
● Use STOP/START SLAVE 'box02'

8. www.percona.com
#: box01 - multi-source slave
[client]
port=3306
socket=/var/lib/mysql/mysql.sock
[mysqld]
user=mysql
port=3306
socket=/var/lib/mysql/mysql.sock
basedir=/usr
datadir=/var/lib/mysql
read_only=1
#: repl vars
server_id=1
report_host=box01
report_port=3306
report_user=repl
log_bin=mysql-bin
log_bin_index=mysql.index
log_slave_updates=true
binlog_format=ROW
#: verify checksum on master
master_verify_checksum=1
#: gtid vars
gtid_domain_id=1
gtid_ignore_duplicates=ON
gtid_strict_mode=1
#: msr slave parallel mode *
box02.slave_parallel_mode=optimistic
box03.slave_parallel_mode=optimistic
box04.slave_parallel_mode=optimistic
#: other slave variables
slave_parallel_threads=16
slave_domain_parallel_threads=4
slave_parallel_max_queued=512M
slave_net_timeout=15
slave_sql_verify_checksum=1
slave_compressed_protocol=1
#: binary log group commit behavior
#binlog_commit_wait_usec=100000
#binlog_commit_wait_count=20

9. www.percona.com
#: box01
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 1
@@gtid_domain_id: 1
1 row in set (0.00 sec)
#: box02
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 2
@@gtid_domain_id: 2
1 row in set (0.00 sec)
#: box03
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 3
@@gtid_domain_id: 3
1 row in set (0.00 sec)
#: box04
MariaDB [(none)]> select @@server_id,@@gtid_domain_idG
*************** 1. row ***************************
@@server_id: 4
@@gtid_domain_id: 4
1 row in set (0.00 sec)
box01
Multi-Source Slave
gtid_domain_id=1
gtid_domain_id=2
gtid_domain_id=3
gtid_domain_id=4

10. www.percona.com
•
● Make sure the replication user is set on all the servers;
● Make sure all the servers has unique server_id and gtid_domain_id;
•
#: Connection name with box02
MariaDB [(none)]> change master 'box02' to master_host='192.168.0.102',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
#: Connection name with box03
MariaDB [(none)]> change master 'box03' to master_host='192.168.0.103',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;
#: Connection name with box04
MariaDB [(none)]> change master 'box04' to master_host='192.168.0.104',
master_user='repl', master_password='Bi@nchI', master_use_gtid=current_pos;

11. www.percona.com
•
# start all slaves
MariaDB [(none)]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.02 sec)
MariaDB [(none)]> show warnings;
+-------+------+-----------------------+
| Level | Code | Message |
+-------+------+-----------------------+
| Note | 1937 | SLAVE 'box04' started |
| Note | 1937 | SLAVE 'box03' started |
| Note | 1937 | SLAVE 'box02' started |
+-------+------+-----------------------+
3 rows in set (0.00 sec)
MariaDB [(none)]> stop all slaves;
Query OK, 0 rows affected, 3 warnings (0.02 sec)

12. www.percona.com
•
#: setting the @@default_master_connection
MariaDB [(none)]> set @@default_master_connection='box02';
Query OK, 0 rows affected (0.00 sec)
#: showing some box02 Connection name's status replication variables
MariaDB [(none)]> pager egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos"
PAGER set to 'egrep "Slave_IO_State|Using_Gtid|Gtid_IO_Pos"'
MariaDB [(none)]> show slave statusG
Slave_IO_State: Waiting for master to send event
Using_Gtid: Current_Pos
Gtid_IO_Pos: 1-1-64,2-2-1,3-3-1
1 row in set (0.00 sec)
#: starting and stopping just box02, as per @@default_master_connection set
MariaDB [(none)]> stop slave;
Query OK, 0 rows affected (0.01 sec)
MariaDB [(none)]> start slave;
Query OK, 0 rows affected (0.02 sec)

13. www.percona.com
•
#: relay logs - one group for each set connection name - host+relay-bin+connection_name
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box02.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box02.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box02.index
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box03.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box03.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box03.index
-rw-rw---- 1 mysql mysql 306 Mar 23 00:32 maria01-relay-bin-box04.000001
-rw-rw---- 1 mysql mysql 663 Mar 23 00:32 maria01-relay-bin-box04.000002
-rw-rw---- 1 mysql mysql 66 Mar 23 00:32 maria01-relay-bin-box04.index
#: master.info - one per set connection name - master-connection_name.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box02.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box03.info
-rw-rw---- 1 mysql mysql 153 Mar 23 00:32 master-box04.info
#: multi-master.info file, listing all set connections names
-rw-rw---- 1 mysql mysql 18 Feb 15 11:00 multi-master.info
[root@maria01 mysql]# cat multi-master.info #: don't edit this file :)
box02
box03
box04

14. www.percona.com
•
•
•
#: check gtid_current and gtid_slave pos variables
MariaDB [mysql]> select @@gtid_current_pos, @@gtid_slave_posG
*************************** 1. row ***************************
@@gtid_current_pos: 1-1-64,2-2-1,3-3-1
@@gtid_slave_pos: 1-1-64,2-2-1,3-3-1
1 row in set (0.00 sec)
#: check current clave current and slave pos from mysql.gtid_slave_pos table
MariaDB [mysql]> select * from mysql.gtid_slave_pos;
+-----------+--------+-----------+--------+
| domain_id | sub_id | server_id | seq_no |
+-----------+--------+-----------+--------+
| 1 | 16 | 1 | 63 |
| 1 | 20 | 1 | 64 |
| 3 | 1 | 3 | 1 |
+-----------+--------+-----------+--------+
3 rows in set (0.00 sec)

15. www.percona.com
• @@GTID_CURRENT_POS
•
● CHANGE MASTER 'foo' TO … MASTER_USE_GTID=CURRENT_POS;
● @@GTID_STRICT_MODE=1
#: checking global variable @@gtid_current_pos
MariaDB [(none)]> SELECT @@GTID_CURRENT_POS,@@GTID_STRICT_MODEG
*************************** 1. row ***************************
@@GTID_CURRENT_POS: 1-1-68,2-2-149378,3-3-88622,4-4-98365
@@GTID_STRICT_MODE: 1 -- help keeping binlogs identical across multiple servers
1 row in set (0.00 sec)

16. www.percona.com
• @@GTID_SLAVE_POS CHANGE MASTER
TO
• CHANGE MASTER 'foo' TO … MASTER_USE_GTID=SLAVE_POS;
•
#: checking global variable @@gtid_slave_pos
MariaDB [(none)]> SELECT @@GTID_SLAVE_POSG
*************************** 1. row ***************************
@@GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365
1 row in set (0.00 sec)
#: checking global variable @@gtid_slave_pos (can't be set per Connection Name)
MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365';
Query OK, 0 rows affected, 3 warnings (0.01 sec)
#: SHOULD BE DONE AFTER STOPPING REPLICATION CONNECTION NAMES...

17. www.percona.com
•
•
#: comparing both @@gtid_current_pos and @@gtid_slave_pos
MariaDB [(none)]> SELECT @@GLOBAL.GTID_CURRENT_POS,@@GLOBAL.GTID_SLAVE_POSG
*************************** 1. row ***************************
@@GLOBAL.GTID_CURRENT_POS: 1-1-71,2-2-149378,3-3-88622,4-4-98365 -- 5 ADDITIONAL TRXs
@@GLOBAL.GTID_SLAVE_POS: 1-1-66,2-2-149378,3-3-88622,4-4-98365
1 row in set (0.00 sec)
#: setting dynamically @@gtid_slave_pos - TRANSACTIONS CANNOT BE REPLAYED
MariaDB [(none)]> SET GLOBAL GTID_SLAVE_POS='1-1-66,2-2-149378,3-3-88622,4-4-98365';
ERROR 1947 (HY000): Specified GTID 1-1-66 conflicts with the binary log which contains
a more recent GTID 1-1-71. If MASTER_GTID_POS=CURRENT_POS is used, the binlog position
will override the new value of @@gtid_slave_pos.
#: command above was embraced by START/STOP ALL SLAVES.

18. www.percona.com
•
● slave_parallel_threads sets the number of threads will take care
about the updates on relay logs based on their timestamps;
●
●
●
●
●
●
●

19. www.percona.com
•
#: making a multi-source slave multi-threaded
MariaDB [(none)]> stop all slaves;
Query OK, 0 rows affected, 3 warnings (0.00 sec)
MariaDB [(none)]> set global slave_parallel_threads=12;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> set global slave_domain_parallel_threads=4;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.03 sec)
MariaDB [(none)]> select @@slave_parallel_mode,@@slave_parallel_threads,@@slave_domain_parallel_threadsG
*************************** 1. row ***************************
@@slave_parallel_mode: optimistic # will retry transaction in case of parallelism conflicts
@@slave_parallel_threads: 12 # total of threads available for slave to execute relay logs
@@slave_domain_parallel_threads: 4 # minimum # of thread used for a domain_id all time
1 row in set (0.00 sec)

20. www.percona.com
•
#: making a multi-source slave multi-threaded
MariaDB [(none)]> SELECT ID,TIME,STATE,USER FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user';
+----+------+------------------------------------------------------------------+-------------+
| ID | TIME | STATE | USER |
+----+------+------------------------------------------------------------------+-------------+
| 18 | 452 | Waiting for master to send event | system user |
| 17 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 16 | 452 | Waiting for master to send event | system user |
| 15 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 14 | 364 | Slave has read all relay log; waiting for the slave I/O thread t | system user |
| 13 | 452 | Waiting for master to send event | system user |
| 12 | 452 | Waiting for work from SQL thread | system user |
| 11 | 452 | Waiting for work from SQL thread | system user |
| 10 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 9 | 0 | Unlocking tables | system user |
| 8 | 452 | Waiting for work from SQL thread | system user |
| 7 | 452 | Waiting for work from SQL thread | system user |
| 6 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 5 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 4 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
| 3 | 0 | Update_rows_log_event::ha_update_row(-1) | system user |
+----+------+------------------------------------------------------------------+-------------+
16 rows in set (0.07 sec)

21. www.percona.com
•
●
● binlog_commit_wait_usec=100000
● binlog_commit_wait_count=20
●
#: binary logs for group commit
[root@maria02 mysql]# mysqlbinlog mysql-bin.000017 -vvvv | egrep "cid=353579"
#160325 21:37:27 server id 2 end_log_pos 27101572 GTID 2-2-149349 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27103107 GTID 2-2-149350 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27104646 GTID 2-2-149351 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27106181 GTID 2-2-149352 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27107716 GTID 2-2-149353 cid=353579 trans
#160325 21:37:27 server id 2 end_log_pos 27109251 GTID 2-2-149354 cid=353579 trans
MariaDB [(none)]> show global status where variable_name in
('Binlog_commits','Binlog_group_commits');
+----------------------+-------+
| Variable_name | Value |
+----------------------+-------+
| Binlog_commits | 39681 |
| Binlog_group_commits | 5523 |
+----------------------+-------+
2 rows in set (0.02 sec)
the bigger the difference between the two variables
the bigger the apparent group commit efficiency

22. www.percona.com
•
•
•
•
•
•
•
•

23. www.percona.com
•
● One can restart both threads or just SQL_THREAD;
#: adding new schema to box02
box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1;
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
#: adding filters on multi-source slave
box01> stop slave 'box02'; set global box02.replicate_ignore_db='box02_new'; start slave 'box02';
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
#: remove filter
box01> stop slave; set global box02.replicate_ignore_db=''; start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)

24. www.percona.com
• sql_slave_skip_counter
● If parallel replication is enabled:
•
● Stop all slaves, turn off the parallel replication and skip_counter;
● Set a new value for @@GLOBAL.GTID_SLAVE_POS;
MariaDB [box02]> set default_master_connection='box02'; set global sql_slave_skip_counter=1;
Query OK, 0 rows affected (0.00 sec)
ERROR 1966 (HY000): When using parallel replication and GTID with multiple replication domains,
@@sql_slave_skip_counter can not be used. Instead, setting @@gtid_slave_pos explicitly can be used to skip
to after a given GTID position.
MariaDB [box02]> stop all slaves; set @@global.gtid_slave_pos='1-1-71,2-2-149382,3-3-88623,4-4-98365';
Query OK, 0 rows affected, 3 warnings (0.00 sec)
Query OK, 0 rows affected (0.01 sec)
MariaDB [box02]> start all slaves;
Query OK, 0 rows affected, 3 warnings (0.01 sec)

25. www.percona.com

26. www.percona.com
●
● gtid_mode=on
●
●
● PERFORMANCE_SCHEMA
● SHOW SLAVE STATUS
● SHOW SLAVE STATUS FOR CHANNEL
mysql> pager egrep "Slave_IO|Channel"
PAGER set to 'egrep "Slave_IO|Channel"'
mysql> show slave statusG
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box02
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box03
Slave_IO_State: Waiting for master to send event
Slave_IO_Running: Yes
Channel_Name: box04
3 rows in set (0.00 sec)

27. www.percona.com
box01
Multi-Source Slave
box02
box03
box04
#: multi-source slave configuration file
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
report_host=box01
report_port=3306
report_user=repl
#: gtid configuration
server_id=1
gtid_mode=on
log_bin=mysql-bin
log_bin_index=mysql-bin.index
log_slave_updates=true
enforce_gtid_consistency
#: binary logs (GROUP COMMIT)
binlog_group_commit_sync_delay=100000
binlog_group_commit_sync_no_delay_cont=20
#binlog_order_commits=0 # increase parallelism
#: repositories for crash-safe
master_info_repository=TABLE
relay_log_info_repository=TABLE
#: slave configs
slave_parallel_workers=4
slave_parallel_type='LOGICAL_CLOCK'
slave_compressed_protocol=1
slave_pending_jobs_size_max=256M

28. www.percona.com
•
•
#: Replication Channel for box02
box01> change master to master_host='192.168.0.12', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box02';
#: Replication Channel for box03
box01> change master to master_host='192.168.0.13', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box03';
#: Replication Channel for box04
box01> change master to master_host='192.168.0.14', master_user='repl',
master_password='Bi@nchI', master_auto_position=1 for channel 'box04';

29. www.percona.com
#: relay logs - one group for each set connection name
-rw-r----- 1 mysql mysql 847 Mar 27 00:51 percona01-relay-bin-box02.000018
-rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box02.000019
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box02.index
-rw-r----- 1 mysql mysql 597 Mar 27 00:51 percona01-relay-bin-box03.000017
-rw-r----- 1 mysql mysql 534 Mar 27 00:51 percona01-relay-bin-box03.000018
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box03.index
-rw-r----- 1 mysql mysql 550 Mar 27 00:51 percona01-relay-bin-box04.000017
-rw-r----- 1 mysql mysql 487 Mar 27 00:51 percona01-relay-bin-box04.000018
-rw-r----- 1 mysql mysql 70 Mar 27 00:51 percona01-relay-bin-box04.index
#:No master.info file as it needs to be configured with crash-safe, repos as TABLE
mysql> show variables where variable_name in
('master_info_repository','relay_log_info_repository');
+---------------------------+-------+
| Variable_name | Value |
+---------------------------+-------+
| master_info_repository | TABLE |
| relay_log_info_repository | TABLE |
+---------------------------+-------+
2 rows in set (0.00 sec)
•

30. www.percona.com
•
#: stopping all slaves
mysql> stop slave;
Query OK, 0 rows affected (0.06 sec)
mysql> show slave statusG
Slave_IO_Running: No
Slave_SQL_Running: No
[...snip...]
3 rows in set (0.00 sec)
#: stopping just one specific slave
mysql> stop slave for channel 'box02';
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status for channel 'box02'G
Slave_IO_Running: No
Slave_SQL_Running: No
Slave_SQL_Running_State:
1 row in set (0.00 sec)

31. www.percona.com
•
#: starting all slaves
mysql> start slave;
Query OK, 0 rows affected (0.06 sec)
mysql> show slave statusG
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
[...snip...]
3 rows in set (0.00 sec)
#: starting just one specific slave
mysql> start slave for channel 'box02';
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status for channel 'box02'G
Slave_IO_Running: No
Slave_SQL_Running: No
Slave_SQL_Running_State:
1 row in set (0.00 sec)

32. www.percona.com
•
#: show all slaves status
mysql> show slave statusG
Channel_Name: box02
Channel_Name: box03
Channel_Name: box04
3 rows in set (0.00 sec)
#: show slave status for a specific slave
mysql> show slave status for channel 'box02'G
Slave_IO_State: Waiting for master to send event
Retrieved_Gtid_Set: 61be13a1-d574-11e5-83c7-0800274fb806:1-61...
Executed_Gtid_Set: 4bd77dee-d572-11e5-b09f-0800274fb806:1-56...
Channel_Name: box02
1 row in set (0.00 sec)

33. www.percona.com
#: PERFORMANCE_SCHEMA replication tables
mysql> show tables from performance_schema like 'replication%';
+---------------------------------------------+
| Tables_in_performance_schema (replication%) |
+---------------------------------------------+
| replication_applier_configuration |
| replication_applier_status |
| replication_applier_status_by_coordinator |
| replication_applier_status_by_worker |
| replication_connection_configuration |
| replication_connection_status |
| replication_group_member_stats |
| replication_group_members |
+---------------------------------------------+
8 rows in set (0.00 sec)

34. www.percona.com
#: PERFORMANCE_SCHEMA replication tables
mysql> select channel_name,service_state,last_heartbeat_timestamp
-> from performance_schema.replication_connection_statusG
*************************** 1. row ***************************
channel_name: box02
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:39
*************************** 2. row ***************************
channel_name: box03
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:30
*************************** 3. row ***************************
channel_name: box04
service_state: ON
last_heartbeat_timestamp: 2016-03-26 22:22:31
3 rows in set (0.00 sec)

35. www.percona.com
mysql> select @@slave_parallel_workers, @@slave_parallel_type;
+--------------------------+-----------------------+
| @@slave_parallel_workers | @@slave_parallel_type |
+--------------------------+-----------------------+
| 8 | LOGICAL_CLOCK |
+--------------------------+-----------------------+
1 row in set (0.00 sec)
#: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves)
mysql> select channel_name, service_state
-> from performance_schema.replication_applier_status_by_coordinatorG
*************************** 1. row ***************************
channel_name: box02
service_state: ON
*************************** 2. row ***************************
channel_name: box03
service_state: ON
*************************** 3. row ***************************
channel_name: box04
service_state: ON
3 rows in set (0.00 sec)

36. www.percona.com
#: PERFORMANCE_SCHEMA, checking coordinators state (multi-threaded slaves)
mysql> select channel_name,thread_id,service_state,last_seen_transaction
-> from performance_schema.replication_applier_status_by_worker
-> where last_seen_transaction<>''G
*************************** 1. row ***************************
channel_name: box03
thread_id: 71
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2536
*************************** 2. row ***************************
channel_name: box03
thread_id: 72
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2537
*************************** 3. row ***************************
channel_name: box03
thread_id: 73
service_state: ON
last_seen_transaction: ab001313-d573-11e5-bc39-0800274fb806:2538
...

37. www.percona.com
• CHANGE REPLICATION FILTER
#: adding new schema to box02
box02> set sql_log_bin=0; create database if not exists box02_new; set sql_log_bin=1;
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
#: adding filters on multi-source slave
mysql> stop slave; change replication filter replicate_ignore_db=(box02_new); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)
#: remove filter
mysql> stop slave; change replication filter replicate_ignore_db=(); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.11 sec)

38. www.percona.com
•
• REPLICATE_DO_DB
• REPLICATE_IGNORE_DB
• REPLICATE_DO_TABLE
• REPLICATE_IGNORE_TABLE
• REPLICATE_WILD_DO_TABLE
• REPLICATE_WILD_IGNORE_TABLE
• REPLICATE_REWRITE_DB
•
•
● http://bugs.mysql.com/bug.php?id=80843

39. www.percona.com
#: rewriting updates on db A to db B
box01> stop slave; change replication filter replicate_rewrite_db=((box02,box03)); start slave;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.07 sec)
#: writing some data to A to be routed to B
box02> insert into box02.t1 set i=10;
Query OK, 1 row affected (1.01 sec)
#: checking data on A
box01> select * from box02.t1;
Empty set (0.00 sec)
#: checking data on B
box01> select * from box03.t1G
*************************** 1. row ***************************
i: 10
1 row in set (0.00 sec)
#: remove filter
box01> stop slave; change replication filter replicate_rewrite_db=(); start slave;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.12 sec)
● REPLICATE_REWRITE_DB

40. www.percona.com
•
●
●
box01> SELECT @@HOSTNAME,ID,USER,STATE,TIME,INFO
-> FROM INFORMATION_SCHEMA.PROCESSLIST WHERE USER='system user';
+------------+----+-------------+--------------------------------------------------------+------+------+
| @@HOSTNAME | ID | USER | STATE | TIME | INFO |
+------------+----+-------------+--------------------------------------------------------+------+------+
| box01 | 32 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 33 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 34 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 35 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 36 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 25 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 26 | system user | Waiting for an event from Coordinator | 721 | NULL |
| box01 | 27 | system user | Waiting for master to send event | 721 | NULL |
[...snip...]
+------------+----+-------------+--------------------------------------------------------+------+------+
30 rows in set (0.00 sec)

41. www.percona.com
•
UUID()
mysql> select * from performance_schema.replication_applier_status_by_workerG
*************************** 1. row ***************************
CHANNEL_NAME: box02
WORKER_ID: 1
THREAD_ID: NULL
SERVICE_STATE: OFF
LAST_SEEN_TRANSACTION: fa11b361-d572-11e5-b63e-0800274fb806:66
LAST_ERROR_NUMBER: 1062
LAST_ERROR_MESSAGE: Worker 0 failed executing transaction 'fa11b361-d572-11e5-b63e-
0800274fb806:66' at master log mysql-bin.000008, end_log_pos 793; Could not execute Write_rows event on table
box02.t1; Duplicate entry '1' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the
event's master log mysql-bin.000008, end_log_pos 793
LAST_ERROR_TIMESTAMP: 2016-04-21 00:18:14
mysql> stop slave for channel 'box02'; set gtid_next='fa11b361-d572-11e5-b63e-0800274fb806:66'; begin;
commit; set gtid_next=automatic; start slave for channel 'box02';
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (1.01 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.02 sec)

42. www.percona.com
Feature MariaDB MySQL 5.7
Multi-Source Slave
Creation
CHANGE MASTER 'name' TO...; CHANGE MASTER TO … FOR CHANNEL 'name';
Parallel Threads slave_parallel_mode=optimistic
slave_parallel_threads=16 # ALL
slave_domain_parallel_threads=4
slave_parallel_max_queued=512M
slave_parallel_workers=4 # per RC
slave_parallel_type='LOGICAL_CLOCK'
slave_pending_jobs_size_max=256M
Replication Filters set global box02.
replicate_ignore_db='foo';
CHANGE REPLICATION FILTER ...
Skip Replication Errors SET GLOBAL GTID_SLAVE_POS='1-1-66';
SET GLOBAL
slave_exec_mode='IDEMPOTENT';
SET GLOBAL sql_slave_skip_counter=1;
SET GLOBAL slave_exec_mode='IDEMPOTENT';
SET GTID_NEXT='UUID:TRX_ID'
SET GLOBAL sql_slave_skip_counter=1; start
slave for channel 'xxxx'; (NO GTID)
Possible number of
sources/masters
You can for now only have 64 masters
256 replication channels for any combination
of hostname and port

43. www.percona.com