MySQL InnoDB Cluster: A Complete HA Solution

MySQL InnoDB Cluster
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
@freshdaz
http://dasini.net/blog/
Copyright 2017, Oracle and/or its affiliates. All rights reserved
A complete High Availability solution for MySQL
PHP Tour 2017 - Nantes

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Copyright © 2017 Oracle and/or its affiliates. All rights reserved. |
Me, Myself & I
➢ MySQL Geek
✔ Addicted to MySQL for 15+ years!
✔ Playing with databases for 20+ years
➢ MySQL Writer, Blogger and Speaker
✔ Also former : DBA, Consultant, Architect, Trainer, ...
➢ MySQL Principal Solutions Architect EMEA at Oracle
➢ Stay tuned! :
✔ @freshdaz
✔ http://dasini.net/blog
3
Olivier DASINI

Program Agenda
1
2
3
State of MySQL
High Availability
Demo : Deploying MySQL InnoDB Cluster
4
4

The world's most popular open source database

A Modern Database for the Digital Age
Digital Disruptors & Large Enterprises Rely on MySQL to Innovate
6

They Scale
with MySQL
Mobile Network Supporting
Over 800 Million
Subscribers
1.7 Billion Active Users
100 TB of User Data for PayPal
IDs Processed for
1 Billion Citizens
850 Million Candy Crush Game
Plays/Day
2 Billion Events/Day for Booking.com
7

MySQL Business Model since 2005
• Community Edition – GPL License
– “You must cause any work that you distribute …, that … in part contains … the
Program … to be licensed as a whole at no charge to all third parties under the terms
of this License.”
• Commercial Editions – Commercial License
– Avoid any GPL issues
8
Improve
Performance
&
Scalability
Enhance Agility
& Productivity
Reduce TCO
Mitigate
Risks
Get
Immediate
Help
if/when
Needed
Increase
Customer
Satisfaction

MySQL within Oracle
More Investment, More Innovation
2009 2017
9
... 22 Years MySQL
… 12 Years InnoDB of Oracle Stewardship
… 7 Years MySQL within Oracle
 2x Engineering Staff
 3x QA Staff
 2x Support Staff

MySQL within Oracle : Opensource and Commercial Extensions
10
• Lossless Semi-sync Replication
• Global Transaction Ids
• Multisource Replication
• Multithreaded Replication
• Group Replication
• Performance_schema
• GIS extensions
• Fulltext search in InnoDB
• Massive Scalability improvement
• Secure by default
• Transparent Database Encryption
• Online DDL Operations
• Transportable Tablespaces
• Document Store / X DevAPI
• JSON Datatype
• JSON SQL Functions
• Generated Columns
• Multiple Triggers per Table
• MySQL Utilities
• MySQL Router
• MySQL Shell
• MySQL Cloud Service
• MySQL Enterprise Backup
• MySQL Enterprise TDE
• MySQL Enterprise Audit
• MySQL Enterprise Firewall
• MySQL Enterprise Authentication
• MySQL Enterprise Thread Pooling
• Memcached Interface
• Optimizer Enhancements
• Query Rewrite Plugin
• Fractional Seconds in time
datatypes
• Password Validation Plugin
• InnoDB Bufferpool Dump
• General Tablespaces
• Separation of Tablespaces
• Persistent Optimizer Statistics
• Characterset Improvements
• Sys Schema
• InnoDB SSD optimizations
• InnoDB native Partitioning
• Dynamic Bufferpool Resizing

5.7 since GA December 2015
11

MySQL 5.7 – Improvements across the board
• Replication
• InnoDB
• Optimizer
• Security
• Performance Schema
• GIS
• Triggers
• Partitioning
• Performance
• New! SYS Schema
• New! JSON
• New! Group Replication
• New! Document Store
12
150+ new
features
In total!
http://www.thecompletelistoffeatures.com/

Program Agenda
1
2
3
State of MySQL
High Availability
13
4

High Availability: Factors
• Environment
– Redundant servers in different datacenters and geographical areas will protect you against regional
issues—power grid failures, hurricanes, earthquakes, etc.
• Hardware
– Each part of your hardware stack—networking, storage, servers—should be redundant
• Software
– Every layer of the software stack needs to be duplicated and distributed across separate hardware and
environments
• Data
– Data loss and inconsistency/corruption must be prevented by having multiple copies of each piece of
data, with consistency checks and guarantees for each change
15

High Availability: The Causes of Downtime
16
40.00%
40.00%
20.00%
Software/Application Human Error Hardware
* Source: Gartner Group 1998 survey
A study by the Gartner Group
projected that through 2015, 80% of
downtime will be due to people and
process issues

High Availability: The Business Cost of Downtime
• Calculate a cost per minute of downtime
– Average revenue generated per-minute over a year
– Cost of not meeting any customer SLAs
– Factor in costs that are harder to quantify
1. Revenue
2. Reputation
3. Customer sentiment
4. Stock price
5. Service’s success
6. Company’s very existence
17
THIS is why HA
matters!

Multiple copies of data ? How ?
18

Multiple copies of data with replication process

HA solutions with MySQL : Replication modes
• Asynchronous => MySQL Replication
— MySQL Default; In parallel: Master acks to app and sends transaction to slave
— Fast; Risk of lost changes if master dies
• Semi-Synchronous => MySQL Replication w/ semisynchronous plugin
— MySQL 5.5+ Enhanced in MySQL 5.7; Serially: Master waits for change to be received by slave then In parallel ack to app and apply changes on slave
— Intermediate latency; Lossless (MySQL 5.7)
• Virtual-Synchronous => MySQL InnoDB Cluster w/ MySQL Group Replication
— MySQL Group Replication; Multi-masters updates on nodes applied in parallel
— MySQL Plugin delivered by MySQL for MySQL :)
— Intermediate latency; Best suited to small transactions; Lossless
• Synchronous => MySQL NDB Cluster
— Only available with MySQL Cluster; Serially: Master waits for change to be applied on all slaves before ack to app
— Higher latency; If Active/Active, best suited to small transactions; Lossless

MySQL Database Replication: Some Notes
Coordination Between Servers

Replication
22

Copyright 2017, Oracle and/or its affiliates. All rights reserved 25
B
C
CrashCrash
B is the
new master
Ouch!!! Whew!
https://dev.mysql.com/downloads/utilities/
B
C
A
C
B
Slave promotion managed by mysqlrpladmin or mysqlfailover

NDB Cluster
26

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 27
MySQL NDB Cluster Data Nodes
Clients
Application Layer
Data Layer
Management
MySQL NDB Cluster Architecture

• MySQL NDB Cluster
–In memory distributed database
–NDB storage engine
–Automatic sharding
–Native access via several API
–Read/write consistency
–Read/write scalability
–Management via MCM/mgm-tool
MySQL NDB Cluster
Rock Solid
Availability
In-Memory
Real-Time
Performance
Extreme
Read & Write
Scalability
Elasticity

Program Agenda
1
2
3
State of MySQL
High Availability
30
4

Collection of products that work together to provide a complete High Availability solution for MySQL

InnoDB Cluster
32
App Servers with
MySQL Router
MySQL Group Replication
“High Availability becomes a core
first class feature of MySQL!”
MySQL Shell
Setup, Manage,
Orchestrate
http://dasini.net/blog/2017/05/11/tutoriel-deployer-mysql-innodb-cluster/

A single product: MySQL
• All components created together
• Tested together
• Packaged together
Flexible and Modern
• C++ 11
• Protocol Buffers
• Developer friendly
MySQL InnoDB Cluster – Goals
Easy to use
• A single client: MySQL Shell
• Easy packaging
• Homogenous servers
Scale-out
• Sharded clusters
• Federated system of N replica sets
• Each replica set manages a shard

MySQL InnoDB Cluster: The Big Picture
P
App
S S
MySQL Shell and Orchestration Tooling
App
Simple Mapping,
State and Extra Metadata
Control, Coordinate,
Provision
Monitoring (MEM)
MySQL Router
MySQL Router

Native, built-in High Availability for your MySQL databases
36

Group Replication
37
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

MySQL Group Replication: What Is It?
• Group Replication library
– Implementation of Replicated Database State Machine theory
• MySQL GCS is based on Paxos (variant of Mencius)
– Provides virtually synchronous replication for MySQL 5.7+
– Supported on all MySQL platforms
• Linux, Windows, Solaris, OSX, FreeBSD
“Single/Multi-master update everywhere replication plugin for MySQL with built-in automatic
distributed recovery, conflict detection and group membership.”
39
http://dasini.net/blog/2016/11/08/deployer-un-cluster-mysql-group-replication/

• A Highly Available distributed MySQL database service
– Clustering eliminates single points of failure (No SPOF)
• Allows for online maintenance
– Removes the need for handling server fail-over
– Provides fault tolerance
– Enables update everywhere setups
– Automates group reconfiguration (handling of crashes, failures, re-connects)
– Provides a highly available replicated database
– Automatically ensures data consistency ie no data loss
• Detects and handles conflicts
• Prevents data loss
• Prevents data corruption
40
MySQL Group Replication: What Does It Provide?

MySQL Group Replication: Use Cases
• Elastic Replication
– Environments that require a very fluid replication infrastructure, where the
number of servers has to grow or shrink dynamically and with little pain as possible.
• Highly Available Shards
– Sharding is a popular approach to achieve write scale-out. Users can use MySQL
Group Replication to implement highly available shards in a federated system.
Each shard can map into a Replication Group.
• Alternative to Master-Slave Replication
– Single-primary mode provides further automation on such setups
● Automatic PRIMARY/SECONDARY roles assignment
● Automatic new PRIMARY election on PRIMARY failures
● Automatic setup of read/write modes on PRIMARY and SECONDARIES
● Global consistent view of which server is the PRIMARY
42

MySQL Group Replication: Architecture
Node Types
R: Traffic routers/proxies: mysqlrouter, ProxySQL, HAProxy...
M: mysqld nodes participating in Group Replication
44

MySQL Group Replication: Performance Comparison
45

Full stack secure connections
• Following the industry standards, Group Replication supports secure connections along the
complete stack
– Client connections
– Distributed recovery connections
– Connections between members
• IP Whitelisting
– Restrict which hosts are allowed to connect to the group
– By default it is set to the values AUTOMATIC, which allow connections from private subnetworks active
on the host
46
http://mysqlhighavailability.com/mysql-group-replication-securing-the-perimeter/

Parallel applier support
• Group Replication now also takes full advantage of parallel binary log applier infrastructure
– Reduces applier lad and improves replication performance considerably
– Configured in the sale way as asynchronous replication
47
slave_parallel_workers=<NUMBER>
slave_parallel_type=logical_clock
slave_preserve_commit_order=ON
slave_parallel_workers=<NUMBER>
slave_parallel_type=logical_clock
slave_preserve_commit_order=ON

Single Primary Mode
• Configuration mode that makes a single member act as a writeable master (PRIMARY) and the rest of the members
act as hot-standbys (SECONDARIES)
– The group itself coordinates automatically to figure out which is the member that will act as the PRIMARY, through an
automatic primary election mechanism
– Secondaries are automatically set to read-only
• Single_primary mode is the default mode
– Closer to classic asynchronous replication setups, simpler to reason about the beginning
– Avoids some limitations of multi-primary mode by default
• The current PRIMARY member UUID can be know by executing the following SQL statement:
48
mysql> SELECT * FROM performance_schema.global_status WHERE VARIABLE_NAME='group_replication_primary_member'G
*************************** 1. row ***************************
VARIABLE_NAME: group_replication_primary_member
VARIABLE_VALUE: dcd3b36b-79c5-11e6-97b8-00212844d44e
mysql> SELECT * FROM performance_schema.global_status WHERE VARIABLE_NAME='group_replication_primary_member'G
*************************** 1. row ***************************
VARIABLE_NAME: group_replication_primary_member
VARIABLE_VALUE: dcd3b36b-79c5-11e6-97b8-00212844d44e

Multi-primary Mode
• Configuration mode that makes all members writeable
– Enabled by setting option --group_replication_single_primary_mode to OFF
• Any two transactions on different servers can write to the same tuple
• Conflicts will be detected and dealt with
– First committer wins rule
49

Transaction life cycle
50

Synchronous Transaction Delivery
51

Full Transaction life cycle
52

Traditional vs Optimistic locking
53
Traditional locking Optimistic locking
Trx 2 waits
on Trx 1 commit
ERROR 1180 (HY000): Got error 149 during COMMIT

Monitoring
• performance_schema.replication_group_member_stats
– Monitoring the performance of the members connected in the group
• performance_schema.replication_group_members
– Used for monitoring the status of the different server instances that
are tracked in the current view
• performance_schema.replication_connection_status
– Show information regarding Group Replication
• performance_schema.replication_applier_status
– The state of the Group Replication related channels and thread
55

MySQL Enterprise Monitor 3.4
• Native holistic support for Group Replication clusters
– Intelligent monitoring and alerting
– Topology views
– Detailed metrics and graphs
– Best Practice advice
• Monitoring of MySQL Router instances
56

MySQL Enterprise Monitor 3.4
57
• Group Replication with 3 online nodes
• Group Replication with 3 nodes :
– 2 online
– 1 unreachable

Storage Engine InnoDB
Distributed Architecture Shared nothing
Network failure protection Quorum
NoSQL APIs MySQL Document Store / InnoDB Memcached
Replication Virtually Synchronous
Database Failover Automatic
OS Supported Linux, Windows, Solaris, OSX, FreeBSD
Operational Complexity Medium
Administration Standard (MySQL)
MySQL Group Replication overview

MySQL Group Replication wrap-up
• Cloud Friendly
– Great technology for deployments where elasticity is a requirement, such as cloud based infrastructures.
• Integrated
– With server core through a well defined API.
– With GTIDs, row based replication, performance schema tables.
– Share many pieces of MySQL Replication
– Mutli-primary approach to replication
• Autonomic and Operations Friendly
– It is self-healing: no admin overhead for handling server fail-overs.
– Provides fault-tolerance, enables multi-master update everywhere and a dependable MySQL service.
• Plugin GA version available with MySQL 5.7.17+, available on 8.0.1+
59

App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
Router 2.1

MySQL Router 2.1
• MySQL Router is lightweight middleware that provides transparent routing
between your application and back-end MySQL Servers.
• MySQL Router also provides High Availability and Scalability by effectively
routing database traffic to appropriate back-end MySQL Servers.
• The pluggable architecture also enables developers to extend MySQL Router
for custom use cases.
61

MySQL Router 2.1
• Native support for MySQL InnoDB clusters
– Understands Group Replication topology
– Utilizes metadata schema stored on each member
• Bootstraps itself and sets up client routing for the GR cluster
• Allows for intelligent client routing into the GR cluster
• Supports multi-master and single primary modes
• Core improvements
– Logging
– Monitoring
– Performance
– Security
62

Shell
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate

MySQL Shell
64
A single unified client for all administrative and operations tasks
• Advanced command-line client and code editor for the MySQL Server
• Multi-Language: JavaScript, Python, and SQL
– Naturally scriptable
• Supports both Document and Relational models
• Exposes full Development and Admin API
”MySQL Shell provides the developer and DBA with a single intuitive,
flexible, and powerful interface for all MySQL related tasks!”

MySQL Shell: Admin API
• mysql-js> dba.help()
• The global variable 'dba' is used to access the MySQL
AdminAPI
• Perform DBA operations
– Manage MySQL InnoDB clusters
• Create clusters
• Get cluster info
• Start/Stop MySQL Instances
• Validate MySQL instances
• …
Database Administration Interface
App Servers with
MySQL Router
MySQL Shell
Setup, Manage,
Orchestrate
65

S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
MySQL Connector
Application
MySQL Router
MySQL Shell
HA
ReplicaSet(Shard1)
S1 S2 S3 S4 S…
M
M M
MySQL Connector
Application
MySQL Router
HA
ReplicaSet(Shard2)
S1 S2 S3 S4 S…
M
M M
HA
ReplicaSet(Shard3)
MySQL Connector
Application
MySQL Router
MySQL InnoDB Cluster – The End Goal
…
MySQL
InnoDB
cluster

Program Agenda
1
2
3
State of MySQL
High Availability
67
4

Deploying MySQL InnoDB Cluster
in Single-Primary Mode

MySQL InnoDB Cluster: Demo’s architecture

Testing MySQL InnoDB Cluster in a sandbox – Demo 1/6
http://dasini.net/blog/2017/03/13/tester-mysql-innodb-cluster/
1/ Connect to MySQL Shell
$ mysqlsh
2/ Deploy 3 local MySQL instances on ports : 3310 - 3320 - 3330
mysql-js> dba.deploySandboxInstance(3310)
• Deploying local sandbox MySQL Instances
Now we have 3 MySQL server instances
running on ports 3310, 3320 and 3330.

Stop
mysql-js> dba.stopSandboxInstance(3310);
Start
mysql-js> dba.startSandboxInstance(3320);
Kill : simulate an unexpected halt while testing failover
mysql-js> dba.killSandboxInstance(3330);
Delete : Completely removes the sandbox instance from your file system
mysql-js> dba.deleteSandboxInstance(3330);
• Managing local sandbox MySQL Instances
Once a sandbox instance is running, it is possible to change its status at any time :

Check instance configuration before the deployment
mysql-js> dba.checkInstanceConfiguration('root@localhost:3310')
• Checking the configuration
Configuration issues could be fixed manually or with configureLocalInstance
e.g.
mysql-js> dba.configureLocalInstance('root@localhost:3310')

1/ Connect to instance 3310
mysql-js> c root@localhost:3310
2/ Create the cluster
mysql-js> var cluster=dba.createCluster('demoCluster');
3/ Check the transaction set and add node 3320 to the cluster
mysql-js> cluster.checkInstanceState('root@localhost:3320')
mysql-js> cluster.addInstance('root@localhost:3320')
4/ Check the transaction set and add node 3330 to the cluster
mysql-js> cluster.checkInstanceState('root@localhost:3330')
mysql-js> cluster.addInstance('root@localhost:3330')
5/ Check cluster status
mysql-js> cluster.status()
• Deploying the InnoDB Cluster

# Bootstrap MySQL Router
$ mysqlrouter --bootstrap root@localhost:3310 --directory ~/demoSandbox
# Start MySQL router
$ ~/demoSandbox/start.sh
• Deploying the Router
The following connection information can be used to connect to the cluster.
Classic MySQL protocol connections to cluster 'demoCluster':
- Read/Write Connections: localhost : 6446
- Read/Only Connections: localhost : 6447
X protocol connections to cluster 'demoCluster':
- Read/Write Connections: localhost : 64460
- Read/Only Connections: localhost : 64470

1/ Connect to MySQL Router (default port = 6446)
$ mysqlsh --uri=root@localhost:64466446
2/ Select primary node’s port
mysql-js> sql
mysql-sql>
SELECT @@port; -- display 3310
3/ Primary node crash!
mysql-sql> js
mysql-js>
dba.killSandboxInstance(3310)
• Failover
4/ Select primary node’s port
mysql-js> c root@localhost:6446
mysql-sql>
SELECT @@port; -- display 3320
5/ Member is back to the group
mysql-sql> js
mysql-js> dba.startSandboxInstance(3310)
var cluster =
dba.getCluster('demoCluster')
mysql-js>
cluster.rejoinInstance('root@localhost:3310')
mysql-js> cluster.status()

Scale-Out
High Performance
Ease-of-Use
Built-in HA
Out-of-Box Solution
Everything Integrated
MySQL
InnoDB
cluster
MySQL InnoDB Cluster – Goals

Where to go from here?
83
• Packages
– http://www.mysql.com/downloads/
• Documentation
– http://dev.mysql.com/doc/refman/5.7/en/group-replication.html
– https://dev.mysql.com/doc/refman/5.7/en/mysql-innodb-cluster-userguide.html
– https://dev.mysql.com/doc/refman/5.7/en/mysql-shell.html
– https://dev.mysql.com/doc/mysql-router/2.1/en/
– http://dev.mysql.com/doc/refman/8.0/en/group-replication.html
• Blogs from the Engineers (news, technical information, and much more)
– http://mysqlhighavailability.com
– http://mysqlserverteam.com/
• Ressources en français
– http://dasini.net/blog/2017/05/11/tutoriel-deployer-mysql-innodb-cluster/
– http://dasini.net/blog/2017/03/13/tester-mysql-innodb-cluster/
– http://dasini.net/blog/2016/11/08/deployer-un-cluster-mysql-group-replication/

Merci !
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
@freshdaz
http://dasini.net/blog/
PHP Tour 2017 - Nantes

MySQL InnoDB Cluster: A Complete HA Solution

MySQL InnoDB Cluster: A Complete HA Solution

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to MySQL InnoDB Cluster: A Complete HA Solution

Similar to MySQL InnoDB Cluster: A Complete HA Solution (20)

More from Olivier DASINI

More from Olivier DASINI (20)

Recently uploaded

Recently uploaded (20)

MySQL InnoDB Cluster: A Complete HA Solution