SlideShare a Scribd company logo
1 of 255
MySQL::Replication
what is it?
It's a replacement for
MySQL Replication
why?
first...
what is replication?
synchronising data
between multiple databases
db1 db2
why replicate?
examples...
redundancy
database client client client client client client client client
database on client client client client client client client client
database on fire client client client client client client client client
database on fire client client client client client client client client
database on fire client client client client client client client client
arhhhh!
If we had replicated
to a hot standby
clients could be moved
 
 
another example
performance
database client
database client client
database client client client
database client client client client
database client client client client client
database client client client client client client
database client client client client client client client
database client client client client client client client client
database client client client client client client client client
database client client client client client client client client
arhhhh!
If we replicated
for scale out
load would be spread
 
 
 
 
 
 
 
 
so...
why replicate?
because it gives you benefits
compared to having
a single database
so...
how does it work?
in particular...
how does MySQL Replication work?
 
master
master slave
master client slave
master client binlog slave
master client binlog slave relay
master client binlog slave relay
the cool thing is
each slave
can be a master
to other slaves
master slave slave slave
master slave slave master/slave slave
master master/slave slave master/slave slave slave
this works fine
but...
slaves can only have a  single  master
master master/slave slave master/slave slave slave
so?
that means...
no multiple masters
slave master master master
slave master master master
why not?
don't ask me
because I don't know
but...
we can  emulate  multiple masters
to do this
we have to
setup a ring topology
what?
each slave
is a master
to another slave
master slave
master slave master/slave
master master/slave master/slave slave
master/slave master/slave master/slave master/slave
 
so...
how does this achieve
multiple master replication?
by having each master
write  all  queries
to it's binlog
...including the queries
it replicated
from  its  master
huh?
queries are passed
around the ring
kind of like
pass the parcel
 
why is this a problem?
when the ring breaks
 
 
 
 
 
arhhhh!
so does everything else
...and recovery is hard
why?
infinite loops!
what do you mean?
each master
is responsible
for termination
of its own queries
around the ring...
in other words
queries don't replicate infinitely
because
a master won't relay
its own queries twice
 
so...
what happens if...
a database goes down?
we recover by
...reconnecting the links
...and restarting replication
?
 
 
(OpenOffice doesn't do the arrow I wanted)
hang on...
who was responsible
for terminating
the failed database's queries?
hrmmm...
if there are queries
from the failed database
still replicating
that haven't yet terminated
 
arhhhh
Infinite queries, infinitely replicating
so...
how can I achieve
multiple master replication
...that is fault tolerant
...and is easy to recover from?
MySQL::Replication
It's a replacement
for MySQL's built-in replication
it's a client/server model
each master runs the server
slave master master master
slave master master server
slave server master server
slave server server server
each slave runs the client
...and the client
can run multiple instances
(one for each server)
(think peer-to-peer)
slave server server server
client(s) server server server
and if something breaks
client(s) server server server
client(s) server server server
client(s) server server server
only the peer connection is effected
client(s) server server server
client(s) server server server
and if the failed database recovers
we restart from where we left off
client(s) server server server
client(s) server server server
otherwise
we just fail it out
and can take our time rebuilding
it was redundant anyway
right?
:)
ohh
...and no infinite loops
why?
since we connect
directly to the server
no intermediary connections
can break
there's more...
relay caching
say what?
instead of connecting
directly to the master
the client
connects to a local relay
if it doesn't have
the queries we want
it will get them for us
example
multiple clients
in data center  A
...each connect
to a server
in data center  B
...each download the same queries
what a waste
...in bandwidth
...and server load
so...
instead
...each connect
to a local relay
now...
only a single client (the relay)
connects to the server
...saves bandwidth
...saves server load
:)
so...
CPAN?
almost ready
client/server working in production
I just need to do the documentation
then I'll put it on CPAN
what about the relay stuff?
It's nearly complete...
I just need to finish
...a couple more test harnesses
...and write the documentation
FAQ
what happens with collisions?
when two databases
update the same record
It's a race condition
solution?
solve it at the application layer
use a broker for global IDs
or shard writes
questions?
[email_address]

More Related Content

What's hot

CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
Skills Matter Talks
 
CoffeeScript presentation
CoffeeScript presentationCoffeeScript presentation
CoffeeScript presentation
John Lynch
 
AJS — Asynchronous Templating in Node
AJS — Asynchronous Templating in NodeAJS — Asynchronous Templating in Node
AJS — Asynchronous Templating in Node
Evan Owen
 
Phorum MySQL tricks
Phorum MySQL tricksPhorum MySQL tricks
Phorum MySQL tricks
guestd34230
 
How to become a practical Vim user
How to become a practical Vim userHow to become a practical Vim user
How to become a practical Vim user
Kana Natsuno
 

What's hot (19)

2019 PHP Serbia - Boosting your performance with Blackfire
2019 PHP Serbia - Boosting your performance with Blackfire2019 PHP Serbia - Boosting your performance with Blackfire
2019 PHP Serbia - Boosting your performance with Blackfire
 
CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
CukeUp! 2012: Michael Nacos on Just enough infrastructure for product develop...
 
CoffeeScript presentation
CoffeeScript presentationCoffeeScript presentation
CoffeeScript presentation
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
 
STP201 Efficiency at Scale - AWS re: Invent 2012
STP201 Efficiency at Scale - AWS re: Invent 2012STP201 Efficiency at Scale - AWS re: Invent 2012
STP201 Efficiency at Scale - AWS re: Invent 2012
 
Scaling Ruby with Evented I/O - Ruby underground
Scaling Ruby with Evented I/O - Ruby undergroundScaling Ruby with Evented I/O - Ruby underground
Scaling Ruby with Evented I/O - Ruby underground
 
Ruby 1.9 Fibers
Ruby 1.9 FibersRuby 1.9 Fibers
Ruby 1.9 Fibers
 
SmpleGroovyexampleinmule
SmpleGroovyexampleinmuleSmpleGroovyexampleinmule
SmpleGroovyexampleinmule
 
AJS — Asynchronous Templating in Node
AJS — Asynchronous Templating in NodeAJS — Asynchronous Templating in Node
AJS — Asynchronous Templating in Node
 
Top4top Showcase
Top4top ShowcaseTop4top Showcase
Top4top Showcase
 
Message Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHPMessage Queues & Offline Processing with PHP
Message Queues & Offline Processing with PHP
 
Ruby Proxies for Scale, Performance, and Monitoring
Ruby Proxies for Scale, Performance, and MonitoringRuby Proxies for Scale, Performance, and Monitoring
Ruby Proxies for Scale, Performance, and Monitoring
 
Phorum MySQL tricks
Phorum MySQL tricksPhorum MySQL tricks
Phorum MySQL tricks
 
Ruby as a glue language
Ruby as a glue languageRuby as a glue language
Ruby as a glue language
 
Nodejs
NodejsNodejs
Nodejs
 
Server-side JavaScript for the rest of us
Server-side JavaScript for the rest of usServer-side JavaScript for the rest of us
Server-side JavaScript for the rest of us
 
NullMQ @ PDX
NullMQ @ PDXNullMQ @ PDX
NullMQ @ PDX
 
ruby + websocket + haproxy
ruby + websocket + haproxyruby + websocket + haproxy
ruby + websocket + haproxy
 
How to become a practical Vim user
How to become a practical Vim userHow to become a practical Vim user
How to become a practical Vim user
 

Viewers also liked

Replication tutorial presentation
Replication tutorial presentationReplication tutorial presentation
Replication tutorial presentation
colderboy17
 

Viewers also liked (6)

Replication tutorial presentation
Replication tutorial presentationReplication tutorial presentation
Replication tutorial presentation
 
Percona Live 2012PPT: introduction-to-mysql-replication
Percona Live 2012PPT: introduction-to-mysql-replicationPercona Live 2012PPT: introduction-to-mysql-replication
Percona Live 2012PPT: introduction-to-mysql-replication
 
MySQL Cluster performance best practices
MySQL Cluster performance best practicesMySQL Cluster performance best practices
MySQL Cluster performance best practices
 
MySQL Group Replication
MySQL Group ReplicationMySQL Group Replication
MySQL Group Replication
 
MySQL Cluster Basics
MySQL Cluster BasicsMySQL Cluster Basics
MySQL Cluster Basics
 
Percona XtraDB Cluster
Percona XtraDB ClusterPercona XtraDB Cluster
Percona XtraDB Cluster
 

Similar to MySQL::Replication (Melbourne Perl Mongers 2011-07)

Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009
pauldix
 

Similar to MySQL::Replication (Melbourne Perl Mongers 2011-07) (20)

Kickin' Ass with Cache-Fu (with notes)
Kickin' Ass with Cache-Fu (with notes)Kickin' Ass with Cache-Fu (with notes)
Kickin' Ass with Cache-Fu (with notes)
 
Data oriented design and c++
Data oriented design and c++Data oriented design and c++
Data oriented design and c++
 
Austin Cassandra Meetup re: Atomic Counters
Austin Cassandra Meetup re: Atomic CountersAustin Cassandra Meetup re: Atomic Counters
Austin Cassandra Meetup re: Atomic Counters
 
Lessons learnt on a 2000-core cluster
Lessons learnt on a 2000-core clusterLessons learnt on a 2000-core cluster
Lessons learnt on a 2000-core cluster
 
Clustered PHP - DC PHP 2009
Clustered PHP - DC PHP 2009Clustered PHP - DC PHP 2009
Clustered PHP - DC PHP 2009
 
Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009
 
Kafka Summit NYC 2017 - Running Hundreds of Kafka Clusters with 5 People
Kafka Summit NYC 2017 - Running Hundreds of Kafka Clusters with 5 PeopleKafka Summit NYC 2017 - Running Hundreds of Kafka Clusters with 5 People
Kafka Summit NYC 2017 - Running Hundreds of Kafka Clusters with 5 People
 
Introduction to XtraDB Cluster
Introduction to XtraDB ClusterIntroduction to XtraDB Cluster
Introduction to XtraDB Cluster
 
Symfony finally swiped right on envvars
Symfony finally swiped right on envvarsSymfony finally swiped right on envvars
Symfony finally swiped right on envvars
 
HTTP Plugin for MySQL!
HTTP Plugin for MySQL!HTTP Plugin for MySQL!
HTTP Plugin for MySQL!
 
Master master vs master-slave database
Master master vs master-slave databaseMaster master vs master-slave database
Master master vs master-slave database
 
Architectural Overview of MapR's Apache Hadoop Distribution
Architectural Overview of MapR's Apache Hadoop DistributionArchitectural Overview of MapR's Apache Hadoop Distribution
Architectural Overview of MapR's Apache Hadoop Distribution
 
Asynchronous programming - .NET Way
Asynchronous programming - .NET WayAsynchronous programming - .NET Way
Asynchronous programming - .NET Way
 
What is NetDevOps? How? Leslie Carr PuppetConf 2015
What is NetDevOps? How? Leslie Carr PuppetConf 2015What is NetDevOps? How? Leslie Carr PuppetConf 2015
What is NetDevOps? How? Leslie Carr PuppetConf 2015
 
Triple Blitz Strike
Triple Blitz StrikeTriple Blitz Strike
Triple Blitz Strike
 
MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4
MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4
MySQL? Load? Clustering! Balancing! PECL/mysqlnd_ms 1.4
 
Concurrency in ruby
Concurrency in rubyConcurrency in ruby
Concurrency in ruby
 
Background processes and tasks in an async world
Background processes and tasks in an async worldBackground processes and tasks in an async world
Background processes and tasks in an async world
 
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
 
Beautiful code instead of callback hell using ES6 Generators, Koa, Bluebird (...
Beautiful code instead of callback hell using ES6 Generators, Koa, Bluebird (...Beautiful code instead of callback hell using ES6 Generators, Koa, Bluebird (...
Beautiful code instead of callback hell using ES6 Generators, Koa, Bluebird (...
 

Recently uploaded

Recently uploaded (20)

ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 

MySQL::Replication (Melbourne Perl Mongers 2011-07)