MySQL High Availability Deep Dive
•
3 likes•1,492 views
A tutorial on MySQL High Availability on the Pacemaker stack. Covers both MySQL on DRBD, and MySQL with MySQL replication. Presented by Florian Haas and Yves Trudeau at the Percona Live MySQL Conference & Expo, 2012
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to MySQL High Availability Deep Dive
Similar to MySQL High Availability Deep Dive (20)
More from hastexo
More from hastexo (11)
Recently uploaded
Recently uploaded (20)
MySQL High Availability Deep Dive
- 1. MySQL High Availability Deep Dive Florian Haas (hastexo) Yves Trudeau (Percona)
- 2. HA Sprint: Get Prepared! ● Tutorial instructions at: http://goo.gl/8ZLta ● Follow along -- or just watch! (up to you) If you need help: we're the guys with the blue SPEAKER tabs, just holler!
- 3. About Us
- 5. An Overview of the Linux HA stack
- 6. Application Interface MySQL RA Cluster Resource Mgmt Pacemaker Cluster Messaging Corosync Storage Replication (optional) DRBD
- 7. Physical Physical Host Host MySQL Master MySQL MySQL MySQL App App Slave Slave Slave Server Server
- 8. Physical Physical Host Host MySQL Master MySQL MySQL MySQL App App Slave Slave Slave Server Server
- 9. DRBD http://www.hastexo.com/knowledge/drbd http://www.drbd.org DRBD & the DRBD logo are registered trademarks of Linbit Information Technologies
- 10. Creating a DRBD resource ● Name: mysql ● Device name: /dev/drbd0 ● Backing disk: /dev/pacemaker-1/lvMySQL ● Meta data: internal ● Replication: 172.30.222.1 ↔ 2 ● Filesystem: ext3
- 11. Corosync http://www.hastexo.com/knowledge/corosync http://www.corosync.org
- 12. Creating a Corosync Cluster ● Enable secauth ● 1 ring ● No logging to files, syslog only ● Start Corosync ● Check connectivity ● Check membership
- 13. Pacemaker http://www.hastexo.com/knowledge/pacemaker http://www.clusterlabs.org
- 14. Create a Pacemaker Cluster ● Add the “pacemaker” service to /etc/corosync.d ● Use ver: 1 ● Restart corosync ● Start pacemakerd ● Verify both nodes have joined the cluster (crm_mon)
- 15. STONITH/Fencing
- 16. Setting up the cluster ● Open the shell with crm configure ● Set no-quorum-policy to ignore ● Disable STONITH ○ Kids, don't do this at home! ● Commit the configuration
- 18. Started start stop Stopped
- 20. Started Started start stop start stop Stopped Stopped
- 21. Master promote demote Slave Slave start stop start stop Stopped Stopped
- 22. Adding a DRBD Master/Slave Set ● Create the DRBD primitive p_drbd_mysql ○ Use the ocf:linbit:drbd RA ○ Add 2 monitor operations (role=Master, role=Slave) with non-equal intervals ● Wrap it in a master/slave set ○ notify=true, clone-max=2, master-max=1 ● Commit the configuration ● Watch the cluster promote
- 23. Constraints
- 24. Creating a Filesystem ● Add the Filesystem resource ○ Name: p_fs_mysql ○ Device: /dev/drbd0 ○ Directory: /var/lib/mysql ○ File system type: ext3 ● Add required order & colocation constraints
- 25. Groups
- 26. Add a Group ● Create the g_mysql resource group ● Add p_fs_mysql and p_ip_mysql
- 27. Requirements for MySQL/DRBD No MyISAM log-bin sync_binlog Bind or No bind innodb_log_file_size innodb_auto_lru_dump / innodb_buffer_pool_restore_at_startup
- 28. Add the MySQL resource ● Run mysql_install_db ● Create an ocf:heartbeat:mysql primitive p_mysql ● Add resource level monitoring ● Add it to the g_mysql group ● Watch the constraints update automagically ● Commit, Rejoice!
- 30. Physical Physical Host Host MySQL Master MySQL MySQL MySQL App App Slave Slave Slave Server Server
- 31. Physical Physical Host Host MySQL Master MySQL MySQL MySQL App App Slave Slave Slave Server Server
- 32. Physical Physical Host Host MySQL Master InnoDB Recovery Umm.. WTF? Hurry up! MySQL MySQL MySQL App App Slave Slave Slave Server Server Booooring
- 33. Physical Physical Host Host MySQL Master MySQL MySQL MySQL App App Slave Slave Slave Server Server
- 34. Handling MySQL replication Read-only Promotion of the master Attaching slaves Monitoring slave state Demotion
- 35. Promote MySQL1 Notify Slaves Final State Start resources Initial state RVIP1 RVIP1 Write master info RVIP2 WVIP MySQL1 set readerOK RVIP3 enable writer VIP WVIP Turn Off Read-only Start MySQL --skip-slave-start master_host = MySQL1 master_log_file=file1 master_log_pos=1 RVIP2 RVIP3 MySQL2 MySQL3 Read CIB Start MySQL Monitor Start MySQL Read CIB Monitor Start Slave --skip-slave-start Update master score --skipe-slave-start Start slave Update master score Update readerOK Update readerOK Update readerOK
- 36. Promotion failure Failover finalMySQL2 Master of state RVIP1 WVIP MySQL1 Fencing master_host = MySQL1 MySQL2 RVIP2 master_log_file=file12 master_log_file=file1 master_log_pos=123 master_log_pos=1 RVIP1 RVIP2 WVIP RVIP3 RVIP3 MySQL2 MySQL3 Write master info Monitor Stop slave Monitor set readerOK Update master score read CIB Master info Update master score enable readerOK Update writer VIP Start salve Update readerOK Turn Off Read-only
- 37. Setup MySQL Grants Node IP attribute MySQL primitive VIP primitive Master-slave ms resource Location rules Colocation rules Order rule
- 38. Variation on the theme Slave only nodes How to add slave nodes Setup with no VIP Avoid cluster collapse
- 39. Further Information http://www.clusterlabs.org http://www.drbd.org http://www.suse.com/products/highavailability http://www.hastexo.com
- 40. Liked this talk? ● Go to www.hastexo.com/shoutbox ● Log in (you can just use your OpenID from Wordpress, Blogger, Yahoo, Google Apps, Google Profile) ● Leave us a message!