This document provides an overview of new InnoDB scalability improvements in MySQL 8.0, including improved read and write scalability. It discusses how the InnoDB architecture was updated to support read/write workloads on modern hardware more efficiently. The redo log was redesigned to be lock-less. Contention aware transaction scheduling and other new features like instant alter algorithms and temporary session tablespaces were added to enhance performance.

1. InnoDB scalability improvements in
MySQL 8.0
Mydbops Meetup - 4
Presented by
Karthik P R
www.mydbops.com info@mydbops.com 1

2. About Me
2
Karthik.P R,
Founder/CEO Mydbops, Ex-Yahoo!,
Working with MySQL ecosystem from 2010,
Juggles between Business leadership and hands
on technical expertise and a Creative Team
Builder.

3. About Mydbops
● Founded in 2015, HQ in Bangalore India with 150+ customer base across the globe.
● Mydbops is on Database Consulting with core specialization on MySQL and MongoDB Administration and
Support.
● We have expert team with 20+ certified DBA’s providing full time support and currently managing 300+
servers on premises and cloud.
● We help organisations to architect and scale systems in MySQL/MongoDB by implementing the advanced
technologies in industry which are completely open source.
● We are a leading solution provider in the market for all sort of cloud based database deployments and
management.
● An AWS partner for databases.
3

4. Mydbops at a GlanceHappy Customers
4

5. Agenda
● InnoDB Architecture ( Overview )
● InnoDB Read Scalability
● InnoDB Write Scalability
● Lock Scheduling
● InnoDB Features in 8.0
5

6. Innodb Architecture
6

7. InnoDB Architecture
● Structures In Memory
○ Buffer Pool
○ Log Buffer
● Structures on disk
○ Tablespaces
○ Redo log File
○ Undo log file
○ Temporary Tablespace ( Global / Session )
7

8. InnoDB Architecture
8
Buffer Pool Redo Buffer
Tablespaces
(System,
Undo,
Files Per table,
Tmp tables )
Redo logs
Sequential Writes
Random R/W

9. InnoDB Architecture
9

10. InnoDB Architecture
Important Configuration Variables
● Innodb_buffer_pool_size
● Innodb_buffer_pool_instances
● Innodb_log_buffer_size
● innodb_flush_method
● innodb_flush_log_at_trx_commit
● Innodb_log_file_size
● Innodb_max_undo_log_size
10

11. Innodb Read Scalability
11

12. InnoDB Read Scalability
● MySQL 5.7
○ Full RO Transaction
○ Faster Reads than writes
● MySQL 8.0
○ RW Scalability
○ Efficient on same HW
○ High Concurrency Work load
12

13. InnoDB Read Scalability
Read Scalability
● Single mutex Contention ( fil_sys_t::mutex)
Solution :
● Sharding the fil_sys_t::mutex
● 64 Shards
● individual shar for REDO and UNDO Spaces
13

14. InnoDB Read Scalability
14
Read only workload ( 4K pages, IO bound,Point Select )

15. Innodb Write Scalability
15

16. InnoDB Write Scalability
Redo log (WAL) - 5.7
● Major Component to ensure ACID
● Ensure Changes are durable
● Crash Recovery
● Slower at high concurrency ( Modern Hardware )
● Needs a new design
16

17. InnoDB Write Scalability
Redo log (WAL) OLD Design ( 5.7 )
17

18. InnoDB Write Scalability
Lock Less REDO log ( Dedicated Thread )
● log_writer
● log_flusher
● log_write_notifier
● log_flush_notifier
● log_checkpointer
● log_closer
18

19. InnoDB Write Scalability
Redo log (WAL) Improvements (8.0)
19

20. InnoDB Write Scalability
More option to Fine tune (spin waits and CPU usage )
● innodb-log-wait-for-flush-spin-hwm
● innodb-log-spin-cpu-abs-lwm
● innodb-log-spin-cpu-pct-hwm
● innodb-log-buffer-size ( Dynamic resize )
Improved performance with innodb-flush-log-at-trx-commit=1
20

21. InnoDB Write Scalability
Many experimental config in MySQL code # Not for production
● INNODB_LOG_RECENT_WRITTEN_SIZE
● INNODB_LOG_EVENTS
● INNODB_LOG_RECENT_CLOSED_SIZE
● INNODB_LOG_WAIT_FOR_WRITE_SPIN_DELAY
● INNODB_LOG_WAIT_FOR_WRITE_TIMEOUT
● INNODB_LOG_WAIT_FOR_FLUSH_SPIN_DELAY
● INNODB_LOG_WAIT_FOR_FLUSH_TIMEOUT
● INNODB_LOG_WRITE_MAX_SIZE
● INNODB_LOG_WRITER_SPIN_DELAY
● INNODB_LOG_WRITER_TIMEOUT
● INNODB_LOG_FLUSHER_SPIN_DELAY
● INNODB_LOG_FLUSHER_TIMEOUT
● INNODB_LOG_WRITE_NOTIFIER_SPIN_DELAY
● INNODB_LOG_WRITE_NOTIFIER_TIMEOUT
● INNODB_LOG_FLUSH_NOTIFIER_SPIN_DELAY
● INNODB_LOG_FLUSH_NOTIFIER_TIMEOUT
● INNODB_LOG_CLOSER_SPIN_DELAY
● INNODB_LOG_CLOSER_TIMEOUT
mysql/storage/innobase/handler/ha_innodb.cc
21

22. InnoDB Write Scalability
22
R/W Workload ( Updates )

23. Lock Scheduling
23

24. Lock Scheduling
CATS ( Contention Aware Transaction Scheduling )
● University of Michigan Contribution.
● Philosophy: Not all transaction are same.
● FIFO ( Default MySQL 5.7 )
● Switches between CATS from FIFO ( MySQL 8.0 )
● >= 32 Waiting threads
● Reduces lock sys wait mutex
● First database to adopt this Philosophy in Transaction
24

25. Lock Scheduling
25

26. InnoDB Features
26

27. InnoDB Features
Data Dictionary ( SDI )
● A major overhaul in Data Dictionary
● ibd2sbi tool
Alter Algorithm=Instant
● Change Index
● Rename Table
● Modify column
● Add/Drop Virtual columns
27

28. InnoDB Features
Alter table add column
● Tencent Games
● Instant add column (at end)
Temporary Tablespace ( Session )
● Pool of 10 temporary tablespace ( .ibt )
● Each of 80K ( 5 pages ) in size
● 400K Space ID is reserved
28

29. InnoDB Features
Undo logs
● SQL to control Undo logs
○ Create UNDO Tablespace …
○ Alter UNDO Tablespace ….
○ Drop UNDO Tablespace ..
● Default 2 files ( 4 is better )
InnoDB Dedicated Server
● Better defaults
● InnoDB Buffer Pool / InnoDB log File Size / InnoDB Flush method
29

30. InnoDB Features
InnoDB Parallel Thread Count
● Path to parallel query in InnoDB
● Innodb_parallel_read_threads
● Make count() and Check tables faster
● Default 4 threads
30

31. Resources
● https://mysqlserverteam.com/mysql-8-0-new-lock-free-scalable-wal-design/
● https://www.percona.com/live/19/speaker/sunny-bains
● https://mysqlserverteam.com/contention-aware-transaction-scheduling-arriving-in-innodb-to-boost-
performance/
● https://dev.mysql.com/worklog/task/?id=10793
● http://dimitrik.free.fr/blog/posts/mysql-performance-80-and-sysbench-oltp_rw-updatenokey.html
● https://dev.mysql.com/doc/refman/8.0/en/innodb-architecture.html
● https://fossies.org/linux/mysql/storage/innobase/handler/ha_innodb.cc
31

32. Questions
Reach me : prk@mydbops.com
32

33. Thank You
33