MySQL Performance Best Practices

MySQL Performance Best Practices
Implementing best practices will get you the scalability you need
1 © 2020 Oracle MySQL
Olivier Dasini
MySQL Principal Solutions Architect EMEA
olivier.dasini@oracle.com
Twitter : @freshdaz
Blogs : www.dasini.net/blog/en
: www.dasini.net/blog/fr

Safe harbor statement
The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon in making purchasing
decisions.
The development, release, timing, and pricing of any features or functionality described for
Oracle’s products may change and remains at the sole discretion of Oracle Corporation.

3
Me, Myself & I
Olivier DASINI
Copyright © 2020 Oracle and/or its affiliates. All rights reserved.

MySQL Geek
 Addicted to MySQL for 15+ years
 Playing with databases for 20+ years

MySQL Writer, Blogger and Speaker
 Also former : DBA, Consultant, Architect, Trainer, ...

MySQL Principal Solutions Architect EMEA at Oracle

Stay up to date!
 Twitter: @freshdaz
 Blog: www.dasini.net/blog/en

Introduction
Program agenda
1
2
3
4
Monitoring
Configuration
Schema Optimization
Query Tuning5

Why MySQL ?
Performance

The world’s most popular open source database

Some Of Our Customers
 MySQL powers the most demanding Web, E-commerce, SaaS and Online Transaction Processing
(OLTP) applications
 It is a fully integrated transaction-safe, ACID compliant database with full commit, rollback, crash
recovery and row level locking capabilities
 MySQL delivers the ease of use, scalability, and performance to power:

Explosive Data Growth
Source IDC, 2017 Source Domo, 2018

MySQL Performance Tuning
Elementary Guidelines
Performance

Dimitri Kravtchuk
MySQL Performance Architect
The MySQL Performance Best Practice #1
is… ???
USE YOUR BRAIN !!! ;-)

Performance Tuning - Guidelines 1/2
 Think – consider what you are doing!
 Monitor your system
– There is no control without measurement! - MySQL Enterprise Monitor is your best ally
– https://www.mysql.com/products/enterprise/monitor.html
 Be wary about “best practices”
– No two systems are the same
– What was true earlier may no longer hold
 Ensure you test your changes before deploying to production
– The testing must reflect your production usage
 Make incremental changes
– One change at a time
– Relatively small change

Performance Tuning - Guidelines 2/2
 Be mindful of your requirements
– Some options give the choice between performance or data safety – what do you need?
 Often the default value is the best value - we’ve made huge improvement in MySQL 8.0
 Ensure all tables have a PRIMARY KEY - very important for InnoDB
 InnoDB organizes the data according to the PRIMARY KEY:
– The PRIMARY KEY is included in all secondary indexes in order to be able to locate the actual row
– Smaller PRIMARY KEY gives smaller secondary indexes
– A mostly sequential PRIMARY KEY is in general recommended to avoid inserting rows between existing rows

13
Hardware
Schema Changes
Data Growth
Indexes
SQL
90% of
Performance Problems
Source of Database Performance Problems

14
1 2 3 4
• Visually Identify Slow Queries
• Correlation Graphs
• Query Response Time index (QRTi)
• Execution Statistics
• Tune Queries
• Add Indexes
• Tune Schemas
• Improve Caching
• MySQL Explain Plan
• Sample Query
• Query Graphs Better Performance
(hopefully)
Solving Query Performance Problems

Monitoring
https://www.mysql.com/products/enterprise/monitor.html
There is no control without measurement!
Performance

Monitoring
 Your database referential
– Fact-based decision making
 Real-time performance monitoring → What is happening right now
– Take the right decision at the right time
 Go back in time → What has happened
– Useful to investigate performance issues
 Predict the future → Allows you to proactively handle potential issues
– Capacity planning
 Visual Query Analysis
– Examine the query execution plan, see sample queries, see the details of the query execution...
 Helps you to enforce best practices
 Easy to setup and configure
Expectations

MySQL Enterprise Monitor – Query Analyzer
 Real-time query performance
 Visual correlation graphs
 Find & fix expensive queries
 Detailed query statistics
 Query Response Time index (QRTi)
– "Quality of Service" (QoS) measurement for each
query
– QoS measurement for a server, group, or every
instance
– Single metric for query performance
Visual Query Analysis for troubleshooting
“With the MySQL Query Analyzer, we were able to identify and analyze problematic SQL code,
and triple our database performance. More importantly, we were able to accomplish this in
three days, rather than taking weeks.”
Keith Souhrada
Software Development Engineer
Big Fish Games
https://www.mysql.com/products/enterprise/query.html

Query Response Time Index (QRTi)
 “Quality of Service” (QoS) measurement for each query
 QoS measurement for a server, group, or every instance
 Single metric for query performance
Identifies queries with unacceptable response times

Best Practice Advisors
 Provide a set of rules
– Designed to enforce best practices
 14 Advisor categories
 250+ Advisors
 Threshold-based alerts
– Exponential moving averages
– Rate change detection
 Expert problem resolution advice

MySQL Configuration
Performance

Configuration – Paths
 Lots of work has gone into improving the defaults since MySQL 5.6
– The work continues
 Some important paths:
– datadir – where the data lives by default – default location for:
●
Data and indexes for non-InnoDB tables
●
InnoDB file-per-table, general, system, and undo log tablespaces
– innodb_data_home_dir – default location for the InnoDB system tablespace (ibdata1)
– innodb_log_group_home_dir – path to the InnoDB redo log files
– innodb_undo_directory – path to the InnoDB undo tablespaces
– log_bin – dual functionality: enable binary logging and set path/file name prefix
– log_error – makes sure you know where it is

Configuration – Data Caching
 Most important options for performance:
– innodb_buffer_pool_size for InnoDB workloads
– key_buffer_size for MyISAM workloads

InnoDB Configuration – innodb_buffer_pool_size
 Size of the buffer pool, memory area where InnoDB caches table and index data
 A larger buffer pool requires less disk I/O to access the same table data more than once
 Optimal size:
– innodb_buffer_pool_size for InnoDB workloads
– How much memory does the host have?
– Subtract memory required by OS and other processes
– Subtract memory required by MySQL other then the InnoDB buffer pool
– Choose minimum of this and the size of the “working data set”
 Since MySQL 5.7, innodb_buffer_pool_size can be changed dynamically

InnoDB Configuration – Checking the size of your working set
MySQL Enterprise Monitor displays InnoDB Buffer Pool utilization

InnoDB Configuration – Checking the size of your working set
https://lefred.be/content/mysql-and-memory-a-love-story-part-2/
SELECT CONCAT(FORMAT(A.num * 100.0 / B.num,2),"%") BufferPoolFullPct
FROM
(
SELECT variable_value num
FROM performance_schema.global_status
WHERE variable_name = 'Innodb_buffer_pool_pages_data'
) A
INNER JOIN
(
SELECT variable_value num
FROM performance_schema.global_status
WHERE variable_name = 'Innodb_buffer_pool_pages_total'
) B;
Verify how much the InnoDB Buffer Pool is filled with data

InnoDB Configuration – innodb_buffer_pool_instances
 Specifies how many instances to split the buffer pool into
– Can reduce contention for concurrent workload ie improve the scalability on a busy server
– This option takes effect only when you set innodb_buffer_pool_size to a size of 1GB or more.
– For best efficiency, specify a combination of innodb_buffer_pool_instances & innodb_buffer_pool_size so that
each buffer pool instance is at least 1GB
– Max allowed value: 64
 Dimitri suggestion
– # BP instances = max ( 4, CPU cores / 2 )
 But test with your workload!
 Requires restart
https://dev.mysql.com/doc/refman/8.0/en/innodb-multiple-buffer-pools.html

InnoDB Configuration – Saving and Restoring the Buffer Pool State
 To reduce the warmup period after restarting the server, InnoDB saves a percentage of the most recently
used pages for each buffer pool at server shutdown and restores these pages at server startup
– innodb_buffer_pool_dump_at_shutdown - default: ON
– innodb_buffer_pool_dump_pct - default: 25 (percentage of the most recently used pages)
– Page loading happens in the background, and does not delay database startup
 You can also save and restore the buffer pool state at any time, while the server is running
– SET GLOBAL innodb_buffer_pool_dump_now=ON; -- Save the state of the buffer pool
– SET GLOBAL innodb_buffer_pool_load_now=ON; -- Restore the buffer pool state
– SET GLOBAL innodb_buffer_pool_load_abort=ON; -- Abort a buffer pool load operation
– SHOW STATUS LIKE 'Innodb_buffer_pool_dump_status'; -- Display progress when saving the buffer pool state
– SHOW STATUS LIKE 'Innodb_buffer_pool_load_status'; -- Display progress when loading the buffer pool
https://dev.mysql.com/doc/refman/8.0/en/innodb-preload-buffer-pool.html

InnoDB Configuration – innodb_dedicated_server
 When innodb_dedicated_server is enabled, InnoDB automatically configures the following variables:
– innodb_buffer_pool_size
– innodb_log_file_size
– innodb_log_files_in_group
– innodb_flush_method
 Only consider enabling innodb_dedicated_server if the MySQL instance resides on a dedicated server
where it can use all available system resources
– e.g. Running MySQL Server in a Docker container or dedicated VM that only runs MySQL
 Enabling innodb_dedicated_server is not recommended if the MySQL instance shares system resources
with other applications
Enabling Automatic Configuration for a Dedicated MySQL Server
https://dev.mysql.com/doc/refman/8.0/en/innodb-dedicated-server.html

Configuration – Other
 max_connections
– Maximum permitted number of simultaneous client connections
– Be careful setting this too large as each connection requires memory
 table_open_cache
– Number of open tables for all threads
– Increasing this value increases the number of file descriptors that mysqld requires
– Each table can be open more than once
– You can check whether you need to increase the table cache by checking the Opened_tables status variable
●
If the value of Opened_tables is large and you do not use FLUSH TABLES often, then you should increase the value
of the table_open_cache variable
 table_open_cache & max_connections affect the maximum number of files the server keeps open
– If you increase one or both of these values, you may run up against a limit imposed by your operating system on
the per-process number of open file descriptors
– open_files_limit
●
The number of file descriptors available to mysqld from the operating system
●
On Unix, the value cannot be set greater than the value displayed by the ulimit -n command

Configuration – join_buffer_size
 Minimum size of the buffer that is used for plain index scans, range index scans, joins that do not
use indexes, hash joins and when Block Nested-Loop or Batched Key Access algorithm is used
 It is better to keep the global setting small and change the session setting to a larger value only in
sessions
 Or change the setting on a per-query basis by using a SET_VAR optimizer hint
 Memory allocation time can cause substantial performance drops if the global size is larger than
needed by most queries that use it
 Usually default value is OK

Configuration – sort_buffer_size
 Used by each session that must perform a sort
 At minimum the sort_buffer_size value must be large enough to accommodate 15 tuples in the sort
buffer
 If many sort_merge_passes per second, consider increasing the sort_buffer_size value to speed up
ORDER BY or GROUP BY operations that cannot be improved with query optimization or improved
indexing
 Best to keep the global setting small and change the session setting to a larger value only in sessions
 Memory allocation time can cause substantial performance drops if the global size is larger than needed
by most queries that use it

InnoDB Configuration – InnoDB Redo Log
 Disk-based data structure used during crash recovery to correct data written by incomplete transactions
 Physically represented on disk by two files named ib_logfile0 and ib_logfile1
 Total redo log size defined by two options: innodb_log_file_size & innodb_log_files_in_group
 Total size = innodb_log_file_size * innodb_log_files_in_group
– Max supported total redo log size is just below 512G since MySQL 5.6
 Should be large enough to avoid “excessive” checkpointing
– Make your redo log files big, even as big as the buffer pool
– The larger the value, the less checkpoint flush activity is required in the buffer pool, saving disk I/O
– Larger log files also make crash recovery slower
 With large write transactions increase the size of the log buffer - innodb_log_buffer_size
– A large log buffer enables large transactions to run without a need to write the log to disk before the transactions
commit
– if you have transactions that update, insert, or delete many rows, making the log buffer larger saves disk I/O
 Dedicated log writer threads can improve performance on high-concurrency systems, but for low-
concurrency systems, disabling dedicated log writer threads provides better performance
– Enable or disable log writer threads using : innodb_log_writer_threads .
https://dev.mysql.com/doc/refman/8.0/en/optimizing-innodb-logging.html

InnoDB Configuration – InnoDB Redo Log
MySQL Enterprise Monitor displays InnoDB Redo Log usage

InnoDB Configuration – Disabling Redo Logging
 As of MySQL 8.0.21, you can disable redo logging using
– ALTER INSTANCE DISABLE INNODB REDO_LOG
 This functionality is intended for loading data into a new MySQL instance
 Disabling redo logging speeds up data loading by avoiding redo log writes and doublewrite buffering
 This feature is intended only for loading data into a new MySQL instance
 Do not disable redo logging on a production system
 Details:
– https://dev.mysql.com/doc/refman/8.0/en/innodb-redo-log.html#innodb-disable-redo-logging

Data Consistency vs Performance
Trading safety for performance?
Performance

innodb_flush_log_at_trx_commit
 Controls the balance between strict ACID compliance for commit operations & higher performance
 You can achieve better performance by changing the default value but then you can lose transactions
in a crash
 Possible values in order of data safety:
– 1: Logs are written and flushed to disk at each transaction commit
●
Is theoretically the slowest, but with fast SSD it may be around as fast as 2 and 0
– 2: Logs are written after each transaction commit and flushed to disk once per second
●
Transactions for which logs have not been flushed can be lost in a crash (host)
– 0: NOT TO USE (basically nothing is written on any commit and log buffer is written when filled)
●
You can easily loose everything...
 Defaults to 1; Logs are written and flushed to disk at each transaction commit
– Required for full ACID compliance - D in ACID
– For this reason it is the recommended value
 Log flushing frequency is controlled by innodb_flush_log_at_timeout

innodb_flush_log_at_trx_commit
 What if innodb_flush_log_at_trx_commit = 1 is too slow and you want the D in ACID?
– Make sure the redo logs are on separate disks
●
Do not be tempted to have the individual redo logs on different disks
– Consider SSD particularly if you have a high commit rate
– Battery-backed disk cache also makes flushes cheaper

sync_binlog
 Controls how often the MySQL server synchronizes the binary log to disk
– 0: Disables synchronization of the binary log to disk
●
MySQL relies on the operating system to flush the binary log to disk
●
Provides the best performance, but in the event of a power failure or operating system crash, it is possible that the
server has committed transactions that have not been synchronized to the binary log
– 1: Enables synchronization of the binary log to disk before transactions are committed
●
The safest setting but can have a negative impact on performance due to the increased number of disk writes
●
Guarantees that no transaction is lost from the binary log
– N: The binary log is synchronized to disk after N binary log commit groups have been collected
●
Can have a negative impact on performance due to the increased number of disk writes
●
A higher value improves performance, but with an increased risk of data loss
 Default value: 1

sync_binlog
 sync_binlog != 1 means that replicas most likely will have to be rebuild if the primary crashes
– CLONE plugin, for cloning data from a MySQL server instance, will help
 But sync_binlog = 0 must be the best performance wise, right?
– By default: max_binlog_size = 1G
– 1G is not a lot of memory nowadays
– So the OS may buffer the entire binary log
– When rotating the binary log, up to 1G will be flushed to disk
●
Stalls all other commits until the binary log rotation is done
 So in short:
– sync_binlog = 0 may give the best throughput
– sync_binlog = 1 may give the most predictable performance

Durability & consistency with MySQL Replication
 For durability and consistency in a replication setup that uses InnoDB with transactions:
– If binary logging is enabled, set sync_binlog=1
– Always set innodb_flush_log_at_trx_commit=1

Schema Optimization
Most important factor in making
a database application fast
Performance

Design Questions
 Starting with an efficient database design makes it easier for team members to write high-performing
application code
– Do the columns have the right data types & clauses?
– Does each table have the appropriate columns for the type of work?
– Do the tables have a primary key?
– Are the right indexes in place to make queries efficient?

Schema Optimization Tips 1/2
 Design your tables to minimize their space on the disk <=> use more caching, thus minimize disk seeks
– Reduce the amount of data written to and read from disk
– Smaller tables normally require less main memory
– Smaller indexes that can be processed faster
 Use the most efficient (smallest) data types possible
– e.g. MEDIUMINT vs INT (MEDIUMINT uses 25% less space than INT) or BIGINT, TINYINT (UNSIGNED ?)
 Declare columns to be NOT NULL if possible
– It makes SQL operations faster, by enabling better use of indexes & eliminating overhead for testing whether each
value is NULL
– You also save some storage space
Look for the most efficient way to organize your data

Schema Optimization Tips 2/2
 The primary key of a table should be as short as possible
– This makes identification of each row easy and efficient
– For InnoDB tables, the PK columns are duplicated in each secondary index entry, so a short PK saves considerable
space if you have many secondary indexes.
 Create only the indexes that you need to improve query performance
– Indexes are good for retrieval, but slow down insert and update operations
– Indexes use disk & memory spaces
– Sometime it can be beneficial to use column prefix key parts (especially for long string column)
 Declare columns with identical information in different tables with identical data types, charsets &
collations
– Speed up joins based on the corresponding columns
Look for the most efficient way to organize your data

MySQL Query Tuning
Make the best use of your data
Performance

MySQL Optimizer (simplified)

Input to Cost Model
 IO-cost
– Estimates from storage engine based on
number of pages to read
– Both index and data pages
 Statistics
– Number of records in table
– Key distribution/Cardinality:
●
Average number of records per key value
●
Only for indexed columns
●
Maintained by storage engine
– Number of records in an index range
– Percentage of table/index in InnoDB buffer pool
 Schema
– Length of records and keys
– Uniqueness for indexes
– Nullability

InnoDB Persistent Optimizer Statistics
 Persisted across server restarts: innodb_stats_persistent (ON by default)
– Greater plan stability
– More accurate statistics
– Consistent query performance
– Less variance between servers
 Automatically recalculates statistics after significant changes: innodb_stats_auto_recalc
– Threshold value is 10% of the rows in the table
– May turn off automatic recalculations
 ANALYZE TABLE forces recalculation of statistics
 May manage precision by changing number of samples: innodb_stats_persistent_sample_pages
– If statistics are not accurate enough and the optimizer chooses suboptimal plans
– If ANALYZE TABLE is too slow

Indexes
 The best way to improve the performance of SELECT operations is to create relevant indexes on
one or more of the columns that are tested in the query
 The index entries act like pointers to the table rows, allowing the query to quickly determine which
rows match a condition in the WHERE clause, and retrieve the other column values for those rows
 All MySQL data types can be indexed
 Although it can be tempting to create an indexes for every possible column used in a query,
unnecessary indexes waste space and waste time for MySQL to determine which indexes to use
 Indexes also add to the cost of INSERT, UPDATE & DELETE because each index must be updated
 You must find the right balance to achieve fast queries using the optimal set of indexes

Multi-Valued Indexes
 InnoDB supports multi-valued indexes
 A multi-valued index is a secondary index defined on a column that stores an array of values
 A multi-valued index can have multiple index records for a single data record
 Multi-valued indexes are intended for indexing JSON arrays
 The optimizer uses a multi-valued index to fetch records when the following functions are specified
in a WHERE clause:
– MEMBER OF()
– JSON_CONTAINS()
– JSON_OVERLAPS()
https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-multi-valued
SELECT * FROM customers -> WHERE 94507 MEMBER OF(custinfo->'$.zipcode');
+----+---------------------+-------------------------------------------------------------------+
| id | modified | custinfo |
+----+---------------------+-------------------------------------------------------------------+
| 2 | 2019-06-29 22:23:12 | {"user": "Jill", "user_id": 22, "zipcode": [94568, 94507, 94582]} |
| 3 | 2019-06-29 22:23:12 | {"user": "Bob", "user_id": 31, "zipcode": [94477, 94507]} |
| 5 | 2019-06-29 22:23:12 | {"user": "Ted", "user_id": 56, "zipcode": [94507, 94582]} |
+----+---------------------+-------------------------------------------------------------------+

Unused & Redundant Indexes
 These 2 views, from sys schema, will help you to spot index issues
– schema_redundant_indexes: indexes that duplicate other indexes or are made redundant by them
– schema_unused_indexes: indexes for which there are no events, which indicates that they are not being used
● Most useful when the server has been up and processing long enough that its workload is representative. Otherwise,
presence of an index in this view may not be meaningful.
 The next 2 indicates statements that have done full table scans:
– schema_tables_with_full_table_scans: display which tables are being accessed with full table scans
– statements_with_full_table_scans: display normalized statements that have done full table scans
Unnecessary indexes waste space and waste time

Some Reasons Why Index can NOT be Used
 Indexed column is used as argument to function
– YEAR(o_orderdate) = 1997
– => Use functional indexes : https://dasini.net/blog/2019/03/14/mysql-functional-indexes
 Looking for a suffix :
– name LIKE '%son'
 First column(s) of compound index NOT used
– b = 10 when index defined over (a, b)
 Type mismatch
– my_string = 10
 Character set / collation mismatch
– t1 LEFT JOIN t2 ON t1.utf8_string = t2. latin1_string

Histogram
 The server stores histogram statistics about column values, for use by the optimizer in constructing
query execution plans
 The optimizer applies histogram statistics to determine row estimates based on the selectivity (filtering
effect) of column value comparisons against constant values
 Histogram statistics are useful primarily for nonindexed columns
– Adding an index to a column for which histogram statistics are applicable might also help the optimizer make row
estimates
– A histogram is created or updated only on demand, so it adds no overhead when table data is modified
– On the other hand, the statistics become progressively more out of date when table modifications occur, until the
next time they are updated
 The optimizer prefers range optimizer row estimates to those obtained from histogram statistics
– If the optimizer determines that the range optimizer applies, it does not use histogram statistics
 To perform histogram management, use the ANALYZE TABLE
https://dev.mysql.com/doc/refman/8.0/en/optimizer-statistics.html

Histogram Tips
https://dev.mysql.com/doc/refman/8.0/en/analyze-table.html#analyze-table-histogram-statistics-analysis
-- Update (or create) an histogram on column salaries.to_date
ANALYZE TABLE salaries UPDATE HISTOGRAM ON to_date;
-- Extract values from the histogram objects
SELECT
TABLE_NAME,
COLUMN_NAME,
HISTOGRAM->>'$."data-type"' AS 'data-type',
JSON_LENGTH(HISTOGRAM->>'$."buckets"') AS 'bucket-count'
FROM INFORMATION_SCHEMA.COLUMN_STATISTICS;

InnoDB Tips
 InnoDB primary key is included in all secondary indexes in order to be able to locate the actual row
– Smaller primary key gives smaller secondary indexes
– A mostly sequential primary key is in general recommended to avoid inserting rows between existing rows
– Alternative: create an AUTO_INCREMENT column as the primary key if your primary key is long, or index a prefix of
a long VARCHAR column instead of the entire column
 InnoDB automatically extends each secondary index by appending the primary key columns to it
– The optimizer takes into account the primary key columns of the extended secondary index when determining how
and whether to use that index
– This can result in more efficient query execution plans and better performance
– Could be disable with : SET optimizer_switch = 'use_index_extensions=off';
 When you use a randomly generated value as a primary key in an InnoDB table, prefix it with an
ascending value such as the current date and time if possible
– When consecutive primary values are physically stored near each other, InnoDB can insert and retrieve them faster
 It is usually better to create a small number of concatenated indexes rather than a large number of
single-column indexes
– If an index contains all the columns needed for the result set (ie covering index), the query might be able to avoid
reading the table data at all

MySQL Query Tuning
Analyzing Queries
Performance

Analyzing Queries – MySQL Enterprise Monitor - Query Analyzer
Visual Query Analysis for troubleshooting
 Real-time query performance
 Find & fix expensive queries
 Detailed query statistics
 Query Response Time index (QRTi)

Analyzing Queries – EXPLAIN
 The set of operations that the optimizer chooses to perform the most efficient query is called the
Query Execution Plan or EXPLAIN plan
 MySQL explains how it would process the statement, including information about how tables are
joined and in which order
 Returns a row of information for each table used in the statement
 Lists the tables in the output in the order that MySQL would read them while processing the statement
– You can see where you should add indexes to tables so that the statement executes faster by using indexes to
find rows
– You can check whether the optimizer joins the tables in an optimal order
 EXPLAIN works with SELECT, DELETE, INSERT, REPLACE, and UPDATE statements
Provide information about how MySQL executes statements

EXPLAIN SELECT distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: salaries
partitions: NULL
type: ALL
possible_keys: PRIMARY
key: NULL
key_len: NULL
ref: NULL
rows: 2838426
filtered: 0.00
Extra: Using where; Using temporary
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: titles
partitions: NULL
type: ref
possible_keys: PRIMARY
key: PRIMARY
key_len: 4
ref: employees.salaries.emp_no
rows: 1
filtered: 100.00
Extra: Using index
No index used for table salaries
Full table scan
Almost 3M rows examined (estimation)

Analyzing Queries – MySQL Workbench Visual EXPLAIN
Cost per table
Table & index
Rows per lookup
Total query cost
Accumulated cost

MySQL Workbench EE
 Database migrations
– From Microsoft SQL Server, PostgreSQL, Sybase ASE,
Sybase SQL Anywhere, SQLite, Microsoft Access, and
more
 Manage migration projects
 Source and target selection
 Object migration
 Data migration
 MySQL version upgrades
Database Migration Wizard
for SQL Server, Sybase, SQLite, Sybase & PostgreSQL
https://www.mysql.com/products/workbench/

Analyzing Queries – EXPLAIN ANALYZE
 Run a statement and produces EXPLAIN output along with timing and additional, iterator-based,
information about how the optimizer's expectations matched the actual execution
 The following information is provided:
– Estimated execution cost
– Estimated number of returned rows
– Time to return first row
– Time to return all rows (actual cost), in milliseconds
– Number of rows returned by the iterator
– Number of loops
 Can be used with SELECT statements, as well as with multi-table UPDATE and DELETE statements
https://dev.mysql.com/doc/refman/8.0/en/explain.html

EXPLAIN ANALYZE SELECT distinct title FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000
*************************** 1. row ***************************
EXPLAIN: -> Table scan on <temporary> (actual time=0.002..0.003 rows=2 loops=1)
-> Temporary table with deduplication (cost=10.62 rows=10) (actual time=5.939..5.939 rows=2 loops=1)
-> Nested loop inner join (cost=10.62 rows=10) (actual time=4.087..5.880 rows=12 loops=1)
-> Filter: (salaries.salary > 155000) (cost=2.42 rows=7) (actual time=0.069..0.085 rows=7 loops=1)
-> Index range scan on salaries using idx_salary (cost=2.42 rows=7) (actual time=0.065..0.075 rows=7 loops=1)
-> Index lookup on titles using PRIMARY (emp_no=salaries.emp_no) (cost=1.04 rows=1) (actual time=0.823..0.826 rows=2 loops=7)

MySQL Query Tuning
Controlling the Query Optimizer
Performance

Controlling the Query Optimizer
When the optimizer does not do what you think it should do
 Affect how query plans are evaluated
 Add indexes and/or performs a key distribution analysis: run ANALYZE TABLE
 Optimizer Hints - can be specified within individual statements
– Join-Order Optimizer Hints
– Table-Level Optimizer Hints
– Index-Level Optimizer Hints
– Subquery Optimizer Hints
– Statement Execution Time Optimizer Hints...
 Switchable Optimizations:
– optimizer_switch flags: enable control over optimizer behavior
 Controlling Query Plan Evaluation
– Control how exhaustive the optimizer is in its search for an optimal query evaluation plan
– The fewer plans that are investigated by the optimizer, the less time it spends in compiling a query
●
But it may miss finding an optimal plan
– optimizer_prune_level: tells the optimizer to skip certain plans based on estimates of the number of rows accessed
for each table
– optimizer_search_depth: tells how far into the “future” of each incomplete plan the optimizer should look to
evaluate whether it should be expanded further https://dev.mysql.com/doc/refman/8.0/en/controlling-optimizer.html

Index-Level Optimizer Hints - example
https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html#optimizer-hints-index-level
SELECT /*+ NO_INDEX(salaries idx_salary) */ distinct title
FROM titles INNER JOIN salaries USING (emp_no) WHERE salary > 155000;
SELECT /*+ NO_INDEX(salaries PRIMARY) */ distinct title
SELECT /*+ NO_INDEX(salaries) */ distinct title
SELECT /*+ INDEX(salaries PRIMARY) */ distinct title
SELECT /*+ INDEX(salaries PRIMARY) NO_INDEX(titles PRIMARY) */ distinct title

Variable-Setting Hint Syntax
 The SET_VAR hint sets the session value of a system variable temporarily ie for the duration of a
single statement
 It permits setting only a single variable, but multiple hints can be given to set multiple variables
https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html#optimizer-hints-set-var
SELECT /*+ SET_VAR(join_buffer_size=524288) */
distinct title
SELECT /*+ SET_VAR(join_buffer_size=524288) SET_VAR(sort_buffer_size=786432) */
distinct title

And also...
Performance

Query Rewrite Plugin
 Rewrite problematic queries without the need to make application changes
– Add hints
– Modify join order...
 Statements subject to rewriting: SELECT, INSERT, REPLACE, UPDATE & DELETE
 Standalone statements and prepared statements are subject to rewriting
 Statements occurring within view definitions or stored programs are not subject to rewriting
 e.g. Adding Rewrite Rules:
INSERT INTO query_rewrite.rewrite_rulesquery_rewrite.rewrite_rules (pattern, replacement ) VALUES
("SELECT sum(salary) FROM employees.salaries WHERE from_date BETWEEN ? AND ?",
"SELECT /*+ INDEX(salaries idx_from_date) */ sum(salary) FROM employees.salaries
WHERE from_date BETWEEN ? AND ?");
 Information:
– https://dev.mysql.com/doc/refman/8.0/en/rewriter-query-rewrite-plugin.html
– www.dasini.net/blog/2016/02/25/30-mins-with-mysql-query-rewriter/

Consider the Whole Stack
The issue may not be in MySQL,
or may be an interaction issue
Performance

Consider the Whole Stack
 The issue may occur at any place in the stack, e.g.:
– Application
– Application host/hardware
– Network between application and host with MySQL
– MySQL host/hardware
– MySQL
 This should also be considered when monitoring
 Consider the OS/hardware settings

Take away
 MySQL performance tuning is similar to all other performance tuning:
– Premature optimization is bad
– Only change one thing at a time and don’t make too large changes
– One size does not fit all
– Base your decisions on measurements (monitoring)
– Understand what an option does before changing it
– Understand your system (data)
– Understand what you need
– Consider the whole stack
 MySQL team can help you
Performance

MySQL Enterprise Edition
https://www.youtube.com/watch?v=CtRIc6Hsia8
We’ve Got You Covered
Performance

MySQL Enterprise Support
 Largest MySQL engineering and support organization
 Backed by the MySQL developers
 World-class support, in 29 languages
 Hot fixes & maintenance releases
 24x7x365
 Unlimited incidents
 Consultative support
 Global scale and reach
Get immediate help for any
MySQL issue, plus expert advice
Get immediate help for any
MySQL issue, plus expert advice
https://www.mysql.com/fr/support/

MySQL Enterprise Consultative Support
 Remote troubleshooting
 Replication review
 Partitioning review
 Schema review
 Query review
 Performance tuning
 ...and more
https://www.mysql.com/fr/support/consultative.html
Make the Most of your Deployments

Questions?

Resources – MySQL Blogs
• MySQL 8.0 Optimization Documentation
– https://dev.mysql.com/doc/refman/8.0/en/optimization.html
• Optimization for InnoDB tables
– https://dev.mysql.com/doc/refman/8.0/en/optimizing-innodb.html
• MySQL Server Team
– http://mysqlserverteam.com/
• MySQL Performance Blog
– http://dimitrik.free.fr/blog/
• MySQL High Availability
– http://mysqlhighavailability.com/
• MySQL Community Manager
– https://lefred.be/
• MySQL Engineering Team
– https://insidemysql.com/
• Oracle MySQL Blog
– https://blogs.oracle.com/mysql/

Resources – MySQL Features
• MySQL Document Store
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://github.com/mysql/mysql-connector-nodejs
– https://www.npmjs.com/package/@mysql/xdevapi
– https://dev.mysql.com/doc/refman/8.0/en/document-store.html
– https://dev.mysql.com/doc/x-devapi-userguide/en/
– https://dev.mysql.com/doc/dev/connector-nodejs/8.0/
– https://www.mysql.com/news-and-events/web-seminars/mysql-document-store-and-node-js/
• MySQL Performance by Dimitri
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-over-18m-qps-with-80-ga-on-2s-skylake.html
– http://dimitrik.free.fr/blog/archives/2018/04/mysql-performance-80-and-utf8-impact.html
• MySQL InnoDB Cluster
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-ga-is-available-now/
– https://mysqlserverteam.com/mysql-innodb-cluster-whats-new-in-the-8-0-ga-release/
– https://mysqlserverteam.com/mysql-innodb-cluster-8-0-a-hands-on-tutorial/

Resources – MySQL Features
• Common Table Expressions (CTE)
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-two-how-to-generate-series/
– http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes-part-three-hierarchies/
– http://mysqlserverteam.com/mysql-8-0-1-recursive-common-table-expressions-in-mysql-ctes-part-four-depth-first-or-breadth-first-traversal-
transitive-closure-cycle-avoidance/
• Window Functions
– http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
– https://mysqlserverteam.com/row-numbering-ranking-how-to-use-less-user-variables-in-mysql-queries/
– https://fr.slideshare.net/DagHWanvik/dublin-4x3finalslideshare-80423777
• Security
– https://www.mysql.com/news-and-events/web-seminars/checking-and-hardening-mysql-8-0-security-via-sql-and-without-os-access/
– https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/
– http://lefred.be/content/mysql-8-0-listing-roles/
– https://dev.mysql.com/doc/refman/8.0/en/password-management.html
• Upgrade from 5.7 to 8.0
– https://mysqlserverteam.com/inplace-upgrade-from-mysql-5-7-to-mysql-8-0/
– https://mysqlserverteam.com/upgrading-to-mysql-8-0-here-is-what-you-need-to-know/

Thank you
Olivier Dasini
MySQL Principal Solutions Architect EMEA
Twitter : @freshdaz
Blog : www.dasini.net/blog/en

MySQL Performance Best Practices

MySQL Performance Best Practices

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to MySQL Performance Best Practices

Similar to MySQL Performance Best Practices (20)

More from Olivier DASINI

More from Olivier DASINI (20)

Recently uploaded

Recently uploaded (20)

MySQL Performance Best Practices