SlideShare a Scribd company logo
1 of 121
Download to read offline
MySQL Query Tuning
for Dev[Op]s. Introduction
•Basics
•Indexes
How Do They Work?
When MySQL Uses Indexes
Optimizer Histograms
•Diagnostics
EXPLAIN: How Optimizer Works
Real Numbers: Inside Storage Engine
Real Numbers: Inside the Server
•How to Affect Query Plans
Table of
Contents
2 ©2021 Percona
• Sveta Smirnova
• MySQL Support engineer
• Author of
• MySQL Troubleshooting
• JSON UDF functions
• FILTER clause for MySQL
• Speaker
• Percona Live, OOW, Fosdem,
DevConf, HighLoad...
3 ©2021 Percona
Basics
Concurrency
Environment
Query Tuning
Troubleshooting
Workflow
5 ©2021 Percona
Concurrency
Environment
Query Tuning
Troubleshooting
Workflow: This
Session
6 ©2021 Percona
$system = System::factory()
->setName($this->form->get(Field::NAME))
->setDescription(
$this->form->get(Field::DESCRIPTION)
);
DAO::system()->take($system);
The Query
7 ©2021 Percona
$system = System::factory()
->setName($this->form->get(Field::NAME))
->setDescription(
$this->form->get(Field::DESCRIPTION)
);
DAO::system()->take($system);
The Query
7 ©2021 Percona
cursor = conn.cursor()
q = ’’’UPDATE ‘foo‘ SET my_date=NOW(),
subject = %s,
msg = %s,
address = %s,
updated_at = NOW()
WHERE id=%s
’’’
cursor.execute(q, [
remote_resp.get(’subject’),
remote_resp.get(’msg’),
remote_resp.get(’address’),
my_id
])
The Query
7 ©2021 Percona
cursor = conn.cursor()
q = ’’’UPDATE ‘foo‘ SET my_date=NOW(),
subject = %s,
msg = %s,
address = %s,
updated_at = NOW()
WHERE id=%s
’’’
cursor.execute(q, [
remote_resp.get(’subject’),
remote_resp.get(’msg’),
remote_resp.get(’address’),
my_id
])
The Query
7 ©2021 Percona
SELECT dept_name, title, gender,
min(salary) AS mins,
max(salary) AS maxs
FROM employees
JOIN salaries USING(emp_no)
JOIN titles USING(emp_no)
JOIN dept_emp USING(emp_no)
JOIN departments USING(dept_no)
JOIN dept_manager USING(dept_no)
WHERE dept_manager.to_date = ’9999-01-01’
GROUP BY dept_name, title, gender
ORDER BY gender, maxs DESC;
The Query
7 ©2021 Percona
SELECT dept_name, title, gender,
min(salary) AS mins,
max(salary) AS maxs
FROM employees
JOIN salaries USING(emp_no)
JOIN titles USING(emp_no)
JOIN dept_emp USING(emp_no)
JOIN departments USING(dept_no)
JOIN dept_manager USING(dept_no)
WHERE dept_manager.to_date = ’9999-01-01’
GROUP BY dept_name, title, gender
ORDER BY gender, maxs DESC;
The Query
7 ©2021 Percona
• PMM QAN
Always Tune
Raw Query
8 ©2021 Percona
• PMM QAN
• Slow Query Log
Always Tune
Raw Query
8 ©2021 Percona
• PMM QAN
• Slow Query Log
• Application log
Always Tune
Raw Query
8 ©2021 Percona
• PMM QAN
• Slow Query Log
• Application log
• ...
Always Tune
Raw Query
8 ©2021 Percona
• Mind your data!
• 75,000,000 rows
• (INT, INT)
 75,000,000 * (4 + 4) = 600,000,000 B = 572 MB
• (INT, INT, DATETIME,
VARCHAR(255), VARCHAR(255))
 75,000,000 * (4 + 4 + 8 + 256 + 256) =
39,600,000,000 bytes = 37 G
• 39,600,000,000 / 600,000,000 = 66
Slow is Relative
9 ©2021 Percona
• Mind your data!
• Mind use case
• Popular website
• Admin interface
• Weekly cron job
Slow is Relative
9 ©2021 Percona
• Mind your data!
• Mind use case
• Mind location
• Server, used by multiple connections
• Dedicated for OLAP queries
Slow is Relative
9 ©2021 Percona
• MySQL performs a job to execute it
Why is
the Query
Slow
10 ©2021 Percona
• MySQL performs a job to execute it
• In the worst case scenario it will do
a full table scan
• CREATE INDEX
• ANALYZE TABLE ...
UPDATE HISTOGRAM ON ...
Why is
the Query
Slow
10 ©2021 Percona
• MySQL performs a job to execute it
• In the worst case scenario it will do
a full table scan
• CREATE INDEX
• ANALYZE TABLE ...
UPDATE HISTOGRAM ON ...
• Incorrect index can be used
Why is
the Query
Slow
10 ©2021 Percona
Query sent
Connection Pool:
Authentication, Caches;
SQL interface; Parser
Optimizer
Storage engines
Hardware
Query Execution
Workflow
11 ©2021 Percona
• Selecting a lot of data
• SELECT * FROM
many columns table
You May not Be
Able to Change
a Slow Query
12 ©2021 Percona
• Selecting a lot of data
• SELECT * FROM
many columns table
• Badly written
• LEFT JOIN instead of INNER JOIN
• Many values in IN()
• Retrieving large result set, then
discarding
• Not effective SQL
 For particular MySQL version
You May not Be
Able to Change
a Slow Query
12 ©2021 Percona
• Selecting a lot of data
• SELECT * FROM
many columns table
• Badly written
• LEFT JOIN instead of INNER JOIN
• Many values in IN()
• Retrieving large result set, then
discarding
• Not effective SQL
 For particular MySQL version
• We can still improve performance
You May not Be
Able to Change
a Slow Query
12 ©2021 Percona
Indexes
Indexes
How Do They Work?
SELECT name FROM users WHERE id=12
1 2 5 6 7 9 12 16 18 21
Full Table Scan
15 ©2021 Percona
SELECT name FROM users WHERE id=12
1 2 5 6 7 9 12 16 18 21
Index Access
16 ©2021 Percona
d001
d003
d008
d009
d003******
d009******
d008******
d009******
d001******
d003******
d009******
d008******
d009******
d001******
d008******
d008******
MySQL Indexes
- B-Tree Mostly
- LSM Tree
- Fractal Tree
- R-Tree Spatial
- Hash Memory SE
- Engine’s
17 ©2021 Percona
• Single column
CREATE INDEX index name ON
the table(the column)
• Multiple columns
CREATE INDEX index name ON
the table(column1, column2)
How to Create
an Index
18 ©2021 Percona
• Single column
ALTER TABLE table name ADD
INDEX [index name]
(the column)
• Multiple columns
ALTER TABLE table name ADD
INDEX [index name] (column1,
column2)
How to Create
an Index
18 ©2021 Percona
Indexes
When MySQL Uses Indexes
• WHERE the column = a value
• WHERE the column IN(value1,
value2, value3)
• WHERE the column LIKE
’value%’
• WHERE the column LIKE ’%value’
Conditions
20 ©2021 Percona
• WHERE left part = value1
AND right part = value2
• WHERE left part = value1 OR
right part = value2
• WHERE right part = value1
AND left part = value2
• WHERE right part = value1 OR
left part = value2
Conditions
20 ©2021 Percona
• table1 JOIN table2 ON
table1.column1 =
table2.column2
Joins
21 ©2021 Percona
• table1 JOIN table2 ON
table1.column1 =
table2.column2
• Same as FROM table1, table2
WHERE table1.column1 =
table2.column2
Joins
21 ©2021 Percona
• GROUP BY the column
• GROUP BY left part,
right part
• GROUP BY right part, left part
• GROUP BY the index, another index
GROUP BY
22 ©2021 Percona
• ORDER BY the column
• ORDER BY left part,
right part
• ORDER BY right part, left part
• ORDER BY the index, another index
ORDER BY
23 ©2021 Percona
5.7 ORDER BY left part DESC,
right part ASC
8.0 ORDER BY left part DESC,
right part ASC
• left part must be descending
• right part must be ascending
• the index(left part DESC,
right part ASC)
ORDER BY
23 ©2021 Percona
• Deterministic, built-in
• Return same value for the same
argument
• WHERE the column =
FLOOR(123.45)
Expressions
24 ©2021 Percona
• Deterministic, built-in
• Return same value for the same
argument
• WHERE the column =
FLOOR(123.45)
• Non-deterministic
• Return different values for different
invocations
• WHERE the column = RAND() ∗ 100
Expressions
24 ©2021 Percona
• Deterministic, built-in
• Return same value for the same
argument
• WHERE the column =
FLOOR(123.45)
• Non-deterministic
• Return different values for different
invocations
• WHERE the column = RAND() ∗ 100
• Stored functions and UDFs
• Indexes are not used
Use generated column indexes
Expressions
24 ©2021 Percona
• Identify queries not using indexes
• Status variables
mysql select * from performance_schema.session_status
- where variable_name in (’Created_tmp_tables’,
- ’Created_tmp_disk_tables’, ’Select_full_join’,
- ’Select_full_range_join’, ’Select_range’,
- ’Select_range_check’, ’Select_scan’,
- ’Sort_merge_passes’, ’Sort_range’, ’Sort_rows’,
- ’Sort_scan’) and variable_value  0;
+------------------------+----------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+------------------------+----------------+
| Select_scan | 2 |
+------------------------+----------------+
1 row in set (0,00 sec)
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
mysql show global status like ’Handler_read%’;
+-----------------------+-------+
| Variable_name | Value |
+-----------------------+-------+
| Handler_read_first | 31 |
| Handler_read_key | 1909 |
| Handler_read_last | 0 |
| Handler_read_next | 4393 |
| Handler_read_prev | 0 |
| Handler_read_rnd | 0 |
| Handler_read_rnd_next | 1135 |
+-----------------------+-------+
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• PERFORMANCE SCHEMA.EVENTS STATEMENTS *
mysql select * from events_statements_history_long
- where sql_text like ’select count(*) from employees join %’G
*************************** 1. row ****************************
...
ROWS_SENT: 1 SELECT_RANGE_CHECK: 0
ROWS_EXAMINED: 541058 SELECT_SCAN: 1
CREATED_TMP_DISK_TABLES: 0 SORT_MERGE_PASSES: 0
CREATED_TMP_TABLES: 0 SORT_RANGE: 0
SELECT_FULL_JOIN: 0 SORT_ROWS: 0
SELECT_FULL_RANGE_JOIN: 0 SORT_SCAN: 0
SELECT_RANGE: 0 NO_INDEX_USED: 0
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• Slow query log
# Time: 2020-10-11T23:34:03.701871Z
# User@Host: root[root] @ localhost [127.0.0.1] Id: 506
# Query_time: 0.024106 Lock_time: 0.000091
Rows_sent: 1 Rows_examined: 1000
SET timestamp=1602459243;
SELECT c FROM sbtest1 WHERE id=996;
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• QAN in PMM
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• Analyze if adding an index will help
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• Analyze if adding an index will help
• Add the index
When to Add
Indexes?
25 ©2021 Percona
• Identify queries not using indexes
• Analyze if adding an index will help
• Add the index
• Test first!
• ADD INDEX is an expensive operation
Use pt-online-schema-change
When to Add
Indexes?
25 ©2021 Percona
Indexes
Optimizer Histograms
• Since version 8.0
Histogram
Statistics
27 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
Histogram
Statistics
27 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
• Visible in Information Schema
mysql select HISTOGRAM from information_schema.column_statistics
- where table_name=’example’G
*************************** 1. row ***************************
HISTOGRAM: {buckets: [[1, 0.6], [2, 0.8], [3, 1.0]],
data-type: int, null-values: 0.0, collation-id: 8,
last-updated: 2018-11-07 09:07:19.791470,
sampling-rate: 1.0, histogram-type: singleton,
number-of-buckets-specified: 3}
1 row in set (0.00 sec)
Histogram
Statistics
27 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
• Visible in Information Schema
• Affects query execution plan, but
not access
• SELECT ... FROM a JOIN b
vs SELECT ... FROM b JOIN a
Histogram
Statistics
27 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
• Visible in Information Schema
• Affects query execution plan, but
not access
More details
Histogram
Statistics
27 ©2021 Percona
1 2 3 4 5 6 7 8 9 10
0
200
400
600
800
Indexes:
Number of
Items with Same
Value
28 ©2021 Percona
1 2 3 4 5 6 7 8 9 10
0
200
400
600
800
Indexes:
Cardinality
29 ©2021 Percona
1 2 3 4 5 6 7 8 9 10
0
200
400
600
800
Histograms:
Number of
Values in Each
Bucket
30 ©2021 Percona
1 2 3 4 5 6 7 8 9 10
0
0.2
0.4
0.6
0.8
1
Histograms:
Data in the
Histogram
31 ©2021 Percona
• Data distribution is far from uniform
• Query accesses two or more tables
• Indexes cannot be used
• Maintaining them is too expensive
• They are already used for different
conditions
Create
Histograms
32 ©2021 Percona
Diagnostics
Diagnostics
EXPLAIN: How Optimizer Works
• EXPLAIN
• Estimates the query execution plan
5.6- EXTENDED
5.6- PARTITIONS
5.6+ FORMAT=JSON
8.0+ FORMAT=TREE
How to Find
how MySQL
Uses Indexes
35 ©2021 Percona
• EXPLAIN
• Estimates the query execution plan
5.6- EXTENDED
5.6- PARTITIONS
5.6+ FORMAT=JSON
8.0+ FORMAT=TREE
• INFORMATION SCHEMA.OPTIMIZER TRACE
• Real data, collected when query was
executing
• Advanced topic
How to Find
how MySQL
Uses Indexes
35 ©2021 Percona
mysql EXPLAIN SELECT first_name, last_name, title, salary FROM employees
- JOIN salaries USING(emp_no) JOIN titles USING(emp_no)
- WHERE salary = (SELECT MAX(salary) FROM salaries
- JOIN titles WHERE titles.to_date  CURDATE());
+----+-------------+-----------+------------+------+---------------+---------+---------+
| id | select_type | table | partitions | type | possible_keys | key | key_len |
+----+-------------+-----------+------------+------+---------------+---------+---------+
| 1 | PRIMARY | employees | NULL | ALL | PRIMARY | NULL | NULL ...
| 1 | PRIMARY | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ...
| 1 | PRIMARY | titles | NULL | ref | PRIMARY | PRIMARY | 4 ...
| 2 | SUBQUERY | titles | NULL | ALL | NULL | NULL | NULL ...
| 2 | SUBQUERY | salaries | NULL | ALL | NULL | NULL | NULL ...
+----+-------------+-----------+------------+------+---------------+---------+---------+
+----+-...-+----------------------------+---------+----------+-------------------------------+
| id | ... | ref | rows | filtered | Extra |
+----+-...-+----------------------------+---------+----------+-------------------------------+
| 1 | ... | NULL | 299113 | 100.00 | NULL |
| 1 | ... | employees.employees.emp_no | 9 | 10.00 | Using where |
| 1 | ... | employees.employees.emp_no | 1 | 100.00 | Using index |
| 2 | ... | NULL | 442189 | 33.33 | Using where |
| 2 | ... | NULL | 2838426 | 100.00 | Using join buffer (hash join) |
+----+-...-+----------------------------+---------+----------+-------------------------------+
5 rows in set, 1 warning (0,01 sec)
299113 * 9 * 1 * 442189 * 2838426 = 3,378,806,408,204,514,738
EXPLAIN: rows
36 ©2021 Percona
mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees
- JOIN salaries USING(emp_no) JOIN titles USING(emp_no)
- GROUP BY title ORDER BY maxs DESC;
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| id | select_type | table | partitions | type | possible_keys | key | key_len
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ...
| 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ...
| 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ...
+----+-------------+-----------+------------+-------+---------------+---------+---------+
EXPLAIN: type
37 ©2021 Percona
ALL Read all rows
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
 index subquery
 unique subquery
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
merge Combination of two indexes
 index merge
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
merge Combination of two indexes
ref Comparision with single value
 ref or null
 ref
 eq ref
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
merge Combination of two indexes
ref Comparision with single value
fulltext Fulltext index
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
merge Combination of two indexes
ref Comparision with single value
fulltext Fulltext index
const Single row
Query
Execution Type
38 ©2021 Percona
ALL Read all rows
index Read all rows from the index
range Range: N ... M
subquery Subqueries
merge Combination of two indexes
ref Comparision with single value
fulltext Fulltext index
const Single row
system Table has single row
Query
Execution Type
38 ©2021 Percona
mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees
- JOIN salaries USING(emp_no) JOIN titles USING(emp_no)
- GROUP BY title ORDER BY maxs DESC;
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| id | select_type | table | partitions | type | possible_keys | key | key_len
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ...
| 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ...
| 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ...
+----+-------------+-----------+------------+-------+---------------+---------+---------+
EXPLAIN:
key and
possible keys
39 ©2021 Percona
mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees
- JOIN salaries USING(emp_no)
- JOIN titles USING(emp_no)
- GROUP BY title ORDER BY maxs DESC;
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| id | select_type | table | partitions | type | possible_keys | key | key_len
+----+-------------+-----------+------------+-------+---------------+---------+---------+
| 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ...
| 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ...
| 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ...
+----+-------------+-----------+------------+-------+---------------+---------+---------+
EXPLAIN:
JOIN order
40 ©2021 Percona
mysql EXPLAIN SELECT first_name, last_name, title, salary FROM employees
- JOIN salaries USING(emp_no) JOIN titles USING(emp_no)
- WHERE salary IN (SELECT MAX(salary) FROM salaries
- JOIN titles WHERE titles.to_date  CURDATE());
...
5 rows in set, 1 warning (0,01 sec)
mysql SHOW WARNINGSG
*************************** 1. row ***************************
Level: Note
Code: 1003
Message: /* select#1 */ select ‘employees‘.‘employees‘.‘first_name‘ AS ‘first_name‘,
‘employees‘.‘employees‘.‘last_name‘ AS ‘last_name‘,‘employees‘.‘salaries‘.‘salary‘ AS ‘salary‘
from ‘employees‘.‘employees‘ join ‘employees‘.‘salaries‘
where ((‘employees‘.‘salaries‘.‘emp_no‘ = ‘employees‘.‘employees‘.‘emp_no‘)
and in_optimizer(‘employees‘.‘salaries‘.‘salary‘,‘employees‘.‘salaries‘.‘salary‘ in
( materialize (/* select#2 */
select max(‘employees‘.‘salaries‘.‘salary‘)
from ‘employees‘.‘salaries‘ join ‘employees‘.‘titles‘
where (‘employees‘.‘titles‘.‘to_date‘  cache(curdate())) having true ),
primary_index_lookup(‘employees‘.‘salaries‘.‘salary‘ in
temporary table on auto_distinct_key
where ((‘employees‘.‘salaries‘.‘salary‘ = ‘materialized_subquery‘.‘MAX(salary)‘))))))
1 row in set (0,00 sec)
Actual Query
41 ©2021 Percona
mysql explain select * from t1G
*************************** 1. row ***************************
...
rows: 12
Extra: NULL
mysql explain select * from t1 where f2=12G
*************************** 1. row ***************************
...
key: NULL
...
rows: 12
Extra: Using where
Effect of
Indexes: Before
42 ©2021 Percona
mysql alter table t1 add index(f2);
Query OK, 12 rows affected (0.07 sec)
Records: 12 Duplicates: 0 Warnings: 0
mysql explain select * from t1 where f2=12G
*************************** 1. row ***************************
...
key: f2
key_len: 5
ref: const
rows: 1
Extra: NULL
1 row in set (0.00 sec)
Effect of
Indexes: After
43 ©2021 Percona
Diagnostics
Real Numbers: Inside Storage Engine
• EXPLAIN is optimistic
mysql explain select * from ol
- where thread_id=10432 and site_id != 9939
- order by id limit 3G
*************************** 1. row ***************************
id: 1 | ref: NULL
select_type: SIMPLE | rows: 33
table: ol | filtered: 8.07
partitions: NULL | Extra: Using where
type: index
possible_keys: thread_id
key: PRIMARY
key_len: 4
1 row in set, 1 warning (0,00 sec)
Handler *
Status Variables
45 ©2021 Percona
• ’Handler *’ show truth
mysql FLUSH STATUS;
mysql select * from ol
- where thread_id=10432 and site_id != 9939
- order by id limit 3;
mysql show status like ’Handler%’;
+----------------------------+--------+
| Variable_name | Value |
+----------------------------+--------+
...
| Handler_read_first | 1 |
| Handler_read_key | 1 |
| Handler_read_last | 0 |
| Handler_read_next | 100000 |
...
Handler *
Status Variables
45 ©2021 Percona
Diagnostics
Real Numbers: Inside the Server
• SHOW [FULL] PROCESSLIST
• INFORMATION SCHEMA.PROCESSLIST
• performance schema.THREADS
PROCESSLIST
47 ©2021 Percona
• SHOW [FULL] PROCESSLIST
• INFORMATION SCHEMA.PROCESSLIST
• performance schema.THREADS
• Your first alert of performance issue
PROCESSLIST
47 ©2021 Percona
• SHOW [FULL] PROCESSLIST
• INFORMATION SCHEMA.PROCESSLIST
• performance schema.THREADS
• Your first alert of performance issue
• Shows all running queries
PROCESSLIST
47 ©2021 Percona
• Can be seen in the PROCESSLIST
mysql show processlistG
************************ 1. row ************************
Id: 7
User: root
Host: localhost:48799
db: employees
Command: Query
Time: 2
State: Sending data
Info: select count(*) from employees
join titles using(emp_no)
where title=’Senior Engineer’
...
Execution
Stages
48 ©2021 Percona
• Can be seen in the PROCESSLIST
• Very useful when you need to answer
on the question: ”What is my server
doing now?”
Execution
Stages
48 ©2021 Percona
• PERFORMANCE SCHEMA.EVENTS STAGES *
mysql select eshl.event_name, substr(sql_text, 1, 15) ‘sql‘,
- eshl.timer_wait/1000000000000 w_s
- from events_stages_history_long
- eshl join events_statements_history_long esthl on
- (eshl.nesting_event_id = esthl.event_id) where
- esthl.current_schema=’employees’ and sql_text like
- ’select count(*) from employees%’
- order by eshl.timer_start asc;
+--------------------------------+-----------------+--------+
| event_name | sql | w_s |
+--------------------------------+-----------------+--------+
| stage/sql/starting | select count(*) | 0.0002 |
| stage/sql/checking permissions | select count(*) | 0.0000 |
...
Execution
Stages
48 ©2021 Percona
• PERFORMANCE SCHEMA.EVENTS STAGES *
...
| stage/sql/checking permissions | select count(*) | 0.0000 |
| stage/sql/Opening tables | select count(*) | 0.0000 |
| stage/sql/init | select count(*) | 0.0001 |
| stage/sql/System lock | select count(*) | 0.0000 |
| stage/sql/optimizing | select count(*) | 0.0000 |
| stage/sql/statistics | select count(*) | 0.0001 |
| stage/sql/preparing | select count(*) | 0.0000 |
| stage/sql/executing | select count(*) | 0.0000 |
| stage/sql/Sending data | select count(*) | 5.4915 |
| stage/sql/end | select count(*) | 0.0000 |
...
Execution
Stages
48 ©2021 Percona
• Status variables
mysql flush status;
Query OK, 0 rows affected (0,01 sec)
mysql select count(*) from employees join titles using(emp_no)
- where title=’Senior Engineer’;
+----------+
| count(*) |
+----------+
| 97750 |
+----------+
1 row in set (5,44 sec)
Temporary
Tables
and
Other Job
49 ©2021 Percona
• Status variables
mysql select * from performance_schema.session_status
- where variable_name in (’Created_tmp_tables’,
- ’Created_tmp_disk_tables’, ’Select_full_join’,
- ’Select_full_range_join’, ’Select_range’,
- ’Select_range_check’, ’Select_scan’, ’Sort_merge_passes’,
- ’Sort_range’, ’Sort_rows’, ’Sort_scan’)
- and variable_value  0;
+------------------------+----------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+------------------------+----------------+
| Select_scan | 2 |
+------------------------+----------------+
1 row in set (0,00 sec)
Temporary
Tables
and
Other Job
49 ©2021 Percona
• PERFORMANCE SCHEMA.EVENTS STATEMENTS *
mysql select * from events_statements_history_long
- where sql_text like
- ’select count(*) from employees join %’G
*************************** 1. row ****************************
...
ROWS_SENT: 1 SELECT_RANGE_CHECK: 0
ROWS_EXAMINED: 541058 SELECT_SCAN: 1
CREATED_TMP_DISK_TABLES: 0 SORT_MERGE_PASSES: 0
CREATED_TMP_TABLES: 0 SORT_RANGE: 0
SELECT_FULL_JOIN: 0 SORT_ROWS: 0
SELECT_FULL_RANGE_JOIN: 0 SORT_SCAN: 0
SELECT_RANGE: 0 NO_INDEX_USED: 0
Temporary
Tables
and
Other Job
49 ©2021 Percona
• sys.statement analysis
mysql select * from statement_analysis where query like
- ’SELECT COUNT ( * ) FROM ‘emplo%’ and db=’employees’G
*************************** 1. row ***************************
query: SELECT COUNT ( * ) FROM ‘emplo ... ‘emp_no‘ ) WHE...
db: employees max_latency: 5.59 s
full_scan: avg_latency: 5.41 s
exec_count: 7 lock_latency: 2.24 ms
err_count: 0 rows_sent: 7
warn_count: 0 rows_sent_avg: 1
total_latency: 37.89 s rows_examined: 3787406
Temporary
Tables
and
Other Job
49 ©2021 Percona
• sys.statement analysis
rows_examined_avg: 541058
rows_affected: 0
rows_affected_avg: 0
tmp_tables: 0
tmp_disk_tables: 0
rows_sorted: 0
sort_merge_passes: 0
digest: 4086bc3dc6510a1d9c8f2fe1f59f0943
first_seen: 2016-04-14 15:19:19
last_seen: 2016-04-14 16:13:14
Temporary
Tables
and
Other Job
49 ©2021 Percona
How to Affect Query
Plans
• Index statistics
• Histogram statistics
• Optimizer switches
• Bugs in optimizer
What has an
Effect on Query
Optimizer
Plans?
51 ©2021 Percona
• Collected by storage engine
Index Statistics
52 ©2021 Percona
• Collected by storage engine
• Used by Optimizer
Index Statistics
52 ©2021 Percona
• Can be examined by SHOW INDEX
mysql show index from sbtest1;
+---------+----------+-------------+-------------+
| Table | Key_name | Column_name | Cardinality |
+---------+----------+-------------+-------------+
| sbtest1 | k_1 | k | 49142 |
+---------+----------+-------------+-------------+
mysql select count(distinct id),
- count(distinct k) from sbtest1;
+--------------------+-------------------+
| count(distinct id) | count(distinct k) |
+--------------------+-------------------+
| 100000 | 17598 |
+--------------------+-------------------+
Index Statistics
52 ©2021 Percona
• Can be updated
• ANALYZE TABLE
• If does not help: rebuild the table
 OPTIMIZE TABLE
 ALTER TABLE ENGINE=INNODB; ANALYZE TABLE
Index Statistics
52 ©2021 Percona
• Since version 8.0
Histogram
Statistics
53 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
Histogram
Statistics
53 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
• Visible in Information Schema
mysql select HISTOGRAM from information_schema.column_statistics
- where table_name=’example’G
*************************** 1. row ***************************
HISTOGRAM: {buckets: [[1, 0.6], [2, 0.8], [3, 1.0]],
data-type: int, null-values: 0.0, collation-id: 8,
last-updated: 2018-11-07 09:07:19.791470,
sampling-rate: 1.0, histogram-type: singleton,
number-of-buckets-specified: 3}
1 row in set (0.00 sec)
Histogram
Statistics
53 ©2021 Percona
• Since version 8.0
• Collected and used by the Optimizer
• Visible in Information Schema
More details
Histogram
Statistics
53 ©2021 Percona
mysql select @@optimizer_switchG
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
index_merge_sort_union=on,index_merge_intersection=on,
engine_condition_pushdown=on,index_condition_pushdown=on,
mrr=on,mrr_cost_based=on,
block_nested_loop=on,batched_key_access=off,
materialization=on,semijoin=on,loosescan=on,firstmatch=on,
duplicateweedout=on,subquery_materialization_cost_based=on,
use_index_extensions=on,condition_fanout_filter=on,
derived_merge=on
1 row in set (0,00 sec)
Optimizer
Switches
54 ©2021 Percona
• Turn ON and OFF particular
optimization
Optimizer
Switches
54 ©2021 Percona
• Turn ON and OFF particular
optimization
• Can be unhelpful
• Especially for queries, tuned for
previous versions
Optimizer
Switches
54 ©2021 Percona
• Turn ON and OFF particular
optimization
• Can be unhelpful
• Work with them as with any other
option
• Turn OFF and try
SET optimizer_switch = ’use_index_extensions=off’;
SELECT ...
EXPLAIN SELECT ...
Optimizer
Switches
54 ©2021 Percona
• Turn ON and OFF particular
optimization
• Can be unhelpful
• Work with them as with any other
option
• If helps implement in queries
SELECT /*+ SEMIJOIN(FIRSTMATCH, LOOSESCAN) */ * FROM t1 ...;
SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
Optimizer
Switches
54 ©2021 Percona
• Optimizer choses wrong plan for no
reason
Bugs in
Optimizer
55 ©2021 Percona
• Optimizer choses wrong plan for no
reason
• Statistics are up to date
• Histograms are not usable
Bugs in
Optimizer
55 ©2021 Percona
• Optimizer choses wrong plan for no
reason
• Statistics are up to date
• Histograms are not usable
• Solution
• Use index hints
 FORCE INDEX
 IGNORE INDEX
Bugs in
Optimizer
55 ©2021 Percona
• Optimizer choses wrong plan for no
reason
• Statistics are up to date
• Histograms are not usable
• Solution
• On every upgrade
• Remove index hints
• Test if the query improved
• You must do it even for minor
version upgrades!
Bugs in
Optimizer
55 ©2021 Percona
• EXPLAIN is essential for query tuning
• Real job is done by storage engine
• Index statistics affect query
execution plan
• All index hints, optimizer hints and
other workarounds must be validated
on each upgrade
Summary
56 ©2021 Percona
EXPLAIN Syntax
EXPLAIN FORMAT=JSON is Cool!
Troubleshooting add-ons
Optimizer Statistics aka Histograms
Optimizer Hints
Tracing the Optimizer
More
Information
57 ©2021 Percona
www.slideshare.net/SvetaSmirnova
twitter.com/svetsmirnova
github.com/svetasmirnova
Thank you!
58 ©2021 Percona

More Related Content

What's hot

MySQL Performance Schema in Action
MySQL Performance Schema in ActionMySQL Performance Schema in Action
MySQL Performance Schema in ActionSveta Smirnova
 
Preparse Query Rewrite Plugins
Preparse Query Rewrite PluginsPreparse Query Rewrite Plugins
Preparse Query Rewrite PluginsSveta Smirnova
 
Modern solutions for modern database load: improvements in the latest MariaDB...
Modern solutions for modern database load: improvements in the latest MariaDB...Modern solutions for modern database load: improvements in the latest MariaDB...
Modern solutions for modern database load: improvements in the latest MariaDB...Sveta Smirnova
 
Need for Speed: Mysql indexing
Need for Speed: Mysql indexingNeed for Speed: Mysql indexing
Need for Speed: Mysql indexingFromDual GmbH
 
Why Use EXPLAIN FORMAT=JSON?
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON? Sveta Smirnova
 
Performance Schema in Action: demo
Performance Schema in Action: demoPerformance Schema in Action: demo
Performance Schema in Action: demoSveta Smirnova
 
Optimizing queries MySQL
Optimizing queries MySQLOptimizing queries MySQL
Optimizing queries MySQLGeorgi Sotirov
 
MySQL Query tuning 101
MySQL Query tuning 101MySQL Query tuning 101
MySQL Query tuning 101Sveta Smirnova
 
Performance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingPerformance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingSveta Smirnova
 
Billion Goods in Few Categories: how Histograms Save a Life?
Billion Goods in Few Categories: how Histograms Save a Life?Billion Goods in Few Categories: how Histograms Save a Life?
Billion Goods in Few Categories: how Histograms Save a Life?Sveta Smirnova
 
Optimizer Histograms: When they Help and When Do Not?
Optimizer Histograms: When they Help and When Do Not?Optimizer Histograms: When they Help and When Do Not?
Optimizer Histograms: When they Help and When Do Not?Sveta Smirnova
 
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013Sergey Petrunya
 
New SQL features in latest MySQL releases
New SQL features in latest MySQL releasesNew SQL features in latest MySQL releases
New SQL features in latest MySQL releasesGeorgi Sotirov
 
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Jaime Crespo
 
Performance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingPerformance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingSveta Smirnova
 
MySQL Performance Schema in 20 Minutes
 MySQL Performance Schema in 20 Minutes MySQL Performance Schema in 20 Minutes
MySQL Performance Schema in 20 MinutesSveta Smirnova
 
Basic MySQL Troubleshooting for Oracle Database Administrators
Basic MySQL Troubleshooting for Oracle Database AdministratorsBasic MySQL Troubleshooting for Oracle Database Administrators
Basic MySQL Troubleshooting for Oracle Database AdministratorsSveta Smirnova
 
New features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in actionNew features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in actionSveta Smirnova
 
Fosdem2012 mariadb-5.3-query-optimizer-r2
Fosdem2012 mariadb-5.3-query-optimizer-r2Fosdem2012 mariadb-5.3-query-optimizer-r2
Fosdem2012 mariadb-5.3-query-optimizer-r2Sergey Petrunya
 
MySQL Performance Schema in Action
MySQL Performance Schema in ActionMySQL Performance Schema in Action
MySQL Performance Schema in ActionSveta Smirnova
 

What's hot (20)

MySQL Performance Schema in Action
MySQL Performance Schema in ActionMySQL Performance Schema in Action
MySQL Performance Schema in Action
 
Preparse Query Rewrite Plugins
Preparse Query Rewrite PluginsPreparse Query Rewrite Plugins
Preparse Query Rewrite Plugins
 
Modern solutions for modern database load: improvements in the latest MariaDB...
Modern solutions for modern database load: improvements in the latest MariaDB...Modern solutions for modern database load: improvements in the latest MariaDB...
Modern solutions for modern database load: improvements in the latest MariaDB...
 
Need for Speed: Mysql indexing
Need for Speed: Mysql indexingNeed for Speed: Mysql indexing
Need for Speed: Mysql indexing
 
Why Use EXPLAIN FORMAT=JSON?
 Why Use EXPLAIN FORMAT=JSON?  Why Use EXPLAIN FORMAT=JSON?
Why Use EXPLAIN FORMAT=JSON?
 
Performance Schema in Action: demo
Performance Schema in Action: demoPerformance Schema in Action: demo
Performance Schema in Action: demo
 
Optimizing queries MySQL
Optimizing queries MySQLOptimizing queries MySQL
Optimizing queries MySQL
 
MySQL Query tuning 101
MySQL Query tuning 101MySQL Query tuning 101
MySQL Query tuning 101
 
Performance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingPerformance Schema for MySQL Troubleshooting
Performance Schema for MySQL Troubleshooting
 
Billion Goods in Few Categories: how Histograms Save a Life?
Billion Goods in Few Categories: how Histograms Save a Life?Billion Goods in Few Categories: how Histograms Save a Life?
Billion Goods in Few Categories: how Histograms Save a Life?
 
Optimizer Histograms: When they Help and When Do Not?
Optimizer Histograms: When they Help and When Do Not?Optimizer Histograms: When they Help and When Do Not?
Optimizer Histograms: When they Help and When Do Not?
 
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
 
New SQL features in latest MySQL releases
New SQL features in latest MySQL releasesNew SQL features in latest MySQL releases
New SQL features in latest MySQL releases
 
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
Query Optimization with MySQL 5.6: Old and New Tricks - Percona Live London 2013
 
Performance Schema for MySQL Troubleshooting
Performance Schema for MySQL TroubleshootingPerformance Schema for MySQL Troubleshooting
Performance Schema for MySQL Troubleshooting
 
MySQL Performance Schema in 20 Minutes
 MySQL Performance Schema in 20 Minutes MySQL Performance Schema in 20 Minutes
MySQL Performance Schema in 20 Minutes
 
Basic MySQL Troubleshooting for Oracle Database Administrators
Basic MySQL Troubleshooting for Oracle Database AdministratorsBasic MySQL Troubleshooting for Oracle Database Administrators
Basic MySQL Troubleshooting for Oracle Database Administrators
 
New features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in actionNew features in Performance Schema 5.7 in action
New features in Performance Schema 5.7 in action
 
Fosdem2012 mariadb-5.3-query-optimizer-r2
Fosdem2012 mariadb-5.3-query-optimizer-r2Fosdem2012 mariadb-5.3-query-optimizer-r2
Fosdem2012 mariadb-5.3-query-optimizer-r2
 
MySQL Performance Schema in Action
MySQL Performance Schema in ActionMySQL Performance Schema in Action
MySQL Performance Schema in Action
 

Similar to Introduction into MySQL Query Tuning for Dev[Op]s

Developers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman OracleDevelopers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman OraclemCloud
 
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0mCloud
 
Migration from mysql to elasticsearch
Migration from mysql to elasticsearchMigration from mysql to elasticsearch
Migration from mysql to elasticsearchRyosuke Nakamura
 
Scaling MySQL Strategies for Developers
Scaling MySQL Strategies for DevelopersScaling MySQL Strategies for Developers
Scaling MySQL Strategies for DevelopersJonathan Levin
 
How Database Convergence Impacts the Coming Decades of Data Management
How Database Convergence Impacts the Coming Decades of Data ManagementHow Database Convergence Impacts the Coming Decades of Data Management
How Database Convergence Impacts the Coming Decades of Data ManagementSingleStore
 
Streaming ETL - from RDBMS to Dashboard with KSQL
Streaming ETL - from RDBMS to Dashboard with KSQLStreaming ETL - from RDBMS to Dashboard with KSQL
Streaming ETL - from RDBMS to Dashboard with KSQLBjoern Rost
 
My sql 56_roadmap_april2012
My sql 56_roadmap_april2012My sql 56_roadmap_april2012
My sql 56_roadmap_april2012sqlhjalp
 
Confoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesConfoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesDave Stokes
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)Aurimas Mikalauskas
 
DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2Alex Zaballa
 
DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2Alex Zaballa
 
NoSQL on MySQL - MySQL Document Store by Vadim Tkachenko
NoSQL on MySQL - MySQL Document Store by Vadim TkachenkoNoSQL on MySQL - MySQL Document Store by Vadim Tkachenko
NoSQL on MySQL - MySQL Document Store by Vadim TkachenkoData Con LA
 
MariaDB Cassandra Interoperability
MariaDB Cassandra InteroperabilityMariaDB Cassandra Interoperability
MariaDB Cassandra InteroperabilityColin Charles
 
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019Dave Stokes
 
Aioug vizag oracle12c_new_features
Aioug vizag oracle12c_new_featuresAioug vizag oracle12c_new_features
Aioug vizag oracle12c_new_featuresAiougVizagChapter
 
Dan Hotka's Top 10 Oracle 12c New Features
Dan Hotka's Top 10 Oracle 12c New FeaturesDan Hotka's Top 10 Oracle 12c New Features
Dan Hotka's Top 10 Oracle 12c New FeaturesEmbarcadero Technologies
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresJitendra Singh
 
Optimizing your Database Import!
Optimizing your Database Import! Optimizing your Database Import!
Optimizing your Database Import! Nabil Nawaz
 

Similar to Introduction into MySQL Query Tuning for Dev[Op]s (20)

Developers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman OracleDevelopers' mDay 2017. - Bogdan Kecman Oracle
Developers' mDay 2017. - Bogdan Kecman Oracle
 
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
Developers’ mDay u Banjoj Luci - Bogdan Kecman, Oracle – MySQL Server 8.0
 
Migration from mysql to elasticsearch
Migration from mysql to elasticsearchMigration from mysql to elasticsearch
Migration from mysql to elasticsearch
 
Scaling MySQL Strategies for Developers
Scaling MySQL Strategies for DevelopersScaling MySQL Strategies for Developers
Scaling MySQL Strategies for Developers
 
How Database Convergence Impacts the Coming Decades of Data Management
How Database Convergence Impacts the Coming Decades of Data ManagementHow Database Convergence Impacts the Coming Decades of Data Management
How Database Convergence Impacts the Coming Decades of Data Management
 
MySQL Ecosystem in 2018
MySQL Ecosystem in 2018MySQL Ecosystem in 2018
MySQL Ecosystem in 2018
 
Streaming ETL - from RDBMS to Dashboard with KSQL
Streaming ETL - from RDBMS to Dashboard with KSQLStreaming ETL - from RDBMS to Dashboard with KSQL
Streaming ETL - from RDBMS to Dashboard with KSQL
 
My sql 56_roadmap_april2012
My sql 56_roadmap_april2012My sql 56_roadmap_april2012
My sql 56_roadmap_april2012
 
Confoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesConfoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New Features
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
 
DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2
 
DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2DBA Commands and Concepts That Every Developer Should Know - Part 2
DBA Commands and Concepts That Every Developer Should Know - Part 2
 
NoSQL on MySQL - MySQL Document Store by Vadim Tkachenko
NoSQL on MySQL - MySQL Document Store by Vadim TkachenkoNoSQL on MySQL - MySQL Document Store by Vadim Tkachenko
NoSQL on MySQL - MySQL Document Store by Vadim Tkachenko
 
MariaDB Cassandra Interoperability
MariaDB Cassandra InteroperabilityMariaDB Cassandra Interoperability
MariaDB Cassandra Interoperability
 
Navicat
NavicatNavicat
Navicat
 
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019
MySQL Baics - Texas Linxufest beginners tutorial May 31st, 2019
 
Aioug vizag oracle12c_new_features
Aioug vizag oracle12c_new_featuresAioug vizag oracle12c_new_features
Aioug vizag oracle12c_new_features
 
Dan Hotka's Top 10 Oracle 12c New Features
Dan Hotka's Top 10 Oracle 12c New FeaturesDan Hotka's Top 10 Oracle 12c New Features
Dan Hotka's Top 10 Oracle 12c New Features
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and Underscores
 
Optimizing your Database Import!
Optimizing your Database Import! Optimizing your Database Import!
Optimizing your Database Import!
 

More from Sveta Smirnova

Database in Kubernetes: Diagnostics and Monitoring
Database in Kubernetes: Diagnostics and MonitoringDatabase in Kubernetes: Diagnostics and Monitoring
Database in Kubernetes: Diagnostics and MonitoringSveta Smirnova
 
MySQL Database Monitoring: Must, Good and Nice to Have
MySQL Database Monitoring: Must, Good and Nice to HaveMySQL Database Monitoring: Must, Good and Nice to Have
MySQL Database Monitoring: Must, Good and Nice to HaveSveta Smirnova
 
MySQL Cookbook: Recipes for Developers
MySQL Cookbook: Recipes for DevelopersMySQL Cookbook: Recipes for Developers
MySQL Cookbook: Recipes for DevelopersSveta Smirnova
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOpsSveta Smirnova
 
MySQL Test Framework для поддержки клиентов и верификации багов
MySQL Test Framework для поддержки клиентов и верификации баговMySQL Test Framework для поддержки клиентов и верификации багов
MySQL Test Framework для поддержки клиентов и верификации баговSveta Smirnova
 
MySQL Cookbook: Recipes for Your Business
MySQL Cookbook: Recipes for Your BusinessMySQL Cookbook: Recipes for Your Business
MySQL Cookbook: Recipes for Your BusinessSveta Smirnova
 
Производительность MySQL для DevOps
 Производительность MySQL для DevOps Производительность MySQL для DevOps
Производительность MySQL для DevOpsSveta Smirnova
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOpsSveta Smirnova
 
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterHow to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterSveta Smirnova
 
How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
 
Современному хайлоду - современные решения: MySQL 8.0 и улучшения Percona
Современному хайлоду - современные решения: MySQL 8.0 и улучшения PerconaСовременному хайлоду - современные решения: MySQL 8.0 и улучшения Percona
Современному хайлоду - современные решения: MySQL 8.0 и улучшения PerconaSveta Smirnova
 
How to Avoid Pitfalls in Schema Upgrade with Galera
How to Avoid Pitfalls in Schema Upgrade with GaleraHow to Avoid Pitfalls in Schema Upgrade with Galera
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
 
How Safe is Asynchronous Master-Master Setup?
 How Safe is Asynchronous Master-Master Setup? How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?Sveta Smirnova
 
Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Sveta Smirnova
 
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...Sveta Smirnova
 
Что нужно знать о трёх топовых фичах MySQL
Что нужно знать  о трёх топовых фичах  MySQLЧто нужно знать  о трёх топовых фичах  MySQL
Что нужно знать о трёх топовых фичах MySQLSveta Smirnova
 
Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Sveta Smirnova
 
MySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialSveta Smirnova
 
Why MySQL Replication Fails, and How to Get it Back
Why MySQL Replication Fails, and How to Get it BackWhy MySQL Replication Fails, and How to Get it Back
Why MySQL Replication Fails, and How to Get it BackSveta Smirnova
 
Эффективная отладка репликации MySQL
Эффективная отладка репликации MySQLЭффективная отладка репликации MySQL
Эффективная отладка репликации MySQLSveta Smirnova
 

More from Sveta Smirnova (20)

Database in Kubernetes: Diagnostics and Monitoring
Database in Kubernetes: Diagnostics and MonitoringDatabase in Kubernetes: Diagnostics and Monitoring
Database in Kubernetes: Diagnostics and Monitoring
 
MySQL Database Monitoring: Must, Good and Nice to Have
MySQL Database Monitoring: Must, Good and Nice to HaveMySQL Database Monitoring: Must, Good and Nice to Have
MySQL Database Monitoring: Must, Good and Nice to Have
 
MySQL Cookbook: Recipes for Developers
MySQL Cookbook: Recipes for DevelopersMySQL Cookbook: Recipes for Developers
MySQL Cookbook: Recipes for Developers
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
 
MySQL Test Framework для поддержки клиентов и верификации багов
MySQL Test Framework для поддержки клиентов и верификации баговMySQL Test Framework для поддержки клиентов и верификации багов
MySQL Test Framework для поддержки клиентов и верификации багов
 
MySQL Cookbook: Recipes for Your Business
MySQL Cookbook: Recipes for Your BusinessMySQL Cookbook: Recipes for Your Business
MySQL Cookbook: Recipes for Your Business
 
Производительность MySQL для DevOps
 Производительность MySQL для DevOps Производительность MySQL для DevOps
Производительность MySQL для DevOps
 
MySQL Performance for DevOps
MySQL Performance for DevOpsMySQL Performance for DevOps
MySQL Performance for DevOps
 
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB ClusterHow to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
How to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
 
How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?
 
Современному хайлоду - современные решения: MySQL 8.0 и улучшения Percona
Современному хайлоду - современные решения: MySQL 8.0 и улучшения PerconaСовременному хайлоду - современные решения: MySQL 8.0 и улучшения Percona
Современному хайлоду - современные решения: MySQL 8.0 и улучшения Percona
 
How to Avoid Pitfalls in Schema Upgrade with Galera
How to Avoid Pitfalls in Schema Upgrade with GaleraHow to Avoid Pitfalls in Schema Upgrade with Galera
How to Avoid Pitfalls in Schema Upgrade with Galera
 
How Safe is Asynchronous Master-Master Setup?
 How Safe is Asynchronous Master-Master Setup? How Safe is Asynchronous Master-Master Setup?
How Safe is Asynchronous Master-Master Setup?
 
Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?
 
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...
A Billion Goods in a Few Categories: When Optimizer Histograms Help and When ...
 
Что нужно знать о трёх топовых фичах MySQL
Что нужно знать  о трёх топовых фичах  MySQLЧто нужно знать  о трёх топовых фичах  MySQL
Что нужно знать о трёх топовых фичах MySQL
 
Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?Billion Goods in Few Categories: How Histograms Save a Life?
Billion Goods in Few Categories: How Histograms Save a Life?
 
MySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete TutorialMySQL Performance Schema in Action: the Complete Tutorial
MySQL Performance Schema in Action: the Complete Tutorial
 
Why MySQL Replication Fails, and How to Get it Back
Why MySQL Replication Fails, and How to Get it BackWhy MySQL Replication Fails, and How to Get it Back
Why MySQL Replication Fails, and How to Get it Back
 
Эффективная отладка репликации MySQL
Эффективная отладка репликации MySQLЭффективная отладка репликации MySQL
Эффективная отладка репликации MySQL
 

Recently uploaded

Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 

Recently uploaded (20)

Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 

Introduction into MySQL Query Tuning for Dev[Op]s

  • 1. MySQL Query Tuning for Dev[Op]s. Introduction
  • 2. •Basics •Indexes How Do They Work? When MySQL Uses Indexes Optimizer Histograms •Diagnostics EXPLAIN: How Optimizer Works Real Numbers: Inside Storage Engine Real Numbers: Inside the Server •How to Affect Query Plans Table of Contents 2 ©2021 Percona
  • 3. • Sveta Smirnova • MySQL Support engineer • Author of • MySQL Troubleshooting • JSON UDF functions • FILTER clause for MySQL • Speaker • Percona Live, OOW, Fosdem, DevConf, HighLoad... 3 ©2021 Percona
  • 9. cursor = conn.cursor() q = ’’’UPDATE ‘foo‘ SET my_date=NOW(), subject = %s, msg = %s, address = %s, updated_at = NOW() WHERE id=%s ’’’ cursor.execute(q, [ remote_resp.get(’subject’), remote_resp.get(’msg’), remote_resp.get(’address’), my_id ]) The Query 7 ©2021 Percona
  • 10. cursor = conn.cursor() q = ’’’UPDATE ‘foo‘ SET my_date=NOW(), subject = %s, msg = %s, address = %s, updated_at = NOW() WHERE id=%s ’’’ cursor.execute(q, [ remote_resp.get(’subject’), remote_resp.get(’msg’), remote_resp.get(’address’), my_id ]) The Query 7 ©2021 Percona
  • 11. SELECT dept_name, title, gender, min(salary) AS mins, max(salary) AS maxs FROM employees JOIN salaries USING(emp_no) JOIN titles USING(emp_no) JOIN dept_emp USING(emp_no) JOIN departments USING(dept_no) JOIN dept_manager USING(dept_no) WHERE dept_manager.to_date = ’9999-01-01’ GROUP BY dept_name, title, gender ORDER BY gender, maxs DESC; The Query 7 ©2021 Percona
  • 12. SELECT dept_name, title, gender, min(salary) AS mins, max(salary) AS maxs FROM employees JOIN salaries USING(emp_no) JOIN titles USING(emp_no) JOIN dept_emp USING(emp_no) JOIN departments USING(dept_no) JOIN dept_manager USING(dept_no) WHERE dept_manager.to_date = ’9999-01-01’ GROUP BY dept_name, title, gender ORDER BY gender, maxs DESC; The Query 7 ©2021 Percona
  • 13. • PMM QAN Always Tune Raw Query 8 ©2021 Percona
  • 14. • PMM QAN • Slow Query Log Always Tune Raw Query 8 ©2021 Percona
  • 15. • PMM QAN • Slow Query Log • Application log Always Tune Raw Query 8 ©2021 Percona
  • 16. • PMM QAN • Slow Query Log • Application log • ... Always Tune Raw Query 8 ©2021 Percona
  • 17. • Mind your data! • 75,000,000 rows • (INT, INT) 75,000,000 * (4 + 4) = 600,000,000 B = 572 MB • (INT, INT, DATETIME, VARCHAR(255), VARCHAR(255)) 75,000,000 * (4 + 4 + 8 + 256 + 256) = 39,600,000,000 bytes = 37 G • 39,600,000,000 / 600,000,000 = 66 Slow is Relative 9 ©2021 Percona
  • 18. • Mind your data! • Mind use case • Popular website • Admin interface • Weekly cron job Slow is Relative 9 ©2021 Percona
  • 19. • Mind your data! • Mind use case • Mind location • Server, used by multiple connections • Dedicated for OLAP queries Slow is Relative 9 ©2021 Percona
  • 20. • MySQL performs a job to execute it Why is the Query Slow 10 ©2021 Percona
  • 21. • MySQL performs a job to execute it • In the worst case scenario it will do a full table scan • CREATE INDEX • ANALYZE TABLE ... UPDATE HISTOGRAM ON ... Why is the Query Slow 10 ©2021 Percona
  • 22. • MySQL performs a job to execute it • In the worst case scenario it will do a full table scan • CREATE INDEX • ANALYZE TABLE ... UPDATE HISTOGRAM ON ... • Incorrect index can be used Why is the Query Slow 10 ©2021 Percona
  • 23. Query sent Connection Pool: Authentication, Caches; SQL interface; Parser Optimizer Storage engines Hardware Query Execution Workflow 11 ©2021 Percona
  • 24. • Selecting a lot of data • SELECT * FROM many columns table You May not Be Able to Change a Slow Query 12 ©2021 Percona
  • 25. • Selecting a lot of data • SELECT * FROM many columns table • Badly written • LEFT JOIN instead of INNER JOIN • Many values in IN() • Retrieving large result set, then discarding • Not effective SQL For particular MySQL version You May not Be Able to Change a Slow Query 12 ©2021 Percona
  • 26. • Selecting a lot of data • SELECT * FROM many columns table • Badly written • LEFT JOIN instead of INNER JOIN • Many values in IN() • Retrieving large result set, then discarding • Not effective SQL For particular MySQL version • We can still improve performance You May not Be Able to Change a Slow Query 12 ©2021 Percona
  • 29. SELECT name FROM users WHERE id=12 1 2 5 6 7 9 12 16 18 21 Full Table Scan 15 ©2021 Percona
  • 30. SELECT name FROM users WHERE id=12 1 2 5 6 7 9 12 16 18 21 Index Access 16 ©2021 Percona
  • 32. • Single column CREATE INDEX index name ON the table(the column) • Multiple columns CREATE INDEX index name ON the table(column1, column2) How to Create an Index 18 ©2021 Percona
  • 33. • Single column ALTER TABLE table name ADD INDEX [index name] (the column) • Multiple columns ALTER TABLE table name ADD INDEX [index name] (column1, column2) How to Create an Index 18 ©2021 Percona
  • 35. • WHERE the column = a value • WHERE the column IN(value1, value2, value3) • WHERE the column LIKE ’value%’ • WHERE the column LIKE ’%value’ Conditions 20 ©2021 Percona
  • 36. • WHERE left part = value1 AND right part = value2 • WHERE left part = value1 OR right part = value2 • WHERE right part = value1 AND left part = value2 • WHERE right part = value1 OR left part = value2 Conditions 20 ©2021 Percona
  • 37. • table1 JOIN table2 ON table1.column1 = table2.column2 Joins 21 ©2021 Percona
  • 38. • table1 JOIN table2 ON table1.column1 = table2.column2 • Same as FROM table1, table2 WHERE table1.column1 = table2.column2 Joins 21 ©2021 Percona
  • 39. • GROUP BY the column • GROUP BY left part, right part • GROUP BY right part, left part • GROUP BY the index, another index GROUP BY 22 ©2021 Percona
  • 40. • ORDER BY the column • ORDER BY left part, right part • ORDER BY right part, left part • ORDER BY the index, another index ORDER BY 23 ©2021 Percona
  • 41. 5.7 ORDER BY left part DESC, right part ASC 8.0 ORDER BY left part DESC, right part ASC • left part must be descending • right part must be ascending • the index(left part DESC, right part ASC) ORDER BY 23 ©2021 Percona
  • 42. • Deterministic, built-in • Return same value for the same argument • WHERE the column = FLOOR(123.45) Expressions 24 ©2021 Percona
  • 43. • Deterministic, built-in • Return same value for the same argument • WHERE the column = FLOOR(123.45) • Non-deterministic • Return different values for different invocations • WHERE the column = RAND() ∗ 100 Expressions 24 ©2021 Percona
  • 44. • Deterministic, built-in • Return same value for the same argument • WHERE the column = FLOOR(123.45) • Non-deterministic • Return different values for different invocations • WHERE the column = RAND() ∗ 100 • Stored functions and UDFs • Indexes are not used Use generated column indexes Expressions 24 ©2021 Percona
  • 45. • Identify queries not using indexes • Status variables mysql select * from performance_schema.session_status - where variable_name in (’Created_tmp_tables’, - ’Created_tmp_disk_tables’, ’Select_full_join’, - ’Select_full_range_join’, ’Select_range’, - ’Select_range_check’, ’Select_scan’, - ’Sort_merge_passes’, ’Sort_range’, ’Sort_rows’, - ’Sort_scan’) and variable_value 0; +------------------------+----------------+ | VARIABLE_NAME | VARIABLE_VALUE | +------------------------+----------------+ | Select_scan | 2 | +------------------------+----------------+ 1 row in set (0,00 sec) When to Add Indexes? 25 ©2021 Percona
  • 46. • Identify queries not using indexes mysql show global status like ’Handler_read%’; +-----------------------+-------+ | Variable_name | Value | +-----------------------+-------+ | Handler_read_first | 31 | | Handler_read_key | 1909 | | Handler_read_last | 0 | | Handler_read_next | 4393 | | Handler_read_prev | 0 | | Handler_read_rnd | 0 | | Handler_read_rnd_next | 1135 | +-----------------------+-------+ When to Add Indexes? 25 ©2021 Percona
  • 47. • Identify queries not using indexes • PERFORMANCE SCHEMA.EVENTS STATEMENTS * mysql select * from events_statements_history_long - where sql_text like ’select count(*) from employees join %’G *************************** 1. row **************************** ... ROWS_SENT: 1 SELECT_RANGE_CHECK: 0 ROWS_EXAMINED: 541058 SELECT_SCAN: 1 CREATED_TMP_DISK_TABLES: 0 SORT_MERGE_PASSES: 0 CREATED_TMP_TABLES: 0 SORT_RANGE: 0 SELECT_FULL_JOIN: 0 SORT_ROWS: 0 SELECT_FULL_RANGE_JOIN: 0 SORT_SCAN: 0 SELECT_RANGE: 0 NO_INDEX_USED: 0 When to Add Indexes? 25 ©2021 Percona
  • 48. • Identify queries not using indexes • Slow query log # Time: 2020-10-11T23:34:03.701871Z # User@Host: root[root] @ localhost [127.0.0.1] Id: 506 # Query_time: 0.024106 Lock_time: 0.000091 Rows_sent: 1 Rows_examined: 1000 SET timestamp=1602459243; SELECT c FROM sbtest1 WHERE id=996; When to Add Indexes? 25 ©2021 Percona
  • 49. • Identify queries not using indexes • QAN in PMM When to Add Indexes? 25 ©2021 Percona
  • 50. • Identify queries not using indexes • Analyze if adding an index will help When to Add Indexes? 25 ©2021 Percona
  • 51. • Identify queries not using indexes • Analyze if adding an index will help • Add the index When to Add Indexes? 25 ©2021 Percona
  • 52. • Identify queries not using indexes • Analyze if adding an index will help • Add the index • Test first! • ADD INDEX is an expensive operation Use pt-online-schema-change When to Add Indexes? 25 ©2021 Percona
  • 54. • Since version 8.0 Histogram Statistics 27 ©2021 Percona
  • 55. • Since version 8.0 • Collected and used by the Optimizer Histogram Statistics 27 ©2021 Percona
  • 56. • Since version 8.0 • Collected and used by the Optimizer • Visible in Information Schema mysql select HISTOGRAM from information_schema.column_statistics - where table_name=’example’G *************************** 1. row *************************** HISTOGRAM: {buckets: [[1, 0.6], [2, 0.8], [3, 1.0]], data-type: int, null-values: 0.0, collation-id: 8, last-updated: 2018-11-07 09:07:19.791470, sampling-rate: 1.0, histogram-type: singleton, number-of-buckets-specified: 3} 1 row in set (0.00 sec) Histogram Statistics 27 ©2021 Percona
  • 57. • Since version 8.0 • Collected and used by the Optimizer • Visible in Information Schema • Affects query execution plan, but not access • SELECT ... FROM a JOIN b vs SELECT ... FROM b JOIN a Histogram Statistics 27 ©2021 Percona
  • 58. • Since version 8.0 • Collected and used by the Optimizer • Visible in Information Schema • Affects query execution plan, but not access More details Histogram Statistics 27 ©2021 Percona
  • 59. 1 2 3 4 5 6 7 8 9 10 0 200 400 600 800 Indexes: Number of Items with Same Value 28 ©2021 Percona
  • 60. 1 2 3 4 5 6 7 8 9 10 0 200 400 600 800 Indexes: Cardinality 29 ©2021 Percona
  • 61. 1 2 3 4 5 6 7 8 9 10 0 200 400 600 800 Histograms: Number of Values in Each Bucket 30 ©2021 Percona
  • 62. 1 2 3 4 5 6 7 8 9 10 0 0.2 0.4 0.6 0.8 1 Histograms: Data in the Histogram 31 ©2021 Percona
  • 63. • Data distribution is far from uniform • Query accesses two or more tables • Indexes cannot be used • Maintaining them is too expensive • They are already used for different conditions Create Histograms 32 ©2021 Percona
  • 66. • EXPLAIN • Estimates the query execution plan 5.6- EXTENDED 5.6- PARTITIONS 5.6+ FORMAT=JSON 8.0+ FORMAT=TREE How to Find how MySQL Uses Indexes 35 ©2021 Percona
  • 67. • EXPLAIN • Estimates the query execution plan 5.6- EXTENDED 5.6- PARTITIONS 5.6+ FORMAT=JSON 8.0+ FORMAT=TREE • INFORMATION SCHEMA.OPTIMIZER TRACE • Real data, collected when query was executing • Advanced topic How to Find how MySQL Uses Indexes 35 ©2021 Percona
  • 68. mysql EXPLAIN SELECT first_name, last_name, title, salary FROM employees - JOIN salaries USING(emp_no) JOIN titles USING(emp_no) - WHERE salary = (SELECT MAX(salary) FROM salaries - JOIN titles WHERE titles.to_date CURDATE()); +----+-------------+-----------+------------+------+---------------+---------+---------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | +----+-------------+-----------+------------+------+---------------+---------+---------+ | 1 | PRIMARY | employees | NULL | ALL | PRIMARY | NULL | NULL ... | 1 | PRIMARY | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ... | 1 | PRIMARY | titles | NULL | ref | PRIMARY | PRIMARY | 4 ... | 2 | SUBQUERY | titles | NULL | ALL | NULL | NULL | NULL ... | 2 | SUBQUERY | salaries | NULL | ALL | NULL | NULL | NULL ... +----+-------------+-----------+------------+------+---------------+---------+---------+ +----+-...-+----------------------------+---------+----------+-------------------------------+ | id | ... | ref | rows | filtered | Extra | +----+-...-+----------------------------+---------+----------+-------------------------------+ | 1 | ... | NULL | 299113 | 100.00 | NULL | | 1 | ... | employees.employees.emp_no | 9 | 10.00 | Using where | | 1 | ... | employees.employees.emp_no | 1 | 100.00 | Using index | | 2 | ... | NULL | 442189 | 33.33 | Using where | | 2 | ... | NULL | 2838426 | 100.00 | Using join buffer (hash join) | +----+-...-+----------------------------+---------+----------+-------------------------------+ 5 rows in set, 1 warning (0,01 sec) 299113 * 9 * 1 * 442189 * 2838426 = 3,378,806,408,204,514,738 EXPLAIN: rows 36 ©2021 Percona
  • 69. mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees - JOIN salaries USING(emp_no) JOIN titles USING(emp_no) - GROUP BY title ORDER BY maxs DESC; +----+-------------+-----------+------------+-------+---------------+---------+---------+ | id | select_type | table | partitions | type | possible_keys | key | key_len +----+-------------+-----------+------------+-------+---------------+---------+---------+ | 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ... | 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ... | 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ... +----+-------------+-----------+------------+-------+---------------+---------+---------+ EXPLAIN: type 37 ©2021 Percona
  • 70. ALL Read all rows Query Execution Type 38 ©2021 Percona
  • 71. ALL Read all rows index Read all rows from the index Query Execution Type 38 ©2021 Percona
  • 72. ALL Read all rows index Read all rows from the index range Range: N ... M Query Execution Type 38 ©2021 Percona
  • 73. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries index subquery unique subquery Query Execution Type 38 ©2021 Percona
  • 74. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries merge Combination of two indexes index merge Query Execution Type 38 ©2021 Percona
  • 75. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries merge Combination of two indexes ref Comparision with single value ref or null ref eq ref Query Execution Type 38 ©2021 Percona
  • 76. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries merge Combination of two indexes ref Comparision with single value fulltext Fulltext index Query Execution Type 38 ©2021 Percona
  • 77. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries merge Combination of two indexes ref Comparision with single value fulltext Fulltext index const Single row Query Execution Type 38 ©2021 Percona
  • 78. ALL Read all rows index Read all rows from the index range Range: N ... M subquery Subqueries merge Combination of two indexes ref Comparision with single value fulltext Fulltext index const Single row system Table has single row Query Execution Type 38 ©2021 Percona
  • 79. mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees - JOIN salaries USING(emp_no) JOIN titles USING(emp_no) - GROUP BY title ORDER BY maxs DESC; +----+-------------+-----------+------------+-------+---------------+---------+---------+ | id | select_type | table | partitions | type | possible_keys | key | key_len +----+-------------+-----------+------------+-------+---------------+---------+---------+ | 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ... | 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ... | 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ... +----+-------------+-----------+------------+-------+---------------+---------+---------+ EXPLAIN: key and possible keys 39 ©2021 Percona
  • 80. mysql EXPLAIN SELECT title, MAX(salary) AS maxs FROM employees - JOIN salaries USING(emp_no) - JOIN titles USING(emp_no) - GROUP BY title ORDER BY maxs DESC; +----+-------------+-----------+------------+-------+---------------+---------+---------+ | id | select_type | table | partitions | type | possible_keys | key | key_len +----+-------------+-----------+------------+-------+---------------+---------+---------+ | 1 | SIMPLE | employees | NULL | index | PRIMARY | PRIMARY | 4 ... | 1 | SIMPLE | titles | NULL | ref | PRIMARY,title | PRIMARY | 4 ... | 1 | SIMPLE | salaries | NULL | ref | PRIMARY | PRIMARY | 4 ... +----+-------------+-----------+------------+-------+---------------+---------+---------+ EXPLAIN: JOIN order 40 ©2021 Percona
  • 81. mysql EXPLAIN SELECT first_name, last_name, title, salary FROM employees - JOIN salaries USING(emp_no) JOIN titles USING(emp_no) - WHERE salary IN (SELECT MAX(salary) FROM salaries - JOIN titles WHERE titles.to_date CURDATE()); ... 5 rows in set, 1 warning (0,01 sec) mysql SHOW WARNINGSG *************************** 1. row *************************** Level: Note Code: 1003 Message: /* select#1 */ select ‘employees‘.‘employees‘.‘first_name‘ AS ‘first_name‘, ‘employees‘.‘employees‘.‘last_name‘ AS ‘last_name‘,‘employees‘.‘salaries‘.‘salary‘ AS ‘salary‘ from ‘employees‘.‘employees‘ join ‘employees‘.‘salaries‘ where ((‘employees‘.‘salaries‘.‘emp_no‘ = ‘employees‘.‘employees‘.‘emp_no‘) and in_optimizer(‘employees‘.‘salaries‘.‘salary‘,‘employees‘.‘salaries‘.‘salary‘ in ( materialize (/* select#2 */ select max(‘employees‘.‘salaries‘.‘salary‘) from ‘employees‘.‘salaries‘ join ‘employees‘.‘titles‘ where (‘employees‘.‘titles‘.‘to_date‘ cache(curdate())) having true ), primary_index_lookup(‘employees‘.‘salaries‘.‘salary‘ in temporary table on auto_distinct_key where ((‘employees‘.‘salaries‘.‘salary‘ = ‘materialized_subquery‘.‘MAX(salary)‘)))))) 1 row in set (0,00 sec) Actual Query 41 ©2021 Percona
  • 82. mysql explain select * from t1G *************************** 1. row *************************** ... rows: 12 Extra: NULL mysql explain select * from t1 where f2=12G *************************** 1. row *************************** ... key: NULL ... rows: 12 Extra: Using where Effect of Indexes: Before 42 ©2021 Percona
  • 83. mysql alter table t1 add index(f2); Query OK, 12 rows affected (0.07 sec) Records: 12 Duplicates: 0 Warnings: 0 mysql explain select * from t1 where f2=12G *************************** 1. row *************************** ... key: f2 key_len: 5 ref: const rows: 1 Extra: NULL 1 row in set (0.00 sec) Effect of Indexes: After 43 ©2021 Percona
  • 85. • EXPLAIN is optimistic mysql explain select * from ol - where thread_id=10432 and site_id != 9939 - order by id limit 3G *************************** 1. row *************************** id: 1 | ref: NULL select_type: SIMPLE | rows: 33 table: ol | filtered: 8.07 partitions: NULL | Extra: Using where type: index possible_keys: thread_id key: PRIMARY key_len: 4 1 row in set, 1 warning (0,00 sec) Handler * Status Variables 45 ©2021 Percona
  • 86. • ’Handler *’ show truth mysql FLUSH STATUS; mysql select * from ol - where thread_id=10432 and site_id != 9939 - order by id limit 3; mysql show status like ’Handler%’; +----------------------------+--------+ | Variable_name | Value | +----------------------------+--------+ ... | Handler_read_first | 1 | | Handler_read_key | 1 | | Handler_read_last | 0 | | Handler_read_next | 100000 | ... Handler * Status Variables 45 ©2021 Percona
  • 88. • SHOW [FULL] PROCESSLIST • INFORMATION SCHEMA.PROCESSLIST • performance schema.THREADS PROCESSLIST 47 ©2021 Percona
  • 89. • SHOW [FULL] PROCESSLIST • INFORMATION SCHEMA.PROCESSLIST • performance schema.THREADS • Your first alert of performance issue PROCESSLIST 47 ©2021 Percona
  • 90. • SHOW [FULL] PROCESSLIST • INFORMATION SCHEMA.PROCESSLIST • performance schema.THREADS • Your first alert of performance issue • Shows all running queries PROCESSLIST 47 ©2021 Percona
  • 91. • Can be seen in the PROCESSLIST mysql show processlistG ************************ 1. row ************************ Id: 7 User: root Host: localhost:48799 db: employees Command: Query Time: 2 State: Sending data Info: select count(*) from employees join titles using(emp_no) where title=’Senior Engineer’ ... Execution Stages 48 ©2021 Percona
  • 92. • Can be seen in the PROCESSLIST • Very useful when you need to answer on the question: ”What is my server doing now?” Execution Stages 48 ©2021 Percona
  • 93. • PERFORMANCE SCHEMA.EVENTS STAGES * mysql select eshl.event_name, substr(sql_text, 1, 15) ‘sql‘, - eshl.timer_wait/1000000000000 w_s - from events_stages_history_long - eshl join events_statements_history_long esthl on - (eshl.nesting_event_id = esthl.event_id) where - esthl.current_schema=’employees’ and sql_text like - ’select count(*) from employees%’ - order by eshl.timer_start asc; +--------------------------------+-----------------+--------+ | event_name | sql | w_s | +--------------------------------+-----------------+--------+ | stage/sql/starting | select count(*) | 0.0002 | | stage/sql/checking permissions | select count(*) | 0.0000 | ... Execution Stages 48 ©2021 Percona
  • 94. • PERFORMANCE SCHEMA.EVENTS STAGES * ... | stage/sql/checking permissions | select count(*) | 0.0000 | | stage/sql/Opening tables | select count(*) | 0.0000 | | stage/sql/init | select count(*) | 0.0001 | | stage/sql/System lock | select count(*) | 0.0000 | | stage/sql/optimizing | select count(*) | 0.0000 | | stage/sql/statistics | select count(*) | 0.0001 | | stage/sql/preparing | select count(*) | 0.0000 | | stage/sql/executing | select count(*) | 0.0000 | | stage/sql/Sending data | select count(*) | 5.4915 | | stage/sql/end | select count(*) | 0.0000 | ... Execution Stages 48 ©2021 Percona
  • 95. • Status variables mysql flush status; Query OK, 0 rows affected (0,01 sec) mysql select count(*) from employees join titles using(emp_no) - where title=’Senior Engineer’; +----------+ | count(*) | +----------+ | 97750 | +----------+ 1 row in set (5,44 sec) Temporary Tables and Other Job 49 ©2021 Percona
  • 96. • Status variables mysql select * from performance_schema.session_status - where variable_name in (’Created_tmp_tables’, - ’Created_tmp_disk_tables’, ’Select_full_join’, - ’Select_full_range_join’, ’Select_range’, - ’Select_range_check’, ’Select_scan’, ’Sort_merge_passes’, - ’Sort_range’, ’Sort_rows’, ’Sort_scan’) - and variable_value 0; +------------------------+----------------+ | VARIABLE_NAME | VARIABLE_VALUE | +------------------------+----------------+ | Select_scan | 2 | +------------------------+----------------+ 1 row in set (0,00 sec) Temporary Tables and Other Job 49 ©2021 Percona
  • 97. • PERFORMANCE SCHEMA.EVENTS STATEMENTS * mysql select * from events_statements_history_long - where sql_text like - ’select count(*) from employees join %’G *************************** 1. row **************************** ... ROWS_SENT: 1 SELECT_RANGE_CHECK: 0 ROWS_EXAMINED: 541058 SELECT_SCAN: 1 CREATED_TMP_DISK_TABLES: 0 SORT_MERGE_PASSES: 0 CREATED_TMP_TABLES: 0 SORT_RANGE: 0 SELECT_FULL_JOIN: 0 SORT_ROWS: 0 SELECT_FULL_RANGE_JOIN: 0 SORT_SCAN: 0 SELECT_RANGE: 0 NO_INDEX_USED: 0 Temporary Tables and Other Job 49 ©2021 Percona
  • 98. • sys.statement analysis mysql select * from statement_analysis where query like - ’SELECT COUNT ( * ) FROM ‘emplo%’ and db=’employees’G *************************** 1. row *************************** query: SELECT COUNT ( * ) FROM ‘emplo ... ‘emp_no‘ ) WHE... db: employees max_latency: 5.59 s full_scan: avg_latency: 5.41 s exec_count: 7 lock_latency: 2.24 ms err_count: 0 rows_sent: 7 warn_count: 0 rows_sent_avg: 1 total_latency: 37.89 s rows_examined: 3787406 Temporary Tables and Other Job 49 ©2021 Percona
  • 99. • sys.statement analysis rows_examined_avg: 541058 rows_affected: 0 rows_affected_avg: 0 tmp_tables: 0 tmp_disk_tables: 0 rows_sorted: 0 sort_merge_passes: 0 digest: 4086bc3dc6510a1d9c8f2fe1f59f0943 first_seen: 2016-04-14 15:19:19 last_seen: 2016-04-14 16:13:14 Temporary Tables and Other Job 49 ©2021 Percona
  • 100. How to Affect Query Plans
  • 101. • Index statistics • Histogram statistics • Optimizer switches • Bugs in optimizer What has an Effect on Query Optimizer Plans? 51 ©2021 Percona
  • 102. • Collected by storage engine Index Statistics 52 ©2021 Percona
  • 103. • Collected by storage engine • Used by Optimizer Index Statistics 52 ©2021 Percona
  • 104. • Can be examined by SHOW INDEX mysql show index from sbtest1; +---------+----------+-------------+-------------+ | Table | Key_name | Column_name | Cardinality | +---------+----------+-------------+-------------+ | sbtest1 | k_1 | k | 49142 | +---------+----------+-------------+-------------+ mysql select count(distinct id), - count(distinct k) from sbtest1; +--------------------+-------------------+ | count(distinct id) | count(distinct k) | +--------------------+-------------------+ | 100000 | 17598 | +--------------------+-------------------+ Index Statistics 52 ©2021 Percona
  • 105. • Can be updated • ANALYZE TABLE • If does not help: rebuild the table OPTIMIZE TABLE ALTER TABLE ENGINE=INNODB; ANALYZE TABLE Index Statistics 52 ©2021 Percona
  • 106. • Since version 8.0 Histogram Statistics 53 ©2021 Percona
  • 107. • Since version 8.0 • Collected and used by the Optimizer Histogram Statistics 53 ©2021 Percona
  • 108. • Since version 8.0 • Collected and used by the Optimizer • Visible in Information Schema mysql select HISTOGRAM from information_schema.column_statistics - where table_name=’example’G *************************** 1. row *************************** HISTOGRAM: {buckets: [[1, 0.6], [2, 0.8], [3, 1.0]], data-type: int, null-values: 0.0, collation-id: 8, last-updated: 2018-11-07 09:07:19.791470, sampling-rate: 1.0, histogram-type: singleton, number-of-buckets-specified: 3} 1 row in set (0.00 sec) Histogram Statistics 53 ©2021 Percona
  • 109. • Since version 8.0 • Collected and used by the Optimizer • Visible in Information Schema More details Histogram Statistics 53 ©2021 Percona
  • 110. mysql select @@optimizer_switchG *************************** 1. row *************************** @@optimizer_switch: index_merge=on,index_merge_union=on, index_merge_sort_union=on,index_merge_intersection=on, engine_condition_pushdown=on,index_condition_pushdown=on, mrr=on,mrr_cost_based=on, block_nested_loop=on,batched_key_access=off, materialization=on,semijoin=on,loosescan=on,firstmatch=on, duplicateweedout=on,subquery_materialization_cost_based=on, use_index_extensions=on,condition_fanout_filter=on, derived_merge=on 1 row in set (0,00 sec) Optimizer Switches 54 ©2021 Percona
  • 111. • Turn ON and OFF particular optimization Optimizer Switches 54 ©2021 Percona
  • 112. • Turn ON and OFF particular optimization • Can be unhelpful • Especially for queries, tuned for previous versions Optimizer Switches 54 ©2021 Percona
  • 113. • Turn ON and OFF particular optimization • Can be unhelpful • Work with them as with any other option • Turn OFF and try SET optimizer_switch = ’use_index_extensions=off’; SELECT ... EXPLAIN SELECT ... Optimizer Switches 54 ©2021 Percona
  • 114. • Turn ON and OFF particular optimization • Can be unhelpful • Work with them as with any other option • If helps implement in queries SELECT /*+ SEMIJOIN(FIRSTMATCH, LOOSESCAN) */ * FROM t1 ...; SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...; Optimizer Switches 54 ©2021 Percona
  • 115. • Optimizer choses wrong plan for no reason Bugs in Optimizer 55 ©2021 Percona
  • 116. • Optimizer choses wrong plan for no reason • Statistics are up to date • Histograms are not usable Bugs in Optimizer 55 ©2021 Percona
  • 117. • Optimizer choses wrong plan for no reason • Statistics are up to date • Histograms are not usable • Solution • Use index hints FORCE INDEX IGNORE INDEX Bugs in Optimizer 55 ©2021 Percona
  • 118. • Optimizer choses wrong plan for no reason • Statistics are up to date • Histograms are not usable • Solution • On every upgrade • Remove index hints • Test if the query improved • You must do it even for minor version upgrades! Bugs in Optimizer 55 ©2021 Percona
  • 119. • EXPLAIN is essential for query tuning • Real job is done by storage engine • Index statistics affect query execution plan • All index hints, optimizer hints and other workarounds must be validated on each upgrade Summary 56 ©2021 Percona
  • 120. EXPLAIN Syntax EXPLAIN FORMAT=JSON is Cool! Troubleshooting add-ons Optimizer Statistics aka Histograms Optimizer Hints Tracing the Optimizer More Information 57 ©2021 Percona