Talk is going to be about mainly query optimization and identifying slow queries in MySQL. We will talk about analyzing queries causing bottlenecks and how to find a solution with proper indexing. We will also talk about how to improve indexing and MySQL behavior on them. I will cover open source tooling and new developments in this subject.

1. © 2017 Percona1
MySQL Query Optimization Best
Practices
and Indexing
Alkin Tezuysal – Sr. Technical Manager
Percona

2. © 2017 Percona2
Who am I? @ask_dba

3. © 2017 Percona3
About Percona
Solutions for your success with MySQL , MongoDB and PostgreSQL
Support, Managed Services, Software
Our Software is 100% Open Source
Support Broad Ecosystem – MySQL, MariaDB, Amazon RDS
In Business for 12 years
More than 3000 customers, including top Internet companies and enterprises

4. © 2017 Percona4
About This Presentation
Indexing Basics
Finding and
Identifying
Slow Queries
Utilizing Explain
Plan
Advanced
Indexing
Tooling and
more

5. © 2017 Percona5
Indexing Basics
• What it does?
• Increase speed of given lookup (SQL)
• Access and maintain changes
• Helps Optimizer to reach its goal

6. © 2017 Percona6
Why do we need indexes?
• Data persists on disks
• Disks cheap but slow
• Data can be in memory
• Memory fast but expensive
Index is the answer to access data fast.
CREATE INDEX part_of_name ON customer (name(10));

7. © 2017 Percona7
Traversal
1. Tree Traversal
2. Follow leaf node chain
3. Fetch the table data

8. © 2017 Percona8
Leaf Nodes
1. Establish doubly linked list
2. Connect index leaf nodes
3. Indexed columns

9. © 2017 Percona9
B-tree Structure

10. © 2017 Percona10
Slow Index Lookups
• Low cardinality
• Large data sets
• Multiple index traversal
• Index column used as argument
• Looking for suffix
• Non-leading column lookup
• Data type mismatch
• Character Set / Collation mismatch
• MySQL Bug

11. © 2017 Percona11
The Optimizer

12. © 2017 Percona12
MySQL Optimizer
• Cost based
• Assign costs to select operations
• Assign costs to partial or alternate plans
• Seek for lowest cost
Access Method Join Order Subquery Strategy

13. © 2017 Percona13
Cost Model

14. © 2017 Percona14
Finding and Identifying Slow Queries
• Slow Query Log
• PMM/QAN
• Network sniff
• Others (Licensed)
• MySQL EM
• Vividcortex
• Solarwinds
• Monyog

15. © 2017 Percona15
Slow Query Tools
• Explain Plan
• Tabular
• JSON (5.7)
• Visual (Workbench)
• Running Query (5.7)
• pt-query-digest
• pt-visual-explain
• Performance Schema
• MySQL Sys Schema
• Optimizer Trace
• MySQL Workbench
• Status Variables
• show status like ‘Sort%’
• show status like ‘Handler%’

16. © 2017 Percona16
PMM/QAN

17. © 2017 Percona17
PMM/QAN

18. © 2017 Percona18
PMM/QAN

19. © 2017 Percona19
PMM Demo - https://pmmdemo.percona.com/

20. © 2017 Percona20
Explain Plan

21. © 2017 Percona21
Explain Plan (JSON)
> EXPLAIN format=JSON SELECT CONCAT(customer.last_name, ', ', customer.first_name) AS customer, address.phone, film.title FROM rental INNER JOIN customer ON rental.customer_id =
customer.customer_id INNER JOIN address ON customer.address_id = address.address_id INNER JOIN inventory ON rental.inventory_id = inventory.inventory_id INNER JOIN film ON inventory.film_id =
film.film_id WHERE rental.return_date IS NULL AND rental_date + INTERVAL film.rental_duration DAY < CURRENT_DATE() LIMIT 5G
*************************** 1. row ***************************
EXPLAIN: {
"query_block": {
"select_id": 1,
"nested_loop": [
{
"table": {
"table_name": "film",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"rows": 1000,
"filtered": 100
}
},
…
…

22. © 2017 Percona22
Explain Plan (pt-visual-explain)
JOIN
+- Bookmark lookup
| +- Table
| | table address
| | possible_keys PRIMARY
| +- Unique index lookup
| key address->PRIMARY
| possible_keys PRIMARY
| key_len 2
| ref sakila.customer.address_id
| rows 1
+- JOIN
+- Bookmark lookup
| +- Table
| | table customer
| | possible_keys PRIMARY,idx_fk_address_id
| +- Unique index lookup
| key customer->PRIMARY
| possible_keys PRIMARY,idx_fk_address_id
| key_len 2
| ref sakila.rental.customer_id
| rows 1
...

23. © 2017 Percona23
Cost Based Access Method
1. Find the optimal method
2. Check if access method useful
3. Estimate the cost of using access method
4. Select low cost access method

24. © 2017 Percona24
Query Execution
Table Scan
Index Scan
Index
Lookup
Range
Scan
Index
Merge
Loose
Index Scan

25. © 2017 Percona25
Indexing Best Practices
• Always have Primary Key
• Physical order of table, if not created explicitly, MySQL will create
hidden one (Global Mutex)
• Fastest lookup is PK

26. © 2017 Percona26
Indexing Best Practices
• Single index with multiple columns
• Left most first and each additional field in a composite key
• Composite indexes better a.k.a Covering indexes
• PK is already part of composite indexes

27. © 2017 Percona27
Indexing Best Practices
• Equality first, range next
• Ex:
select first_name, last_name, birth_date from
employees
where date_of_birth => to_date (?, `YYYY-MM-DD`)
and date_of_birth <= to_date (?, `YYYY-MM-DD`)
and branch_id = ?

28. © 2017 Percona28
Indexing Best Practices
• One index scan is faster than two
• Avoid duplicate indexes pt-duplicate-key-checker

29. © 2017 Percona29
Indexing Best Practices
• Data types matter. Numeric for numbers.
• Ex:
select …
from …
where numeric_value = `48`

30. © 2017 Percona30
Query Optimization Best Practices
• Negative clauses and subqueries aren’t as good as positive
clauses
• Ex:
• IS NOT
• IS NOT NULL
• NOT IN
• NOT LIKE

31. © 2017 Percona31
Query Optimization Best Practices
• User INNER instead of LEFT where you can

32. © 2017 Percona32
Query Optimization Best Practices
• UNION ALL is better than UNION
UNION
UNION ALL

33. © 2017 Percona33
Query Optimization Best Practices
• ORDER BY can be expensive
SELECT * FROM t1
ORDER BY idx_c1, idx_c2;
• Avoid while sorting small set of data (Use code)
cust
_id
first_na
me
last_
name
email
1 Billy Joel bb7@bluen
ot.com
2 Jane Fond
a
jf1950@yah
oo.com
3 Mark Welt
on
markW1912
@gmail.co
m
4 Linda Joey linda.joey@
yandex.com
5 Sidney Travo
r
sidney.travo
r@icloud.co
m
6 Jordan Velez jordanv@a
mazon.com

34. © 2017 Percona34
Query Optimization Best Practices
• Watch out those ORDER BY + LIMIT operations
• These usually return small set of data with big cost (filesort)
SELECT col1, ... FROM t1 ... ORDER BY name LIMIT
10;
SELECT col1, ... FROM t1 ... ORDER BY RAND()
LIMIT 15;

35. © 2017 Percona35
Query Optimization Best Practices
• Watch out those ORDER BY + LIMIT operations
• These usually return small set of data with big cost (filesort)
SELECT col1, ... FROM t1 ... ORDER BY name LIMIT
10;
SELECT col1, ... FROM t1 ... ORDER BY RAND()
LIMIT 15;

36. © 2017 Percona36
MySQL Index Types
• B-tree (Common)
• Fractal Tree
• LSM Tree
• R-Tree (Spatial)
• Hash (Memory)
• Engine-dependent

37. © 2017 Percona37
Advanced Indexing
• Optimizer hints
• Global: The hint affects the entire statement
• Query block: The hint affects a particular query block within a statement
• Table-level: The hint affects a particular table within a query block
• Index-level: The hint affects a particular index within a table
• Index hints
• SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX FOR
ORDER BY (i2) ORDER BY a;

38. © 2017 Percona38
If indexes not enough
• Query Re-write
• ProxySQL
• https://www.percona.com/blog/2018/05/02/proxysql-query-rewrite-
use-case/
• MySQL 5.7: Query Rewrite Plugin
• Add hints
• Modify join order

39. © 2017 Percona39
Advanced Queries with ProxySQL – Query rewrite
engine
• Most wanted feature by DBAs
• Rewrite queries overloading the database on the fly.
Application A
ProxySQL
• Simply buy time until application can be modified
Application B
MySQL
Master
MySQL
Slave
MySQL
Slave
MySQL
Slave
Query
Rewriting
MySQL
Slave
MySQL
Slave

40. © 2017 Percona40
Final Thoughts
Optimizer is not smart as DBAs
• Help to choose best possible path
• Improve throughput
Add only indexes you need
• Avoid duplicate indexing
• Avoid overhead disk space, extra i/o ops
Stay on current version of MySQL
• Several bugs fixed
• Optimizer and Engine improvements in place

41. © 2017 Percona41
References and Credits
• Markus Winand (2018) - SQL Performance Explained (2018)
• Otstein Grovlen (2017- How to Analyze and Tune MySQL Queries for
Better Performance
• Sveta Smirnova (2018) – Introduction into MySQL Query Tuning
• Oracle Reference Manual
• Jeremy Cole (2013) - How does InnoDB behave without a Primary
Key?
• Tata McDaniel (2018) - Visualize This! MySQL Tools That Explain
Queries
• Reviewers: Daniel G Burgos, Tate McDaniel, Janos Ruszo

42. DATABASE PERFORMANCE
MATTERS
Database Performance MattersDatabase Performance MattersDatabase Performance MattersDatabase Performance MattersDatabase Performance Matters