MariaDB Optimizer Guide

Copyright(c)2014 by ora-sysdba. All Page content is property of dbjongjin.lee
Blog : http://ora-sysdba.tistory.com/
1
MariaDB : Optimizer
Jong Jin Lee
SYS4U I&C EC Solution Lab / Software Engineer

2
Agenda
• Query execution process
• Optimizer
• Statistics
• Histogram-based Statistics
• Join Optimizer
• Analysis of the execution plan
• Optimizer Hint
• Tuning Point

3
MariaDB : Query execution process
SQL Parsing Optimization and Plan Join and Sort
Using SQL Parser
• Split from the SQL
statement
• Grammatical error
checking
• Create a parse tree
Using SQL Parse tree
• Select the tables and
indexes
• Simplify operations
• Join the ordering
• Set whether to use a
temporary table
Reading Record
• Reading Record From
Storage Engine
• Join & Sort Operations

4
MariaDB : Optimizer
 RBO(Rule-based optimizer)
 It is only present to provide backwards compatibility during the migration
to the query optimizer (Cost Based Optimizer).
 Do not collect statistics.
 CBO(Cost-based optimizer)
 The optimizer can use a rules-based approach to work without statistical
information, but this approach is less intelligent than the cost-based
approach.
 The optimizer factors in statistical information about the contents of the
particular schema objects (tables, clusters, or indexes) being accessed.

5
MariaDB : Statistics
 MariaDB 5.5 Version
 The statistics provided for each storage engine does not have sufficient
information about the contents.
 Column statistics are not collected (Only Index) – Using MYISAM engine
 MySQL 5.6 Version
 MariaDB 10.0
MariaDB [(none)]> use mysql;
MariaDB [(mysql)]> show tables like ‘%_stats’;
innodb_index_stats
innodb_table_stats
MariaDB [(none)]> use mysql;
MariaDB [(mysql)]> show tables like ‘%_stats’;
column_stats
index_stats
table_stats

6
 Statistics collection
 DB : my.cnf (INNODB_STATS_AUTO_RECALE > ON)
 Table : STATS_AUTO_RECALE option
– STATS_AUTO_RECALC is available only in MariaDB 10.0+. It indicates whether to automatically
recalculate persistent statistics (see STATS_PERSISTENT, below) for an InnoDB table. If set to 1,
statistics will be recalculated when more than 10% of the data has changed. When set to 0, stats will be
recalculated only when an ANALYZE TABLE is run. If set to DEFAULT, or left out, the value set by the
innodb_stats_auto_recalc system variable applies.

7
 CLIENT_STATISTICS
– The CLIENT_STATISTICS table holds statistics about client connections.
MariaDB [(mysql)]> SELECT * FROM INFORMATION_SCHEMA.CLIENT_STATISTICSG
*************************** 1. row ***************************
CLIENT: localhost
TOTAL_CONNECTIONS: 3
CONCURRENT_CONNECTIONS: 0
CONNECTED_TIME: 4883
BUSY_TIME: 0.009722
CPU_TIME: 0.0102131
BYTES_RECEIVED: 841
BYTES_SENT: 13897
BINLOG_BYTES_WRITTEN: 0
ROWS_READ: 0
ROWS_SENT: 214
ROWS_DELETED: 0
ROWS_INSERTED: 207
ROWS_UPDATED: 0
SELECT_COMMANDS: 10
UPDATE_COMMANDS: 0
OTHER_COMMANDS: 13
COMMIT_TRANSACTIONS: 0
ROLLBACK_TRANSACTIONS: 0
DENIED_CONNECTIONS: 0
LOST_CONNECTIONS: 0
ACCESS_DENIED: 0
EMPTY_QUERIES: 1
1 row in set (0.00 sec)

8
 USER_STATISTICS
– The USER_STATISTICS table holds statistics about user activity. You can use this table to find out such
things as which user is causing the most load and which users are being abusive. You can also use this table
to measure how close to capacity the server may be.
SELECT * FROM INFORMATION_SCHEMA.USER_STATISTICSG
*************************** 1. row ***************************
USER: root
TOTAL_CONNECTIONS: 1
CONCURRENT_CONNECTIONS: 0
CONNECTED_TIME: 297
BUSY_TIME: 0.001725
CPU_TIME: 0.001982
BYTES_RECEIVED: 388
BYTES_SENT: 2327
BINLOG_BYTES_WRITTEN: 0
ROWS_READ: 0
ROWS_SENT: 12
ROWS_DELETED: 0
ROWS_INSERTED: 13
ROWS_UPDATED: 0
SELECT_COMMANDS: 4
UPDATE_COMMANDS: 0
OTHER_COMMANDS: 3
COMMIT_TRANSACTIONS: 0
ROLLBACK_TRANSACTIONS: 0
DENIED_CONNECTIONS: 0
LOST_CONNECTIONS: 0
ACCESS_DENIED: 0
EMPTY_QUERIES: 1

9
 INDEX_STATISTICS
– The INDEX_STATISTICS table shows statistics on index usage and makes it possible to do such things as
locating unused indexes and generating the commands to remove them.
SELECT * FROM INFORMATION_SCHEMA.INDEX_STATISTICS WHERE TABLE_NAME = "author";
+--------------+------------+------------+-----------+
| TABLE_SCHEMA | TABLE_NAME | INDEX_NAME | ROWS_READ |
+--------------+------------+------------+-----------+
| books | author | by_name | 15 |
+--------------+------------+------------+-----------+
Field Type Notes
TABLE_SCHEMA varchar(192) The schema (database) name.
TABLE_NAME varchar(192) The table name.
INDEX_NAME varchar(192) The index name (as visible in SHOW CREATE TABLE).
ROWS_READ int(21) The number of rows read from this index.

10
 TABLE_STATISTICS
– The TABLE_STATISTICS table is similar to the INDEX_STATISTICS table. It shows statistics on table usage.
SELECT * FROM INFORMATION_SCHEMA.TABLE_STATISTICS WHERE TABLE_NAME='user';
+--------------+------------+-----------+--------------+------------------------+
| TABLE_SCHEMA | TABLE_NAME | ROWS_READ | ROWS_CHANGED | ROWS_CHANGED_X_INDEXES |
+--------------+------------+-----------+--------------+------------------------+
| mysql | user | 5 | 2 | 2 |
+--------------+------------+-----------+--------------+------------------------+w
Field Type Notes
TABLE_SCHEMA varchar(192) The schema (database) name.
TABLE_NAME varchar(192) The table name.
ROWS_READ int(21) The number of rows read from the table.
ROWS_CHANGED int(21) The number of rows changed in the table.
ROWS_CHANGED_X_INDEXES int(21)
The number of rows changed in the table,
multiplied by the number of indexes changed.

11
 MariaDB 10.0 Version
– Histogram-based statistics were introduced in MariaDB 10.0.2 as a
mechanism to improve the query plan chosen by the optimizer in certain
situations. Until then, all conditions on non-indexed columns were
ignored when searching for the best execution plan. Histograms can be
collected for both indexed and non-indexed columns, and are made
available to the optimizer.
– MySQL : column_stats table
MariaDB : Histogram-based Statistics

12
 Height-Balanced Histogram algorithm
MariaDB : Histogram-based Statistics
A
B
C
B
C
C
B
A
A
B
B
B
C
C
A
A
1 B
2 C
3 A
4 B
Column Order Histogram
Column Bucket Size = System Variables(histogram_size)

13
 Exhaustive Search
– The combination of all the tables in the execution plan cost calculation
– Ex) Table : 10 , Join : 20! (Factorial, 3628800)
MariaDB : Join Optimizer

14
 Heuristic Search(Greedy Search)
– Join (optimizer_prune_level, optimizer_search_depth)
MariaDB : Join Optimizer

15
 HOW TO USE
– The EXPLAIN statement can be used either as a synonym for
DESCRIBE or as a way to obtain information about how MariaDB
executes a SELECT (as well as UPDATE and DELETE since MariaDB
10.0.5) statement:
MariaDB : Analysis of the execution plan
EXPLAIN
SELECT e.emp_no, e.first_name, s.from_date, s.salary
FROM employees e, salaries s
WHERE e.emp_no=s.emp_no
LIMIT 10;
Outer
Inner

16
 The columns in EXPLAIN ... SELECT
Column name Description
id Sequence number that shows in which order tables are joined.
select_type What kind of SELECT the table comes from.
table Alias name of table. Materialized temporary tables for sub queries are named <subquery#>
type How rows are found from the table (join type).
possible_keys keys in table that could be used to find rows in the table
key The name of the key that is used to retrieve rows. NULL is no key was used.
key_len How many bytes of the key that was used (shows if we are using only parts of the multi-column key).
ref The reference that is used to as the key value.
rows An estimate of how many rows we will find in the table for each key lookup.
Extra Extra information about this join.

17
 ID column
Select
( ( select count(*) from employees) + (select count(*) from departments) ) as total_counts;
select e.emp_no, e.first_name, s.from_Date, s.salary from employees e, salaries s where e.emp_no=s.emp_nolimit 10;
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

18
 Select_Type column
Value Description
UNION
The SELECT is a UNION of the
PRIMARY.
DEPENDENT UNION The UNION is DEPENDENT.
UNION RESULT The result of the UNION.
SUBQUERY
The SELECT is a SUBQUERY of the
PRIMARY.
DEPENDENT SUBQUERY The SUBQUERY is DEPENDENT.
DERIVED
The SELECT is DERIVED from the
PRIMARY.
UNCACHEABLE SUBQUERY The SUBQUERY is UNCACHEABLE.
PRIMARY The SELECT is a PRIMARY one.
SIMPLE The SELECT is a SIMPLE one.
UNCACHEABLE UNION The UNION is UNCACHEABLE.
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

19
 Select_Type column [UNION]
select *
from (
(select emp_no from employees e1 limit 10)
union all
union all
) tb;

20
 Select_Type column [DEPENDENT UNION]
select *
from employees e1 where e1.emp_no in
(
select e2.emp_no from employees e2 where e2.first_name='Matt‘ (where e2.emp_no=e1.emp_no)
union
select e3.emp_no from employees e3 where e3.first_name='Matt‘ (where e3.emp_no=e1.emp_no)
)
Optimization

21
 Select_Type column [UNION RESULT]
select emp_no
from salaries
where salary>100000
union all
select emp_no
from dept_emp
where from_date >'2001-01-01';
UNION RESULT < UNION 1,2 >

22
 Select_Type column [SUBQUERY]
select e.first_name,
(select count(*) from dept_emp de, dept_manager dm where dm.dept_no=de.dept_no) as cnt
from employees e
where e.emp_no=10001;
SELECT : Nested Query
WHERE : Sub Query
FROM : DERIVED

23
 Select_Type column [DEPENDENT SUBQUERY]
(select count(*)from dept_emp de, dept_manager dm where dm.dept_no=de.dept_no and de.emp_no=e.emp_no)
as cnt
from employees e
where e.first_name='Matt';

24
 Select_Type column [DERIVED]
select *
from
(select de.emp_no from dept_emp de group by de.emp_no) tb, (DERIVED)
employees e
where e.emp_no=tb.emp_no;
Tuning Point : 1. Check SELECT_TYPE (DERIVED)
2. SUBQUERY -> JOIN
Optimization

25
 Select_Type column [UNCACHEABLE SUBQUERY]
select *
from employees e
where e.emp_no =
(
select @status from dept_emp de where de.dept_no ='d005'
);
UNCACHEABLE SUBQERY CASE :
1. Including User Variables
2. Including NOT-DETERMINISTIC
3. Including UUID(), RAND() Function
> Compare the number of records generated as a function call

26
 Select_Type column [MATERIALIZED]
select *
from employees e
where e.emp_no in (select emp_no from salaries where salary between 100 and 1000); (DERIVED  Materialization)

27
 Table column [derived N]
ID SELECT_TYPE TABLE …
1 PRIMARY <derived 2>
1 PRIMARY E
2 DERIVED Dept_emp
＊ Must Have : Temporary tables should have a alias
<derived N> = <Derived Table Number(ID)>
MariaDB [employees]> select dttm from (select now() as dttm);
ERROR 1248 (42000): Every derived table must have its own alias
MariaDB [employees]> select dttm from (select now() as dttm) derived_table_alias;
+---------------------+
| dttm |
+---------------------+
| 2014-07-08 21:10:29 |
+---------------------+
1 row in set (0.19 sec)
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

28
 Type column
Value Description
ALL
A full table scan is done for the table
(all rows are read)
const
There is only one possibly matching
row in the table. (UNIQUE INDEX
SCAN)
eq_ref
A unique index is used to find the
rows. (Best Practice)
fulltext
A fulltext index is used to access the
rows.
index_merge
A 'range' access is done for several
index and the found rows are merged
index_subquery
This is similar as ref, but used for sub
queries that are transformed to key
lookups.
index
A full scan over the used index.
(Better than ALL but still bad)
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

29
 Type column
Value Description
range
The table will be accessed with a key
over one or more value ranges.
ref_or_null ref or null comparison(is null)
ref
A non unique index or prefix of an
unique index is used to find the rows.
system The table has 0 or 1 rows.
unique_subquery
This is similar as eq_ref, but used for
sub queries that are transformed to
key lookups

30
 Type column [ALL]
select * from employees;
ALL = Full Table Scan

31
 Type column [const]
select * from employees where emp_no=10001;
select * from dept_emp where dept_no='d005';
select * from dept_emp where dept_no='d005' and emp_no=10001;
One of Consisting of a multi-column primary key, unique key

32
 Type column [ref]
1. const
primary key/unique key column (equal) -> recode only one
2. eq_req
using read the column values of first table the second table values the retrieve condition -> recode only one
3. ref
Equal -> recode only one X

33
 Type column [eq_ref]
select * from dept_emp de, employees e
where e.emp_no=de.emp_no and de.dept_no='d005';
1. Join Table (id = 1)
2. Process (dept_emp -> employee)
3. emp.no (primary) -> eq_ref

34
 Type column [fulltext]
select *
from employee_name
where emp_no=10001 #const
and emp_no between 10001 and 10005 #range
and match(first_name, last_name) against('Facello' in boolean mode);
select *
from employee_name
#where emp_no=10001 #const
where emp_no between 10001 and 10005 #range
and match(first_name, last_name) against('Facello' in boolean mode);
1. A full-text index in MariaDB is an index of type FULLTEXT
2. Full-text searching is performed using MATCH() ... AGAINST syntax.

35
 Type column [index_merge]
select *
from employees
where emp_no between 10001 and 11000
or first_name='Smith';
1. several index and the found rows are merged
2. efficiency is less than the range scan
3. Do not work “full-text” search

36
 Type column [index_subquery]
select *
from departments where dept_no in (
select dept_no from dept_emp where dept_no between 'd001' and 'd003');
1. unique_subquery
IN(subquery): Deduplication does not need work
2. index_subquery
IN(subquery) : Possible duplicate values, Index(Remove duplicate values)
MariaDB 5.6 Over : index_subquery => ref

37
 Type column [index]
select * from departments order by dept_name desc limit 10;
1. Type(index) = Index Full Scan (But, Not Data File Full Scan -> Index File Full Scan)
2. Best Practice : Operation is possible only index

38
 Possible_keys column
– To make the best execution plan that was nominated for an index list
– Actually unused indexes
– Ignore column
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

39
 Key column
– The selected index in the best execution plan
– Tuning Point key (index)
– One or more indexes are "," separated (type - index_merge)
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

40
 Key_len column
– The selected index in the best execution plan
– Tuning Point key (index)
– One or more indexes are "," separated (type - index_merge)
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra

41
 Key_len column
* Description : Character – utf 8(1~3byte), dept_no – char(4 byte)
12 = 3 * 4
select * from dept_emp where dept_no='d005' and emp_no=10001;
* Description : Character – utf 8(1~3byte), dept_no – char(4 byte), emp_no – integer (4 byte)
16 = (4*3) + 4

42
 Key_len column
select * from titles where to_date<='1985-10-10';
* Description : Character – utf 8(1~3byte), to_date – date(3 byte)
4 ? -> why? : to_date (3 byte + nullable(1 byte))

43
 Ref column
– reference column
– ex, Ref (Equal), other column (Table name, Column name), const
 “Func”
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
select *
from employees e, dept_emp de
where e.emp_no=de.emp_no;
select *
from employees e, dept_emp de
where e.emp_no=(de.emp_no-1);

44
 Rows column
– Anticipated counting of records
– Based on statistical information (Record count, distribution of index
values … )
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
# ix_formdate
select * from dept_emp where from_Date >='1985-01-01';
select * from dept_emp where from_Date >='2002-07-01'; Why ?

45
Column name
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
 Extra column
– Displays important information about the performance-related
– 2-3 show generally

46
 Extra column [const row not found]
– Display type : const
– Fact : 0 rows

47
 Extra column [Distinct]
select distinct d.dept_no
from departments d, dept_emp de where de.dept_no=d.dept_no;
How to Handling Distinct on MariaDB ?

48
 Extra column [Full scan on Null key]
select d.dept_no, null in (select id.dept_name from departments id)
from departments d; Solutions : Definition column Attribute (is not null)

49
 Extra column [Impossible HAVING]
select e.emp_no, count(*) as cnt
from employees e
where e.emp_no=10001
group by e.emp_no
having e.emp_no is null; e.emp_no = primary key & not null columns

50
 Extra column [Impossible WHERE]
select * from employees where emp_no is null; e.emp_no = primary key & not null columns

51
 Extra column [Impossible WHERE noticed after reading const tables]
select * from employees where emp_no=0; Comparison(Equal) => const type

52
 Extra column [No matching min/max row]
select min(dept_no), max(dept_no)
from dept_emp where dept_no=''; Not matching = 0 rows

53
 Extra column [No matching row in const table]
select *
from dept_emp de,
(select emp_no from employees where emp_no=0) tb1
where tb1.emp_no=de.emp_no and de.dept_no='d005'; Not matching when const approaching

54
 Extra column [No tables used]
select 1;
select 1 from dual; Handled internally by the optimizer

55
 Extra column [Not exists]
select *
from dept_emp de
left join departments d on de.dept_no=d.dept_no
where d.dept_no is null; Anti-Join (When a lot of data : use Outer Join)
Optimization

56
 Extra column [Range checked for each recode(index map:N)]
select *
from employees e1, employees e2
where e2.emp_no >= e1.emp_no;
Index map: 0x1 (16 Hexadecimal) => 1 (Transfer to 2 Hexadecimal)
1 => first index of e2(employees) table
Type ALL : if first index not good Performance => Plan change : ALL(Full Table Scan)

57
 Extra column [Select tables optimized away]
select max(emp_no), min(emp_no) from employees;
# salaries table (pk : emp_no+from_date)
select max(from_date), min(from_date) from salaries where emp_no=10001;
Optimization
10002
10003
…
10009
10010
10001
MIN(emp_no)
MAX(emp_no)
CASE : Optimization (if NOT WHERE statement)
1996-01-01
…
2001-08-14
2014-07-09
1995-01-01
MIN(from_date)
Max(from_date)
CASE : Optimization (if the WHERE statement)
1991-03-15

58
 Extra column [Skip_open_table, Open_frm_only, Open_trigger_only,
Open_full_table]
– = “Scanned N databases”
– Select INFORMATION_SCHEMA (meta data)
1. Skip_open_table : No need to read
2. Open_frm_only : Read only files stored in the meta information file(*.FRM)
3. Open_trigger_only : Read only files stored in the trigger information file (*.TRG)
4. Open_full_table : Not optimized , Read all meta information file(*FRM) & data file(*MYD), index file(*MYI)

59
 Extra column [Unique row not found]
# // test table creation
create table tb_test1 (fdpk int, primary key(fdpk));
create table tb_test2 (fdpk int, primary key(fdpk));
# // insert sample data
insert into tb_test1 values (1), (2);
insert into tb_test2 values (1);
select t2.fdpk
from tb_test1 t1
left join tb_test2 t2 on t2.fdpk=t1.fdpk
where t1.fdpk =2;
* Each table has unique (PK included) columns in a query that performs an outer join, the outer table has a
matching record does not exist

60
 Extra column [Using filesort]
select * from employees order by last_name desc;
* Use Quick soft algorithm
* Processing
1. read recode
2. Copy to soft buffer
3. Order by
4. Send to client
* Overhead -> create index, query tuning

61
 Quick soft algorithm
Ex) Pivot = p, List Left Index I, List Right Index J
1.
5 - 3 - 7 - 6 - 2 - 1 - 4
p
2.
5 - 3 - 7 - 6 - 2 - 1 - 4
i j p
1 - 3 - 7 - 6 - 2 - 5 - 4
i j p
3.
1 - 3 - 7 - 6 - 2 - 5 - 4
i j p
4.
1 - 3 - 7 - 6 - 2 - 5 - 4
i j p
1 - 3 - 2 - 6 - 7 - 5 - 4
i j p
5.
1 - 3 - 2 - 6 - 7 - 5 - 4
p
1 - 3 - 2 - 4 - 7 - 5 - 6
p
6.
1 - 3 - 2
1 - 2 - 3

62
 Extra column [Using index(covering index)]
# use index (ix_first_name) & disk access
select first_name, birth_date
from employees where first_name between 'Babette' and 'Gad';
# use index (ix_first_name)
select first_name
from employees where first_name between 'Babette' and 'Gad';
* Operation to Only index page
* Innodb table (default cluster index)

63
 Extra column [Using index for group-by]
#tight index scan
select first_name, count(*) as counter from employees group by first_name;
#loose index scan
select emp_no, min(from_date) as first_changed_date, max(from_date) as last_changed_date
from salaries
group by emp_no;
1. Tight index scan
- A tight index scan may be either a full index scan or a range index scan, depending on the query conditions.
2. Loose index scan
- The most efficient way is when the index is used to directly retrieve the group fields
Since this access method considers only a fraction of the keys in an index, it is called a loose index scan
#loose index scan
select emp_no
from salaries where emp_no between 10001 and 10099
group by emp_no;
* Good performance in a large records.
* The optimizer to determine the break-even point

64
 Extra column [Using join buffer(Block Nested Loop),
Using join buffer(Batched Key Access)]
select *
from dept_emp de, employees e
where de.from_date>'2005-01-01' and e.emp_no<10904;
Join Processing
- Using Join buffer
- Using Block Nested Loop (Nested Loop)
Recommend : Driven Table (Create Index)
 prevention (Full Tabe Scan, Index Full Scan)
* Using Join buffer
- Place to temporarily store the records
- OLTP Recommend Buffer Size : 1MB
- MariaDB 5.3 Over : Hash Join(Based on Block)
Batched Key Access = Multi Range Read(MRR)

65
 Extra column [Using soft_union, Using union, Using intersect,
Using sort_intersecion]
– Using intersect(…)
• AND operation (Intersection)
– Using union(…)
• OR operation (union)
– Using sort_union(…)
• OR operation associated with the relatively large quantity of RANGE operation
 1. Reading Primary key
 2. Order by
 3. Merge
 4. Output
– Using sort_intersection(…)
• 1. Order by
• 2. Intersection operation
* MariaDB 5.3 Over (equal operator, the range of comparison operators)
* Only If the Plan Type Column = index_merge

66
 Extra column [Using temporary]
select * from employees group by gender order by min(emp_no);
Create Temporary Table (Memory or Disk)
* When create a temporary table
- Subquery (From …) = Derived table
- COUNT(DISTINCT column1) (If the index is not available)
- Union & Union all
- Using filesort (Large sort operations)

67
 Extra column [Using where]
select * from employees where emp_no between 1001 and 10100 and gender='F'
Engine
MariaDB Engine
(Join, Filtering, Aggregation)
100 rows
100 rows
3 rows
InnoDB
MyISAM
User
* Only when filtering process work "Using where" display

68
 Extra column [Using where with pushed condition]
– = “Condition push down”
Management Node
SQL Node(MariaDB Engine)
Data Node
(Storage Engine)
TCP/IP
100 rows 100 rows
Passing

69
 Extra column [Deleting all rows]
– If you delete all of the records in the table : Deleting all rows
– If you delete of the records with where statement : Using where

70
 Extra column [FirstMatch(tbl_name)]
– MariaDB 5.3 & MySQL 5.6 Over : Subquery optimization
select *
from departments where dept_no in (
select dept_no from dept_emp where dept_no between 'd001' and 'd003');
Optimization
FirstMatch(tbl_name) : departments is outer table
* Depending on version

71
 Extra column [LooseScan(m..n)]
– IN(subquery) : If the results of sub-queries create duplicate records
– “Loose Index Scan” : delete duplicate records after Join(It’s driving table)
– Does not require a separate temporary table

72
 Extra column [Materialize, Scan]
– Materialize : MySQL 5.6 & MariaDB 10.0 Over
– Scan : Materialized table without index -> Materialized table full scan

73
 Extra column [Start temporary, End temporary]
– Duplicate Weedout (Display Start temporary, End temporary)
• 1. Select Subquery
• 2. Join with Outer table
 First table : Start temporary
 Last table : End temporary
• 3. Stored in a temporary table
• 4. Remove duplicate records

74
 Extra column [Using index condition]
select *
from employees
where first_name like 'Lee%' and first_name like '%matt';
Engine InnoDB
first_name like ‘Lee%’
and first_name like ‘%matt’ (ix_firstname)
Engine InnoDB
Index condition
Pushdown, ICP
SQL
first_name like ‘Lee%’ (Random Access – all recode)
and first_name like ‘%matt’(Efficient use of non)
SQL

75
 Extra column [Rowid-ordered scan, key-ordered scan]
B-Tree
Index
Data Table
...
Where
=
...
Index Range Scan
(Random access)
MRR(Multi Range Read)
B-Tree
Index
Data Table
...
Where
=
...
Sort
By
Primary
Key
Optimization

76
Column name
… (SKIP)
Extra
EXTENDED
EXTENTED
Partitions
 EXTENDED(Filtered) column
Engine
MariaDB Engine
100 rows
3 rows
InnoDB
User
93 Rows => Filtered
MySQL
(After)
DATA
employees(DATA + INDEX))
RANGE SCAN
(emp_no BETWEEN 10001 AND 10100)
Filtering
Gender=‘F’
FILTERD
ROWS
FILTERD
MySQL
(Before)
Storage
Engine

77
Column name
… (SKIP)
Extra
EXTENDED
EXTENTED
Partitions
 EXTENDED(Additional information about the optimizer) column
EXPLAIN EXTENDED
(select count(*) from dept_emp de, dept_manager dm where dm.dept_no=de.dept_no) as cnt
from employees e
where e.emp_no=10001;
Show warnings;
select 'Georgi' AS `first_name`,
(select count(0)from èmployees`.`dept_emp` `de` join èmployees`.`dept_manager` `dm`
where (èmployees`.`de`.`dept_no` = èmployees`.`dm`.`dept_no`)) AS `cnt`
from èmployees`.èmployees` è`where 1
How to interpret the query optimizer,
How to you convert the query,
What happened to the special processing is performed

78
Column name
… (SKIP)
Extra
EXTENDED
EXTENTED
Partitions
 EXTENDED(Additional information about the optimizer) column
create table tb_partition (
reg_date date default null,
id int default null,
name varchar(50) default null
) engine=innodb
partition by range (year(reg_date)) (
partition p0 values less than (2008) engine = innodb,
partition p3 values less than (2011) engine = innodb
);
explain partitions
select * from tb_partition
where reg_date between '2010-01-01' and '2010-12-30';

79
 HOW TO USE
 MariaDB Version Control
– Divided into three parts: Major version, Minor version, Patch version
(ex MariaDB 5.5.8)
– Comment(Major(1), Minor(2), Patch(2) = 5-digit)
“/*!50508 TEMPORARY */”
MariaDB : Optimizer Hint
select * from employees USE INDEX (primary) where emp_no=1001;
select * from employees /*! USE INDEX (PRIMARY) */ where emp_no=1001;
SELECT /*!32302 temporary */ TABLE TEMP_EMP_STAT(hire_year INT NOT NULL, emp_count INT, PRIMARY KEY (hire_year));
SELECT TEMPORARY TABLE TEMP_EMP_STAT(hire_year INT NOT NULL, emp_count INT, PRIMARY KEY (hire_year));
=> (Version specific comment)

80
 STRAIGHT_JOIN
– This hint will tell MySQL to join the tables in the order that they are
specified in the FROM clause.
– Use EXPLAIN to make sure that MySQL has not already figured out the
optimal join order. And if you specify an ill order you can make MySQL do
a lot more work than it needs to.

81
 STRAIGHT_JOIN
– Fixed the order of the join (looks like ordered hint at oracle)
select *
from employees e, dept_emp de, departments d
where e.emp_no=de.emp_no and d.dept_no=de.dept_no;
DRIVING_TABLE
(matching where statement and small counting of records

82
 STRAIGHT_JOIN
select straight_join e.first_name, e.last_name, d.dept_name
from employees e, dept_emp de, departments d
Join Access Path : employees -> dept_emp -> departments
select straight_join e.first_name, e.last_name, d.dept_name
from employees e, departments d, dept_emp de
Join Access Path : employees -> departments -> dept_emp
Recode Count : (299920 * 1 * 1)
Recode Count : (299920 * 9 * 1)

83
 STRAIGHT_JOIN
– Temporary tables and regular tables join
• A temporary table is good driving table
– Join between the temp table
• A small size table is good driving table
– Join between the regular tables join
• If there is index columns of both sides or no index columns of both
 small size table
• Else
 no index table
Recode counting : The counting of records that satisfy the conditions

84
 USE INDEX / FORCE INDEX /IGNORE INDEX
– USE INDEX
• Recommended to use that index
• Using the index but it is not always
– FORCE INDEX
• More powerful than USE INDEX Hint
– IGNORE INDEX
• Prevent the use of the that index
• To use the full table scan.

85
– USE INDEX FOR JOIN
• Purpose of Join & select record
– USE INDEX FOR ORDER BY
• Order by statement use only
– USE INDEX FOR GROUP BY
• Group by statement use only
Depending on the type

86
select * from employees where emp_no=10001;
select * from employees force index(primary) where emp_no=10001;
select * from employees use index(primary) where emp_no=10001;
select * from employees ignore index(primary) where emp_no=10001;
select * from employees force index(ix_firstname) where emp_no=10001;

87
 SQL_CACHE / SQL_NO_CACHE
– if you have setup MySQL Query Caching to explicit mode (set
query_cache_type = 2) then you can use the SQL_CACHE hint to tell
MySQL which queries to cache.
– The SQL_NO_CACHE hint turns off MySQL's builtin query caching
mechanism for a particular query.
Query_cache_type(System variable settings)
0 or OFF 1 or ON 2 or DEMAND
No Hint NO Caching Caching NO Caching
SQL_CACHE NO Caching Caching Caching
SQL_NO_CACHE NO Caching NO Caching NO Caching

88
 Tuning Point [Select_Type column]
– DERIVED
• Large size data table when it is stored on disk temporary tables
– UNCACHEABLE SUBQUERY
• Leads to caching as much as possible
• Leads to reusing as much as possible (ex. Remove a user variable)
– DEPENDENT SUBQUERY
• Leads to Join operation
• Remove the dependence of the outer table
MariaDB : Tuning Point

89
 Tuning Point [Type column]
– ALL, index
• Index is Index Full Scan, ALL is Full Table Scan
• Add new index or Query changing

90
 Tuning Point [Key column]
– When not using the index is not displayed.
– Add new index or Change the WHERE statement condition

91
 Tuning Point [Rows column]
– If you see a much larger value, check the index & configure the index
column
– If index is not efficient, re-generated.

92
 Tuning Point [Extra]
– That word is displayed, the more detailed review is preferred.
– If the query is not good requirements
• Full scan on Null key
• Impossible HAVING (MariaDB 5.1 Over)
• Impossible WHERE (MariaDB 5.1 Over)
• Impossible WHERE noticed after reading const tables
• No matching min/max row (MariaDB 5.1 Over)
• No matching row in const table (MariaDB 5.1 Over)
• Unique row not found (MariaDB 5.1 Over)

93
 Tuning Point [Extra]
– If the plan is not good enough
• Range checked for each record (index map:N)
• Using filesort
• Using join buffer (MariaDB 5.1 Over)
• Using temporary
• Using where
– If the plan is good
• Distinct
• Using index (best practice : Covering index)
• Using index for group-by
Recommend that a covering index.

94
Select_Type Type Extra
SIMPLE system Distinct
PRIMARY const Using index
UNION eq_ref Using index for group-by
DEPENDENT UNION ref Range checked for each record
UNION RESULT fulltext Using filesort
SUBQUERY ref_or_null Using join buffer
DEPENDENT SUBQUERY unique_subquery Using temporary
DERIVED index_subquery Using where
UNCACHEABLE SUBQUERY range Full scan on Null key
UNCACHEABLE UNION index_merge Impossible HAVING
MATERIALIZED index Impossible WHERE
ALL Impossible WHERE noticed after reading const tables
No matching min/max row
No matching row in const table
Unique row not found

95
Reference
 Site : MariaDB Knowledge Base(EXPLAIN)
PETE FREITAG(MySQL Optimization Hints)
 Book : “Real MariaDB”

96
Q & A

97

MariaDB Optimizer Guide

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to MariaDB Optimizer Guide

Similar to MariaDB Optimizer Guide (20)

Recently uploaded

Recently uploaded (20)

MariaDB Optimizer Guide