More Related Content Similar to MySQLerの7つ道具 plus (20) 15. 目次
第 1 章 MySQL クエリーチューニングことはじめ
第 2 章 スローログの集計に便利な「pt-query-digest」
を使ってみよう
第 3 章 SQL 実⾏計画の疑問解決には「とりあえず
EXPLAIN」しよう
第 4 章 「PMP for Cacti」で MySQL のステータスを可
視化する
14/98
16. 目次
第 5 章 MySQL のリアルタイムモニタリングに
innotop
第 6 章 再現性のあるスロークエリーには「SHOW
PROFILE」を試してみよう
第 7 章 performance̲schema を sys で使い倒す
第 8 章 MySQL のチューニングを戦う⽅へ
15/98
27. 先頭のセクション
$ pt-query-digest /path/to/slowlog
..
# Time range: 2016-03-16 17:29:32 to 2016-11-22 11:37:24
# Attribute total min max avg 95% stddev median
# ============ ======= ======= ======= ======= ======= ======= =======
# Exec time 45155s 500ms 117s 3s 7s 9s 640ms
# Lock time 2s 29us 14ms 110us 176us 136us 93us
# Rows sent 156.96M 0 5.85M 10.33k 54.03k 74.89k 0.99
# Rows examine 36.18G 0 26.40M 2.38M 22.38M 5.41M 298.06k
# Rows affecte 0 0 0 0 0 0 0
# Bytes sent 9.86G 0 362.91M 664.55k 4.26M 4.87M 1.09k
# Query size 3.29M 25 24.47k 221.80 346.17 425.46 136.99
..
26/98
30. Anemometerの弱点
テーブル上UNIQUE KEY (hostname_max, checksum, ts_min,
ts_max)で、Anemometerはts_minでプロットするため、そ
のクエリーがts_minに集中したことになってしまう。
mysql> SELECT * FROM global_query_review_history LIMIT 1G
*************************** 1. row ***************************
hostname_max: xxx
db_max: xxx
checksum: 1233945238822708500
sample: xxx
ts_min: 2015-09-14 11:32:12
ts_max: 2015-10-28 15:51:01
ts_cnt: 31
Query_time_sum: 651.778
Query_time_min: 2.07993
Query_time_max: 197.678
29/98
32. そこでanemo eat er
スローログをスプリットして pt-query-digest を呼びまくる
1分ぶんずつ pt-query-digest に⾷わせれば、ts_min と ts_max の差
は最⼤でも1分
-
AnemometerをDockerコンテナーとして起動する-
既存のAnemometerがあれば単にスローログを分割して⾷わせる pt-
query-digest のラッパーとして呼べる
-
31/98
35. anemo eat er
現在のところ docker と pt-query-digest はホストにインス
トールしておかないとダメ
$ git clone https://github.com/yoku0825/anemoeater
$ cd anemoeater
$ ./anemoeater slow_log_file
Docker container starts with 172.17.0.43.
URL will be http://xxxx:32780/anemometer
34/98
36. With anemo eat er
$ scp -c arcfour target_server:/data/../slow.log ./target_server_
slow.log
$ ./anemoeater target_server_slow.log
Docker container starts with 172.17.0.3.
URL will be http://192.168.230.241:32789/anemometer
processing 2016-12-16 01:08:00 at target_server_slow.log.
..
デフォルトでは直近1か⽉分をCPUスレッド数 * 1.5 パラレ
ルにして5分単位で分割する
35/98
45. EXPLAIN EXTENDED
5.7からはデフォルトでEXTENDED & PARTITIONSがつい
てくる
EXPLAIN直後に SHOW WARNINGS でオプティマイザーが最適化した後
のクエリーが⾒える
-
5.6まではEXTENDEDとPARTITIONSを同時に指定できなかったけ
ど、5.7はまとめて出してくれる
-
想像したのと違う遅くなり⽅ をしたらEXTENDED⾒た⽅が
良い
特に5.6の蝉ジョインとか蝉ジョインとか蝉ジョインとか-
MySQL :: MySQL 5.6 リファレンスマニュアル :: 8.8.3
EXPLAIN EXTENDED 出⼒フォーマット
44/98
46. EXPLAIN EXTENDED
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: hg
type: ref
possible_keys: PRIMARY,hogehoge_idx05,hogehoge_idx09,hogehoge_idx10,hogehoge_idx11,hogehoge_idx12,ho
gehoge_idx13,hogehoge_idx14,hogehoge_idx15,hogehoge_idx16,hogehoge_idx17,hogehoge_idx19,idx_col2_col
3_col1
key: hogehoge_idx05
key_len: 5
ref: const
rows: 170287
filtered: 100.00
Extra: Using index condition; Using where; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: fg
type: ref
possible_keys: fugafuga_idx1,idx_fugafuga_04,idx_col1_col2_col3
key: idx_fugafuga_04
key_len: 7
ref: hg.col2,const
rows: 5
filtered: 100.00
Extra: Using index
2 rows in set, 1 warning (0.00 sec)
45/98
47. SHOW WARNINGS
*************************** 1. row ***************************
Level: Note
Code: 1003
Message: /* select#1 */ select `fg`.`fugafuga_id` AS `fugafuga_id
`,`hg`.`col3` AS `col3`,`hg`.`title` AS `hogehoge_title`,`hg`.`co
l2` AS `col2`,`hg`.`col4` AS `col4`,`hg`.`fugafuga_count` AS `fug
afuga_count` from `fugafuga` `fg` join `hogehoge` `hg` where ((`f
g`.`col1` = `hg`.`col2`) and (`hg`.`col2` = 210) and (`hg`.`col2
` = 1) and (`fg`.`col5` = 1) and (`hg`.`col6` in (0,100,101))) or
der by `fg`.`fugafuga_id` desc limit 0,5
46/98
49. EXPLAIN format=json
EXPLAIN: {
"query_block": {
"select_id": 1,
"ordering_operation": {
"using_temporary_table": true,
"using_filesort": true,
"nested_loop": [
{
"table": {
..
"rows": 170287,
"filtered": 100,
"index_condition": "(`hg`.`col2` is not null)",
"attached_condition": "((`hg`.`col2` = 1) and (`hg`.`
col6` in (0,100,101)))"
}
..
}
1 row in set, 1 warning (0.00 sec)
48/98
52. optimizer_trace
> SELECT * FROM information_schema.optimizer_traceG
*************************** 1. row ***************************
QUERY: ..
TRACE: {
"steps": [
{
"join_preparation": {
"select#": 1,
"steps": [
{
"expanded_query": ..
{
"join_optimization": {
"select#": 1,
"steps": [
{
"transformations_to_nested_joins": {
"transformations": [
"outer_join_to_inner_join",
"JOIN_condition_to_WHERE",
"parenthesis_removal"
],
..
{
"index": "hogehoge_idx18",
"usable": false,
"cause": "not_applicable"
},
MISSING_BYTES_BEYOND_MAX_MEM_SIZE: 22143
INSUFFICIENT_PRIVILEGES: 0
51/98
57. Data Input Methodのデフォルトを⼀気にSQLで書き換
えるとか
mysql> UPDATE data_input_fields JOIN data_input_data ON data_input_fields.id= data_i
nput_data.data_input_field_id JOIN data_input ON data_input_fields.data_input_id = d
ata_input.id
-> SET data_input_data.value = 'pmp' WHERE data_input.name LIKE 'Percona %' AND
data_input_fields.name = 'Username';
mysql> UPDATE data_input_fields JOIN data_input_data ON data_input_fields.id= data_i
nput_data.data_input_field_id JOIN data_input ON data_input_fields.data_input_id = d
ata_input.id
-> SET data_input_data.value = 'pmp_pass' WHERE data_input.name LIKE 'Percona %
' AND data_input_fields.name = 'Password';
mysql> UPDATE data_input_fields JOIN data_input_data ON data_input_fields.id= data_i
nput_data.data_input_field_id JOIN data_input ON data_input_fields.data_input_id = d
ata_input.id
-> SET data_input_data.value = '3306', data_input_data.t_value = 'on' WHERE data
_input.name LIKE 'Percona %' AND data_input_fields.name = 'Port';
56/98
62. innotop
みんな⼤好き、topライクに SHOW PROCESSLIST を表⽰して
くれるinnotop
地味に “M” (Replication Status) も便利
なんとMSR対応してるんだぜ
-
“L” (InnoDB Locks) とか-
“T” (InnoDB Transaction) とか-
“M”(Replications) も便利だな-
pt-osc してる間だと “D” (InnoDB Deadlocks) を眺めることもある-
tmuxでばっちんばっちんターミナル割って、 dstat とか流
しながら⾒るのが好き
61/98
68. SHOW PROFILE
mysql> SET @@profiling= 1;
Query OK, 0 rows affected, 1 warning (0.01 sec)
mysql> ..;
mysql> SHOW PROFILE;
+----------------------+----------+
| Status | Duration |
+----------------------+----------+
| starting | 0.000206 |
| checking permissions | 0.000024 |
| Opening tables | 0.000039 |
| init | 0.000089 |
| System lock | 0.000027 |
| optimizing | 0.000037 |
| statistics | 0.000245 |
| preparing | 0.000058 |
| Creating tmp table | 0.000119 |
| Sorting result | 0.000023 |
| executing | 0.000019 |
| Sending data | 2.619037 |
| Creating sort index | 0.000821 |
| end | 0.000014 |
| removing tmp table | 0.000017 |
| end | 0.000013 |
| query end | 0.000015 |
| closing tables | 0.000022 |
| freeing items | 0.000028 |
| logging slow query | 0.000109 |
| cleaning up | 0.000013 |
+----------------------+----------+
21 rows in set (0.00 sec)
67/98
73. p̲s⾃体のモニタリング
mysql> SHOW ENGINE PERFORMANCE_SCHEMA STATUS;
+--------------------+-------------------------------------------------------------
+----------+
| Type | Name
| Status |
+--------------------+-------------------------------------------------------------
+----------+
| performance_schema | events_waits_current.size
| 176 |
| performance_schema | events_waits_current.count
| 1536 |
| performance_schema | events_waits_history.size
| 176 |
| performance_schema | events_waits_history.count
| 2560 |
| performance_schema | events_waits_history.memory
| 450560 |
..
| performance_schema | performance_schema.memory
| 94739320 |
+--------------------+-------------------------------------------------------------
+----------+
229 rows in set (0.00 sec)
72/98
74. 最近お気に⼊りのp̲sクエリー
SELECT
thread_id,
event_name,
sql_text,
@progress:= (work_completed / work_estimated) * 100 AS progress,
@elapsed:= (timer_current - timer_start) / power(10, 12) AS elapsed,
@elapsed * (100 / @progress) - @elapsed AS estimated
FROM
(SELECT
stage.thread_id,
stage.event_name,
work_completed,
work_estimated,
(SELECT timer_start
FROM events_statements_current JOIN threads USING(thread_id)
WHERE processlist_id = @@pseudo_thread_id) AS timer_current,
statement.timer_start,
sql_text
FROM
events_stages_current AS stage JOIN events_statements_current AS statement USING(th
read_id)
) AS dummy;
⽇々の覚書: MySQL 5.7.6のPerformance Schemaで
InnoDBのALTER TABLE進捗どうですか
73/98
75. ALTER TABLE が⾒える…⾒えるぞ…
mysql> SELECT ..;
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
| thread_id | event_name | sql_text | progress | elapsed | estimated |
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
| 28 | stage/innodb/alter table (read PK and internal sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 7.330386000 | 1.877416142 | 23.734006530694177 |
+-----------+------------------------------------------------------+-------------------------------------+-------------+-------------+--------------------+
1 row in set (0.15 sec)
mysql> SELECT ..;
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
| thread_id | event_name | sql_text | progress | elapsed | estimated |
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
| 28 | stage/innodb/alter table (read PK and internal sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 46.969643300 | 33.385053874 | 37.6928839778295 |
+-----------+------------------------------------------------------+-------------------------------------+--------------+--------------+------------------+
1 row in set (0.01 sec)
mysql> SELECT ..;
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
| thread_id | event_name | sql_text | progress | elapsed | estimated |
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
| 28 | stage/innodb/alter table (merge sort) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 50.831565800 | 40.169081343 | 38.854810033960106 |
+-----------+---------------------------------------+-------------------------------------+--------------+--------------+--------------------+
1 row in set (0.00 sec)
mysql> SELECT ..;
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
| thread_id | event_name | sql_text | progress | elapsed | estimated |
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
| 28 | stage/innodb/alter table (insert) | ALTER TABLE t1 ADD UNIQUE KEY (val) | 83.429283200 | 61.092267798 | 12.134140789914134 |
+-----------+-----------------------------------+-------------------------------------+--------------+--------------+--------------------+
1 row in set (0.00 sec)
74/98
77. sysのインストール(5.6向け)
$ git clone https://github.com/mysql/mysql-sys.git
Initialized empty Git repository in /root/mysql-sys/.git/
remote: Counting objects: 3009, done.
remote: Total 3009 (delta 0), reused 0 (delta 0), pack-reused 3008
Receiving objects: 100% (3009/3009), 1.17 MiB | 466 KiB/s, done.
Resolving deltas: 100% (1768/1768), done.
$ cd mysql-sys
$ mysql -uroot -p < sys_56.sql
$ mysql -uroot -p
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.01 sec)
76/98
78. ⾖知識
MySQL 5.7の mysql_install_db には --skip-sys-schema オ
プションがある
https://dev.mysql.com/doc/refman/5.7/en/mysql-install-
db.html#option̲mysql̲install̲db̲skip-sys-schema
-
mysql_upgrade にもある
https://dev.mysql.com/doc/refman/5.7/en/mysql-
upgrade.html#option̲mysql̲upgrade̲skip-sys-schema
-
77/98
81. create_synonym_db
ストアド
SHOW TABLES FROM .. の結果をそのまま新しいスキーマに
CREATE VIEW .. AS SELECT * FROM .. するという雑な作り
p_s と i_s を作るのにすごく便利だ
mysql> CALL sys.create_synonym_db('performance_schema', 'p_s');
+----------------------------------------+
| summary |
+----------------------------------------+
| Created 87 views in the `p_s` database |
+----------------------------------------+
1 row in set (0.35 sec)
Query OK, 0 rows affected (0.35 sec)
mysql> use p_s
Database changed
mysql> SHOW TABLES;
..
80/98
93. MySQL Casual Talks vol.10
もっと深く浅く、広く狭くMySQLを使っていこうとい
う趣旨のイベントです。
多⽅⾯から多様なMySQLの使い⽅、運⽤、Tipsなどな
どのTalkを集めたいと思っております。
http://mysql-casual-slackin.herokuapp.com/ から
Slackチャンネルへjoinできますので、ご参加くださ
い。
92/98
