15 tips taken from the field on how to achieve a better performance on your MySQL server.

1. 15 Pro-tips for
MySQL Users
PHPBenelux Meeting - March 2011
Enrise HQ - Amersfoort
http://joind.in/talk/view/2947
1

2. Who am I?
2

3. Who am I?
Joshua Thijssen (32)
2

4. Who am I?
Joshua Thijssen (32)
Senior Software Engineer @ Enrise
2

5. Who am I?
Joshua Thijssen (32)
Senior Software Engineer @ Enrise
Development in PHP, Python, Perl,
C, Java....
2

6. Who am I?
Joshua Thijssen (32)
Senior Software Engineer @ Enrise
Development in PHP, Python, Perl,
C, Java....
Blogs: http://www.adayinthelifeof.nl
http://www.enrise.com/blog
2

7. Who am I?
Joshua Thijssen (32)
Senior Software Engineer @ Enrise
Development in PHP, Python, Perl,
C, Java....
Blogs: http://www.adayinthelifeof.nl
http://www.enrise.com/blog
Email: joshua@enrise.com
2

8. Who am I?
Joshua Thijssen (32)
Senior Software Engineer @ Enrise
Development in PHP, Python, Perl,
C, Java....
Blogs: http://www.adayinthelifeof.nl
http://www.enrise.com/blog
Email: joshua@enrise.com
Twitter: @jaytaph
Identi.ca: jaytaph
2

9. What are we going to discuss?
‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD
3

10. What are we going to discuss?
‣ 15 MySQL Pro-tips
‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD
3

11. What are we going to discuss?
‣ 15 MySQL Pro-tips
‣ No “theoretical tips”, all taken from
the field.
‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD
3

12. What are we going to discuss?
‣ 15 MySQL Pro-tips
‣ No “theoretical tips”, all taken from
the field.
‣ Starting simple - ending “complex”
‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD
3

13. Tip 1
1) Know how to use explain.
‣ EXPLAIN IS YOUR BESTEST FRIEND
4

14. Tip 1: Know your EXPLAIN (1)
5

15. Tip 1: Know your EXPLAIN (1)
‣ I will not show you how to use EXPLAIN.
5

16. Tip 1: Know your EXPLAIN (1)
‣ I will not show you how to use EXPLAIN.
‣ Use EXPLAIN and EXPLAIN EXTENDED/
SHOW WARNINGS;
5

17. Tip 1: Know your EXPLAIN (1)
6

18. Tip 1: Know your EXPLAIN (1)
6

19. Tip 1: Know your EXPLAIN (1)
6

20. Tip 2
2) Know the most basic my.cnf settings
‣ THERE ARE ONLY A FEW “BASIC” ONES.
7

21. Tip 2: My.cnf settings (1)
Know the most important ones:
key_buffer_size, innodb_buffer_pool_size,
sort_buffer_size, max_connections
8

22. Tip 2: My.cnf settings (2)
9

23. Tip 2: My.cnf settings (2)
‣ Some settings work on global level,
some per connection!
9

24. Tip 2: My.cnf settings (2)
‣ Some settings work on global level,
some per connection!
‣ Know some quirks:
(max_heap_table_size vs tmp_table_size, binlog-
do-db, replicate-ignore-db etc)
9

25. Tip 2: My.cnf settings (3)
‣ http://www.omh.cc/mycnf/
‣ http://rackerhacker.com/mysqltuner/
‣ http://www.day32.com/MySQL/
‣ phpmyadmin
10

26. Tip 3
3) Backup on table level
‣ RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE
11

27. Tip 3: Backup on table level (1)
‣ COULD YOU RESTORE TABLE x? YES! YES I CAN!
12

28. Tip 3: Backup on table level (1)
‣ mysqldump can dump per database OR
by table.
‣ COULD YOU RESTORE TABLE x? YES! YES I CAN!
12

29. Tip 3: Backup on table level (1)
‣ mysqldump can dump per database OR
by table.
‣ Simple scripts to scan/dump tables.
‣ COULD YOU RESTORE TABLE x? YES! YES I CAN!
12

30. Tip 3: Backup on table level (1)
‣ mysqldump can dump per database OR
by table.
‣ Simple scripts to scan/dump tables.
‣ Easy restore for single table (or part of
table)
‣ COULD YOU RESTORE TABLE x? YES! YES I CAN!
12

31. Tip 4
4) Don’t use “SELECT *” when you only
need 1 or 2 fields.
‣ DON’T ASK WHAT YOU DON’T NEED
13

32. Tip 4: Select * (1)
‣ DON’T ASK WHAT YOU DON’T NEED
14

33. Tip 4: Select * (1)
‣ Much more data to be read from disk
‣ DON’T ASK WHAT YOU DON’T NEED
14

34. Tip 4: Select * (1)
‣ Much more data to be read from disk
‣ Much more data will be send over,
thus slower (blobs/texts)
‣ DON’T ASK WHAT YOU DON’T NEED
14

35. Tip 4: Select * (1)
‣ Much more data to be read from disk
‣ Much more data will be send over,
thus slower (blobs/texts)
‣ Cannot use covering indices
‣ DON’T ASK WHAT YOU DON’T NEED
14

36. Tip 4: Select * (2)
15

37. Tip 4: Select * (2)
15

38. Tip 5
5) Use triggers and stored procedures
‣ ENFORCE CONSISTENCY
16

39. Tip 5: Triggers and stored procedures (1)
‣ ENFORCE CONSISTENCY
17

40. Tip 5: Triggers and stored procedures (1)
‣ 6 triggers per table (insert, update,
delete, before and after the mutation)
‣ ENFORCE CONSISTENCY
17

41. Tip 5: Triggers and stored procedures (1)
‣ 6 triggers per table (insert, update,
delete, before and after the mutation)
‣ 3rd party tools (phpmyadmin etc) can
also use the database without
loosing data consistency.
‣ ENFORCE CONSISTENCY
17

42. Tip 5: Triggers and stored procedures (1)
‣ 6 triggers per table (insert, update,
delete, before and after the mutation)
‣ 3rd party tools (phpmyadmin etc) can
also use the database without
loosing data consistency.
‣ Watch out with (phpmyadmin) table
dumps!
‣ ENFORCE CONSISTENCY
17

43. Tip 6
6) Don’t use FULLTEXT searches
‣ THERE ARE MUCH BETTER SOLUTIONS
18

44. Tip 6: Don’t use FULLTEXT search (1)
‣ THERE ARE MUCH BETTER SOLUTIONS
19

45. Tip 6: Don’t use FULLTEXT search (1)
‣ They only work for MyISAM tables.
‣ THERE ARE MUCH BETTER SOLUTIONS
19

46. Tip 6: Don’t use FULLTEXT search (1)
‣ They only work for MyISAM tables.
‣ Not compatible with other DB’s.
‣ THERE ARE MUCH BETTER SOLUTIONS
19

47. Tip 6: Don’t use FULLTEXT search (1)
‣ They only work for MyISAM tables.
‣ Not compatible with other DB’s.
‣ Slow (especially compared to Solr, Sphinx).
‣ THERE ARE MUCH BETTER SOLUTIONS
19

48. Tip 6: Don’t use FULLTEXT search (1)
‣ They only work for MyISAM tables.
‣ Not compatible with other DB’s.
‣ Slow (especially compared to Solr, Sphinx).
‣ No extra features (faceted search, spell
checking etc).
‣ THERE ARE MUCH BETTER SOLUTIONS
19

49. Tip 7
7) Wildcard searches (%item%) are
bad for performance
‣ IT LOOKS LIKE YOU NEED MORE ADVANCED
20

50. Tip 7: Wildcard searches (1)
‣ THERE ARE MUCH BETTER SOLUTIONS
21

51. Tip 7: Wildcard searches (1)
‣ MySQL cannot use indexes!
‣ THERE ARE MUCH BETTER SOLUTIONS
21

52. Tip 7: Wildcard searches (1)
‣ MySQL cannot use indexes!
‣ Revert your data:
search for ‘moc.esirne@%’ instead of
‘%@enrise.com’.
‣ THERE ARE MUCH BETTER SOLUTIONS
21

53. Tip 7: Wildcard searches (1)
‣ MySQL cannot use indexes!
‣ Revert your data:
search for ‘moc.esirne@%’ instead of
‘%@enrise.com’.
‣ Use a better solution (solr, sphinx).
You probably want it.
‣ THERE ARE MUCH BETTER SOLUTIONS
21

54. Tip 8
8) Shard your volatile and non-
volatile data.
‣ MAKE CACHING AND LOCKING HAPPY AGAIN
22

55. Tip 8: Sharding (1)
23

56. Tip 8: Sharding (2)
24

57. Tip 8: Sharding (2)
‣ Remember: an update on a table will
invalidate ALL queries referring to that
table.
24

58. Tip 8: Sharding (2)
‣ Remember: an update on a table will
invalidate ALL queries referring to that
table.
‣ UPDATE pages SET hit_count =
hit_count + 1;
24

59. Tip 8: Sharding (2)
‣ Remember: an update on a table will
invalidate ALL queries referring to that
table.
‣ UPDATE pages SET hit_count =
hit_count + 1;
‣ Thus: page table will NEVER be cached.
24

60. Tip 8: Sharding (3)
25

61. Tip 8: Sharding (3)
‣ Define hot data (volatile, changes
often) and cold data (static, changes
never or infrequently)
25

62. Tip 8: Sharding (3)
‣ Define hot data (volatile, changes
often) and cold data (static, changes
never or infrequently)
‣ move to different tables
25

63. Tip 8: Sharding (3)
‣ Define hot data (volatile, changes
often) and cold data (static, changes
never or infrequently)
‣ move to different tables
‣ UPDATE page_stats SET hit_count =
hit_count + 1;
25

64. Tip 8: Sharding (3)
‣ Define hot data (volatile, changes
often) and cold data (static, changes
never or infrequently)
‣ move to different tables
‣ UPDATE page_stats SET hit_count =
hit_count + 1;
‣ Query cache is happy again
25

65. Tip 9
9) Don’t use a large Primary Key for
InnoDB tables.
‣ PK’S ARE ON EVERY INDEX
26

66. Tip 9: Large primary keys (1)
27

67. Tip 9: Large primary keys (1)
‣ InnoDB adds the primary key to EACH
index.
27

68. Tip 9: Large primary keys (1)
‣ InnoDB adds the primary key to EACH
index.
‣ No primary key given? It uses an
internal 6(!)-byte key.
27

69. Tip 10
10) Don’t “SELECT COUNT(*) FROM
TABLE” on InnoDB.
‣ COUNT(*) => MYISAM = FAST
‣ COUNT(*) => INNODB = SLOW
28

70. Tip 10: SELECT COUNT(*) (1)
29

71. Tip 10: SELECT COUNT(*) (1)
‣ InnoDB implements MVCC (multi-
version concurrency control).
29

72. Tip 10: SELECT COUNT(*) (1)
‣ InnoDB implements MVCC (multi-
version concurrency control).
‣ COUNT(*) must be counted and is not
fetched from metadata.
29

73. Tip 10: SELECT COUNT(*) (1)
30

74. Tip 10: SELECT COUNT(*) (1)
‣ What do you want to COUNT(*)?
30

75. Tip 10: SELECT COUNT(*) (1)
‣ What do you want to COUNT(*)?
‣ Just for displaying purposes (there
are X amount of pages): do you need
the EXACT amount?
30

76. Tip 11
11) Don’t rely on the VARCHAR()
‣ IT ISN’T THAT VARIABLE AS YOU MIGHT THINK
31

77. Tip 11: VARCHAR() (1)
‣ IT DOESN’T SOLVE YOUR PROBLEMS
32

78. Tip 11: VARCHAR() (3)
‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER
33

79. Tip 11: VARCHAR() (3)
‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER
33

80. Tip 12
12) UTF-8 is not the enemy, but it
certainly isn’t your friend.
‣ DON’T EXCHANGE ONE PROBLEM FOR ANOTHER
34

81. Tip 12: UTF-8 (1)
35

82. Tip 12: UTF-8 (1)
‣ Dr Jeckyl and Mr Hyde
35

83. Tip 12: UTF-8 (1)
‣ Dr Jeckyl and Mr Hyde
‣ Solves all your multi-language
problems!
35

84. Tip 12: UTF-8 (1)
‣ Dr Jeckyl and Mr Hyde
‣ Solves all your multi-language
problems!
‣ But gives back performance issues.
35

85. Tip 12: UTF-8 (1)
36

86. Tip 12: UTF-8 (1)
37

87. Tip 12: UTF-8 (1)
37

88. Tip 12: UTF-8 (1)
ALL temporary buffers are allocated
for worst-case scenario’s. This
means a varchar(255) in UTF-8 uses
255*3 + 2 = 767 bytes PER row, even
if you have only 1 single char inside.
38

89. Tip 13
13) Know your cardinality &
selectivity
‣ WHY LOOKUP DATA WHEN YOU ALREADY HAVE IT?
39

90. Tip 13: Cardinality & Selectivity (1)
40

91. Tip 13: Cardinality & Selectivity (1)
‣ Cardinality: the number of unique
entries inside the index.
40

92. Tip 13: Cardinality & Selectivity (1)
‣ Cardinality: the number of unique
entries inside the index.
‣ Selectivity: percentage of unique
entries.
40

93. Tip 13: Cardinality & Selectivity (1)
‣ Cardinality: the number of unique
entries inside the index.
‣ Selectivity: percentage of unique
entries.
‣ S(I) = cardinality / count * 100%
40

94. Tip 13: Cardinality & Selectivity (2)
41

95. Tip 13: Cardinality & Selectivity (3)
country_id (max +-200, but effectively +- 50, maybe less)
42

96. Tip 13: Cardinality & Selectivity (3)
country_id (max +-200, but effectively +- 50, maybe less)
‣ with 10 records: 5/10 * 100% = 50%
42

97. Tip 13: Cardinality & Selectivity (3)
country_id (max +-200, but effectively +- 50, maybe less)
‣ with 10 records: 5/10 * 100% = 50%
‣ with 1000 records: 75/1000 * 100 = 7.5%
42

98. Tip 13: Cardinality & Selectivity (3)
country_id (max +-200, but effectively +- 50, maybe less)
‣ with 10 records: 5/10 * 100% = 50%
‣ with 1000 records: 75/1000 * 100 = 7.5%
‣ with 10.000 records: 200/10000 * 100% = 2%
42

99. Tip 13: Cardinality & Selectivity (4)
‣ A selectivity < 30% ? Full table scan!
‣ ANALYZE TABLE frequently.
43

100. Tip 13: Cardinality & Selectivity (5)
44

101. Tip 13: Cardinality & Selectivity (5)
‣ Adding records changes your
cardinality and selectivity.
44

102. Tip 13: Cardinality & Selectivity (5)
‣ Adding records changes your
cardinality and selectivity.
‣ Develop against a “real” dataset (10K
records instead of 10 for instance).
44

103. Tip 14
14) Non-deterministic functions do
not go well with query caching
‣ NOW(), RAND(), UUID(), CONNECTION_ID() ETC..
45

104. Tip 14: Query caching (1)
46

105. Tip 14: Query caching (1)
46

106. Tip 14: Query caching (2)
47

107. Tip 14: Query caching (2)
47

108. Tip 14: Query caching (3)
48

109. Tip 14: Query caching (3)
48

110. Tip 14: Query caching (4)
SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW());
vs
SELECT * FROM table WHERE YEAR(created_dt) < ‘2010’;
49

111. Tip 15
15) Certify yourself as a DBA and/or DBE.
‣ AND GET SOME NICE TITLES WHILE YOU’RE AT IT...
50

112. Tip 15: Certify yourself (1)
‣ Oracle Certified MySQL Associate
‣ Oracle Certified Professional MySQL
5.0 Developer
‣ Oracle Certified Professional MySQL
5.0 Database Administrator
‣ Oracle Certified Expert, MySQL 5.1
Cluster Database Administrator.
‣ Get them all!
‣ THEY ARE NOT EASY EXAMS, BUT WELL WORTH IT
51

113. Let’s summarize
(1) Know how to use explain. (9) Don’t use a large Primary Key for
InnoDB tables.
(2) Know the most basic my.cnf settings.
(10) Don’t “Select COUNT(*)” on InnoDB.
(3) Backup on table level.
(11) Don’t rely on the VARCHAR().
(4) Don’t use “SELECT *” when you only
need 1 or 2 fields. (12) UTF-8 is not the enemy, but it
certainly isn’t your friend.
(5) Use triggers and stored procedures.
(13) Know your cardinality & selectivity.
(6) Don’t use FULLTEXT searches.
(14) Non-deterministic functions do not
(7) Wildcard searches (%item%) are bad for go well with query caching.
performance.
(15) Certify yourself as a DBA and/or
(8) Shard your volatile and non-volatile DBE.
data.
52

114. Shameless plug
‣ Enrise MySQL training/workshop
‣ Day of training into basics/DBA/DBE
‣ When, how much, what?
‣ Depends on interests.

115. Any questions?
∂ QUESTIONS?
http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg

116. ‣ THANK YOU FOR YOUR ATTENTION
‣ Please rate my talk: http://joind.in/talk/view/2947
55