Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Instrumenting Plugins for
Performance Schema
Mark L...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Yea but “Why?”
The Interfaces
An exa...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Yea but “Why?”
• There’s a bunch of great interface...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
The Performance Schema
Event Horizon
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
The Interfaces
• The main interface is within ./inc...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
The Interfaces
• The API is broken down by the type...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
An Example
• Let us dream of an audit plugin
• It w...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
pthread_mutex_t LOCK_client_error;
static int clien...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
FILE *fp;
static int client_error_plugin_init(…)
{
...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
static int client_error_notify(…)
{
…
if (error_eve...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
An Example - Adding Stages
• Now you know where you...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
PSI_stage_info opening_client_error_log_file=
{ 0, ...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
An Example - Adding Stages
• Note that when adding ...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
An Example - Stage Progress
• You can also track pr...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
An Example - Performance Schema Output
15
( SELECT ...
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Question and feedback time!
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to ...
Instrumenting plugins for Performance Schema
Upcoming SlideShare
Loading in …5
×

Instrumenting plugins for Performance Schema

15,947 views

Published on

Instrumenting plugins for Performance Schema

Published in: Technology
  • Login to see the comments

Instrumenting plugins for Performance Schema

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Instrumenting Plugins for Performance Schema Mark Leith Senior Software Development Manager MySQL Enterprise Tools, Oracle Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
  2. 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Program Agenda Yea but “Why?” The Interfaces An example Questions 1 2 3 4
  3. 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Yea but “Why?” • There’s a bunch of great interfaces in to the MySQL Server now, from the well known storage engines, through pre/post parser plugins, auditing, full text search, INFORMATION_SCHEMA tables, UDFs, replication interfaces, and more.. • We’ve made great strides in instrumenting the core server, but if you want to use plugins, and don’t also add to the instrumentation in the new standard ways, this will cause new black holes in your available performance data
  4. 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Performance Schema Event Horizon
  5. 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Interfaces • The main interface is within ./include/mysql/psi/ • The ABI is maintained via ./include/mysql/psi/psi.h • Great Doxygen based docs here: • https://dev.mysql.com/doc/dev/mysql-server/8.0.0/PAGE_PFS_PSI.html
  6. 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | The Interfaces • The API is broken down by the type of instrumentation • Threads, sync points (mutexes, rwlocks, conditions etc.) in include/mysql/psi/mysql_thread.h • File IO in include/mysql/psi/mysql_file.h • Memory in include/mysql/psi/mysql_memory.h • Network IO in include/mysql/psi/mysql_socket.h • These all use the underlying versioned performance schema interfaces via the psi.h ABI
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | An Example • Let us dream of an audit plugin • It will: • Log statements that cause errors to a file • Track some error stats with some global status variables • That requires: • Some file operations • A mutex to protect concurrent access to the variables and file
  8. 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | pthread_mutex_t LOCK_client_error; static int client_error_plugin_init(…) { pthread_mutex_init(&LOCK_client_error, NULL); … } 8 An Example - Mutex init mysql_mutex_t LOCK_client_error; PSI_mutex_key key_LOCK_client_error; static PSI_mutex_info client_error_mutexes[]= { { &key_LOCK_client_error, "LOCK_client_error", PSI_FLAG_GLOBAL } }; static int client_error_plugin_init(…) { const char* category= "client_error"; int count; count= array_elements(client_error_mutexes); mysql_mutex_register(category, client_error_mutexes, count); mysql_mutex_init(key_LOCK_client_error, &LOCK_client_error, MY_MUTEX_INIT_FAST); … } P_S Instrument Info Register within P_S
  9. 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | FILE *fp; static int client_error_plugin_init(…) { if(!(fp = fopen(“client_errors.log”, “a+”))) return true; … } 9 An Example - File init PSI_file_key key_file_client_error_log; static PSI_file_info client_error_files[]= { { &key_file_client_error_log, "client_error_log", 0} }; static int client_error_plugin_init(…) { … count= array_elements(client_error_files); mysql_file_register(category, client_error_files, count); … if (!(client_error_log= mysql_file_create(key_file_client_error_log, “client_errors.log”, 0, O_WRONLY | O_APPEND, MYF(MY_WME)))) } See my_sys.h, in this case, write message on error
  10. 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | static int client_error_notify(…) { … if (error_event->event_subclass == MYSQL_AUDIT_GENERAL_ERROR) { pthread_mutex_lock(&LOCK_client_error); … /* Increment some counters */ /* Create some log line buffer in buff */ … write(fp, (uchar*) buff, len); pthread_mutex_unlock(&LOCK_client_error); } } 10 An Example - Writing to the file static int client_error_notify(…) { … if (error_event->event_subclass == MYSQL_AUDIT_GENERAL_ERROR) { mysql_mutex_lock(&LOCK_client_error); … /* Increment some counters */ /* Create some log line buffer in buff */ … mysql_file_write(client_error_log, (uchar*) buff, len, MYF_RW); mysql_mutex_unlock(&LOCK_client_error); } } Write message on error, or if not all bytes are processed
  11. 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | An Example - Adding Stages • Now you know where you may be waiting for thread synchronisation or file IO (or network IO, I won’t talk about that here, look at the API if you’re doing some Daemon plugin) • You can track more major sections of code with Stages - the thread states that exposed via process lists and profiling interfaces • API is within include/mysql/psi/mysql_stage.h
  12. 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | PSI_stage_info opening_client_error_log_file= { 0, "Opening client error log file", 0 }; PSI_stage_info recording_error_statistics= { 0, "Recording error statistics", 0 }; PSI_stage_info *client_error_stages[]= { &opening_client_error_log_file, &recording_error_statistics }; 12 An Example - Stages static int client_error_log_open() { mysql_set_stage(opening_client_error_log_file.m_key); mysql_mutex_lock(&LOCK_client_error_log); /* Open the file, maybe write some header */ return 0; } static int client_error_notify(…) { … if (error_event->event_subclass == MYSQL_AUDIT_GENERAL_ERROR) { mysql_set_stage(recording_error_statistics.m_key); /* Update stats, write log file etc. */ } return 0; }
  13. 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | An Example - Adding Stages • Note that when adding stages, you are hijacking the current stage • Think about where in the flow your stage can be added • Consider adding a “continuation” stage • This can show code executed after your plugin (or face the blame game)
  14. 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | An Example - Stage Progress • You can also track progress of stages within 5.7+ • Call mysql_set_stage(…) • Then mysql_stage_set_work_estimated(<stage>, <estimate count>) • Then while doing the estimated work, at the right interval: • mysql_stage_set_work_completed(<stage>, <completed count>);
  15. 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | An Example - Performance Schema Output 15 ( SELECT thread_id, nesting_event_id, event_id, event_name, sys.format_time(timer_wait) AS time FROM performance_schema.events_stages_history_long ORDER BY event_id ) UNION ALL ( SELECT thread_id, nesting_event_id, event_id, concat('-> ', event_name), sys.format_time(timer_wait) AS time FROM performance_schema.events_waits_history_long WHERE event_name != 'idle' ORDER BY event_id ) ORDER BY thread_id, event_id; +-----------+------------------+----------+--------------------------------------------------------+-----------+ | thread_id | nesting_event_id | event_id | event_name | time | +-----------+------------------+----------+--------------------------------------------------------+-----------+ | 24 | 427 | 428 | stage/sql/starting | 120.46 us | … | 24 | 428 | 437 | -> wait/synch/rwlock/sql/LOCK_grant | 437.32 ns | | 24 | 427 | 438 | stage/sql/Opening tables | 28.21 us | | 24 | 427 | 440 | stage/client_error/Recording error statistics | 23.14 us | | 24 | 440 | 441 | -> wait/synch/mutex/client_error/LOCK_client_error_log | 203.58 ns | | 24 | 440 | 442 | -> wait/io/file/client_error/client_error_log | 11.25 us | | 24 | 427 | 443 | stage/sql/query end | 3.52 us | | 24 | 443 | 444 | -> wait/synch/mutex/sql/THD::LOCK_query_plan | 158.34 ns | | 24 | 427 | 445 | stage/sql/closing tables | 1.68 us | | 24 | 427 | 446 | stage/sql/freeing items | 50.63 us |
  16. 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Question and feedback time!
  17. 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

×