Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

MariaDB Temporal Tables Slide 1 MariaDB Temporal Tables Slide 2 MariaDB Temporal Tables Slide 3 MariaDB Temporal Tables Slide 4 MariaDB Temporal Tables Slide 5 MariaDB Temporal Tables Slide 6 MariaDB Temporal Tables Slide 7 MariaDB Temporal Tables Slide 8 MariaDB Temporal Tables Slide 9 MariaDB Temporal Tables Slide 10 MariaDB Temporal Tables Slide 11 MariaDB Temporal Tables Slide 12 MariaDB Temporal Tables Slide 13 MariaDB Temporal Tables Slide 14 MariaDB Temporal Tables Slide 15 MariaDB Temporal Tables Slide 16 MariaDB Temporal Tables Slide 17 MariaDB Temporal Tables Slide 18 MariaDB Temporal Tables Slide 19 MariaDB Temporal Tables Slide 20 MariaDB Temporal Tables Slide 21 MariaDB Temporal Tables Slide 22 MariaDB Temporal Tables Slide 23 MariaDB Temporal Tables Slide 24 MariaDB Temporal Tables Slide 25 MariaDB Temporal Tables Slide 26 MariaDB Temporal Tables Slide 27 MariaDB Temporal Tables Slide 28 MariaDB Temporal Tables Slide 29 MariaDB Temporal Tables Slide 30 MariaDB Temporal Tables Slide 31 MariaDB Temporal Tables Slide 32 MariaDB Temporal Tables Slide 33 MariaDB Temporal Tables Slide 34 MariaDB Temporal Tables Slide 35 MariaDB Temporal Tables Slide 36 MariaDB Temporal Tables Slide 37 MariaDB Temporal Tables Slide 38 MariaDB Temporal Tables Slide 39
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

MariaDB Temporal Tables

Download to read offline

MariaDB 10.3 supports system-versioned tables, or temporal tables. This allows us to query data as they were at any point in time, or how they evolved in a certain time period.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

MariaDB Temporal Tables

  1. 1. MariaDB Temporal Tables Federico Razzoli
  2. 2. € whoami ● Federico Razzoli ● Freelance consultant ● Writing SQL since MySQL 2.23 ● info@federico-razzoli.com ● I love open source, sharing, Collaboration, win-win, etc ● I love MariaDB, MySQL, Postgres, etc ○ Even Db2, somehow
  3. 3. Methods for Versioning Data
  4. 4. Data versioning… why? Several reasons: ● Auditing ● Travel back in time ○ Which / how many products were we selling in Dec 2016? ● Track a row’s history ○ History of the relationship with a customer ● Compare today’s situation with 6 month ago ○ How many EU employees did we lose because of Brexit? ● Statistics on data changes ○ Sales trends ● Find correlations ○ Sales decrease because we invest less in web marketing
  5. 5. Example SELECT * FROM users WHERE id = 24 G *************************** 1. row id: 24 first_name: Jody last_name: Whittaker email: first_lady@doctorwho.co.uk gender: F birth_date: NULL 1 row in set (0.00 sec)
  6. 6. Method 1: track row versions SELECT * FROM user_changes G *************************** 1. row id: 1 first_name: Jody last_name: Whittaker email: first_lady@doctorwho.co.uk gender: F valid_from: 2018-10-07 valid_to: NULL 1 row in set (0.00 sec)
  7. 7. Method 1: track row versions What we can do (easily): ● Undo a column change ● Undo an UPDATE/DELETE ● Get the full state of a row at a given time ● See how often a row changes Harder to do: ● Audit changes ● See how often a value changed over time
  8. 8. Method 2: track field changes SELECT * FROM user_changes G *************************** 1. row id: 1 user_id: 24 field: email old_value: jody@gmail.com new_value: first_lady@doctorwho.co.uk valid_from: 2018-10-07 valid_to: NULL 1 row in set (0.00 sec)
  9. 9. Method 2: track field changes What we can do (easily): ● Undo a column change ● Audit changes ● See how a certain value changed over time Harder to do: ● Undo an UPDATE/DELETE ● Get/restore an old row version ● See how often a row changes over time
  10. 10. System-Versioned Tables They automagically implement the Keep Row Changes method ● You INSERT, DELETE, UPDATE and SELECT data, getting the same results you would get with a regular table ● Old versions of the rows are stored in the same (logical) table ● To get old data, you need to use a special syntaxes, like: SELECT … AS OF TIMESTAMP '2018/01/01 16:30:00';
  11. 11. System-Versioned Tables Implementations
  12. 12. Where are sysver tables implemented? In the proprietary DBMS world: ● Oracle 11g (2007) ● Db2 (2012) ● SQL Server 2016 Sometimes they are called Temporal Tables In Db2, a temporal table can use system-period or application-period
  13. 13. Where are sysver tables implemented? In the open source world: ● PostgreSQL, as an extension ● CockroachDB ● MariaDB 10.3 PostgreSQL and CockroachDB implementations have important limitations
  14. 14. Where are sysver tables implemented? In the NoSQL world: ● In HBase, rows have a version property
  15. 15. System-Versioned Tables In MariaDB
  16. 16. Overview ● Implemented in MariaDB 10.3 (stable since Apr 2017) ● You must have row-start and row-end Generated Columns ○ Type: TIMESTAMP(6) or DATETIME(6) ○ You decide the names ○ These are Invisible Columns (10.3 feature) ● Any storage engine ○ Except CONNECT (MDEV-15968)
  17. 17. ALTER TABLE ● Forbidden by default ○ Changes that only affect metadata also forbidden ○ system_versioning_alter_history='KEEP’ ● ADD COLUMN adds a column that is set to the current DEFAULT value or NULL for all old version rows ○ When was the column added? ● DROP COLUMN also affects old versions of the rows ○ The column’s history is lost ● CHANGE COLUMN also affects old versions of the rows ○ The column’s history is modified
  18. 18. Example CREATE OR REPLACE TABLE employee ( ... valid_from TIMESTAMP(6) GENERATED ALWAYS AS ROW START COMMENT ‘When the row was INSERTed', valid_to TIMESTAMP(6) GENERATED ALWAYS AS ROW END COMMENT 'When row was DELETEd or UPDATEd', PERIOD FOR SYSTEM_TIME (valid_from, valid_to) ) WITH SYSTEM VERSIONING, ENGINE InnoDB;
  19. 19. Adding versioning to an existing table ALTER TABLE employee LOCK = SHARED, ALGORITHM = COPY, ADD COLUMN valid_from TIMESTAMP(6) GENERATED ALWAYS AS ROW START, ADD COLUMN valid_to TIMESTAMP(6) GENERATED ALWAYS AS ROW END, ADD PERIOD FOR SYSTEM_TIME(valid_from, valid_to), ADD SYSTEM VERSIONING ; ● Notice ALGORITHM=COPY and LOCK=SHARED
  20. 20. Querying historical data: point in time SELECT * FROM my_table FOR SYSTEM_TIME ● AS OF TIMESTAMP'2018-10-01 12:00:00' ● FROM '2018-10-01 00:00:00' TO '2018-11-01 00:00:00' ● BETWEEN (NOW() - INTERVAL 1 YEAR) AND NOW() ALL SELECT * FROM my_table FOR SYSTEM_TIME ALL WHERE valid_from < @end AND valid_to > @start;
  21. 21. Querying historical data: point in time SELECT * FROM my_table FOR SYSTEM_TIME ● AS OF TIMESTAMP'2018-10-01 12:00:00' ● AS OF (SELECT valid_from FROM employee WHERE id=50) ● AS OF @some_event_timestamp
  22. 22. Querying historical data: time range SELECT * FROM my_table FOR SYSTEM_TIME ● FROM '2018-10-01 00:00:00' TO '2018-11-01 00:00:00' ● FROM (SELECT ...) TO (SELECT ...) ● FROM @some_event TO @another_event
  23. 23. Querying historical data: time range SELECT * FROM my_table FOR SYSTEM_TIME ● BETWEEN '2018-10-01 00:00:00' AND '2018-11-01 00:00:00' ● BETWEEN (NOW() - INTERVAL 1 YEAR) AND NOW() ● BETWEEN @some_event AND (SELECT ...)
  24. 24. Querying historical data: example INSERT INTO customer (id, first_name, last_name, email) VALUES (1, 'William', 'Hartnell', 'the_first@gmail.com'), (2, 'Tom', 'Baker', 'tom.baker@gmail.com'); SET @beginning_of_time := NOW(6); DELETE FROM customer WHERE id = 1; UPDATE customer SET email = 'tom.baker@hotmail.com' WHERE id=2; INSERT INTO customer (id, first_name, last_name, email) VALUES (3, 'Peter', 'Capaldi', 'capaldi.petey@gmail.com'); SET @twelve_regeneration := NOW(6); INSERT INTO customer (id, first_name, last_name, email) VALUES (4, 'Jody', 'Wittaker', 'jody@gmail.com');
  25. 25. Querying historical data: example SELECT id, first_name, last_name, valid_from, valid_to FROM customer FOR SYSTEM_TIME BETWEEN @beginning_of_time AND @twelve_regeneration; +----+------------+-----------+----------------------------+----------------------------+ | id | first_name | last_name | valid_from | valid_to | +----+------------+-----------+----------------------------+----------------------------+ | 1 | William | Hartnell | 2018-11-04 13:50:33.627753 | 2018-11-04 13:50:33.633414 | | 2 | Tom | Baker | 2018-11-04 13:50:33.627753 | 2018-11-04 13:50:33.638419 | | 2 | Tom | Baker | 2018-11-04 13:50:33.638419 | 2038-01-19 03:14:07.999999 | | 3 | Peter | Capaldi | 2018-11-04 13:50:33.644880 | 2038-01-19 03:14:07.999999 | +----+------------+-----------+----------------------------+----------------------------+
  26. 26. Partitions ● By default, the history is stored together with current data ● You can put the history on separate partitions ● And limit them: ○ By rows number ○ By time
  27. 27. Indexes ● The ROW END column is appended to UNIQUE indexes and the PK ● Other indexes are untouched ○ You may consider adding ROW END to some of your indexes ○ This is a good reason to define temporal columns explicitly
  28. 28. Practical Use Cases
  29. 29. Basic Examples ● First examples take advantage of temporal columns to identify INSERTs, DELETEs and UPDATEs
  30. 30. Get DELETEd rows ● Get canceled orders: SET @eot := '2038-01-19 03:14:07.999999'; SELECT * FROM `order` FOR SYSTEM_TIME ALL WHERE valid_to < @eot; ● Get orders canceled today: SELECT COUNT(*) FROM `order` FOR SYSTEM_TIME WHERE valid_to = @ AND valid_from DATE(NOW()) AND (DATE(NOW()) + INTERVAL 1 DAY);
  31. 31. Get INSERTed rows ● Get orders generated today: SELECT id, MIN(valid_from) AS insert_time FROM `order` FOR SYSTEM_TIME ALL GROUP BY id HAVING DATE(MIN(valid_from)) = DATE(NOW()) ORDER BY MIN(valid_from);
  32. 32. Get UPDATEd rows ● How many times orders were modified: SELECT -- exclude the INSERTions id, (COUNT(*) – 1) AS how_many_edits FROM `order` FOR SYSTEM_TIME ALL -- exclude DELETions WHERE valid_to < @eot GROUP BY id ORDER BY how_many_edits;
  33. 33. Debug mistakes in a single row ● Wrong data has been found in a row. The original version of the row was correct, so we want to know when the mistake (or malicious change) happened
  34. 34. Debug mistakes in a single row ● Find all versions of a row: SELECT id, status, valid_from, valid_to FROM `order` FOR SYSTEM_TIME ALL WHERE id = 24;
  35. 35. Debug mistakes in a single row ● When an order was blocked: SELECT DATE(MIN(valid_from)) AS block_date FROM `order` FOR SYSTEM_TIME ALL WHERE status = 'BLOCKED' AND id = 24;
  36. 36. Debug mistakes in a single row ● Status is wrong. To find out how the problem happened, we want to check the INSERT and all following status changes: SELECT id, status, valid_from, valid_to FROM ( SELECT NOT (status <=> @prev_status) AS status_changed, @prev_status := status, id, status, valid_from, valid_to FROM `order` FOR SYSTEM_TIME ALL WHERE id = 24 ) t WHERE status_changed = 1;
  37. 37. Debug mistakes in a single row ● We want to know if the status is the same as one month ago: SELECT present.id, present.status AS current_status, past.status AS past_status FROM `order` present INNER JOIN `order` FOR SYSTEM_TIME AS OF TIMESTAMP NOW() - INTERVAL 1 MONTH AS past ON present.id = past.id ORDER BY present.id;
  38. 38. Stats on data changes SELECT AVG(amount), STDDEV(amount), MAX(amount), MIN(amount), COUNT(amount) FROM account FOR SYSTEM_TIME ALL WHERE customer_id = 24 AND valid_from BETWEEN '2016-00-00' AND NOW() GROUP BY customer_id;
  39. 39. Questions?

MariaDB 10.3 supports system-versioned tables, or temporal tables. This allows us to query data as they were at any point in time, or how they evolved in a certain time period.

Views

Total views

281

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

5

Shares

0

Comments

0

Likes

0

×