For way too many application developers, SQL equates that SQL-92 they learned in college. Fortunately, SQL has seen many additions over the past 30 years.
In this presentation, we are going to see what present-day SQL looks like and learn new SQL tricks that even the DBAs will be impressed.
7. @vlad_mihalcea vladmihalcea.com
SQL – ‘70s
1970
E.F. Codd’s from IBM
Relational Model
1974
D. D. Chamberlin & R. F. Boyce
IBM System R
1974
Michael Stonebraker
Berkeley Ingres
1979
Relational Software, Inc.
Oracle V2 Marketing
8. @vlad_mihalcea vladmihalcea.com
SQL – ‘80s
IBM System R
1981
IBM SQL/DS 1983
IBM DB2
1987
IBM DB2 for OS/2
1985
Robert Epstein
Sybase
Berkeley Ingres
1988
Microsoft
Sybase SQL Server
1985
Michael Stonebraker
POSTGRES
9. @vlad_mihalcea vladmihalcea.com
SQL – ‘90s
1995
David Axmark & Michael Widenius
MySQL
Sybase SQL Server
1993
Microsoft
SQL Server
POSTGRES
1996
PostgreSQL
17. @vlad_mihalcea vladmihalcea.com
SQL:92
• In 1992, a major version of the standard was released. The
specification grew from 115 pages to 580.
• SQL:92 adds support for:
• DATE, TIME, TIMESTAMP, INTERVAL, BIT, VARCHAR, NVARCHAR,
• UNION, NATURAL JOIN,
• CASE expressions,
• CHECK constraints,
• Type casting (e.g., CAST (expr AS type))
18. @vlad_mihalcea vladmihalcea.com
SQL:92
• Other SQL:92 features include:
• INFORMATION_SCHEMA,
• Dynamic query execution (e.g., java.sql.Statement). Previously, only
prepared statements were supported.
• Temporary tables,
• Custom isolation levels,
• Database cursors,
• Scalar operations like String concatenation, date manipulation.
20. @vlad_mihalcea vladmihalcea.com
Fetching one-to-many relationship with pagination
We need to get the first 2 post entries
starting with a given string value, along
with all their associated post_comment
child records.
21. @vlad_mihalcea vladmihalcea.com
post
id review post_id
1 SQL:2016 is great! 1
2 SQL:2016 is excellent! 1
3
SQL:20016 is awesome!
1
4 SQL:2011 is great! 2
5 SQL:2011 is excellent! 2
7 SQL:2008 is great! 3
id title
1 SQL:2016
2 SQL:2011
3 SQL:2008
Fetching one-to-many relationship with pagination
post_comment
post_id post_title comment_id comment_review
1 SQL:2016 1 SQL:2016 is great!
1 SQL:2016 2 SQL:2016 is excellent!
1 SQL:2016 3
SQL:20016 is awesome!
Result set
22. @vlad_mihalcea vladmihalcea.com
post
id review post_id
1 SQL:2016 is great! 1
2 SQL:2016 is excellent! 1
3
SQL:20016 is awesome!
1
4 SQL:2011 is great! 2
5 SQL:2011 is excellent! 2
7 SQL:2008 is great! 3
id title
1 SQL:2016
2 SQL:2011
3 SQL:2008
Fetching one-to-many relationship with pagination
post_comment
post_id post_title comment_id comment_review
1 SQL:2016 1 SQL:2016 is great!
1 SQL:2016 2 SQL:2016 is excellent!
1 SQL:2016 3
SQL:20016 is awesome!
2 SQL:2011 4 SQL:2011 is great!
2 SQL:2011 5 SQL:2011 is excellent!
Result set
23. @vlad_mihalcea vladmihalcea.com
JPQL – Join fetch with Pagination
List<Post> posts = entityManager
.createQuery(
"""
select p
from Post p
left join fetch p.comments pc
where p.title like :title
order by p.id, pc.id
""", Post.class)
.setParameter("title", "SQL%")
.setMaxResults(2)
.getResultList();
24. @vlad_mihalcea vladmihalcea.com
JPQL – Join fetch with Pagination
WARN [main]: QueryTranslatorImpl - HHH000104:
firstResult/maxResults specified with collection fetch;
applying in memory!
SELECT
p.id AS id1_0_0_,
p.title AS title3_0_0_,
pc.id AS id1_1_0__,
pc.post_id AS post_id4_1_1_,
pc.review AS review3_1_1_
FROM post p
LEFT OUTER JOIN post_comment pc ON p.id = pc.post_id
WHERE p.title LIKE 'SQL%'
ORDER BY p.id, pc.id
25. @vlad_mihalcea vladmihalcea.com
SQL – Join fetch with Pagination
SELECT
p.id AS post_id,
p.title AS post_title,
pc.id AS comment_id,
pc.review AS comment_review
FROM post p
LEFT JOIN post_comment pc ON p.id = pc.post_id
WHERE p.id IN (
SELECT id
FROM post
WHERE p.title LIKE 'SQL%'
ORDER BY id
LIMIT 2
)
ORDER BY post_id, comment_id
26. @vlad_mihalcea vladmihalcea.com
SQL – Derived tables – Oracle Top N
SELECT t.*
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t
WHERE ROWNUM <= 5
ORDER BY ROWNUM
We must sort the result set prior to assigning the
row ranking numbers.
27. @vlad_mihalcea vladmihalcea.com
SQL – Derived tables – Oracle Top N
SELECT t.*
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t
WHERE ROWNUM <= 5
ORDER BY ROWNUM
The outer query can now limit the result set size.
28. @vlad_mihalcea vladmihalcea.com
SQL – Derived tables – Oracle Next N
SELECT t2.*
FROM (
SELECT t1.*, ROWNUM AS ROW_NUM
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t1
WHERE ROWNUM <= 10
) t2
WHERE ROW_NUM > 5
ORDER BY ROW_NUM
We must sort the result set prior to assigning the
row ranking numbers.
29. @vlad_mihalcea vladmihalcea.com
SQL – Derived tables – Oracle Next N
SELECT t2.*
FROM (
SELECT t1.*, ROWNUM AS ROW_NUM
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t1
WHERE ROWNUM <= 10
) t2
WHERE ROW_NUM > 5
ORDER BY ROW_NUM
The following outer query can now limit the result
set size.
30. @vlad_mihalcea vladmihalcea.com
SQL – Derived tables – Oracle Next N
SELECT t2.*
FROM (
SELECT t1.*, ROWNUM AS ROW_NUM
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t1
WHERE ROWNUM <= 10
) t2
WHERE ROW_NUM > 5
ORDER BY ROW_NUM
The outer-most query can set the offset where we
want to start streaming the result set back to the
DB client.
34. @vlad_mihalcea vladmihalcea.com
SQL:1999
• In 1999, a new major version of the standard was released.
• It added support for:
• Boolean column type,
• WITH CTE (Common Table Expression) queries and WITH RECURSIVE queries
for hierarchic queries,
• ROLLUP, CUBE, GROUPING SETS for GROUP BY,
• Initial support for ARRAY types (e.g., UNNEST)
35. @vlad_mihalcea vladmihalcea.com
Pyramid of Doom
SELECT t2.*
FROM (
SELECT t1.*, ROWNUM AS ROW_NUM
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t1
WHERE ROWNUM <= 10
) t2
WHERE ROW_NUM > 5
ORDER BY ROW_NUM
37. @vlad_mihalcea vladmihalcea.com
SQL – CTE
WITH
p AS (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
),
p_limit AS (
SELECT p.*, ROWNUM AS ROW_NUM
FROM p
WHERE ROWNUM <= 10
)
SELECT title
FROM p_limit
WHERE ROW_NUM > 5
ORDER BY ROWNUM
38. @vlad_mihalcea vladmihalcea.com
SQL – CTE
WITH
p AS (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
),
p_limit AS (
SELECT p.*, ROWNUM AS ROW_NUM
FROM p
WHERE ROWNUM <= 10
)
SELECT title
FROM p_limit
WHERE ROW_NUM > 5
ORDER BY ROWNUM
39. @vlad_mihalcea vladmihalcea.com
SQL – CTE
WITH
p AS (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
),
p_limit AS (
SELECT p.*, ROWNUM AS ROW_NUM
FROM p
WHERE ROWNUM <= 10
)
SELECT title
FROM p_limit
WHERE ROW_NUM > 5
ORDER BY ROWNUM
42. @vlad_mihalcea vladmihalcea.com
Fetch the top 3 comment hierarchies by total score
We need to get the top 3 post_comment
hierarchies based on their total
post_comment score.
53. @vlad_mihalcea vladmihalcea.com
Fetching hierarchical data – Nth level
This approach of merging manually (using
UNION ALL) the result sets of each
individual level is cumbersome and requires
us knowing the number of levels upfront.
54. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
)
The result set table built recursively
55. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
)
We want to propagate the root_id
to all comments belonging to the
same hierarchy.
56. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
SELECT
id, id, post_id, parent_id, review, created_on, score
FROM post_comment
WHERE post_id = 1 AND parent_id IS NULL
)
Anchor member – the 1st level
57. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
SELECT
id, id, post_id, parent_id, review, created_on, score
FROM post_comment
WHERE post_id = 1 AND parent_id IS NULL
)
Anchor member – the 1st level
58. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
SELECT
id, id, post_id, parent_id, review, created_on, score
FROM post_comment
WHERE post_id = 1 AND parent_id IS NULL
UNION ALL
SELECT
pc.id, pcs.root_id, pc.post_id, pc.parent_id, pc.review, pc.created_on, pc.score
FROM post_comment pc
INNER JOIN post_comment_score pcs ON pc.parent_id = pcs.id
)
Recursive member – the 2nd to Nth level
59. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
SELECT
id, id, post_id, parent_id, review, created_on, score
FROM post_comment
WHERE post_id = 1 AND parent_id IS NULL
UNION ALL
SELECT
pc.id, pcs.root_id, pc.post_id, pc.parent_id, pc.review, pc.created_on, pc.score
FROM post_comment pc
INNER JOIN post_comment_score pcs ON pc.parent_id = pcs.id
)
Joining the current and previous levels
60. @vlad_mihalcea vladmihalcea.com
SQL – Recursive CTE
WITH RECURSIVE
post_comment_score(id, root_id, post_id, parent_id, review, created_on, score) AS (
SELECT
id, id, post_id, parent_id, review, created_on, score
FROM post_comment
WHERE post_id = 1 AND parent_id IS NULL
UNION ALL
SELECT
pc.id, pcs.root_id, pc.post_id, pc.parent_id, pc.review, pc.created_on, pc.score
FROM post_comment pc
INNER JOIN post_comment_score pcs ON pc.parent_id = pcs.id
)
SELECT
id, parent_id, root_id, review, created_on, score
FROM post_comment_score
Result set projection query
69. @vlad_mihalcea vladmihalcea.com
SQL:2003
• SQL:2003 was a minor version of SQL:1999.
• It added support for:
• Window Functions,
• MERGE statement
• SEQUNCE generator and IDENTITY column type,
• XML column type
71. @vlad_mihalcea vladmihalcea.com
Summing up comment scores by root_id
total_score_comment
AS (
SELECT
id,
parent_id,
review,
created_on,
score,
SUM(score) OVER (
PARTITION BY root_id
) AS total_score
FROM post_comment_score
),
82. @vlad_mihalcea vladmihalcea.com
Ranking comment total scores
total_score_ranking
AS (
SELECT
id,
parent_id,
review,
created_on,
score,
total_score,
DENSE_RANK() OVER (
ORDER BY total_score DESC
) ranking
FROM total_score_comment
),
89. @vlad_mihalcea vladmihalcea.com
Filtering by the comment total score
SELECT
id,
parent_id,
review,
created_on,
score,
total_score
FROM total_score_ranking
WHERE
ranking <= 3
ORDER BY
total_score DESC,
id ASC
93. @vlad_mihalcea vladmihalcea.com
SQL:2003
• SQL:2003 was a minor version of SQL:1999.
• It added support for:
• Window Functions,
• MERGE statement
• SEQUNCE generator and IDENTITY column type,
• XML column type
95. @vlad_mihalcea vladmihalcea.com
Oracle MERGE – Ignore on constraint violation
MERGE INTO book
USING (SELECT 1 FROM dual) ON (id = 1)
WHEN NOT MATCHED THEN
INSERT (
id,
title,
isbn
)
VALUES (
1,
'High-Performance Java Persistence',
'978-9730228236'
)
96. @vlad_mihalcea vladmihalcea.com
Oracle MERGE – UPDATE on constraint violation
MERGE INTO book
USING (SELECT 1 FROM dual) ON (id = 1)
WHEN MATCHED THEN
UPDATE SET
title = 'High-Performance Java Persistence, 2nd Edition',
isbn = '978-9730228236'
WHEN NOT MATCHED THEN
INSERT (
id,
title,
isbn
)
VALUES (
1,
'High-Performance Java Persistence',
'978-9730228236'
)
98. @vlad_mihalcea vladmihalcea.com
SQL:2008
• SQL:2008 added support for:
• TRUNCATE TABLE,
• Multiple WHEN clauses in CASE expressions,
• INSTEAD OF database triggers (override INSERT, UPDATE, DELETE),
• XQuery regular expression/pattern matching,
• Derived column list to override a derived table column names,
• Standard pagination using:
• FETCH FIRST N ROWS ONLY
• OFFSET M ROWS
99. @vlad_mihalcea vladmihalcea.com
SQL – Oracle legacy Top N
SELECT t.*
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t
WHERE ROWNUM <= 5
ORDER BY ROWNUM
101. @vlad_mihalcea vladmihalcea.com
SQL – Oracle legacy Next N
SELECT t2.*
FROM (
SELECT t1.*, ROWNUM AS ROW_NUM
FROM (
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
) t1
WHERE ROWNUM <= 10
) t2
WHERE ROW_NUM > 5
ORDER BY ROW_NUM
102. @vlad_mihalcea vladmihalcea.com
SQL – Standard Next N
SELECT title
FROM post
ORDER BY created_on DESC, id DESC
OFFSET 5 ROWS
FETCH NEXT 5 ROWS ONLY
104. @vlad_mihalcea vladmihalcea.com
SQL:2016
• SQL: 2016 added support for:
• JSON,
• MATCH_RECOGNIZE (Row Pattern Recognition),
• LISTAGG (aggregate multiple values to a delimited string value),
• DECFLOAT column type (e.g., decimal floating arithmetic),
• Date and Time formatting (e.g., CAST(.. AS .. FORMAT ..),
EXTRACT(.. FROM <datetime>))
105. @vlad_mihalcea vladmihalcea.com
• Supported databases:
• Oracle 12c
• SQL Server 2016
• PostgreSQL 9.2
• MySQL 5.6
• PostgreSQL and MySQL offer non-standard alternatives.
JSON
107. @vlad_mihalcea vladmihalcea.com
MySQL – Get scalar attribute
SELECT
id,
CAST(
properties -> '$.price'
AS DECIMAL(4, 2)
) AS price
FROM book
WHERE
properties -> '$.title' = 'High-Performance Java Persistence'
Without casting to DECIMAL, the result will be
returned as a String.
108. @vlad_mihalcea vladmihalcea.com
MySQL – Get scalar attribute
id isbn properties
1 978-9730228236
{
"title":"High-Performance Java Persistence",
"author":"Vlad Mihalcea",
"publisher":"Amazon",
"price":44.99,
"reviews":[
…
]
}
book
SELECT
id,
CAST(
properties -> '$.price'
AS DECIMAL(4, 2)
) AS price
FROM book
WHERE
properties -> '$.title' =
'High-Performance Java Persistence'
id price
1 44.99 Result set
110. @vlad_mihalcea vladmihalcea.com
MySQL – Get JSON object
SELECT
id,
properties -> '$.reviews' AS reviews
FROM book
WHERE
isbn = '978-9730228236'
id isbn properties
1 978-9730228236
{
"title":"High-Performance Java Persistence",
"author":"Vlad Mihalcea",
"publisher":"Amazon",
"price":44.99,
"reviews":[
{
"reviewer":"Cristiano",
"review":"Excellent book to understand Java Persistence",
"date":"2017-11-14",
"rating":5
},
{
"reviewer":"T.W",
"review":"The best JPA ORM book out there",
"date":"2019-01-27",
"rating":5
},
{
"reviewer":"Shaikh",
"review":"The most informative book",
"date":"2016-12-24",
"rating":4
}
]
}
book
id reviews
1
[
{..},
{..},
{..}
] Result set
112. @vlad_mihalcea vladmihalcea.com
SELECT r.*
FROM
book, JSON_TABLE(
properties,
'$.reviews[*]'
COLUMNS(
reviewer VARCHAR(4000) PATH '$.reviewer',
review VARCHAR(4000) PATH '$.review',
review_date DATETIME PATH '$.date’,
rating INT PATH '$.rating'
)
) AS r
WHERE isbn = '978-9730228236'
MySQL – JSON to SQL table
"reviews":[
{
"reviewer":"Cristiano",
"review":"Excellent book to understand Java Persistence",
"date":"2017-11-14",
"rating":5
},
{
"reviewer":"T.W",
"review":"The best JPA ORM book out there",
"date":"2019-01-27",
"rating":5
},
{
"reviewer":"Shaikh",
"review":"The most informative book",
"date":"2016-12-24",
"rating":4
}
]
reviewer review review_date rating
book
113. @vlad_mihalcea vladmihalcea.com
SELECT r.*
FROM
book, JSON_TABLE(
properties,
'$.reviews[*]'
COLUMNS(
reviewer VARCHAR(4000) PATH '$.reviewer',
review VARCHAR(4000) PATH '$.review',
review_date DATETIME PATH '$.date’,
rating INT PATH '$.rating'
)
) AS r
WHERE isbn = '978-9730228236'
MySQL – JSON to SQL table
"reviews":[
{
"reviewer":"Cristiano",
"review":"Excellent book to understand Java Persistence",
"date":"2017-11-14",
"rating":5
},
{
"reviewer":"T.W",
"review":"The best JPA ORM book out there",
"date":"2019-01-27",
"rating":5
},
{
"reviewer":"Shaikh",
"review":"The most informative book",
"date":"2016-12-24",
"rating":4
}
]
reviewer review review_date rating
Cristiano
Excellent book to understand Java
Persistence
14-Nov-2017 5
book
114. @vlad_mihalcea vladmihalcea.com
SELECT r.*
FROM
book, JSON_TABLE(
properties,
'$.reviews[*]'
COLUMNS(
reviewer VARCHAR(4000) PATH '$.reviewer',
review VARCHAR(4000) PATH '$.review',
review_date DATETIME PATH '$.date’,
rating INT PATH '$.rating'
)
) AS r
WHERE isbn = '978-9730228236'
MySQL – JSON to SQL table
"reviews":[
{
"reviewer":"Cristiano",
"review":"Excellent book to understand Java Persistence",
"date":"2017-11-14",
"rating":5
},
{
"reviewer":"T.W",
"review":"The best JPA ORM book out there",
"date":"2019-01-27",
"rating":5
},
{
"reviewer":"Shaikh",
"review":"The most informative book",
"date":"2016-12-24",
"rating":4
}
]
reviewer review review_date rating
Cristiano
Excellent book to understand Java
Persistence
14-Nov-2017 5
T.W The best JPA ORM book out there 27-Jan-2019 5
book
115. @vlad_mihalcea vladmihalcea.com
SELECT r.*
FROM
book, JSON_TABLE(
properties,
'$.reviews[*]'
COLUMNS(
reviewer VARCHAR(4000) PATH '$.reviewer',
review VARCHAR(4000) PATH '$.review',
review_date DATETIME PATH '$.date’,
rating INT PATH '$.rating'
)
) AS r
WHERE isbn = '978-9730228236'
MySQL – JSON to SQL table
"reviews":[
{
"reviewer":"Cristiano",
"review":"Excellent book to understand Java Persistence",
"date":"2017-11-14",
"rating":5
},
{
"reviewer":"T.W",
"review":"The best JPA ORM book out there",
"date":"2019-01-27",
"rating":5
},
{
"reviewer":"Shaikh",
"review":"The most informative book",
"date":"2016-12-24",
"rating":4
}
]
reviewer review review_date rating
Cristiano
Excellent book to understand Java
Persistence
14-Nov-2017 5
T.W The best JPA ORM book out there 27-Jan-2019 5
Shaikh The most informative book 24-Dec-2016 4book