MERGE SQL Statement: Lesser Known Facets

1. blog.sqlora.com@Andrej_SQL
MERGE SQL Statement: Lesser Known Facets
Andrej Pashchenko

2. About me
• Working at Trivadis Germany, Düsseldorf
• Focusing on Oracle:
• Data Warehousing
• Application Development
• Application Performance
• Course instructor „Oracle New Features for
Developers“
@Andrej_SQL blog.sqlora.com

5. • MERGE is a part of SQL 2003 and
has been introduced in Oracle 9i
• Since then, the MERGE has been
well adopted and widely used,
but sometimes still has some
confusing or unexpected
behavior

6. ORA-30926

7. ORA-30926
• ORA-30926 is definitely the most confusing error related to MERGE
• The error description is somewhat confusing too:
• One of the reasons is clearly documented:

8. • Create an „overlaping“ bonus list and try to merge it in employee table
INSERT INTO scott.bonus (ename, job, sal, comm)
SELECT ename, job, sal, sal*0.2 comm
FROM scott.emp
WHERE deptno = 30;
INSERT INTO scott.bonus (ename, job, sal, comm)
SELECT ename, job, sal, sal*0.1 comm
FROM scott.emp
WHERE job = 'MANAGER’;
SQL> MERGE INTO scott.emp e
2 USING scott.bonus b
3 ON (b.ename = e.ename)
4 WHEN MATCHED THEN UPDATE set e.comm = b.comm;
USING scott.bonus b
*
ERROR at line 2:
ORA-30926: unable to get a stable set of rows in the source tables
ORA-30926 - Example
Sales department 20%
Each manager 10%
But SALES also has a manager:
BLAKE will be updated twice

9. • Check the duplicates in the source with respect to the ON-keys:
SQL> MERGE INTO scott.emp e
2 USING scott.bonus b
3 ON (b.ename = e.ename)
4 WHEN MATCHED THEN
5 UPDATE SET e.comm = b.comm;
USING scott.bonus b
*
ERROR at line 2:
ORA-30926: unable to get a stable
set of rows in the source tables
ORA-30926 - How to find the problem?
SQL> SELECT ename
2 FROM scott.bonus b
3 GROUP BY ename
4 HAVING COUNT(*) > 1;
ENAME
----------
BLAKE
SQL>SELECT ename, MAX(comm) comm FROM scott.bonus b GROUP BY ename;
• Find the correct way to avoid duplicates. Often this is a business question, e.g. use MAX or
SUM:

10. • Fix the problem in the source data or directly in your query:
SQL> MERGE INTO scott.emp e
2 USING (SELECT ename, MAX(comm) comm
3 FROM scott.bonus b
4 GROUP BY ename) b
5 ON (b.ename = e.ename)
6 WHEN MATCHED THEN UPDATE set e.comm = b.comm;
8 rows merged.
ORA-30926 Fixing the problem
• What does the documentation say about ORA-30926:
ORA-30926: unable to get a stable set of rows in the source tables
Cause: A stable set of rows could not be got because of large dml activity or a non-
deterministic where clause.
Action: Remove any non-deterministic where clauses and reissue the dml.

11. The whole execution three times?
• Have you noticed that the execution takes much longer if you get ORA-30926?
SQL> MERGE INTO scott.emp e
2 USING scott.bonus b
3 ON (b.ename = e.ename)
4 WHEN MATCHED THEN UPDATE SET e.comm = b.comm;
USING scott.bonus b
*
ERROR at line 2:
ORA-30926: unable to get a stable set of rows in the source tables
-----------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows |
-----------------------------------------------------------------
| 0 | MERGE STATEMENT | | 3 | | 0 |
| 1 | MERGE | EMP | 3 | | 0 |
| 2 | VIEW | | 3 | | 21 |
|* 3 | HASH JOIN | | 3 | 9 | 21 |
| 4 | TABLE ACCESS FULL| BONUS | 3 | 9 | 27 |
| 5 | TABLE ACCESS FULL| EMP | 3 | 14 | 26 |
-----------------------------------------------------------------

12. Write Consistency
and DML restarts

13. Write Consistency and DML Restarts (UPDATE)
Session 1 Session 2
SQL> UPDATE emp
2 SET sal = 1000
3 WHERE ename = 'JAMES';
t1
SQL> UPDATE emp
2 SET comm = nvl(comm,0) + 1000
3 WHERE sal < 1000;
t2
COMMIT;t3
t4
SQL> SELECT ename, sal, comm ENAME SAL COMM
FROM emp WHERE sal < 1000; ---------- ---------- ----------
SMITH 800
JAMES 950
ENAME SAL COMM
---------- ---------- ----------
SMITH 800 1000
JAMES 1000
Cannot update JAMES‘s row an waits
Session 2 can now update JAMES, but JAMES‘s
salary is not less than 1000 anymore

14. Write Consistency and DML Restart
Get SCN1
Identify rows to be updated
in consistent mode (per SCN1)
Get row in
current mode
tracked columns
unchanged?
Rollback all changes
made so far
Request new
SCN
SELECT FOR UPDATE
(current mode)
Identify rows to be updated
in consistent mode (new SCN)
tracked columns
unchanged?
Update locked rows
Update the row
Yes
YesNo
Up to
5000
times
No
Tracked: columns in
WHERE and :OLD, :NEW
values in BEFORE EACH
ROW trigger

15. Write Consistency and DML Restarts (MERGE)
Session 1 Session 2
SQL> UPDATE emp
2 SET sal = 1000
3 WHERE ename = 'JAMES';
t1
SQL> MERGE INTO emp t
2 USING (SELECT 1000 comm FROM dual) q
3 ON (t.sal < 1000)
4 WHEN MATCHED THEN UPDATE
5 SET t.comm = nvl(t.comm,0)+q.comm;
t2
COMMIT;t3
t4
ENAME SAL COMM
---------- ---------- ----------
SMITH 800 1000
JAMES 1000 1000
Waits for locked row
Session 2 can now update JAMES, but JAMES‘s
salary is not less than 1000 anymore

16. Write Consistency and DML Restarts (MERGE)
Session 1 Session 2
SQL> UPDATE emp
2 SET sal = 1000
3 WHERE ename = 'JAMES';
t1
SQL> MERGE INTO emp t
2 USING (SELECT 1000 comm FROM dual) q
3 ON (t.sal < 1000)
4 WHEN MATCHED THEN UPDATE
5 SET t.comm = nvl(t.comm,0)+q.comm
6 WHERE (t.sal < 1000) ;
t2
COMMIT;t3
t4
ENAME SAL COMM
---------- ---------- ----------
SMITH 800 1000
JAMES 1000
Waits for locked row

17. Write Consistency and DML Restarts (MERGE)
Session 1 Session 2
SQL> UPDATE emp
2 SET comm = 500
3 WHERE ename = 'JAMES';
t1
SQL> MERGE INTO emp t
2 USING (SELECT 1000 comm FROM dual) q
3 ON (t.sal < 1000)
4 WHEN MATCHED THEN UPDATE
5 SET t.comm = nvl(t.comm,0)+ q.comm;
t2
COMMIT;t3
t4
ENAME SAL COMM
---------- ---------- ----------
SMITH 800 1000
JAMES 950 1500
Waits for locked row

18. Write Consistency and DML Restart
• MERGE can show a different behavior
regarding DML restarts
• There was a bug until 18c about not
tracking ON-columns, but it is still
there even in 19c in some cases
• But MERGE is tracking columns in SET
clause thus preventing “lost updates”
during running statements (no replace
for locking strategy in your app)
• In case of DML restart triggers can fire
multiple times, so avoid any non-
transactional logic or autonomous
transactions in triggers!

19. ORA-30926 revisited

20. Write Consistency and DML Restart
• Obviously Oracle is using the same mechanism of mini rollbacks as with write
consistency also to ensure its deterministic behavior
• SET-columns are tracked
• Even within the same session updating the column to another value will be detected
• The MERGE statement will be restarted
• As a result, we can observe the mentioned triple effort: MERGE and than rollback,
SELECT FOR UPDATE and then MERGE again

21. ORA-38104

22. ORA-38104
• Oracle doesn’t allow to update columns used in ON clause
• If you feel like you have to do this, verify your requirements carefully
• Sometimes useful for ad hoc data manipulation, fixing erroneous data, etc.
EMPNO ENAME
7839 KING
7782 CLARK
7934 MILLER
EMPNO ROLE_NAME
7839 DEFAULT
7782 SALES
EMP
EMP_ROLES
• Assign the role
ACCOUNTING to each
employee of deptno=10
• If an employee is
assigned the DEFAULT
role, overwrite this
assignment.
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
MERGE?

23. ORA-38104
• The straightforward approach doesn’t work:
SQL> MERGE INTO emp_roles t
2 USING (SELECT empno, 'ACCOUNTING' role_name, 'DEFAULT' old_role
3 FROM emp
4 WHERE deptno = 10
5 ) q
6 ON (t.empno = q.empno and t.role_name = q.old_role)
7 WHEN MATCHED THEN UPDATE SET t.role_name = q.role_name
8 WHEN NOT MATCHED THEN INSERT VALUES (q.empno, q.role_name) ;
ON (t.empno = q.empno and t.role_name = q.old_role)
*
ERROR at line 6:
ORA-38104: Columns referenced in the ON Clause cannot be updated:
"T"."ROLE_NAME"

24. ORA-38104
• Using WHERE clause instead of ON only seems to work, because the result is wrong:
no new role assignment for MILLER (7782)
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO emp_roles t
2 USING (SELECT empno, 'ACCOUNTING' role_name
3 FROM emp
4 WHERE deptno =10
5 ) q
6 ON (t.empno = q.empno)
7 WHEN MATCHED THEN UPDATE
8 SET role_name = q.role_name
9 WHERE t.role_name = 'DEFAULT'
10 WHEN NOT MATCHED THEN INSERT (empno, role_name)
11 VALUES (q.empno, q.role_name) ;
2 rows merged.

25. ORA-38104 – Using ROWID
• Doing the whole logic in USING subquery and merging on ROWID
• At the price of increased complexity and performance penalty of one more join
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO emp_roles t
2 USING (SELECT r.rowid rid, 'ACCOUNTING' new_role_name
3 , e.empno
4 FROM emp e LEFT JOIN emp_roles r
5 ON e.empno = r.empno
6 AND r.role_name = 'DEFAULT'
7 WHERE e.deptno = 10
8 ) q
9 ON (t.rowid = q.rid )
10 WHEN MATCHED THEN UPDATE
11 SET role_name = q.new_role_name
12 WHEN NOT MATCHED THEN INSERT (empno, role_name)
13 VALUES (q.empno, q.new_role_name) ;
3 rows merged.

26. ORA-38104 – Fooling the Parser
• Using a subquery
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO emp_roles t
2 USING (SELECT empno, 'ACCOUNTING' role_name
3 , 'DEFAULT' old_role
4 FROM emp
5 WHERE deptno = 10
6 ) q
7 ON ( t.empno = q.empno
8 AND (SELECT t.role_name FROM dual) = q.old_role )
9 WHEN MATCHED THEN UPDATE SET role_name = q.role_name
10 WHEN NOT MATCHED THEN INSERT (empno, role_name)
11 VALUES (q.empno, q.role_name) ;
3 rows merged.
Idee: Blog Lukas Eder

27. ORA-38104 – Fooling the Parser
• Using a view as a merge target and hiding a column inside NVL()
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO
2 (SELECT empno, role_name
3 , nvl(role_name,'NEVER') check_role_name
4 FROM emp_roles) t
5 USING (SELECT empno, 'ACCOUNTING' role_name
6 , 'DEFAULT' old_role
7 FROM emp
8 WHERE deptno = 10 ) q
9 ON ( t.empno = q.empno
10 AND t.check_role_name = q.old_role )
11 WHEN MATCHED THEN UPDATE SET role_name = q.role_name
12 WHEN NOT MATCHED THEN INSERT (empno, role_name)
13 VALUES (q.empno, q.role_name) ;
3 rows merged.
Idee: Blog Lukas Eder

28. ORA-38104 – Fooling the Parser
• Using row value expressions
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO emp_roles t
2 USING (SELECT empno, 'ACCOUNTING' role_name
3 , 'DEFAULT' old_role
4 FROM emp
5 WHERE deptno = 10 ) q
6 ON ( 1=2 OR
7 (t.empno, t.role_name) = ((q.empno, q.old_role)) )
8 WHEN MATCHED THEN UPDATE SET role_name = q.role_name
9 WHEN NOT MATCHED THEN INSERT (empno, role_name)
10 VALUES (q.empno, q.role_name) ;
3 rows merged.
Idee: Blog Lukas Eder

29. ORA-38104 – Can be executed multiple times
• All previous examples could only be executed once! This one is “multi-executable”
EMPNO ROLE_NAME
7839 ACCOUNTING
7782 SALES
7782 ACCOUNTING
7934 ACCOUNTING
EMP_ROLES
SQL> MERGE INTO emp_roles t
2 USING (
3 SELECT DISTINCT
4 FIRST_VALUE(r.rowid)
5 OVER(PARTITION BY e.empno
6 ORDER BY DECODE(r.role_name,
7 'ACCOUNTING',1,2)) rid
8 , e.empno, 'ACCOUNTING' new_role_name
9 FROM emp e LEFT JOIN emp_roles r
10 ON e.empno = r.empno
11 AND r.role_name in ('DEFAULT', 'ACCOUNTING')
12 WHERE e.deptno = 10
13 ) q
14 ON (t.rowid = q.rid )
15 WHEN MATCHED THEN UPDATE SET role_name = q.new_role_name
16 WHEN NOT MATCHED THEN INSERT (empno, role_name)
17 VALUES (q.empno, q.new_role_name) ;
3 rows merged.

30. Direct-Path Insert
&
Parallel DML

31. Direct Path and MERGE?
SQL> MERGE /*+ append */
2 INTO tx2 USING tx ON (tx.n = tx2.n)
3 WHEN NOT MATCHED THEN INSERT (N)
VALUES (tx.n) ;
10,000 rows merged.
----------------------------------------
| Id | Operation | Name |
----------------------------------------
| 0 | MERGE STATEMENT | |
| 1 | MERGE | TX2 |
| 2 | VIEW | |
|* 3 | HASH JOIN RIGHT OUTER| |
| 4 | TABLE ACCESS FULL | TX2 |
| 5 | TABLE ACCESS FULL | TX |
----------------------------------------
SQL> SELECT count(*) FROM tx2;
ORA-12838: cannot read/modify an object
after modifying it in parallel
SQL> INSERT /*+ append*/ INTO tx2
2 SELECT n FROM tx
3 WHERE n NOT IN (SELECT n FROM tx2);
10,000 rows inserted.
-----------------------------------------------
| Id | Operation | Name
-----------------------------------------------
| 0 | INSERT STATEMENT |
| 1 | LOAD AS SELECT | TX2
| 2 | OPTIMIZER STATISTICS GATHERING |
|* 3 | HASH JOIN RIGHT ANTI NA |
| 4 | TABLE ACCESS FULL | TX2
| 5 | TABLE ACCESS FULL | TX
-----------------------------------------------
direct path write has happend

32. Space Management with PDML and MERGE?
SQL> MERGE /*+ append parallel*/
2 INTO t_tgt_join t0
3 USING ( SELECT ...
----------------------------------------
| Id | Operation |
----------------------------------------
| 0 | MERGE STATEMENT |
| 1 | PX COORDINATOR |
| 2 | PX SEND QC (RANDOM) |
| 3 | MERGE |
| 4 | PX RECEIVE |
SEGMENT_NAME BLOCKS CNT
------------ ------ -------
T_TGT_JOIN 8 1088
... 7 rows ...
T_TGT_JOIN 128 4647
... 20 rows ...
T_TGT_JOIN 1024 34
30 rows selected.
SQL> INSERT /*+ append parallel */
2 INTO t_tgt_join t0
3 SELECT ...
--------------------------------------------------
|Id | Operation
--------------------------------------------------
| 0 | INSERT STATEMENT
| 1 | PX COORDINATOR
| 2 | PX SEND QC (RANDOM)
| 3 | LOAD AS SELECT (HIGH WATER MARK BROKERED)
| 4 | OPTIMIZER STATISTICS GATHERING
SEGMENT_NAME BLOCKS CNT
------------ ---------- ---------
T_TGT_JOIN 8 1024
T_TGT_JOIN 128 4216
T_TGT_JOIN 256 20
T_TGT_JOIN 384 2
T_TGT_JOIN 512 8
T_TGT_JOIN 640 8
T_TGT_JOIN 768 4
T_TGT_JOIN 896 1
T_TGT_JOIN 1024 134

33. Conclusion
• It‘s easy to find out the reason for ORA-30926
• But be careful when fixing it only by technical methods. Another discussion with business
stakeholders may be necessary.
• Don’t execute heavy batch DML in environments with large user activity.
• Don‘t use any non-transactional logic inside the triggers
• Be careful overcoming ORA-38104 restriction. Rethink your use case if possible.
• Keep in mind a different behavior for PDML and Online Statistics Gathering

34. Links
• Oracle documentation, MERGE
• Tom Kyte about DML restarts I and II
• Ruslan Dautkhanov, Oracle’s Write Consistency
• Lukas Eder about ORA-38104
• MERGE and ORA-30926