SlideShare a Scribd company logo
1 of 79
Download to read offline
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js and Oracle Database
Christopher Jones
Data Access Development
Oracle Database
22 May 2019
christopher.jones@oracle.com
@ghrd
New Development Techniques
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5
About Me
Christopher Jones
Product Manger for Oracle Database,
Scripting Language Drivers (and related
technology)
Contact information
christopher.jones@oracle.com
@ghrd
blogs.oracle.com/opal
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
8
Introduction
It Starts and Ends With Connections
Getting Data Out of the Database
Putting Data Into the Database
JSON and Simple Oracle Document Access
What’s coming in node-oracledb 4
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
9
Introduction1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js
• Middle-tier JavaScript runtime
– Built on Chrome’s V8 JavaScript engine
– Lightweight and efficient
– Event-driven, non-blocking I/O model
• Allows one language, front-end and back-end
• npm package ecosystem
– World’s largest repo of open-source libraries
• Open Source
– Under Node.js Foundation
10
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Node.js module for accessing Oracle Database
• Development is under Apache 2.0 license
– GitHub repository for source code
– Ongoing features and maintenance by Oracle
• Installable from npm registry
Users can contribute under the Oracle Contributor Agreement.
Thanks to all who have contributed code, documentation and ideas
11
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12
Under the Covers
app.js
node-oracledb
ODPI-C
Oracle Call Interface
in Oracle Client
libraries
Oracle Net Provides:
• Connectivity, Failover, Encryption,
and more
• Configure with tnsnames.ora,
sqlnet.ora
Oracle Client Provides:
• Data Access, Performance, High
Availability, and more
• Configure with oraaccess.xml
Node.js
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb Features
• Async/Await, Promises, Callbacks and Streams
• SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs
• Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution
• Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams
• Smart mapping between JavaScript and Oracle types with manual mapping also available
• Query results as JavaScript objects or arrays
• Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining
• External Authentication / Privileged Connections / Proxy Connections / Password Changing
• Row Prefetching / Statement Caching / Client Result Caching
• End-to-End Tracing / Edition-Based Redefinition
• Continuous Query Notification
• Simple Oracle Document Access (SODA)
• Oracle Database High Availability Features and Oracle Net Features
13
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14
node-oracledb Classes
Base class
• Get connections or create pools
• Set configuration parameters
Connection Pooling
• Dramatically increases performance
• Built-in pool cache for convenience
SQL and PL/SQL Execution
• Transaction support w/data type conversion
• Bind using JavaScript objects or arrays
Read-consistent, pageable cursor
• Used for large result sets
• Callbacks, Promises or Node.js streams
Large object support
• Stream large LOBs with this class
• Can fetch smaller LOBs as string/buffer
Oracledb
Pool
Connection
ResultSet
Lob
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Starting with node-oracledb
1. Install Node.js
2. Install Oracle Instant Client
3. Create package.json
4. Run npm install
Confidential – Oracle Internal/Restricted/Highly Restricted 16
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb fine print
• Prebuilt node-oracledb 3.1 binary modules exist for:
– Windows, Linux and macOS
– Node.js 6, 8, 10, 11
• Upcoming node-oracledb 4.0 will be for Node.js 8+
• You can also build node-oracledb from source
• LTS strategy
Confidential – Oracle Internal/Restricted/Highly Restricted 17
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
oracledb.getConnection(
{user:"hr", password:"welcome", connectString:"localhost/orclpdb1"},
function(err, connection) {
if (err) {
console.error(err.message);
return;
}
connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`,
[110], { outFormat: oracledb.ARRAY },
function(err, result) {
if (err) {
console.error(err.message);
return;
}
console.log(result.rows);
connection.close(function(err) {
if (err) {
console.error(err.message);
}
});
});
});
18
node-oracledb – Node.js Callback Style
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19
node-oracledb - Node.js 8’s Async/Await
async function getEmployee(empid) {
let conn;
try {
connection = await oracledb.getConnection(
{user: "hr", password: "welcome", connectString: "localhost/orclpdb1" });
const result = await connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`, [empid] );
console.log(result.rows);
} catch (err) {
console.error(err);
} finally {
if (connection) {
try {
await connection.close();
} catch (err) {
console.error(err);
}
}
}
}
$ node select.js
[ [ 60, 'IT' ],
[ 90, 'Executive' ],
[ 100, 'Finance' ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip:
• Using a fixed Oracle time zone helps avoid machine and deployment
differences
process.env.ORA_SDTZ = 'UTC';
Confidential – Oracle Internal/Restricted/Highly Restricted 20
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
21
It Starts and Ends With Connections
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Standalone Connections
const oracledb = require("oracledb");
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"};
let connection = await oracledb.getConnection(dbConfig);
// ... Use connection
await connection.close();
22
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pooled Connections
// Application initialization
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60};
let pool = await oracledb.createPool(dbConfig);
// General runtime
let connection = await pool.getConnection();
// ... Use connection
await connection.close();
// Application termination
await pool.close();
23
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24
Node.js Architecture (not entirely accurate)
Timers
TCP/UDP
Evented
File I/O
DNS
User Code
Thread Pool
Async API CallsMain Thread
Event / Callback Queue
Callback
Functions
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• oracledb.createPool({ . . .,
poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . .
– 10 users access the app
• Result: slower than expected throughput. May see deadlocks
– In-use connections wait for DB responses so they hold a thread in use
– Node.js has maximum 4 worker threads by default (upper limit is 128)
• Solution: increase Node.js thread limit before Node.js threadpool starts:
export UV_THREADPOOL_SIZE=10
– May want more threads than connections, to do non-database work
25
Scenario 1: Want to Allow 10 Concurrent DB Users
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– UV_THREADPOOL_SIZE=4
– App opens one connection
– promise.all on 1 x long running SELECT and 3 x INSERTs
• Result
– Each execute() gets thread from worker thread pool
– But the connection can only do one thing
• So the query will block the inserts, blocking the threads from doing other work
• Solution
– Keep control in the JavaScript layer
26
Scenario 2: promise.all
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– Each iteration of promise.all gets its own connection
– Each connection used to insert some data
• Can the database even handle (lots of) multiple connections?
– You have to balance workload expectations with actual resources available
• Transactional consistency not possible
• Data load solution shown in later slides!
27
Scenario 3: Parallelizing Data Load
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid Connection Storms on DB Server
• Set poolMin = poolMax:
oracledb.createPool(
{ user: "hr", password: "welcome", connectString: "localhost/XE”,
poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 })
• Or use Oracle Net Listener rate limit settings
– CONNECTION_RATE_listener name
– RATE_LIMIT
28
Node.js DB
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Basic High Availability
• Configure operating system network TCP timeouts
• Configure Oracle Net options on the DB and/or node-oracledb side e.g.
– SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT,
SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc.
• connection.break()
– may need: DISABLE_OOB=on if firewall blocks out-of-band breaks
• connection.callTimeout with 18c Oracle Client libraries
29
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Immediately Usable Connections
• Connections in pools can become unusable due to network dropouts etc
• Use 19, 18, or 12.2 Oracle Client libraries
– these have an always-on, network check during pool.getConnection()
• Tune createPool()’s pingInterval setting
– Round-trip ping for detecting session issues
Confidential – Oracle Internal/Restricted/Highly Restricted 31
c = p.getConnection()
c.execute() // error
c = p.getConnection()
c.execute() // error
X
X
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
poolPingInterval
poolPingInterval Behavior at pool.getConnection()
< 0 Never check for connection aliveness
= 0 Always check for each pool.getConnection()
> 0
(default is 60)
Checks aliveness with round-trip ping if the connection has been
idle in the pool (not "checked out" to the application by
getConnection()) for at least poolPingInterval seconds.
Confidential – Oracle Internal/Restricted/Highly Restricted 32
If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Usable Connections
• Always do error checking and recovery after execute()
– network failures could occur anytime after getConnection()
– Could disable poolPingInterval to get ultimate scalability
• Avoid firewall timeouts and DBAs killing “idle” sessions
– node-oracledb ping features may mask these problems
• scalability impact
– Use AWR to check connection rate
Confidential – Oracle Internal/Restricted/Highly Restricted 33
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34
Use the pool cache to access pool by name
Pro Tip: No need to pass a pool around
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"};
/* pool = */ await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection("mypool");
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1};
await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection();
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection State
• Many applications set connection state
– e.g. using ALTER SESSION commands to set the date format, or a time zone
– For all practical purposes, a 'session' is the same as a 'connection’
• Pooled connections retain state after being released back to the pool
– Next user of the connection will see the same state
– But will the next user get the same connection or a brand new one?
Confidential – Oracle Internal/Restricted/Highly Restricted 35
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid unnecessary ALTER SESSIONs
connection = await oracledb.getConnection(dbConfig);
connection.execute(`ALTER SESSION . . . `);
Use a connection callback function to set session state
Confidential – Oracle Internal/Restricted/Highly Restricted 36
TIP
X
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
await oracledb.createPool(
{user: un, password: pw, connectString: cs, sessionCallback: initSession});
// Called only when the pool selects a brand new, never-before used connection.
// Called before pool.getConnection() returns.
function initSession(connection, requestedTag, cb) {
connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb);
}
. . .
connection = await oracledb.getConnection(); // get connection in UTC
let result = await connection.execute(sql, binds, options);
await connection.close();
See examples/sessionfixup.js
Confidential – Oracle Internal/Restricted/Highly Restricted 37
Scenario 1: when all connections should have the same state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
Confidential – Oracle Internal/Restricted/Highly Restricted 38
Scenario 1: when all connections should have the same state
Micro-service where each service invocation does one query
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
With a callback 4 1000 1000 4 1000 1004
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Connection Pool sessionCallback
Avoid round-trips
function initSession(connection, requestedTag, cb) {
connection.execute(
`begin
execute immediate
'alter session set nls_date_format = ''YYYY-MM-DD''
nls_language = AMERICAN';
execute immediate
' . . . '; -- other SQL
end;`,
cb);
}
Confidential – Oracle Internal/Restricted/Highly Restricted 39
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Set a connection tag to represent connection state
connection = await pool.getConnection();
// Set state and update the tag
connection.execute(`ALTER SESSION . . . `, cb);
connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’;
await connection.close();
Confidential – Oracle Internal/Restricted/Highly Restricted 40
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Requesting an already tagged pooled connection
conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'});
When using a sessionCallback this will:
– Select an existing connection in the pool that has the tag USER_TZ=UTC
• sessionCallback is NOT called
– Or, select a new, previously unused connection in the pool (which will have no tag)
• sessionCallback is called
– Or, select a previously used connection with a different tag
• The existing session state and tag are cleared
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 41
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
conn = await oracledb.getConnection({poolAlias:'default’,
tag:'USER_TZ=UTC', matchAnyTag: true});
• This will:
– Select an existing connection in the pool that has the requested tag
• sessionCallback is NOT called
– Or, select another connection in the pool
• Any existing connection state and tag are retained
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 42
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Confidential – Oracle Internal/Restricted/Highly Restricted 43
Scenario 2: when connections need differing state
function initSession(connection, requestedTag, cb) {
console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`);
// Parse requestedTag and connection.tag to decide what state to set
. . .
// Set the state
connection.execute(`ALTER SESSION SET . . .`,
(err) => {
connection.tag = requestedTag; // Update to match the new state
cb(err); // Call cb() last
});
}
See examples/sessiontagging1.js and examples/sessiontagging2.js
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
End Connections When No Longer Needed
• Releasing connections is:
– Required for pooled connections
– Best practice for standalone connections
• Don’t release connections until you are done with them, otherwise:
– “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537:
TNS:connection closed”, etc.
– Example scenario: promise.all error handler releasing the connection
• Handler is called on first error
– Example scenario: streaming from a Lob instance
45
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Draining and Force Closing
• Closing the pool closes connections cleanly
• May need: DISABLE_OOB=on if firewall blocks out-of-band breaks
46
// close pool only if no connections in use
await pool.close();
// close pool when no connections are in use, or force it closed after 10 seconds
// No new connections can be created but existing connections can be used for 10 seconds
await pool.close(10);
// force the pool closed immediately
await pool.close(0);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
47
Getting Data Out of the Database
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Query Methods
• Direct Fetches: execute("SELECT . . .", . . . )
– All rows are returned in one big memory-limited array, or limited to maxRows
• ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . )
– getRow(): Returns one row on each call until all rows are returned
– getRows(numRows): Returns batches of rows in each call until all rows are returned
• Stream: queryStream("SELECT . . .", . . . )
– Streams rows until all rows are returned
48
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Direct Fetches
• Default query behavior
– Easy to use
• Tune network transfer performance with fetchArraySize
– Memory can incrementally grow when the number of query rows is unknown, or
varies from execution to execution
– A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst
case' of a large number of rows
• Drawbacks of direct fetches:
– One big array of results is needed
– Concatenation of record batches can use more memory than the final array requires
49
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50
ResultSet Fetches
const result = await connection.execute(
`SELECT * FROM bigtable`,
[], // no bind variables
{ resultSet: true }
);
const rs = result.resultSet;
let row;
while ((row = await rs.getRow())) {
console.log(row);
}
await rs.close(); // always close the ResultSet
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Querying Data
• Use row-limiting SQL clauses:
SELECT last_name FROM employees ORDER BY last_name
OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY
• Use Direct Fetches for known small number of rows
– set fetchArraySize to number of rows, if known
– E.g for single row fetches set fetchArraySize to 1
• Use ResultSets or queryStream() for data sets of unknown or big size
– Always close ResultSets
• Cast date and timestamp bind variables in WHERE clauses:
– . . . WHERE cast(:ts as timestamp) < mytimestampcol
– . . . WHERE cast(:dt as date) < mydatecol
51
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Fetching LOBs
• Fetch and bind CLOB and BLOB as String and Buffer for performance
– Only use Lob Stream class for huge LOBs or if memory is limited
const result = await connection.execute(
`SELECT c FROM mylobs WHERE id = :idbv`,
[1],
{ fetchInfo: {"C": {type: oracledb.STRING}} }
);
const clob = result.rows[0][0];
console.log(clob);
52
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
53
Putting Data Into the Database
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Single Row DML is Easy
SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40));
result = await connection.execute(
"INSERT INTO mytable VALUES (:k, :f)",
{ k: 1, f: 'apple' }, // Bind values
{ autoCommit: true }); // Or use connection.commit() later
console.log("Rows inserted: " + result.rowsAffected); // 1
54
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Use Batch Execution for DML
• Executes one DML statement (e.g INSERT, UPDATE) with many data values
• Reduces round trips: "A server round-trip is defined as the trip from the
client to the server and back to the client."
55
executeMany()
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56
Basic Array DML
sql = "INSERT INTO mytable VALUES (:k, :f)";
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ];
options = { autoCommit: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
{ rowsAffected: 2 }
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" },
{ k: null, f: "cherry" }, { k: 3, f: "pear" },
{ k: null, f: "damson" } ];
options = { batchErrors: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
BatchErrors
{ rowsAffected: 3,
batchErrors:
[ { Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 },
{ Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Batch Errors
• Batch errors are in the result, not the error
– Some classes of error will always trigger real errors
• Any autoCommit will be ignored if there are DML errors
– Valid rows will have been inserted and can be explicitly committed
• Attribute rowsAffected shows 3 rows were inserted
• Array of errors, one for each problematic record
– The offset is the data array position (0-based) of the problem record
58
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
{rowsAffected: 7, dmlRowCounts: [ 5, 2 ]}
59
DML Row Counts
// assume 5 apples and 2 bananas exist in the table
sql = "DELETE FROM mytable WHERE fruit = :f";
data = [ { f: "apple" }, { f: " banana" } ];
options = { dmlRowCounts: true,
bindDefs: {
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60
sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2";
binds = [ ["One"], ["Two"], ["Three"] ];
options = {
bindDefs: [
{ type: oracledb.STRING, maxSize: 5 },
{ type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT }
]
};
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
DMLRETURNING
[ [ [ 'AAAmWkAAMAAAAnWAAA' ] ],
[ [ 'AAAmWkAAMAAAAnWAAB' ] ],
[ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark
• Data: for (var i = 0; i < numRows; i++)
data.push([i, "Test " + i]);
• SQL: sql = "INSERT INTO mytable VALUES (:1, :2)";
• Multiple inserts:
for (var i = 0; i < numRows; i++) {
await connection.execute(sql, data[i],
{autoCommit: (i < numRows-1 ? false : true)}); }
• Single insert:
await connection.executeMany(sql, data, {autoCommit: true});
61
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark – Scenario 1
• 1 number,
1 short string
• Local DB
• YMMV
1 10 100 1000 10000 100000
execute() 10 38 265 2,323 23,914 227,727
executeMany() 16 9 20 16 39 361
62
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63
executeMany() Benchmark – Scenario 2
• 1 x NUMBER,
• 3 x VARCHAR2(1000))
• Remote DB
• YMMV
1 10 100 1,000
execute() 57 510 5,032 50,949
executeMany() 57 155 368 2,537
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: executeMany()
• For huge data sets may need to call executeMany() multiple times with
batches of records
– autoCommit: false for first batches
– autoCommit: true for the last batch
• Use SQL*Loader or Data Pump instead, where appropriate
– These were added to Instant Client 12.2
64
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 66
PL/SQL
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Calling PL/SQL is easy
const result = await connection.execute(
`BEGIN
myproc(:i, :io, :o);
END;`,
{
i: 'Chris', // type found from the data. Default dir is BIND_IN
io: { val: 'Jones', dir: oracledb.BIND_INOUT },
o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
});
console.log(result.outBinds);
Confidential – Oracle Internal/Restricted/Highly Restricted 67
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68
SQL> CREATE OR REPLACE PROCEDURE testproc
(p_in IN NUMBER, p_out OUT NUMBER) AS
BEGIN
p_out := p_in * 2;
END;
plsql = "BEGIN testproc(:1, :2); END;";
binds = [ [1], [2], [3] ];
options = {
bindDefs: [
{ type: oracledb.NUMBER },
{ type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
] };
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
CallingPL/SQL
[ [ 2 ], [ 4 ], [ 6 ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
PL/SQL Collection Associative Array (Index-by) Binds
TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN
FORALL i IN INDICES OF p_vals
INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i));
END;
await connection.execute(
"BEGIN myinproc(:idbv, :valbv); END;",
{ idbv: 1234,
valbv: { type: oracledb.NUMBER,
dir: oracledb.BIND_IN,
val: [1, 2, 23, 4, 10] } };
69
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
70
JSON and Simple Oracle Document Access
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
SQL> CREATE TABLE myjsontab (
c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE);
myContent = {name: "Sally", address: {city: "Melbourne"}};
json = JSON.stringify(myContent);
result = await connection.execute(
'insert into myjsontab (c) values (:cbv)',
{ cbv: json }
);
71
Inserting
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
result = await connection.execute(
`select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax
{ cbv: 'Sally' },
{ fetchInfo: {"C": {type: oracledb.STRING } }});
js = JSON.parse(result.rows[0]);
console.log('Name is: ' + js.name);
console.log('City is: ' + js.address.city);
//sql = "select c FROM myjsontab where json_exists(c, '$.address.city')";
72
Fetching
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Simple Oracle Document Access (SODA)
• Set of NoSQL-style APIs to create and access documents
– Documents are often JSON
– Oracle maps SODA calls to managed tables, indexes and sequences
– Query-by-example access makes data operations easy
• SQL is not necessary
– But could be used for advanced analysis
• SODA APIs also exist in Python, PL/SQL, C and Java
73
node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74
node-oracledb SODA Classes
Base SODA class
• Get SODA database
• Manipulate SODA collections
Set of Documents
• Create and find documents
Operation Builder
• Filters and QBE
• Retrieve, replace or remove documents
Document Cursor
• Iterate over retrieved documents
SODA Document
• Document content
• Access as JSON, object or buffer
SodaDatabase
SodaCollection
SodaOperation
SodaDocumentCursor
SodaDocument
Oracledb
Pool
Connection
ResultSet
Lob
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75
Creating Collections
Using SODA in node-oracledb
SQL> grant SODA_APP to cj;
conn = await oracledb.getConnection({user:"cj",....});
soda = await conn.getSodaDatabase();
collection = await soda.createCollection("mycollection");
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76
Inserting Documents
Using SODA in node-oracledb
content = {name: "Anthony", address: {city: "Edmonton"}};
await collection.insertOne(content);
doc = await collection.insertOneAndGet(content);
console.log("Key is:", doc.key);
// JSON (or Object or Buffer)
doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}');
await collection.insertOne(doc2);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Using SODA in node-oracledb
• Recommendation is to turn on oracle.autocommit
– Like with SQL, avoid auto commit when inserting lots of documents
• SODA’s DDL-like operations occur in autonomous transactions
77
Commit Behavior
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78
Operation Builder – find()
Using SODA in node-oracledb
doc = await collection.find().key(key).getOne(); // A SodaDocument
content = doc.getContent(); // A JavaScript object
console.log('Retrieved SODA document as an object:');
console.log(content);
content = doc.getContentAsString(); // A JSON string
console.log('Retrieved SODA document as a string:');
console.log(content);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79
Operation Builder – filter() Query-by-example (QBE)
Using SODA in node-oracledb
// Find all documents with city names starting with 'S'
console.log('Cities starting with S');
documents = await collection.find()
.filter({"address.city": {"$like": "S%"}})
.getDocuments();
for (let i = 0; i < documents.length; i++) {
content = documents[i].getContent();
console.log(' city is: ', content.address.city);
}
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
80
What’s coming in node-oracledb 4
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
What’s coming in node-oracledb 4.0
• Install (compile) current code snapshot with:
npm install oracle/node-oracledb#dev-4.0
• Already landed:
– N-NAPI code refactor improving Node.js version compatibility
– Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers
– Implicit Results
– SODA bulk insert
• Being investigated (no promises!):
– SQL and PL/SQL object binding & queries
• When? Planned CY2019
Confidential – Oracle Internal/Restricted/Highly Restricted 81
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82
AQ ‘RAW’ Queue Demo
Oracle Advanced Queuing
Dequeue:
queue = connection.queue('MYQUEUE');
msg = await queue.deqOne();
await connection.commit();
console.log(msg.payload.toString());
Enqueue:
queue = connection.queue('MYQUEUE');
messageString = 'This is my message';
await queue.enqOne(messageString);
await connection.commit();
BEGIN
dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW');
dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB');
dbms_aqadm.start_queue('MYQUEUE');
END;
/
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83
Wrap up
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Connection management is the basis for good scalability
• Utilize the best query method for the task, and tune it
• Use executeMany() to insert/update/delete multiple records efficiently
• Use SODA’s NoSQL-style data access to simplify applications
84
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Resources
• Mail: christopher.jones@oracle.com
• Twitter: @ghrd, @AnthonyTuininga, @dmcghan
• Blog: https://blogs.oracle.com/opal
• Office Hours: https://tinyurl.com/NodeHours
• Facebook: https://www.facebook.com/groups/oraclescripting/
• Home: https://oracle.github.io/node-oracledb/
• Code: https://github.com/oracle/node-oracledb
85
Node.js and Oracle Database: New Development Techniques

More Related Content

What's hot

WSO2 Identity Server - Getting Started
WSO2 Identity Server - Getting StartedWSO2 Identity Server - Getting Started
WSO2 Identity Server - Getting StartedIsmaeel Enjreny
 
Oracle Office Hours - Exposing REST services with APEX and ORDS
Oracle Office Hours - Exposing REST services with APEX and ORDSOracle Office Hours - Exposing REST services with APEX and ORDS
Oracle Office Hours - Exposing REST services with APEX and ORDSDoug Gault
 
Domain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with RailsDomain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with RailsDeclan Whelan
 
Advanced Reporting And Charting With Oracle Application Express 4.0
Advanced Reporting And Charting With Oracle Application Express 4.0Advanced Reporting And Charting With Oracle Application Express 4.0
Advanced Reporting And Charting With Oracle Application Express 4.0Rinie Romme
 
JavaOne 2014 - Securing RESTful Resources with OAuth2
JavaOne 2014 - Securing RESTful Resources with OAuth2JavaOne 2014 - Securing RESTful Resources with OAuth2
JavaOne 2014 - Securing RESTful Resources with OAuth2Rodrigo Cândido da Silva
 
Fortify On Demand and ShadowLabs
Fortify On Demand and ShadowLabsFortify On Demand and ShadowLabs
Fortify On Demand and ShadowLabsjasonhaddix
 
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Nelson Calero
 
RESTful Services for your Oracle Autonomous Database
RESTful Services for your Oracle Autonomous DatabaseRESTful Services for your Oracle Autonomous Database
RESTful Services for your Oracle Autonomous DatabaseJeff Smith
 
Ruby Programming Language - Introduction
Ruby Programming Language - IntroductionRuby Programming Language - Introduction
Ruby Programming Language - IntroductionKwangshin Oh
 
Building RESTful applications using Spring MVC
Building RESTful applications using Spring MVCBuilding RESTful applications using Spring MVC
Building RESTful applications using Spring MVCIndicThreads
 
Concurrency Utilities in Java 8
Concurrency Utilities in Java 8Concurrency Utilities in Java 8
Concurrency Utilities in Java 8Martin Toshev
 
Oracle APEX Social Login
Oracle APEX Social LoginOracle APEX Social Login
Oracle APEX Social Loginmsewtz
 
APEX Office Print (AOP)
APEX Office Print (AOP)APEX Office Print (AOP)
APEX Office Print (AOP)Dimitri Gielis
 
Odi 12c-getting-started-guide-2032250
Odi 12c-getting-started-guide-2032250Odi 12c-getting-started-guide-2032250
Odi 12c-getting-started-guide-2032250Udaykumar Sarana
 
Wso2 is integration with .net core
Wso2 is   integration with .net coreWso2 is   integration with .net core
Wso2 is integration with .net coreIsmaeel Enjreny
 
Spring boot Under Da Hood
Spring boot Under Da HoodSpring boot Under Da Hood
Spring boot Under Da HoodMichel Schudel
 
Understanding oracle rac internals part 2 - slides
Understanding oracle rac internals   part 2 - slidesUnderstanding oracle rac internals   part 2 - slides
Understanding oracle rac internals part 2 - slidesMohamed Farouk
 

What's hot (20)

WSO2 Identity Server - Getting Started
WSO2 Identity Server - Getting StartedWSO2 Identity Server - Getting Started
WSO2 Identity Server - Getting Started
 
Oracle Office Hours - Exposing REST services with APEX and ORDS
Oracle Office Hours - Exposing REST services with APEX and ORDSOracle Office Hours - Exposing REST services with APEX and ORDS
Oracle Office Hours - Exposing REST services with APEX and ORDS
 
Kotlin
KotlinKotlin
Kotlin
 
Domain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with RailsDomain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with Rails
 
IBS Score
IBS ScoreIBS Score
IBS Score
 
Advanced Reporting And Charting With Oracle Application Express 4.0
Advanced Reporting And Charting With Oracle Application Express 4.0Advanced Reporting And Charting With Oracle Application Express 4.0
Advanced Reporting And Charting With Oracle Application Express 4.0
 
JavaOne 2014 - Securing RESTful Resources with OAuth2
JavaOne 2014 - Securing RESTful Resources with OAuth2JavaOne 2014 - Securing RESTful Resources with OAuth2
JavaOne 2014 - Securing RESTful Resources with OAuth2
 
Fortify On Demand and ShadowLabs
Fortify On Demand and ShadowLabsFortify On Demand and ShadowLabs
Fortify On Demand and ShadowLabs
 
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
Automate Oracle database patches and upgrades using Fleet Provisioning and Pa...
 
RESTful Services for your Oracle Autonomous Database
RESTful Services for your Oracle Autonomous DatabaseRESTful Services for your Oracle Autonomous Database
RESTful Services for your Oracle Autonomous Database
 
Ruby Programming Language - Introduction
Ruby Programming Language - IntroductionRuby Programming Language - Introduction
Ruby Programming Language - Introduction
 
Building RESTful applications using Spring MVC
Building RESTful applications using Spring MVCBuilding RESTful applications using Spring MVC
Building RESTful applications using Spring MVC
 
Rapid Home Provisioning
Rapid Home ProvisioningRapid Home Provisioning
Rapid Home Provisioning
 
Concurrency Utilities in Java 8
Concurrency Utilities in Java 8Concurrency Utilities in Java 8
Concurrency Utilities in Java 8
 
Oracle APEX Social Login
Oracle APEX Social LoginOracle APEX Social Login
Oracle APEX Social Login
 
APEX Office Print (AOP)
APEX Office Print (AOP)APEX Office Print (AOP)
APEX Office Print (AOP)
 
Odi 12c-getting-started-guide-2032250
Odi 12c-getting-started-guide-2032250Odi 12c-getting-started-guide-2032250
Odi 12c-getting-started-guide-2032250
 
Wso2 is integration with .net core
Wso2 is   integration with .net coreWso2 is   integration with .net core
Wso2 is integration with .net core
 
Spring boot Under Da Hood
Spring boot Under Da HoodSpring boot Under Da Hood
Spring boot Under Da Hood
 
Understanding oracle rac internals part 2 - slides
Understanding oracle rac internals   part 2 - slidesUnderstanding oracle rac internals   part 2 - slides
Understanding oracle rac internals part 2 - slides
 

Similar to Node.js and Oracle Database: New Development Techniques

Confoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetConfoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetDave Stokes
 
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareMulti-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareBruno Borges
 
Why to Use an Oracle Database?
Why to Use an Oracle Database? Why to Use an Oracle Database?
Why to Use an Oracle Database? Markus Michalewicz
 
Serverless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsServerless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsDavid Delabassee
 
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...Trivadis
 
Oracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesOracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesChris Muir
 
Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Oracle Developers
 
Serverless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsServerless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsDavid Delabassee
 
Java EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudJava EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudBruno Borges
 
Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Shaun Smith
 
Nonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SENonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SEDmitry Kornilov
 
UNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New FeaturesUNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New Featuresmsewtz
 
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...Miguel Araújo
 
Presentation oracle exalogic elastic cloud
Presentation   oracle exalogic elastic cloudPresentation   oracle exalogic elastic cloud
Presentation oracle exalogic elastic cloudsolarisyougood
 
Oracle Autonomous Database - introducción técnica y hands on lab
Oracle Autonomous Database  - introducción técnica y hands on labOracle Autonomous Database  - introducción técnica y hands on lab
Oracle Autonomous Database - introducción técnica y hands on lab"Diego \"Perico\"" Sanchez
 
Consolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesConsolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesDLT Solutions
 
SmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsSmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsKoppelaars
 

Similar to Node.js and Oracle Database: New Development Techniques (20)

Confoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetConfoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSet
 
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareMulti-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
 
Why to Use an Oracle Database?
Why to Use an Oracle Database? Why to Use an Oracle Database?
Why to Use an Oracle Database?
 
Cloud based database
Cloud based databaseCloud based database
Cloud based database
 
Serverless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsServerless Java - Challenges and Triumphs
Serverless Java - Challenges and Triumphs
 
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
 
Oracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesOracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best Practices
 
Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data
 
Oracle NoSQL
Oracle NoSQLOracle NoSQL
Oracle NoSQL
 
Serverless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsServerless Java Challenges & Triumphs
Serverless Java Challenges & Triumphs
 
Java EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudJava EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The Cloud
 
Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019
 
con8832-cloudha-2811114.pdf
con8832-cloudha-2811114.pdfcon8832-cloudha-2811114.pdf
con8832-cloudha-2811114.pdf
 
Nonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SENonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SE
 
UNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New FeaturesUNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New Features
 
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
 
Presentation oracle exalogic elastic cloud
Presentation   oracle exalogic elastic cloudPresentation   oracle exalogic elastic cloud
Presentation oracle exalogic elastic cloud
 
Oracle Autonomous Database - introducción técnica y hands on lab
Oracle Autonomous Database  - introducción técnica y hands on labOracle Autonomous Database  - introducción técnica y hands on lab
Oracle Autonomous Database - introducción técnica y hands on lab
 
Consolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesConsolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficiencies
 
SmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsSmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing Concepts
 

Recently uploaded

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Mater
 
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)jennyeacort
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf31events.com
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesPhilip Schwarz
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesŁukasz Chruściel
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfStefano Stabellini
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...Akihiro Suda
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions
 

Recently uploaded (20)

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)
 
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprise
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a series
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New Features
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
 

Node.js and Oracle Database: New Development Techniques

  • 1.
  • 2. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js and Oracle Database Christopher Jones Data Access Development Oracle Database 22 May 2019 christopher.jones@oracle.com @ghrd New Development Techniques
  • 3. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5 About Me Christopher Jones Product Manger for Oracle Database, Scripting Language Drivers (and related technology) Contact information christopher.jones@oracle.com @ghrd blogs.oracle.com/opal
  • 4. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. 6
  • 5. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 8 Introduction It Starts and Ends With Connections Getting Data Out of the Database Putting Data Into the Database JSON and Simple Oracle Document Access What’s coming in node-oracledb 4 1 2 3 4 5 6
  • 6. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 9 Introduction1 2 3 4 5 6
  • 7. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js • Middle-tier JavaScript runtime – Built on Chrome’s V8 JavaScript engine – Lightweight and efficient – Event-driven, non-blocking I/O model • Allows one language, front-end and back-end • npm package ecosystem – World’s largest repo of open-source libraries • Open Source – Under Node.js Foundation 10
  • 8. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Node.js module for accessing Oracle Database • Development is under Apache 2.0 license – GitHub repository for source code – Ongoing features and maintenance by Oracle • Installable from npm registry Users can contribute under the Oracle Contributor Agreement. Thanks to all who have contributed code, documentation and ideas 11
  • 9. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12 Under the Covers app.js node-oracledb ODPI-C Oracle Call Interface in Oracle Client libraries Oracle Net Provides: • Connectivity, Failover, Encryption, and more • Configure with tnsnames.ora, sqlnet.ora Oracle Client Provides: • Data Access, Performance, High Availability, and more • Configure with oraaccess.xml Node.js
  • 10. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb Features • Async/Await, Promises, Callbacks and Streams • SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs • Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution • Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams • Smart mapping between JavaScript and Oracle types with manual mapping also available • Query results as JavaScript objects or arrays • Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining • External Authentication / Privileged Connections / Proxy Connections / Password Changing • Row Prefetching / Statement Caching / Client Result Caching • End-to-End Tracing / Edition-Based Redefinition • Continuous Query Notification • Simple Oracle Document Access (SODA) • Oracle Database High Availability Features and Oracle Net Features 13
  • 11. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14 node-oracledb Classes Base class • Get connections or create pools • Set configuration parameters Connection Pooling • Dramatically increases performance • Built-in pool cache for convenience SQL and PL/SQL Execution • Transaction support w/data type conversion • Bind using JavaScript objects or arrays Read-consistent, pageable cursor • Used for large result sets • Callbacks, Promises or Node.js streams Large object support • Stream large LOBs with this class • Can fetch smaller LOBs as string/buffer Oracledb Pool Connection ResultSet Lob
  • 12. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Starting with node-oracledb 1. Install Node.js 2. Install Oracle Instant Client 3. Create package.json 4. Run npm install Confidential – Oracle Internal/Restricted/Highly Restricted 16
  • 13. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb fine print • Prebuilt node-oracledb 3.1 binary modules exist for: – Windows, Linux and macOS – Node.js 6, 8, 10, 11 • Upcoming node-oracledb 4.0 will be for Node.js 8+ • You can also build node-oracledb from source • LTS strategy Confidential – Oracle Internal/Restricted/Highly Restricted 17
  • 14. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | oracledb.getConnection( {user:"hr", password:"welcome", connectString:"localhost/orclpdb1"}, function(err, connection) { if (err) { console.error(err.message); return; } connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [110], { outFormat: oracledb.ARRAY }, function(err, result) { if (err) { console.error(err.message); return; } console.log(result.rows); connection.close(function(err) { if (err) { console.error(err.message); } }); }); }); 18 node-oracledb – Node.js Callback Style
  • 15. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19 node-oracledb - Node.js 8’s Async/Await async function getEmployee(empid) { let conn; try { connection = await oracledb.getConnection( {user: "hr", password: "welcome", connectString: "localhost/orclpdb1" }); const result = await connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [empid] ); console.log(result.rows); } catch (err) { console.error(err); } finally { if (connection) { try { await connection.close(); } catch (err) { console.error(err); } } } } $ node select.js [ [ 60, 'IT' ], [ 90, 'Executive' ], [ 100, 'Finance' ] ]
  • 16. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: • Using a fixed Oracle time zone helps avoid machine and deployment differences process.env.ORA_SDTZ = 'UTC'; Confidential – Oracle Internal/Restricted/Highly Restricted 20 TIP
  • 17. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 21 It Starts and Ends With Connections 1 2 3 4 5 6
  • 18. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Standalone Connections const oracledb = require("oracledb"); let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"}; let connection = await oracledb.getConnection(dbConfig); // ... Use connection await connection.close(); 22 Node.js DB
  • 19. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pooled Connections // Application initialization let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60}; let pool = await oracledb.createPool(dbConfig); // General runtime let connection = await pool.getConnection(); // ... Use connection await connection.close(); // Application termination await pool.close(); 23 Node.js DB
  • 20. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24 Node.js Architecture (not entirely accurate) Timers TCP/UDP Evented File I/O DNS User Code Thread Pool Async API CallsMain Thread Event / Callback Queue Callback Functions
  • 21. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • oracledb.createPool({ . . ., poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . . – 10 users access the app • Result: slower than expected throughput. May see deadlocks – In-use connections wait for DB responses so they hold a thread in use – Node.js has maximum 4 worker threads by default (upper limit is 128) • Solution: increase Node.js thread limit before Node.js threadpool starts: export UV_THREADPOOL_SIZE=10 – May want more threads than connections, to do non-database work 25 Scenario 1: Want to Allow 10 Concurrent DB Users
  • 22. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – UV_THREADPOOL_SIZE=4 – App opens one connection – promise.all on 1 x long running SELECT and 3 x INSERTs • Result – Each execute() gets thread from worker thread pool – But the connection can only do one thing • So the query will block the inserts, blocking the threads from doing other work • Solution – Keep control in the JavaScript layer 26 Scenario 2: promise.all
  • 23. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – Each iteration of promise.all gets its own connection – Each connection used to insert some data • Can the database even handle (lots of) multiple connections? – You have to balance workload expectations with actual resources available • Transactional consistency not possible • Data load solution shown in later slides! 27 Scenario 3: Parallelizing Data Load
  • 24. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid Connection Storms on DB Server • Set poolMin = poolMax: oracledb.createPool( { user: "hr", password: "welcome", connectString: "localhost/XE”, poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 }) • Or use Oracle Net Listener rate limit settings – CONNECTION_RATE_listener name – RATE_LIMIT 28 Node.js DB TIP
  • 25. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Basic High Availability • Configure operating system network TCP timeouts • Configure Oracle Net options on the DB and/or node-oracledb side e.g. – SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT, SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc. • connection.break() – may need: DISABLE_OOB=on if firewall blocks out-of-band breaks • connection.callTimeout with 18c Oracle Client libraries 29 TIP
  • 26. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Immediately Usable Connections • Connections in pools can become unusable due to network dropouts etc • Use 19, 18, or 12.2 Oracle Client libraries – these have an always-on, network check during pool.getConnection() • Tune createPool()’s pingInterval setting – Round-trip ping for detecting session issues Confidential – Oracle Internal/Restricted/Highly Restricted 31 c = p.getConnection() c.execute() // error c = p.getConnection() c.execute() // error X X
  • 27. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | poolPingInterval poolPingInterval Behavior at pool.getConnection() < 0 Never check for connection aliveness = 0 Always check for each pool.getConnection() > 0 (default is 60) Checks aliveness with round-trip ping if the connection has been idle in the pool (not "checked out" to the application by getConnection()) for at least poolPingInterval seconds. Confidential – Oracle Internal/Restricted/Highly Restricted 32 If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
  • 28. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Usable Connections • Always do error checking and recovery after execute() – network failures could occur anytime after getConnection() – Could disable poolPingInterval to get ultimate scalability • Avoid firewall timeouts and DBAs killing “idle” sessions – node-oracledb ping features may mask these problems • scalability impact – Use AWR to check connection rate Confidential – Oracle Internal/Restricted/Highly Restricted 33 TIP
  • 29. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34 Use the pool cache to access pool by name Pro Tip: No need to pass a pool around dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"}; /* pool = */ await oracledb.createPool(dbConfig); connection = await oracledb.getConnection("mypool"); dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1}; await oracledb.createPool(dbConfig); connection = await oracledb.getConnection(); TIP
  • 30. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection State • Many applications set connection state – e.g. using ALTER SESSION commands to set the date format, or a time zone – For all practical purposes, a 'session' is the same as a 'connection’ • Pooled connections retain state after being released back to the pool – Next user of the connection will see the same state – But will the next user get the same connection or a brand new one? Confidential – Oracle Internal/Restricted/Highly Restricted 35 Node.js DB
  • 31. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid unnecessary ALTER SESSIONs connection = await oracledb.getConnection(dbConfig); connection.execute(`ALTER SESSION . . . `); Use a connection callback function to set session state Confidential – Oracle Internal/Restricted/Highly Restricted 36 TIP X
  • 32. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback await oracledb.createPool( {user: un, password: pw, connectString: cs, sessionCallback: initSession}); // Called only when the pool selects a brand new, never-before used connection. // Called before pool.getConnection() returns. function initSession(connection, requestedTag, cb) { connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb); } . . . connection = await oracledb.getConnection(); // get connection in UTC let result = await connection.execute(sql, binds, options); await connection.close(); See examples/sessionfixup.js Confidential – Oracle Internal/Restricted/Highly Restricted 37 Scenario 1: when all connections should have the same state
  • 33. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 Confidential – Oracle Internal/Restricted/Highly Restricted 38 Scenario 1: when all connections should have the same state Micro-service where each service invocation does one query poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 With a callback 4 1000 1000 4 1000 1004
  • 34. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Connection Pool sessionCallback Avoid round-trips function initSession(connection, requestedTag, cb) { connection.execute( `begin execute immediate 'alter session set nls_date_format = ''YYYY-MM-DD'' nls_language = AMERICAN'; execute immediate ' . . . '; -- other SQL end;`, cb); } Confidential – Oracle Internal/Restricted/Highly Restricted 39 TIP
  • 35. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Set a connection tag to represent connection state connection = await pool.getConnection(); // Set state and update the tag connection.execute(`ALTER SESSION . . . `, cb); connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’; await connection.close(); Confidential – Oracle Internal/Restricted/Highly Restricted 40 Scenario 2: when connections need differing state
  • 36. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Requesting an already tagged pooled connection conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'}); When using a sessionCallback this will: – Select an existing connection in the pool that has the tag USER_TZ=UTC • sessionCallback is NOT called – Or, select a new, previously unused connection in the pool (which will have no tag) • sessionCallback is called – Or, select a previously used connection with a different tag • The existing session state and tag are cleared • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 41 Scenario 2: when connections need differing state
  • 37. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging conn = await oracledb.getConnection({poolAlias:'default’, tag:'USER_TZ=UTC', matchAnyTag: true}); • This will: – Select an existing connection in the pool that has the requested tag • sessionCallback is NOT called – Or, select another connection in the pool • Any existing connection state and tag are retained • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 42 Scenario 2: when connections need differing state
  • 38. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Confidential – Oracle Internal/Restricted/Highly Restricted 43 Scenario 2: when connections need differing state function initSession(connection, requestedTag, cb) { console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`); // Parse requestedTag and connection.tag to decide what state to set . . . // Set the state connection.execute(`ALTER SESSION SET . . .`, (err) => { connection.tag = requestedTag; // Update to match the new state cb(err); // Call cb() last }); } See examples/sessiontagging1.js and examples/sessiontagging2.js
  • 39. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | End Connections When No Longer Needed • Releasing connections is: – Required for pooled connections – Best practice for standalone connections • Don’t release connections until you are done with them, otherwise: – “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537: TNS:connection closed”, etc. – Example scenario: promise.all error handler releasing the connection • Handler is called on first error – Example scenario: streaming from a Lob instance 45
  • 40. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Draining and Force Closing • Closing the pool closes connections cleanly • May need: DISABLE_OOB=on if firewall blocks out-of-band breaks 46 // close pool only if no connections in use await pool.close(); // close pool when no connections are in use, or force it closed after 10 seconds // No new connections can be created but existing connections can be used for 10 seconds await pool.close(10); // force the pool closed immediately await pool.close(0);
  • 41. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 47 Getting Data Out of the Database 1 2 3 4 5 6
  • 42. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Query Methods • Direct Fetches: execute("SELECT . . .", . . . ) – All rows are returned in one big memory-limited array, or limited to maxRows • ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . ) – getRow(): Returns one row on each call until all rows are returned – getRows(numRows): Returns batches of rows in each call until all rows are returned • Stream: queryStream("SELECT . . .", . . . ) – Streams rows until all rows are returned 48
  • 43. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Direct Fetches • Default query behavior – Easy to use • Tune network transfer performance with fetchArraySize – Memory can incrementally grow when the number of query rows is unknown, or varies from execution to execution – A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst case' of a large number of rows • Drawbacks of direct fetches: – One big array of results is needed – Concatenation of record batches can use more memory than the final array requires 49
  • 44. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50 ResultSet Fetches const result = await connection.execute( `SELECT * FROM bigtable`, [], // no bind variables { resultSet: true } ); const rs = result.resultSet; let row; while ((row = await rs.getRow())) { console.log(row); } await rs.close(); // always close the ResultSet
  • 45. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Querying Data • Use row-limiting SQL clauses: SELECT last_name FROM employees ORDER BY last_name OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY • Use Direct Fetches for known small number of rows – set fetchArraySize to number of rows, if known – E.g for single row fetches set fetchArraySize to 1 • Use ResultSets or queryStream() for data sets of unknown or big size – Always close ResultSets • Cast date and timestamp bind variables in WHERE clauses: – . . . WHERE cast(:ts as timestamp) < mytimestampcol – . . . WHERE cast(:dt as date) < mydatecol 51 TIP
  • 46. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Fetching LOBs • Fetch and bind CLOB and BLOB as String and Buffer for performance – Only use Lob Stream class for huge LOBs or if memory is limited const result = await connection.execute( `SELECT c FROM mylobs WHERE id = :idbv`, [1], { fetchInfo: {"C": {type: oracledb.STRING}} } ); const clob = result.rows[0][0]; console.log(clob); 52
  • 47. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 53 Putting Data Into the Database 1 2 3 4 5 6
  • 48. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Single Row DML is Easy SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40)); result = await connection.execute( "INSERT INTO mytable VALUES (:k, :f)", { k: 1, f: 'apple' }, // Bind values { autoCommit: true }); // Or use connection.commit() later console.log("Rows inserted: " + result.rowsAffected); // 1 54
  • 49. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Use Batch Execution for DML • Executes one DML statement (e.g INSERT, UPDATE) with many data values • Reduces round trips: "A server round-trip is defined as the trip from the client to the server and back to the client." 55 executeMany() TIP
  • 50. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56 Basic Array DML sql = "INSERT INTO mytable VALUES (:k, :f)"; data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ]; options = { autoCommit: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); { rowsAffected: 2 }
  • 51. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57 data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" }, { k: null, f: "cherry" }, { k: 3, f: "pear" }, { k: null, f: "damson" } ]; options = { batchErrors: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); BatchErrors { rowsAffected: 3, batchErrors: [ { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 }, { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
  • 52. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Batch Errors • Batch errors are in the result, not the error – Some classes of error will always trigger real errors • Any autoCommit will be ignored if there are DML errors – Valid rows will have been inserted and can be explicitly committed • Attribute rowsAffected shows 3 rows were inserted • Array of errors, one for each problematic record – The offset is the data array position (0-based) of the problem record 58
  • 53. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | {rowsAffected: 7, dmlRowCounts: [ 5, 2 ]} 59 DML Row Counts // assume 5 apples and 2 bananas exist in the table sql = "DELETE FROM mytable WHERE fruit = :f"; data = [ { f: "apple" }, { f: " banana" } ]; options = { dmlRowCounts: true, bindDefs: { f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result);
  • 54. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60 sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2"; binds = [ ["One"], ["Two"], ["Three"] ]; options = { bindDefs: [ { type: oracledb.STRING, maxSize: 5 }, { type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); DMLRETURNING [ [ [ 'AAAmWkAAMAAAAnWAAA' ] ], [ [ 'AAAmWkAAMAAAAnWAAB' ] ], [ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
  • 55. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark • Data: for (var i = 0; i < numRows; i++) data.push([i, "Test " + i]); • SQL: sql = "INSERT INTO mytable VALUES (:1, :2)"; • Multiple inserts: for (var i = 0; i < numRows; i++) { await connection.execute(sql, data[i], {autoCommit: (i < numRows-1 ? false : true)}); } • Single insert: await connection.executeMany(sql, data, {autoCommit: true}); 61
  • 56. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark – Scenario 1 • 1 number, 1 short string • Local DB • YMMV 1 10 100 1000 10000 100000 execute() 10 38 265 2,323 23,914 227,727 executeMany() 16 9 20 16 39 361 62
  • 57. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63 executeMany() Benchmark – Scenario 2 • 1 x NUMBER, • 3 x VARCHAR2(1000)) • Remote DB • YMMV 1 10 100 1,000 execute() 57 510 5,032 50,949 executeMany() 57 155 368 2,537
  • 58. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: executeMany() • For huge data sets may need to call executeMany() multiple times with batches of records – autoCommit: false for first batches – autoCommit: true for the last batch • Use SQL*Loader or Data Pump instead, where appropriate – These were added to Instant Client 12.2 64 TIP
  • 59. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 66 PL/SQL
  • 60. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Calling PL/SQL is easy const result = await connection.execute( `BEGIN myproc(:i, :io, :o); END;`, { i: 'Chris', // type found from the data. Default dir is BIND_IN io: { val: 'Jones', dir: oracledb.BIND_INOUT }, o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } }); console.log(result.outBinds); Confidential – Oracle Internal/Restricted/Highly Restricted 67
  • 61. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68 SQL> CREATE OR REPLACE PROCEDURE testproc (p_in IN NUMBER, p_out OUT NUMBER) AS BEGIN p_out := p_in * 2; END; plsql = "BEGIN testproc(:1, :2); END;"; binds = [ [1], [2], [3] ]; options = { bindDefs: [ { type: oracledb.NUMBER }, { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); CallingPL/SQL [ [ 2 ], [ 4 ], [ 6 ] ]
  • 62. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | PL/SQL Collection Associative Array (Index-by) Binds TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER; PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN FORALL i IN INDICES OF p_vals INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i)); END; await connection.execute( "BEGIN myinproc(:idbv, :valbv); END;", { idbv: 1234, valbv: { type: oracledb.NUMBER, dir: oracledb.BIND_IN, val: [1, 2, 23, 4, 10] } }; 69
  • 63. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 70 JSON and Simple Oracle Document Access 1 2 3 4 5 6
  • 64. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 SQL> CREATE TABLE myjsontab ( c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE); myContent = {name: "Sally", address: {city: "Melbourne"}}; json = JSON.stringify(myContent); result = await connection.execute( 'insert into myjsontab (c) values (:cbv)', { cbv: json } ); 71 Inserting
  • 65. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 result = await connection.execute( `select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax { cbv: 'Sally' }, { fetchInfo: {"C": {type: oracledb.STRING } }}); js = JSON.parse(result.rows[0]); console.log('Name is: ' + js.name); console.log('City is: ' + js.address.city); //sql = "select c FROM myjsontab where json_exists(c, '$.address.city')"; 72 Fetching
  • 66. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Simple Oracle Document Access (SODA) • Set of NoSQL-style APIs to create and access documents – Documents are often JSON – Oracle maps SODA calls to managed tables, indexes and sequences – Query-by-example access makes data operations easy • SQL is not necessary – But could be used for advanced analysis • SODA APIs also exist in Python, PL/SQL, C and Java 73 node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
  • 67. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74 node-oracledb SODA Classes Base SODA class • Get SODA database • Manipulate SODA collections Set of Documents • Create and find documents Operation Builder • Filters and QBE • Retrieve, replace or remove documents Document Cursor • Iterate over retrieved documents SODA Document • Document content • Access as JSON, object or buffer SodaDatabase SodaCollection SodaOperation SodaDocumentCursor SodaDocument Oracledb Pool Connection ResultSet Lob
  • 68. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75 Creating Collections Using SODA in node-oracledb SQL> grant SODA_APP to cj; conn = await oracledb.getConnection({user:"cj",....}); soda = await conn.getSodaDatabase(); collection = await soda.createCollection("mycollection");
  • 69. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76 Inserting Documents Using SODA in node-oracledb content = {name: "Anthony", address: {city: "Edmonton"}}; await collection.insertOne(content); doc = await collection.insertOneAndGet(content); console.log("Key is:", doc.key); // JSON (or Object or Buffer) doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}'); await collection.insertOne(doc2);
  • 70. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Using SODA in node-oracledb • Recommendation is to turn on oracle.autocommit – Like with SQL, avoid auto commit when inserting lots of documents • SODA’s DDL-like operations occur in autonomous transactions 77 Commit Behavior
  • 71. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78 Operation Builder – find() Using SODA in node-oracledb doc = await collection.find().key(key).getOne(); // A SodaDocument content = doc.getContent(); // A JavaScript object console.log('Retrieved SODA document as an object:'); console.log(content); content = doc.getContentAsString(); // A JSON string console.log('Retrieved SODA document as a string:'); console.log(content);
  • 72. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79 Operation Builder – filter() Query-by-example (QBE) Using SODA in node-oracledb // Find all documents with city names starting with 'S' console.log('Cities starting with S'); documents = await collection.find() .filter({"address.city": {"$like": "S%"}}) .getDocuments(); for (let i = 0; i < documents.length; i++) { content = documents[i].getContent(); console.log(' city is: ', content.address.city); }
  • 73. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 80 What’s coming in node-oracledb 4 1 2 3 4 5 6
  • 74. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | What’s coming in node-oracledb 4.0 • Install (compile) current code snapshot with: npm install oracle/node-oracledb#dev-4.0 • Already landed: – N-NAPI code refactor improving Node.js version compatibility – Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers – Implicit Results – SODA bulk insert • Being investigated (no promises!): – SQL and PL/SQL object binding & queries • When? Planned CY2019 Confidential – Oracle Internal/Restricted/Highly Restricted 81
  • 75. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82 AQ ‘RAW’ Queue Demo Oracle Advanced Queuing Dequeue: queue = connection.queue('MYQUEUE'); msg = await queue.deqOne(); await connection.commit(); console.log(msg.payload.toString()); Enqueue: queue = connection.queue('MYQUEUE'); messageString = 'This is my message'; await queue.enqOne(messageString); await connection.commit(); BEGIN dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW'); dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB'); dbms_aqadm.start_queue('MYQUEUE'); END; /
  • 76. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83 Wrap up
  • 77. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Connection management is the basis for good scalability • Utilize the best query method for the task, and tune it • Use executeMany() to insert/update/delete multiple records efficiently • Use SODA’s NoSQL-style data access to simplify applications 84
  • 78. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Resources • Mail: christopher.jones@oracle.com • Twitter: @ghrd, @AnthonyTuininga, @dmcghan • Blog: https://blogs.oracle.com/opal • Office Hours: https://tinyurl.com/NodeHours • Facebook: https://www.facebook.com/groups/oraclescripting/ • Home: https://oracle.github.io/node-oracledb/ • Code: https://github.com/oracle/node-oracledb 85