FuseSource webinar around Apache Camel, persistent component s (jdbc, sql, jpa) and Transactions - part I

1. Database
Integra6on
with
Apache
Camel
Part
I
Charles Moulliard,
October 21th, 2011
twitter : cmoulliard | LinkedIn: charlesmoulliard
http://cmoulliard.blogspot.com
A
Progress
So3ware
Company
1
Copyright
©
2010
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

2. When you joined today’s session …
Audio is broadcast from your computer
Submit your questions
via the Chat Window
Contact today’s Host
via the Chat Window
2
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

3. Speaker
–
Charles
Moulliard
 Engineer
in
Agronomy
&
Master
Follow
me
on
:
in
Zoology
 17
years
of
experience
in
IT
hYp://twiYer.com/cmoulliard
world
development
(J2EE,
.NET)
hYp://www.linkedin.com/in/
• Specialised
in
new
technologies
charlesmoulliard
web/Internet
&
integra6on
• Project
manager
in
bank,
financial,
hYp://cmoulliard.blogspot.com
telco,
insurance
and
transport
world
hYp://www.slideshare.net/
 Solu6on
Architect
at
FuseSource
cmoulliard
 CommiYer
on
Apache
projects
:
• ServiceMix,
Apache
Karaf
(PMC)
,
Camel,
Fuse
Fabric,
Scalate
3
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

4. Agenda
 Goal
:
Present
Database
Integra6on
with
Camel
 This
first
part
is
focused
on
some
Camel
Components
and
Transac6on
• Star6ng
from
"simple"
to
more
"elaborated"
• Describe
some
"interes6ng"
components
• Discuss
Pros
and
Cons
• Introduce
you
through
transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
4
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

5. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
5
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

6. Table
of
contents
 Components
presenta,on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
6
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

7. Components
 We
have
a
big
family
of
SQL
components
allowing
to
design
all
the
required
architectures
Name Type Description Camel
Version
JDBC SQL Simple SQL query >= 1.5
SQL SQL Idem as JDBC but using spring-jdbc template >= 1.5
JPA SQL Use Java Persistence Api in combination with >= 1.6
OpenJPA, EClipseLink, HibernateJPA, …
Hibernate SQL ORM tool > 1.x
iBatis SQL Another ORM tool > =1.6
myIbatis SQL Ibatis ORM engine (left Apache community) >= 2.7
Bean - Integrated with Service/DAO layers > 1.x
7
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

8. Components
 And
also
noSQL
components
Name Type Description Camel
Version
Db4o NoSql NoSQL Database >= 2.5
HawtDB NoSql FuseSource project, key/value embedded DB >= 2.3
MongoDb NoSql Scalable, high-performance, open source,
document-oriented database
Cassandra NoSql highly scalable distributed database using as
data model (Bigtable's ColumnFamily)
8
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

9. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
9
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

10. Component
:
JDBC
 Defini6on
:
"The
jdbc
component
enables
you
to
access
databases
through
JDBC,
where
SQL
queries
and
opera6ons
are
sent
in
the
message
body.
This
component
uses
the
standard
JDBC
API."
 Format
:
jdbc:dataSourceName[?options]!
 Very
simple
component
as
it
only
required
a
datasource
and
SQL
query
defined
in
the
body
of
the
message
10
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

11. Component
:
JDBC
-­‐
Setup
 (Step
0)
Configure
your
database
engine,
tables,
access
rights,
….
 (Step
1)
Define
DataSource
in
Spring
XML
file
Id of the bean is the key used by camel-
jdbc component to find
DataSource manager can be what you
want : Apache Commons DBCP, C3P0,…
11
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

12. Component
:
JDBC
–
Camel
-­‐
SELECT
 (Step
3)
Build
the
"SQL
Body"
to
fetch,
insert,
update
or
delete
data
from
Database
setBody allows to define the SQL Timer component triggers the
Select statement DB every 20s
12
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

13. Component
:
JDBC
–
Camel
–
SELECT
WHERE
 Can
we
do
something
more
dynamic
?
Yes
using
setBody,
Processor
or
a
CamelBean
 We
will
use
an
ID
key
provided
as
a
value
from
a
file
to
select
info
from
Database
Hey ! See How camel with simple language
simplifies your life.
13
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

14. Component
:
JDBC
–
Camel
–
Externalize
query
 With
camel-­‐proper6es,
we
can
externalise
the
SQL
query
(or
a
part)
Hey ! See How camel + properties with
simple language retrieves the key from
file
14
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

15. Demo
using
camel-­‐jdbc
Demo
using
camel-­‐jdbc
with
PropertyPlace
Holder
15
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

16. Component
:
JDBC
–
Pros
&
Cons
 Pros
• Few
dependencies
required,
less
jars
files
• Easy
to
use
and
setup
 Cons:
• This
component
can
only
be
used
to
define
producer
endpoints.
• Bean,
processor,
setBody
or
template
language
(velocity,
freemarker,
scalate)
must
be
used
to
build
complex
SQL
query
• Not
Transac6onal
friendly
–
Transac6on
must
be
managed
"manually"
(e.g.
Add
spring)
16
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

17. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
17
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

18. Component
:
SQL
 Defini6on
:
• Allows
you
to
work
with
databases
using
Spring
JDBC.
• The
difference
between
this
component
and
JDBC
component
is
that

the
SQL
query
is
a
property
of
the
endpoint
and
it
uses
message
payload
as
parameters
passed
to
the
query
 Format
:
sql:query!
 Example
: sql:select * from table where
id=# order by name[?options]!
18
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

19. Component
:
SQL
-­‐
setup
 Step
(0)
and
(1)
are
iden6cal
to
component
JDBC
 Difference
with
JDBC
component

we
must
define
the
SqlComponent
and
inject
reference
of
the
Datasource
Reference
to
the
Datasource
is
provided
as
a
property
and
will
allow
camel
to
create
it
accordingly
when
endpoint
will
be
called
19
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

20. Component
:
SQL
-­‐
setup
 Step
(2)
:
Define
your
camel
route
including
the
query
in
the
URI
defini6on.
Observe
that
in
this
case,
we
do
not
need
to
define
an
exchange
containing
a
query
Body
Query
is
part
of
the
URI
defini6on
Remark
:
query
can
also
be
externalised
using
property
placeholder
!
20
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

21. Component
:
SQL
-­‐
setup
 This
seems
really
sta6c
!
 How
can
we
define
dynamically
the
parameters
of
the
query

The
component
provides
a
placeholder
#
that
we
will
use
to
replace
the
values
received
from
the
Body
(Object[])
File
contains
the
parameter
key
that
we
convert
into
Object[]
Symbol
of
Placeholder
is
#
and
not
?
To
avoid
conflict
with
URI
defini6on
21
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

22. Component
:
SQL
–
Dynamic
Query
 Here
is
another
approach
using
a
camel
bean
component
where
the
query
xxx
is
defined
at
run6me
(and
not
as
before
at
DSL
compile
6me!)
Look
how
we
simplify
the
camel
route.
SQL
component
has
disappeared
but
in
fact
it
is
called
within
camel
bean
(see
next
slide)
and
SQL
query
created
22
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

23. Component
:
SQL
–
Dynamic
Query
 We
use
ProducerTemplate,
@EndpointInject
of
Camel
Query + SQL component are
defined here and a camel
exchange is created by the
template
23
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

24. Demo
using
camel-­‐sql
(spring
JDBC
template)
24
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

25. Component
:
SQL–
Pros
&
Cons
 Pros
• Easy
to
use
and
setup,
very
flexible
–
SQL
• Uses
prepared
statements
• Use
PlaceHolder
to
replace
symbol
#
with
values
• Depends
on
Spring
JDBC
Template
which
simplifies
SQL
management
• Can
be
included
in
Camel
<transacted/>
route

Cons:
• This
component
can
only
be
used
to
define
producer
endpoints
• The
query
is
in
the
URI
which
can
become
long
• More
dependencies
than
JDBC
25
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

26. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
26
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

27. Component
:
JPA
 Defini6on
:
• Enables
you
to
store
and
retrieve
Java
objects
from
persistent
storage
using
Java
Persistence
Architecture
(JPA),
• Allows
to
use
Object/Rela6onal
Mapping
(ORM)
products
such
as
OpenJPA,
Hibernate,
TopLink,
…
 Format
: jpa:[entityClassName][?
options]
27
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

28. Component
:
JPA
 We
can
produce
and
consume
!
 Sending
to
the
JPA
endpoint

You
can
store
a
Java
en6ty
bean
in
a
database.
• The
body
of
the
In
message
is
assumed
to
be
an
en6ty
bean
(that
is,
a
POJO
with
an
@En6ty
annota6on
on
it)
or
a
collec6on
or
array
of
en6ty
beans.
 Consuming
from
the
JPA
endpoint

removes
(or
updates)
en6ty
beans
in
the
database.
• This
allows
you
to
use
a
database
table
as
a
logical
queue:
consumers
take
messages
from
the
queue
and
then
delete/update
them
to
logically
remove
them
from
the
queue.
• If
you
do
not
wish
to
delete
the
en6ty
bean
when
it
has
been
processed,
you
can
specify
consumeDelete=false
on
the
URI.
This
will
result
in
the
en6ty
being
processed
each
poll.
28
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

29. Component
:
JPA
-­‐
setup
 (Step
0)
Configure
your
database,
tables,
…
 (Step
1)
Define
En6ty
Class is enrich with
JPA @Annotations
29
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

30. Component
:
JPA
-­‐
setup
 (Step
2)
Configure
persistence
file
A
B
C
30
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

31. Component
:
JPA
-­‐
setup
 (Step
3)
Configure
Spring
File
–
En6tyManager/TxManager
We use Spring Local JPA
Container & Tx Manager and
provide a ref to DS
According to your JPA
Vendor, select
OpenJPA, Hibernate,
….
31
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

32. Component
:
JPA
-­‐
Consume
 (Step
4)
Configure
Camel
Route,
JPA
component
to
Consume
We can consume directly from JPA endpoint –
(different from JDBC, SQL components)
32
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

33. Component
:
JPA
-­‐
Consume
 Using
namedQuery
namedQuery is passed as
parameter (query can also
place directly in the URI)
and defined here
33
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

34. Component
:
JPA
-­‐
Produce
 (Step
4)
Configure
Camel
Route
to
produce
Extract
incident from
Map and
enrich it
Bindy is used to transform CSV records into Map of Incident
Insert
record in
DB
34
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

35. Demo
using
camel-­‐jpa
35
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

36. Component
:
SQL–
Pros
&
Cons
 Pros
• Allow
to
consume
(insert)
or
produce
(select)
• It
is
a
standard
• Can
do
boilterplate
CRUD
well
(eg
Create,
Update,
Delete
an
en6ty)
• Fully
support
Transac6ons
and
<transacted/>
route

Cons:
• Less
flexible
than
JDBC,
SQL
as
it
requires
more
config,
more
development
6me
is
required
• Higher
learning
curve,
For
simpler
use-­‐cases
its
"overkill"
36
• Only
one
en6ty
can
be
processed
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

37. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac,on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
37
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

38. Transac6on
-­‐
Introduc6on
 Developing
messaging
architecture
involves
also
that
we
handle
errors,
excep6ons,
loss
of
messages,
connec6on
issues,
…
 What
will
happen
to
our
Camel
Exchanges
when
excep6ons
occur
during
DB
connec6on
?

The
response
depends
on
the
ErrorHandler/Excep6on
strategy
put
in
place
and
the
component
used
 Component
like
(File,
JPA,
SQL,
…)
implements
a
fallback
strategy
and
than
it
will
be
possible
to
move
the
messages
to
directory,
DLQ,
…
(they
are
called
Tx
clients)

When
exchanges
are
send
back
to
the
caller,
they
could
be
• Redelivered
x
6mes
(=
DefaultErrorHandler)
• Move
to
a
DeadLeYerQueue
(=DeadLeYerErrorHandler)
• Discarded
38
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

39. Transac6ons
–
Introduc6on
(next)
 Is
it
enough
?
No
as
we
must
also
rollback
info
pushed
into
the
DB.
Tx
is
all
or
nothing
strategy
while
RDMS
rely
on
ACID
principles
 Solu6ons

1)
Transac6ons
can
be
designed
using
Spring
in
the
applica6on
which
are
the
beans/POJO
that
we
call
Cons
-­‐
This
is
not
integrated
with
your
Camel
routes
and
you
must
intercept
excep6ons
and
handle
them
2)
A
beYer
approach
is
to
use
the
Transac6on
support
offered
by
Camel
 This
is
not
a
proprietary
solu6on
 Camel
uses
Spring
Transac6on
Api
to
work
with
Transac6ons
 Transac6on
Manager
can
be
provided
by
• J2EE
Applica6on
Server
(Tomcat,
Jboss,
WebSphere,
…),
OSGI
pla}orm
(Apache
ServiceMix),
Standalone
Applica6on
(Spring)
39
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

40. Transac6ons
-­‐
Setup
A transaction
 To
be
honest,
this
is
really
simple
…..
Manager is
required
Every time an exception occurs,
Camel Route must camel will propagate it to the caller
be defined as (which is a Tx client) and the file will
Transactional using be moved as transaction is marked as
<transacted/> DSL rollback
40
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

41. Demo
using
Transac6on
41
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

42. Summing
up!
42
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

43. Table
of
contents
 Components
presenta6on
 JDBC
Component
 SQL
Component
 JPA
Component
 Transac6on
 Summing-­‐up
 Explain
what
we
will
cover
in
the
2nd
part
43
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

44. Summing
up
 Camel
provides
a
collec6on
of
Persistence
components
• SIMPLE:
JDBC,
SQL
• ELABORATED
–
ORM
:
Hibernate,
Iba6s,
MyIba6s.
• STANDARD
BASED
–
JPA
:
OpenJPA,
Hibernate-­‐EJB
(JPA)
 Your
choice
will
depend
on
constraints
like
:
• Datamodel
(simple
vs
complicated),
• Business,
• Architecture
(Container,
Tx
Manager,
….)
• Database
(new
schema
or
integrate
with
exis6ng)
• Stored
procedure
needed
?
• Sql
Vendor
Trick
 All
components
are
really
simple
to
configure
and
can
be
deployed
in
any
container
:
J2EE,
ServiceMix,
Standalone
 Camel
supports
transac6ons
in
an
transparent
way
44
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

45. Next
Webinars
 See
hYp://fusesource.com/resources/video-­‐archived-­‐
webinars/
 In
the
second
part,
we
will
cover
more
stuffs
about
transac6on,
use
JTA
Tx
manager
of
Fuse
ESB
( jms,
jdbc)
and
cover
persistence
with
Idempotent
Processor
 22nd
of
November
2011:
“ Transac6on
Management
with
Apache
ServiceMix
and
EIPs”
 See
you
then!
45
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

46. Useful
references
 Camel
• hYp://fusesource.com/products/enterprise-­‐camel/#documenta6on
• hYp://fusesource.com/docs/router/2.7/component_ref/index.html
 Transac6on
with
camel
• hYp://fusesource.com/docs/router/2.7/transac6ons/index.html
 Spring
• hYp://sta6c.springsource.org/spring/docs/3.0.6.RELEASE/spring-­‐
framework-­‐reference/html/
• ORM
-­‐
hYp://sta6c.springsource.org/spring/docs/3.0.6.RELEASE/spring-­‐
framework-­‐reference/html/orm.html
• Transac6on
-­‐
hYp://sta6c.springsource.org/spring/docs/3.0.6.RELEASE/
spring-­‐framework-­‐reference/html/transac6on.html
46
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company

47. Learn
More
at
hYp://fusesource.com
47
Copyright
©
2011
Progress
So3ware
Corpora6on
and/or
its
subsidiaries
or
affiliates.
All
rights
reserved.
A
Progress
So3ware
Company