Oracle Data Integrator 12c is a great ELT tool with the ability to work efficiently on different environments. It might be the traditional Development, Test and Production environments but there can be much more. After looking at the different architectures we will see what are the promotion mechanisms available in ODI12c and how we can keep track of the changes. We will also see how different development teams can work efficiently without impacting each other and how to automate the promotion mechanism.
In the second part of this session, we will move on another success factor for any data integration project: Data Governance. Running a business is constantly making decisions. These are often based upon data, upon information about our business. To be sure to make the right decision, we need to be sure it’s based on the right data source and the data is correct. We will discover how Oracle Enterprise Metadata Management and Oracle Enterprise Data Quality can help us to achieve this
2. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Jérôme Françoisse
• Consultant for Rittman Mead
- Oracle BI/DW Architect/Analyst/Developer
• ODI Trainer
• Providing ODI support on OTN Forums
• ODI 12c Beta Program Member
• Blogger at http://www.rittmanmead.com/blog/
• Email : jerome.francoisse@rittmanmead.com
• Twitter : @JeromeFr
2
3. info@rittmanmead.com www.rittmanmead.com @rittmanmead
About Rittman Mead
3
• World’s leading specialist partner for technical
excellence, solutions delivery and innovation in
Oracle Data Integration, Business Intelligence,
Analytics and Big Data
• Providing our customers targeted expertise; we are a
company that doesn’t try to do everything… only
what we excel at
• 70+ consultants worldwide including 1 Oracle ACE
Director and 2 Oracle ACEs
• Founded on the values of collaboration, learning,
integrity and getting things done
Optimizing your investment in Oracle Data Integration
• Comprehensive service portfolio designed to support
the full lifecycle of any analytics solution
• 4 UKOUG Partner of the Year Award 2015
4. info@rittmanmead.com www.rittmanmead.com @rittmanmead
User Engagement
4
Visual Redesign Business User Training
Ongoing SupportEngagement Toolkit
Average user adoption for BI
platforms is below 25%
Rittman Mead’s User Engagement Service can help
7. info@rittmanmead.com www.rittmanmead.com @rittmanmead
ODI 12c Key Components
8
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
• ODI Studio (the main development IDE)
- Designer Navigator
- Operator Navigator
- Topology Navigator
- Security Navigator
• ODI Agents (executing mappings and packages)
• ODI Console (web-based interface)
• ODI Repositories (contain definitions of models,
mappings and other objects)
10. info@rittmanmead.com www.rittmanmead.com @rittmanmead
How many repositories do I need?
11
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
Execution
Execution Repository
(Test)
11. info@rittmanmead.com www.rittmanmead.com @rittmanmead
How many repositories do I need?
12
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
Execution
Execution Repository
(Test)
Security
Topology
Versioning
Master Repository
12. info@rittmanmead.com www.rittmanmead.com @rittmanmead
How many repositories do I need?
13
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
Execution
Execution Repository
(Test)
Security
Topology
Versioning
Master Repository
Execution
Execution Repository
(Pre-Production)
Security
Topology
Versioning
Master Repository
13. info@rittmanmead.com www.rittmanmead.com @rittmanmead
How many repositories do I need?
14
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
Execution
Execution Repository
(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Hotfix)
Execution
Execution Repository
(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
14. info@rittmanmead.com www.rittmanmead.com @rittmanmead
How many repositories do I need?
15
Models
Projects
Execution
Work Repository
(Development)
Execution
Execution Repository
(Production)
Execution
Execution Repository
(Test)
Security
Topology
Versioning
Master Repository
Models
Projects
Execution
Work Repository
(Hotfix)
Execution
Execution Repository
(Pre-Production)
Security
Topology
Versioning
Master Repository
Security
Topology
Versioning
Master Repository
Execution
Execution Repository
(Smoke Test)
15. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Problems
16
• As a developer, how do I create a consistent set of objects for a release?
- For development environments
- For execution environments
• As a release manager, how do I easily promote code from one
environment to another?
• As a release manager, how do I easily keep track of these sets of
objects?
• As a release manager, how do I roll back to a previous release?
16. info@rittmanmead.com www.rittmanmead.com @rittmanmead
The Solution – Deployment Archive (DA)
• Archive file containing a set of consistent ODI objects
• Applied to Deployed Repositories (test/production)
• 3 types :
- Initial
- Patch
- Rollback
• Development or Execution
17
17. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Example
18
Development
environment
QA environment
(Deployed repository)
Developer
Release Manager
Bug
Creates Initial
Release 1.0.0.0
Initialize
repository with
Release 1.0.0.0
Creates Patch
Release 1.1.0.0
to add a
features
Apply Patch
Release 1.1.0.0
and create
rollback DA
Apply Rollback
Release 1.1.0.0
Creates Patch
Release 1.1.0.1
to fix the
previous bug
Apply Patch
Release 1.1.0.1
and create
rollback DA
27. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Automation
28
• Available through ODI SDK
• Scenarios :
- Import and be sure you always generate a rollback DA
- Import daily into a smoke test environment
- Import the same release in Hotfix and Production at the
same time
34. info@rittmanmead.com www.rittmanmead.com @rittmanmead
VCS - Features
35
• Tags
- Snapshot of consistent set of versioned objects
- Ready for promotion – Deployment Archive
- … or for branching
- Created from
‣ Whole repository (Full Tag)
‣ Selected Objects and dependencies (Partial Tag)
35. info@rittmanmead.com www.rittmanmead.com @rittmanmead
VCS - Features
36
• Branches
- Created from a Tag
- Populate a new environment – Master and Work repository
- For a new feature or a new team
- Can be merged back in the trunk
36. info@rittmanmead.com www.rittmanmead.com @rittmanmead
VCS - Features
37
• Branches
Trunk
Branch 1
Branching
Release 1
101 2 3 5
4 6 7 9
Merging
11
Bug Fixes for Release 1
New features for Release 2
New features for Release 2New features for Release 1
8
39. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Data Governance
40
• The right data
Which reports uses
my application data?
Application User Sysadmin
Which reports use
the mainframe data?
ETL Developer
What is the impact if I
change this
mapping?
DBA
Who is using this
database I need to
upgrade?
Data Steward
Where is this data
entered?
Who should I notify if
there is a problem?
Which data are
available?
Data Scientist BI Developer
Where does this data
come from?
Executive
How are sales
figures
calculated?
40. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Oracle Enterprise Metadata Management
41
• Lineage
- From reports to source
• Impact Analysis
- From source to reports
• DB - ETL - BI Metadata
• Business Glossary
• Semantic Linking
• Supporting features
- Harvesting
- Automatic Stitching
- Versioning
- Collaboration - Annotations
53. info@rittmanmead.com www.rittmanmead.com @rittmanmead 55
Summary
• ODI - Deployment Archives
- Promote, Rollback and keep track of your code
- For development or execution-only environment
- Easy for Developers and Release Managers
• ODI - VCS
- Be able to get back to any version to object
- Branching for Team development
• OEMM
- Lineage, impact analysis and business glossary
• OEDQ
- Profile, Cleanse, Match, Monitor, Govern
54. info@rittmanmead.com www.rittmanmead.com @rittmanmead
Further Reading / Testing
56
• ODI Lifecycle
- Managing versions in SVN
https://blogs.oracle.com/dataintegration/entry/oracle_data_integrator_12_21
- Parallel Development using SVN Branches
https://blogs.oracle.com/dataintegration/entry/oracle_data_integrator_12_22
- ODI Getting Started VM 12.2.1
http://www.oracle.com/technetwork/middleware/data-integrator/odi-demo-2032565.html
• OEMM
- Oracle By Example
http://www.oracle.com/technetwork/middleware/oemm/learnmore/index.html
• OEDQ Getting Started 12.2.1
http://www.oracle.com/technetwork/middleware/oedq/downloads/edq-vm-download-2424092.html