Organisations are adopting microservices to keep pace with business innovation; whilst needing to meet the resilience, scalability and security requirements critical for digital solutions. Enterprise relational DBs are often a barrier to this transformation, but they needn’t be.
This presentation delves into the challenges faced by enterprises during digital transformation and modernization initiatives which are often hamstrung by the inherent monolithic nature of enterprise databases.
Many Oracle data-centric applications consist of an intricate web of hundreds of tables, housing hundreds of thousands of lines of PL/SQL code executed within the database via packaged procedures. These relational databases have enabled us to safely and securely manage structured data for several decades, but over time they grow more complex and harder to maintain, slowing down delivery and seriously degrading application performance, business innovation all but grinds to a halt.
Given the impracticality and cost associated with complete rewrites, many organisations are turning to Microservices Architecture, to extract value from existing assets whilst gradually deconstructing the monolithic architecture to facilitate evolutionary changes.
This presentation outlines a systematic and phased approach, based on experience from multiple client initiatives, highlighting the crucial role of this transformation in enabling the creation of APIs that drive new business initiatives. The concept of domain separation, a pivotal element in the migration process, will be introduced, along with options to move certain data retrieval and processing to more appropriate architectures
2. Introduction
👋 Hi! I’m Mark Simpson
👉 Tech Director with Griffiths Waite in Birmingham
📅 > 25 Years Enterprise Software Experience
👨💻 Responsible for Tech Innovation & Adoption at GW
📖 Oracle ACE Director 2012-2022
https://www.linkedin.com/in/marksimpsongw/
mark.simpson@griffiths-waite.co.uk
3. • Founded in 1994
• Over quarter of a century of award-winning innovation
• Situated in the heart of Birmingham, right next to New Street Station
• Enterprise JavaScript Specialists – TypeScript across the stack
• Focus on Custom Software Products - Digital Portals, API Platforms, Data Analytics
• Heritage in Oracle Forms, Oracle SOA, Oracle DB, ADF, API and Oracle JET
• Microsoft Azure Cloud focus
Who are GW?
4. Monolith
Architecture
A software design pattern where an application
is built as a single, self-contained unit, making it
difficult to scale, modify, and maintain.
We commonly see this with Enterprise Database
systems, with intertwined PL/SQL making it
difficult to isolate services without disruption
5. Hidden
Complexities
Tight coupling between the PL/SQL and
database schema is natural.
However, the cascading nature of modular
database code, with encapsulation of
business logic makes dissecting and
extracting logic a challenge
6. Ongoing
Code Growth
Enterprise Databases constantly grow
as the business changes and features
evolve.
Often seen as a legacy nightmare,
exponentially more complex than when
first designed, but they still model and
run the business.
8. Microservices
A microservices architecture is an approach
to software development where applications
are built as a collection of small, independent
services that can be deployed and scaled
separately, allowing for flexibility, faster
development, and easier maintenance.
Often linked to Domain Driven Design, it is a
architectural style rather than a prescriptive
framework.
9. Case Study
All actors in this story are fictional within an
organisation called RetroTech - however the
scenarios are based on experiences I have
observed in many organisations
Digital Transformation
in a large organisation
with a 30-year-old
enterprise relational
Oracle DB
10. Digital
Transformation
• Meet Eric from RetroTech
• Strong history with the Oracle DB
• Looking to modernise the applications
• Challenges with current DB
• Wants to leverage innovation and cloud advances
• Looking for “continuous improvement and product thinking”
T H E C U S T O M E R
11. Digital
Transformation
• Meet James
• Works for a large global consultancy
• Expert in Technology & Architecture
• Read all the theory books !
• Committed to modernising RetroTech systems
• After “A true business transformation”
T H E E X P E R T
13. Digital
Transformation
• Meet Sarah
• Works for an Enterprise Software House
• Experience of many product developments
• Likes to apply innovation to customer use cases
• Committed to building a partnership with RetroTech
• She thinks “Digital Products will bring new business value”
T H E PA R T N E R
14. Top-Down
Design
James recognises that the DB is complex, a
mere glance at the source code shows
thousands of lines of code for what he expects
should be simple.
He concludes that the DB model and code is now
not fit for purpose for the digitally transformed
organisation and produces a fresh API model
based on a pure Domain Driven Design,
Common Data Services and local data stores.
15. Layered
Architecture
As digital requirements increase the database
naturally becomes a layer in the architecture, but
one that requires specific coding skills and
people who understand the data model,
business logic and code syntax.
A gap will often appear in the layer with
separation wider due to the choices of
technology for API and UI development.
16. Role of Data
Data comes in many guises and the use cases
for the consumption of data varies in enterprise
solutions.
Solutions are required for:
- Transactional
- Entity Reads
- Large Lists
- Bulk Uploads
- Large Data Set streaming
- Analytics …Etc.
17. Understanding
the Legacy DB
Recognising there is a lot of hidden complexity in
the Enterprise DB that has grown over the last 30
years.
Sarah spends time with the DB team, especially
Eric, understanding the data model and the
inherent domain structure of the code.
She uses a parser to write typescript modules to
interrogate the database, ensuring tests written
for the new system consider existing structure
and logic where appropriate.
18. PL/SQL
Archaeology
It is imperative that the Service developers
understand the existing systems, extract the key
logic and respect the data model that has served
the business for so long.
Rather than separate the DB and Service
developers, GW wrote a Typescript parser for
PL/SQL to allow types to be generated and for
developers to write Typescript or JavaScript code
to interrogate logic, models and dependencies
from the DB.
21. Transformation
MVP
James is looking for ways to “transform” the
business to fit in with the new system now that the
MVP stories are delivered. The direction is set
and now he is waiting on other teams, has got a
large delivery factory scaled up now. He is
engaged with his Change Management team.
He has delivered the mapping stories to the Data
Migration team to keep the enterprise DB and the
new domain DBs in step.
He has setup a Co-existence team to synchronise
data in the Enterprise DB to support the products
and domains not yet migrated in the MVP.
22. Product
MVP
Sarah has agreed an MVP with the business
stakeholders to solve some current market
challenges.
New services have been written leveraging the
current DB model where appropriate and splitting
out certain services to scale to the new usage,
leveraging some cloud services.
Some PL/SQL has been decommissioned and
logic has been moved to the services layer with
good automated test coverage and good DevOps
practices have been put in place.
23. Domain
Separation
Domain Driven Design is a key strategy in designing a
Microservices Architecture. Splitting the database code into logical
domains can help manage the complexities.
These should be isolated for change impact, deployment, scalability
and resilience but can leverage shared platform capabilities as long
as these shared services do not compromise the above principles.
Other considerations in splitting the services..
- Performance
- Volatility
- Data Sharing
- Response times
Local
Shared
24. Microservices
Responsibility
In our experience the Microservice does not need to
be self-contained from a technology perspective.
It can expose data via APIs, Data Connections,
Events, UI.
The implementation should use the right technology
for the need.
Platform Engineering disciplines will help standardise
the user of the technology, Kubernetes, Enterprise
DB, Data Platforms etc.
Service
API
Data
UI
Event
Platform
AKS
Data
Bricks
ORCL
DB
25. Success!
(Big Bang Hell)
With the co-existence and data migration streams struggling
to fill the gaps created by the new design, a big bang launch
is planned.
The core data model has reverted to the original design but
now running on a cloud database that has been migrated by
the Cloud Competency team, with the team on hand to
support the many teething issues.
The services wrap the existing pl/sql code and the
improvements are focussed on building the UI in JavaScript.
James reminds the team that RetroTech are not Netflix !!!
Project pronounced a success, James can move to his next
client, with the lesson that he should scale the team earlier.
26. Success!
(Product Team)
The product is now in production, monitoring shows good
improvement in metrics.
The Product team are in place for continuous
improvements with Business, DevOps, Platform
Engineering and IT Delivery working together.
There are tech debt objectives in place where complex
legacy code remains and the library of new microservices
with the appropriate data sources is in place. The public
API solution has gained traction, and the conversation UI
is accurate and fast. Cloud is making a significant
different to performance and reliability in busy periods.
Retro Tech have signed a partnership deal with Sarah for
a joint collaborative team to deliver more business value.
27. Leveraging
the Cloud
The redesign of the application and data architecture has
readied Retro Tech for their move to a scalable and
resilient Cloud Architecture.
They are confident of supporting the business change
needed to modernise, with operational readiness baked
into the development process.
They are exploring new channels, partnering with
platforms and are starting to look at how they might
monetise their core capability – their DATA!
29. #1
ACL Layer
An integration pattern to protect services
from dependant data or service changes can
be used within the microservice.
Allows for evolution of the data model and
supports switching of data implementations.
Mocking and Contract based testing of the
data layer within the service is now possible.
30. #2
API Routing
As you break down the PL/SQL monolith into
clearly separated services, you can start to
split the architecture for scalability benefits.
E.g. consumers with large requests could be
diverted to a cloud copy of the data service.
This coupled with the other patterns leads to a
smoother transition to a cloud data
architecture.
31. #3
Large Datasets & Analytics
Move data to a specific, scalable big data and analytics platform for the retrieval of large data sets and analytical views.
32. #4
Logical
Microservice
Microservices should be self contained,
independent and have clear ownership.
In Enterprise ecosystems they will naturally
be layered and leverage dependant services
and platforms.
Have logical bounded context and use
Platform Engineering disciplines to allow the
service owner to deploy, scale and monitor
the service independently.
33. • Automate the analysis of legacy DB – Model generation, Archaeology scripts
• Traditional layered service architecture becomes bloated for a lot of digital use cases
• Be pragmatic as you apply the theory, solution focused
• Get the logical design of your Microservices right, then leverage Platform and Services
• Coexistence and phased change is essential
• Use the right tools for the right data scenarios – you will work with copies
• Be like Sarah, less like James !
Summary
