Very large portion of the world’s business critical systems are considered to be ‘legacy’ –and so is the code underpinning them (ie COBOL, PASCAL, C, to name a few). Although in many cases it is the case that these systems are robust, stable and fit for the main purpose they were originally built, they aren’t flexible and scalable enough to support emerging requirements mainly derived from a more demanding ‘always on the move’ and ‘always connected’ user.
These systems struggle to meet these demands mainly because of the ‘monolithic’ approach on which they were built and the complexity hidden in millions of lines of code that is only understood a very few hand-full of people that still remain active from the teams that several years ago developed these systems.
In almost an equal amount there have also been thousands of failed attempts to modernise these legacy systems. The ‘eating the elephant’ in one go approach certainly didn’t work, and the traditional SOA approach alone although worked till certain extend, it also fell short when it came down to addressing specific requirements around scalability and platform/service inter-dependencies.
In this presentation I will talk about how a legacy modernisation framework based on Microservice Architecture (MSA) in conjunction with some other known SOA patterns (ie. ESB, API Gateway), can be applied to ‘eat the elephant one piece at the time’ but most importantly ‘without getting indigestion’
2. {2}
About me
Luis Weir
Oracle Ace Director – Principal Architect
in assisting organisations define and implement solutions and strategies that can help
them realise the benefits that such technologies have to offer.
I am very passionate about technology. I have be the lead authored of two books (Oracle
SOA Governance 11g Implementation and Oracle API Management 12c Implementation), I am a
regular blogger and speaker in major conferences and events. A well-known industry expert
especially when it comes to Oracle middleware technologies I am also an OTN certified SOA
black belt.
I am an Oracle Ace Director, principal architect and a thought leader
specialised in Oracle Fusion Middleware & Oracle PaaS technologies. With
more than 15 years of experience implementing IT solutions across the
globe, I have been exposed to a wide wide variety of business problems
many of which I’ve helped solved by adopting SOA architectural styles such
as traditional SOA, API management and now Microservices. My current focus
is
2nd Place
1st OTN
Cloud
Hackathon
June, 2016
Cloud
Contribution
Award
SOA Community
March, 2016
Latest Media:
•Oracle Magazine May/June 2016
(http://bit.ly/1RTCAU3)
•Systematic Approach for
Migrating to Oracle Cloud SaaS
(http://bit.ly/1Xr6acs)
•Oracle Magazine Jan/Feb 2016
(http://ora.cl/Vhh)
•API Management Implementation
(http://ora.cl/Gcw)
•A Word About Microservices and
SOA (http://bit.ly/25Dk5go)
5. {5}
… what is a legacy system?
Monolithic applications that significantly
resists modification and evolution [1], seem
frozen in time [6] and thus do not fit with
the organisations future IT strategy [7]
Characteristics:
• They are business critical
[1][2][4][5][6][7][8][12][15]
• Typically been around for a
while (in some cases 30+
years) [2][7][8]
• End of life tech stacks
[1][2][5][8][12][15]
• Not necessarily defined by
age. Lack of vendor support or
inflexibility also mean legacy
[2][5][7][14]
• They are stable, reliable and
performing [4][5][7][14]
6. {6}
Does the world run on legacy?
• 80% of the worlds systems are legacy [3][5]
• 200 billion lines of legacy code are still in use [5][7]
• 1+ million COBOL programmers world wide maintaining an
approximate of 100 billion lines of COBOL code [7][11]
• 75% of financial institutions in Europe are using out-
dated systems (legacy) for core business [9]
• COBOL powers 80% of all daily business transactions [17]
• 30+ years of legacy modernisation research and 80% of
world systems are still “legacy”? [10][7]
[11]
7. {7}
Driving forces
• Lack of flexibility.
Extremely difficult to
change to take new
requirements affecting
new product
development and time
to market
[1][2][3][4][5][7][8][9][12][15]
• High costs of
maintenance and change
[1][2][3][4][5][7][8][9][12][15]
• End of life tech-stacks(out of support, no new
or major feature releases) [1][2][3][5][7][8][9][12][15]
• People [with relevant skills] are scarce
[4][5][7][10][11][17]
• Integration is difficult [1][4][5][6][12][15]
• Security (no new security patches) [6][9][13]
9. {9}
Arguments against modernisation
• High risk of failure – “we’ve tried before
(several times) and failed” [1][2][4][6][7][10][12][15]
• Too costly without clear justification. Ie.
When will we see ROI? [1][2][3][4][6][7][8][12][14][15]
• Lack of knowledge and skills –
[1][4][5][7][12][10][11][14][17]
• “If ain’t broken don’t fix it”, the system is
doing what’s supposed to why change it? [3][6][7][14]
“They [Top management] are
always looking for a short
term Return of Investment.
Once they put the money in,
they want to earn it back”
[7]
We didn’t
fix it last
year, and
survived.
Why should
this year
be
different?
[7]
“I think top management
doesn’t understand the issue
and they don’t give budget
for it [legacy
modernisation]” [7]
11. {11}
Faults are as big as the legacy!!
(2012) 16 million (RBS, NatWest,
Ulster Bank) customer accounts
ended up being frozen
(Dec, 2014) Cancelation of 40
flights in UK affecting 1.9k
other flights and 230k
passengers
(Jan, 2016) Customers couldn't
access accounts for 2 days
12. {12}
Not failing is not success [16]
Insight into legacy modernisation success rate:
• Success rates vary from 24% to 39%, depending
on study and the type of technology project [16]
• 1 in 6 projects (17%) result in "black swan” [16]
• Only 50% of projects deliver 44% of the
intended business benefits [16]
“Healthcare.gov is just a
frontend. The heavy lifting takes
place in the back ends, a
extremely complex array of legacy
systems that span multiple US
federal agencies” [20]
“The worry is that trying to
promise a startup-style
experience on the back of
multiple old and lumbering IT
systems. There's a long way to
go, even after all the initial
issues with accessing the site
itself are resolved” [20]
13. {13}
Usual suspects: reasons for failure
• Lack of documentation and knowledge of the
system leading to issues like bad scoping, poor
estimation and planning, inappropriate target
designs and incomplete testing [1][2][3][4][7][8][12][14]
• Wrong approach specially based on “Eating the
elephant in one go” (cold turkey)”
[1][4][7][8][12][15][16]
• Time pressures. Rushing through project phases
to meet deadlines – [1][4][5][7][12][10][11][14][17]
• Lack of people with relevant skills –
[1][4][5][7][12][10][11][14][17]
15. {15}
What is a Microservice? SOA 2.0?
Loosely coupled service oriented
architecture with bounded context [27]
“The value of the term
microservices is that it
allows to put a label on
a useful subset of the
SOA terminology” [22]
[21]
Functional decomposition
of systems into
manageable and
independently deployable
components[18]
20. {20}
Order & Fulfillment
Domain
Customer Relations
Domain
ERP
Domain
(P2C, R2C,
HR, GL,
Billing,
etc)
MSA – Architectural
API Gateway(s)
Microservice Architecture
Async
Communication
Microservice Architecture Legacy
Message Pipe
Mobile Apps
Adapter
Adapter
Sync
Communication Managed API Microservice
Monolith
ServiceChoreography
Contac
t
Custom
er
Bounded
Context
Shipme
nt
Order
Produc
t
Web Apps
Applications
21. {21}
MSA - Architectural
Pattern Traditional SOA MSA
Monolith pattern (http://bit.ly/1Gjr2Y0) Yes No
Polyglot Programming & Persistence
(http://bit.ly/18BvDIj & http://bit.ly/1XYiak2)
Not traditionally (use
of Suites)
Yes
API gateway pattern (http://bit.ly/1WTyNLJ) Yes Yes
Orchestration (http://bit.ly/1U0SWil) Yes No
Choreography (http://bit.ly/1ssALZQ) No Yes
Event Collaboration (http://bit.ly/25Dk7oE) Yes Yes
Canonical Schema (http://bit.ly/1r6KkfK) Very common No
Schema centralization (http://bit.ly/1sVlqkc) Very common No
Decouple Contract (http://bit.ly/1O8mVpm) Yes Could be….
Bounded Context (http://bit.ly/1o7AK8B) Some times Yes
Ubiquitous Language (http://bit.ly/1c8nXQe) Some times Yes
Bulkhead (http://bit.ly/1c8nXQe) Not really… Yes
Tolerant Reader (http://bit.ly/1aa4mr9) Some times Yes
Client-side Service Discovery
(http://bit.ly/1OunUyq)
Initially only
(service registry)
Recommended
Server-side Service Discovery
(http://bit.ly/1X3RmzA)
Yes Yes
ESB Pattern (http://bit.ly/1ZlSKeT) Yes Across bounded contexts
or domains (dump pipe)
[19]
Yes = Applied most of the time
No = Not applied most of the time
22. {22}
MSA – Organisational
Microservice
Teams organized around business capabilities
Small teams
You build it you
run it
Decentralised governance
Culture of
automation
Products not
projects
23. {23}
MSA – Organizational
Development and support teams
organized by technologies resulting
in siloes(Conway’s law in action)
SOA
Support
Team
DB
Support
Team
UI
Support
team
UI Dev Team
Database
Dev Teams
SOA Dev Team
ProjectTeams
Coms
Gaps
Multi-disciplinary [small] teams
organized by business capability
resulting in modular systems
Customer
U
I D
B
MW
Orders
U
I D
B
MW
Items
U
I D
B
MW
Shipment
U
I D
B
MW
Traditional Operations Model
DevOps /
Continuous Delivery
MSA Operations Model
[19]
Customer
Microservice
Orders
Microservice
Items
Microservice
Shipment
Microservice
24. {24}
Recommend watching
https://www.youtube.com/watch?v=yPvef9R3k-M
DDD & Microservices
By Eric Evans
Goto Berlin, Nov 2014
https://www.youtube.com/watch?v=nMTaS07i3jk
State of the art in MSA
By Adrian Cockcroft
Dockercon Amsterdam, Nov 2015
https://www.youtube.com/watch?v=wgdBVIX9ifA
Microservices
By Martin Fowler
Goto Berlin, Nov 2014
26. {26}
Ok cool, but why MSA for this?
Modularity
Eat the elephant one piece at the time.
Phased implementation approach. Starting
small. Small teams owning full lifecycle
of their piece(a business capability)[23]
Segmented complexity
Separate a big problem into smaller
problems handled by small teams ensures
mental models are retained avoiding a
“legacy in the making” [5]
Easy of deployment / speed
Moving away from entire system deployment
(ie. “one line change to a million-line-long
monolithic”). Deploy services independently
and fast (ie. with containers) and introduce
automation (continuous delivery) [23]
27. {27}
(continuation)
Scalability & resilience
Scale independently and possibly on-demand.
Bulkheads to isolate problems and avoid
whole system failures (avoiding the cascade
effect), Then purposely test resilience [23]
Enabling cloud transition
Building container-based modular applications
whilst adhering to basic principles (like 12
factor [30], Lehman’s law [31], and the
reactive manifesto [33]) cloud adoption is a
real option
Breaking organisational silos
Organise small teams based on business
capabilities in order to avoid
organisational silos being reflected in the
way systems are built (Conway's law [32])
29. {29}
Modernisation Strategies
Cold Turkey
[8]
• Big-bang approach
• High risk and Costly (big
project)
• Reduced business benefits
Monolith
Ie. COBOL
Monolith
Ie. Java
Tool
Refactor / Replace
Convert legacy code and/or
replace with COTS
Monolith
Ie. COBOL
Tools,
manual,
any
other..
Rebuild
Rebuild from ground up in
modern language in one-go
Monolith
Ie. Java
Apps
Chicken
Little [8]
• Phased approach based on
coexisting with legacy using
gateways
• Less risky but complex
Monolith Decomp
ose
A1
Gateways
An
Re-engineer
Decompose legacy into
modules and then rebuild
whilst coexisting with
legacy using gateways
Butterfly [15]
• Phased approach based on
coexisting with legacy
without gateways
• Less risky but complex
Re-engineer
Decompose legacy into
modules and then rebuild
whilst coexisting with
legacy using data access
allocators and chrysaliser
Monolith
Decomp
ose
A1
An
Chrysaliser
Wrapping[15]
• As little modification to
legacy as possible (tactical
approach in nature)
• Used of wrapper technologies
like CICS, screen-scrapers or
ESBs with legacy adapters
• Lower risk, quicker results
• Higher costs, complexity and
risk in the mid-long term
Monolith
A1
Wrappers
An
31. {31}
I) API-fication (wrap)
Onpremise
Terminals
Oracle Cloud PaaS
Mainframe
DB2
Order
Mng
Inventory
Pricing
Product
Shipment
Consignment
s
Sales Purchasing Payables
Receivables GL Etc..
Weblogic
API Gateway (OAPCS)
Oracle SOA Suite
(11g or 12c when
Adapter is certified)
Product API
CICS PROGRAMS Oracle
Connect
JCA Legacy Adapter
REST Adapter
Native
Apps
Web
Apps
OACCS MCSJet on Node Mobile Backend
32. {32}
Oracle Cloud PaaS
APICS Gateway
Native
Apps
Web
Apps
I) API-fication (wrap)
Terminals
Mainframe
DB2
Order
Mng
Inventory
Pricing
Product
Shipment
Consignment
s
Sales Purchasing Payables
Receivables GL Etc..
Weblogic
Oracle SOA Suite
(11g or 12c when
Adapter is certified)
CICS PROGRAMS
Oracle Connect
JCA Legacy Adapter
REST Adapter
OACCS MCS
ICS
Jet on Node
Onpremise Agent
Mobile Backend
Onpremise
Rest endpoint
Product API
33. {33}
Weblogic
Oracle SOA Suite
(11g or 12c when
Adapter is certified)
Oracle Cloud PaaS
APICS Gateway
Native
Apps
Web
Apps
II) Re-engineer (MSA) & coexist
Terminals
Mainframe
DB2
Inventory Orders
Pricing
Product
Shipment
Consignmen
ts
Sales Purchasing Payables
Receivable
s
GL Etc..
CICS PROGRAMS
JCA Legacy Adapter
REST Adapter
OACCS MCS
ICS
Jet on Node
Onpremise Agent
Mobile Backend
Onpremise
Rest endpoint
OACCS ODCS
ProductProduct
Product API
Oracle
Connect
ODI
DB2 KM
Sync
34. {34}
Weblogic
Oracle SOA Suite
(11g or 12c when
Adapter is certified)
Oracle Cloud PaaS
APICS Gateway
Native
Apps
Web
Apps
III) Switch over & start over
Terminals
Mainframe
DB2
Inventory Orders
Pricing
Product
Shipment
Consignment
s
Sales Purchasing Payables
Receivables GL Etc..
CICS PROGRAMSOracle
Connect
JCA Legacy Adapter
REST Adapter
OACCS MCS
ICS
Jet on Node
Onpremise Agent
Mobile Backend
Onpremise
Rest endpoint
OACCS ODCS
Product API
Inventory API
ODI
DB2 KM
updates
ProductProductProducts
Changes in
terminals
blocked
37. {37}
The WAGILE Approach
Sprint
1
Sprint
2
Sprint
3
Sprint
4
Sprint
5
Sprint
N
W-Agile Methodology
Functional
Technical
Solution
Design
New
Requirements
Decompositions
End to
end
testing
SIT,
UAT,
PPT
Release
Support
Several iterations – small scopeFew iterations – large scope
Solution build & component test
Factory Management & Governance
Centre of Excellence
Relea
se
Mgmt.
&
Deplo
yment
Detail Design
Sprint Backlog
Sprints
Scrum Master
DevOps
Pre-SIT
ModernisationFactory
Build & Test
Sprints
DevOps
Work Items Job Card
Cut
over
Go-
Live
Roll
out
39. {39}
I) Avoid the obvious!
Avoid legacy to legacy modernisation!!
A fundamental goal of legacy systems
modernisation is that the target system
doesn’t become a legacy [8]. Modular systems
are easier to evolve
40. {40}
II) No big bangs or cold turkeys
One piece a the time!
Understand the problem. Slice and dice your
elephant by defining boundaries in the
business capabilities. Modernise one piece
at the time. Starting small
Replenishment
Orders
Sales
Orders
Logist
ics
Tracki
ng
41. {41}
III)One domain to rule them all [26]
Gather together those things that change
for the same reason, and separate those
things that change for different reasons
[23][25]
[24]
Domain driven design (DDD) divides up a large system
into Bounded Contexts, each of which can have a unified
model - essentially a way of structuring Multiple
Canonical Models[24]
42. {42}
IV) The main prereq is setting the right goals
Do “You have to be this tall to use
microservices[28]”? – perhaps not!
Sure you need a degree of operational
maturity to adopt microservices, however
maturity is a journey that can be achieved
by setting the right goals and objectives.
So set the right goals!!!
[27]
43. {43}
V) Optimise for speed not efficiency [29]
Speed means learning about your customers and
giving them what they want at a faster
pace[29]. Use simple patterns automated by
tooling [27]. Learn the walk before running.
Speed wins in the
marketplace [27]
44. {44}
VI) 30+ years of research [5][10] proves that:
What you have seen early is just a point of
view (mine!). Do dui-diligence and find the
right approach for your environment. Get
inspiration from lots of research work
There is no silver bullet [26]
45. {45}
VII) The world runs on legacy
So, let’s learn legacy!
Credible research shows that there already
is a huge skill gap in the industry on
COBOL, RPG, PL/I, FORTRAN, PASCAL, C++, etc
[4][5][7][10][11][17].
“Study the past if you would define the future” -
Confucius
46. {46}
References (I)
[1] Legacy Information Systems: Issues and Direction by Bisbal, J.; Lawless, D.; Bing Wu;
Grimson, J.; October 1999 (http://bit.ly/1sVnOYo)
[2] Software Engineering by Ian Sommerville, 10th
edition 2015 (http://amzn.to/1Uo5QnB)
[3] Legacy systems definition by Techopedia(http://bit.ly/1UXRpf8)
[4] Portfolio Analysis, The business case, Aand solution for reducing risk in legacy environments
by Modern Systems (http://bit.ly/22Ezw69)
[5] The burden of legacy by Dr. Toby Sucharov and Philip Rice of Erudine (http://bit.ly/1XYkSWM)
[6] Legacy systems continue to have a place in the enterprise, by computerweekly, June 2008
(http://bit.ly/XeUQ5M)
[7] How Do Professionals Perceive Legacy Systems and Software Modernization? By Utrecht
University, 2014 (http://dl.acm.org/citation.cfm?id=2568318)
[8] DARWIN: On the Incremental Migration of Legacy Information Systems by Michael L. Brodie GTE
laboratories and Michael Stonebreaker University of California Berkeley, March 1993
(http://db.cs.berkeley.edu/papers/S2K-93-25.pdf)
[9] Banks still handicapped by IT legacy, by computerweekly, May 2012(http://bit.ly/1U0UN6N)
[10] Moving from your legacy system- COBOL conversion, cost, and consequences bt Logapps LLC,
August 2015 (http://bit.ly/24orJJ9)
[11] Who Maintains The Legacy Code by Vishwesh Bhat, October 2015 (http://bit.ly/1Y7L7du)
[12] A survey research into legacy system migration by Jesus Bisbal et al, 1997
(https://www.scss.tcd.ie/publications/tech-reports/reports.97/TCD-CS-1997-01.pdf)
[13] Security Concerns for Legacy Systems. An on-going process by Robert Annett, March 2015
(http://www.codingthearchitecture.com/2015/03/07/security_concerns_for_legacy_systems.html)
[14] Working with Legacy Systems, A Practical Guide to the Systems we Inherit and Maintain by
Robert Annett. 80% complete in May 2016 (https://leanpub.com/WorkingWithLegacySystems)
47. {47}
References (II)
[15] The Butterfly Methodology: A Gateway-free Approach for Migrating Legacy Information Systems
by Bing Wu et al (http://dl.acm.org/citation.cfm?id=852129)
[16] Why legacy modernization projects fail, advice on how to how to steer your project toward
Success by David Weldon, October 2014 (http://bit.ly/1t852xy)
[17] Academia needs more support to tackle the IT skills gap by Microfocus, March 2013
(http://bit.ly/22EAb7E)
[18] Microservice Architectures by Dr. Andreas Schroeder (http://bit.ly/1TOGZK8)
[19] Oracle Service Bus for Modernisation strategies by Robert Wunderlich, Ricardo Ferreira, Luis
Weir, October 2015 (http://bit.ly/1ZlWtZR)
[20] Misunderstanding the Problem? By John Marshall, October 2013 (http://bit.ly/1U0TVyT)
[21] Tweet by Adrian Cockcroft (https://twitter.com/adrianco/status/542850261782237184)
[22] Microservices by Martin Fowler (minute 14), GOTO conference, Berlin November 2014
(https://www.youtube.com/watch?v=wgdBVIX9ifA)
[23] Building microservices, designing fine-grained systems by Sam Newman, October 2015
(http://shop.oreilly.com/product/0636920033158.do)
[24] Bounded context by Martin Follower, January 2014
(http://martinfowler.com/bliki/BoundedContext.html)
[25] The single responsibility principle by Robert C. Martin, November 2009
(http://bit.ly/1VDgw79)
[26] The seven deadly sins of Microservices by Daniel Bryant, January 2016
(https://www.infoq.com/presentations/7-sins-microservices)
48. {48}
References (III)
[27] Microservice workshop – craft conference by Adrian Cockcroft, April 2015
(http://bit.ly/24os3aL)
[28] Microservice prerequisites by Martin Fowler, August 2014 (http://bit.ly/1wIjY58)
[29] Adopting Microservices at Netflix: Lessons for Team and Process Design by Tony Mauro, March
2015 (http://bit.ly/25DnkVc)
[30] The twelve factor app by Adam Wiggins (http://12factor.net/)
[31] Lehman's laws of software evolution by Meir "Manny" Lehman and László Bélády, Sep 1980
(https://en.wikipedia.org/wiki/Lehman's_laws_of_software_evolution)
[32] Conway’s law by Melvin Conway, 1967 (https://en.wikipedia.org/wiki/Conway's_law)
[33] The reactive manifesto by Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson, Sep
2014 (http://www.reactivemanifesto.org/)
