EKON20 Conference, November 2016
Starting from Michael C. Feathers “Working Effectively with Legacy Code”, we will introduce the concept of “technical debt”, and some practical patterns to integrate testing, separation of concerns, structure, re-usability, to ease maintenance and evolution of existing projects. Don’t forget that even new projects will soon become legacy… Of course, we will introduce some mORMot features which were developed to reduce your pain in this process.
2. Reduce our Technical Debt
• Notions
– Technical Debt
– Working Effectively with Legacy Code
• Dependency breaking
• Separation of concerns
• Seams
• Testing
3. Reduce our Technical Debt
• Technical Debt
• Working Effectively with Legacy Code
– Software Seams
– Dependency breaking techniques
– Testing is Everything
• mORMot to the rescue
4. Technical Debt
• BBM /Quick & dirty way
sets up with a technical debt
– Like a financial debt,
incurs interest payment
• Extra efforts in any future development
• Higher maintenance costs
5. Technical Debt
• BBM /Quick & dirty way
sets up with a technical debt
– Difficult to measure
• As software productivity is hard to estimate
• No immediate impact on client software
• Mythical man month from management side
7. Technical Debt
• Debt does smell
– Long standing bugs
• Support and customers know workarounds
– Underestimated effort evaluation
• Simple tasks take a lot of time or resources
8. Technical Debt
• Debt does smell
– Unexpected regressions
• “Edit and pray” maintenance style
– Only the initial code writers dare touch it
• Difficult to find people willing to join
• Even the initial coders fail on the task
9. Technical Debt
• Hard to manage
– First step is to face and evaluate it
• Like any debt, could be eventually resolved
– Bankrupt is possible
• Need clear, prioritized and pragmatic approach
10. Technical Debt
• Hard to manage
– Could be integrated into backlog
• Handle debt as part of legacy system
– Full rewrite may not be the solution
• Netscape syndrome
• Technology change mirage
11. Working Effectively with Legacy Code
• Hard problem, but not unsolvable
Working Effectively
with Legacy Code
The book
12. Working Effectively with Legacy Code
“Legacy Code
offers far more possibilities
for the application of design skill
than new feature do”
Michael Feathers, p. 249
13. Working Effectively with Legacy Code
– Identify where code changes are needed
– Get legacy code into a test harness
– Write tests to protect against regressions
– Introduce proper OOP and SOLID patterns
– Interface between old and new code
– Progressive introduction of SOA
16. Working Effectively with Legacy Code
• Identify a Software Seam
– In Clothing
• The place where two parts are stitched together
• The piece on each side
only touches the other right at the seam
• You may be able to replace a piece with another
17. Working Effectively with Legacy Code
• Identify a Software Seam
– In Software
• A place where you can alter behavior in your
program without editing in that place
18. Working Effectively with Legacy Code
• Identify a Software Seam
– Identify the place
where there is a well defined interface
– Allow to replace implementation
without the rest of the software being able to tell
– Prepare Dependency Injection
or Dependency Inversion
e.g. about persistence or third-party feature
19. Working Effectively with Legacy Code
• Identify a Software Seam
– Allow to replace implementation
without the rest of the software being able to tell
– Use Dependency Injection
first implementation may be 1:1
but will reduce coupling for future evolution
20. Working Effectively with Legacy Code
• Identify a Software Seam
– In Real Life Legacy Software
• There is no such place
where there is a well defined interface
• “Let there be light”, thanks to iterative refactoring
• Follow dependency breaking techniques
21. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
– Adapt from existing
– Where you need to (instability, new features)
– Create a production implementation
– Create a fake implementation
– Write associated test cases
– Refine your interfaces
– Iterate
22. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
23. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities big picture
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
24. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
25. Working Effectively with Legacy Code
1. Know your subject
– In a perfect DDD-oriented world
• Reading the code as a book introduces you
into the reality the computer program is modeling
• Legacy code tends to obfuscate reality
in implementation details
due to BBM iteration
was code focused, not business focused
26. Working Effectively with Legacy Code
1. Know your subject
– First gather a high vision of the problem
• From non-tech experts
• Straight to the main points
• Asking for a 5 min presentation from tech experts
27. Working Effectively with Legacy Code
1. Know your subject
– Ask support for long-standing bugs
– Ask customers for their wishes
• Customers have workarounds for bugs,
not for unexpected behavior
– Ask managers for upcoming features
– Ask older coders
• For the positive aspects of the legacy software
• For what should have been done in another way
28. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
29. Working Effectively with Legacy Code
2. Identify responsibilities
– Probably no documentation
or deprecated documentation
– No code conventions
• Naming, style, design
• Procedural style:
– Spaghetti methods
– Stored procedures
– RAD
30. Working Effectively with Legacy Code
2. Identify responsibilities
– Start from a fresh eye
• Even if you work on this code since decades
• This is an expert’s work, not for rookies
• What should we have done?
• Don’t waste time
31. Working Effectively with Legacy Code
2. Identify responsibilities
– Heuristics to find light in the darkness
• Legacy implementation is valuable
– You will learn how the reality is modelized
• Code archeologists enjoy ancient times
– Software “exegesis”
over several layers
with long history
and a lot of material
– Nihil novi sub sole
32. Working Effectively with Legacy Code
2. Identify responsibilities
– Heuristics to find light in the darkness
• Read the code, as a fast book reader
• Keep in mind known bugs and new feature requests
33. Working Effectively with Legacy Code
2. Identify responsibilities
– Heuristics to find light in the darkness
• Read the code, as a fast book reader
• Keep in mind known bugs and new feature requests
• Group classes and methods
• Identify relationships
• Link to stored data structures
34. Working Effectively with Legacy Code
2. Identify responsibilities
– Heuristics to find light in the darkness
• Read the code, as a fast book reader
• Keep in mind known bugs and new feature requests
• Group classes and methods
• Identify relationships
• Link to stored data structures
• Note any change that would be needed
• Ignore the bugs
35. Working Effectively with Legacy Code
2. Identify relationships
– Between variables and methods
– Evaluate if existing coupling is worth it
– Guess abusive class re-use in several contexts
– Guess how it may be uncoupled in the future
– Don’t be fooled by implementation details
– Ignore the bugs
36. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
37. Working Effectively with Legacy Code
3. Study data structures
“Bad programmers worry about the code.
Good programmers worry about data structures
and their relationships.” Linus T.
38. Working Effectively with Legacy Code
3. Study data structures
– Legacy database is where logic remains
• Extract business knowledge from Relational Model
• Using tools, or existing (old) architecture diagrams
– Identify SQL patterns in code
• Copy & pasta blocks may be Seams
• Eventually refactored as Persistence Services
39. Working Effectively with Legacy Code
• Iterative Code Refactoring
4. Break out big classes
5. Break out huge functions
6. Encapsulate global references
40. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
41. Working Effectively with Legacy Code
• Iterative Code Refactoring
4. Break out big classes
• Too much methods are killing the methds
– Confusing to use
– Difficult to expand / adapt
– Breaks Single Responsibility Principle (SOLID)
42. Working Effectively with Legacy Code
• Iterative Code Refactoring
4. Break out big classes
• Refactor by writing new uncoupled classes
executed within the legacy main class
– Old class remains as a Seam
– Old class signature is the Seam interface
– Real implementation moved in new classes
43. Working Effectively with Legacy Code
• Iterative Code Refactoring
4. Break out big classes
• Iterative process, using unitary testing
• Favor composition over inheritance (SRP)
• Introduce interface for abstraction
• Add unit tests on uncoupled software units
44. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
45. Working Effectively with Legacy Code
• Iterative Code Refactoring
5. Break out huge functions
• Refactor a big function/method
into a new class / set of classes
• Preserve the function signature
as class constructors, or public properties
• Local variables as private members
46. Working Effectively with Legacy Code
• Iterative Code Refactoring
5. Break out huge functions
• Iterative process, using unitary testing
• First implement as a single class method
• Then cut it in smaller (private) methods
• Then several classes (SOLID)
• Favor composition over inheritance (SRP)
• Re-use classes when possible
47. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
48. Working Effectively with Legacy Code
• Iterative Code Refactoring
6. Encapsulate global references
Identify global variables in disguise
• Singleton or static variables/functions
• Huge main class (TMainForm TDataModule)
• Everything which prevents reentrance
49. Working Effectively with Legacy Code
• Iterative Code Refactoring
6. Encapsulate global references
Two steps cleaning process
• Identify “families” of global variables or functions
and group them as classes
• Move from a global instance of the class
into an injection mechanism
50. Working Effectively with Legacy Code
• Iterative Code Refactoring
6. Encapsulate global references
• Enable proper unit testing
thanks to Dependency Injection
• Allows server-side concurrent execution
and progressive switch to SOA
51. Working Effectively with Legacy Code
• Refactoring to isolate seams via interfaces
1. Know your subject
2. Identify responsibilities
3. Study data structures
4. Break out big classes
5. Break out huge functions iterative process
6. Encapsulate global references
7. Testing is everything
52. Working Effectively with Legacy Code
7. Testing is everything
– This is perhaps the hardest part
• Especially if unitary test is new for your teams
• Induce new test code to write and debug
– But also the most valuable
• Avoid regressions and refactoring fear
• If you can test, your code is likely to be uncoupled
53. Working Effectively with Legacy Code
7. Testing is everything
– Getting legacy code under test
• To avoid regressions
• To ensure as safe as possible refactoring
• Define reference datasets, for automated testing
– Brute replay of whole application (not yet unit testing)
– Using e.g. Virtual Machines snapshots
• Test “Bug to bug” behavior of the modified code
– Once bugs are fixed, update reference datasets
54. Working Effectively with Legacy Code
7. Testing is everything
– Value the existing
• Numerous applications and proven User Interface
• Integrate new technologies and patterns
• Continuous integration during build process
55. Working Effectively with Legacy Code
7. Testing is everything
– TDD over each Software Seam
• Write a failing test case
• Compile
• Make it pass
• Repeat
56. Working Effectively with Legacy Code
7. Testing is everything
– Seams specific tests
• Unit testing of the well bounded new interface
• Regression testing against the existing
• Prepare integration with the future Seams
57. Working Effectively with Legacy Code
7. Testing is everything
– Uncoupled seams as interfaces
• Could be stubbed/mocked
• Could be reused
• Could be hosted on a Server (SOA/SaaS)
60. mORMot ToolBox
• Cross-cutting features
– Unit Test
– Logging
– TDocVariant JSON
– SynDB SynMongoDB
– SynCrypto SynECC
– Services
– IoC, stubs and mocks
61. mORMot ToolBox
• Cross-cutting features
– Unit Test
• Light and cross-platform
• Convention over configuration
• Integrated with logging
• Integrated with interface Stubs and Mocks
• Sample 07
62. mORMot ToolBox
• Cross-cutting features
– Logging
• Low overhead, fast value serialization
• Local or remote
• Set of events, not levels
• Fast viewer tool with method and threads profiling
• Exception catch, stack trace
• Used by the whole framework
• Sample 11
63. mORMot ToolBox
• Cross-cutting features
– TDocVariant custom type
Stores any transient value as document:
• Object
• Array
• Any nested combination of the two
64. mORMot ToolBox
• Cross-cutting features
– TDocVariant custom type
Low memory overhead
• Data allocation per blocks of variants
• Copy by reference can be enabled
• Instance lifetime managed by the compiler
65. mORMot ToolBox
• Cross-cutting features
– TDocVariant custom type
Direct objects or arrays JSON support
Late-binding magic
Perfect dynamic values for seams
66. mORMot ToolBox
• Cross-cutting features
– TDocVariant custom type
var V: variant; // stored as any variant
...
TDocVariant.New(V); // or slightly slower V := TDocVariant.New;
V.name := 'John'; // property accessed via late-binding
V.year := 1972;
// now V contains {"name":"john","year":1972}
var V1,V2: variant;
...
V1 := _Obj(['name','John','year',1972]);
V2 := _Obj(['name','John','doc',_Obj(['one',1,'two',2.5])]);
V1 := _Json('{"name":"John","year":1982}');
V2 := _Json('{name:"John",doc:["one",1,"two",2.5]}');
67. mORMot ToolBox
• Cross-cutting features
– TDocVariant custom type
writeln('name=',V1.name,' year=',V1.year);
// will write 'name=John year=1972'
writeln('name=',V2.name,' doc.one=',V2.doc.one,' doc.two=',doc.two);
// will write 'name=John doc.one=1 doc.two=2.5
V1.name := 'Mark'; // overwrite a property value
writeln(V1.name); // will write 'Mark'
V1.age := 12; // add a property to the object
writeln(V1.age); // will write '12'
writeln(V1); // implicit conversion to string -> as JSON
// will write '{"name":"Mark","year":1972,"age":12}'
writeln(VariantSaveJSON(V1)); // serialize as JSON text
68. mORMot ToolBox
• Cross-cutting features
• SynDB Direct RDBMS access layer
– Not linked to DB.pas
– Multi providers
– UTF-8 JSON
– Interface based
– Knows SQL dialects (used e.g. by the ORM)
– SynDBExplorer tool
69. mORMot ToolBox
• Cross-cutting features
• SynDB Direct RDBMS access layer
– Less data types
– By-pass TDataSet
– Unicode even before Delphi 2009
– Array binding
– Native JSON support
– Remote access
70. mORMot ToolBox
• Cross-cutting features
• SynMongoDB NoSQL acces
– MongoDB native access
– BSON types - TBSONVariant
– TDocVariant support
– Extended JSON
– MongoDB 3.2 support (SCRAM-SHA1)
71. mORMot ToolBox
• Cross-cutting features
• SynCrypto SynECC
– Introduce safe and fast encryption
• AES, SHA, MD5, PBKDF2, PRNG
• Optimized pascal or asm AES-NI, SSE4
– Certificate-based public key cryptography
• Content signature using ECDSA-secp256r1
• Transmission or file encryption ECDH / ECIES
• JSON-based Public Key Infrastructure (PKI)
72. mORMot ToolBox
• Cross-cutting features
• Services
– Daemonization made easy
– Console, install or service mode
– Remote administration interface
– Windows integration
– Cross-platform
73. mORMot ToolBox
• Cross-cutting features
• Inversion Of Concerns (IoC)
– Leverage interface types for seams
– Runtime class instance resolution
TInjectableObject TInterfaceResolver
– Mocks and stubs
(a killer feature)
74. mORMot ToolBox
• Killer features
– SOA
– ORM / ODM
– Stubs and Mocks
– MVC web portals
– DDD CQRS helpers
– SyNode
76. mORMot ToolBox
• Shared features
– Convention Over Configuration
– Cross-Platform and Cross-Compiler
– Fully integrated and coherent
– Made for scaling and performance
– Modular: pickup what is needed,
integrate with existing
– Open Source with active community
77. mORMot ToolBox
• Don’t be afraid
– Huge feature set
– Exhaustive / endless documentation
– Unusual concepts
• Where to start
– Run the samples
– Use the forum
– Publish your test project on github!
78. mORMot ToolBox
• Resources
– http://synopse.info
– http://synopse.info/forum
– http://blog.synopse.info
– http://github.com/synopse/mORMot
Includes exhaustive documentation,
samples and regression tests.