It is now more challenging than ever before to deliver business critical software in the Finance industry. Test automation is no longer optional, but it's difficult to achieve; it effectively; A strategy that suits one business may not suit another, pay-back times can be long and maintenance expensive. What approach should be taken? How should the team be structured? How should the right level of coverage be determined? This presentation talks to some of these challenges and offers some possible solutions.

1. • Challenges in Finance
• How to Automate
1. Automation Tools
2. Getting the Basics Right
3. One Size Does Not Fit All
4. Build the Team
5. Coverage – Science or Craft?
6. Get Inside the Code!
7. Pick your Arguments
8. Incentivise Good Behaviour
Test Automation
John Stinson

2. Challenges in Finance
• Finance IS technology
• Complexity is higher than ever before
• Mergers have lead to accumulation of systems
• Increased regulations have increased pressure
• Teams are frequently spread globally
• Volumes have increased 20 times since 1995
• Latencies have decreased to milliseconds
• Proliferation of custom software. Approval to use open-source takes
time.
• Operations separated from developers via entitlements
• No sharing of information via blogs or social media
• Different personalities, some driving automation, some not
• Automation may pay off in 2-3 years. Bonus cycle is annual

3. 1) Automation Approaches
• Server
– Unit Tests eg JUnit
– Fluent Tests eg AssertJ, FEST
– Message Record/Replay, eg Rational Integration Tester
– Custom Frameworks
– BDD: Cucumber Ruby or JVM
– Prod Log File Replay
• GUI
– Unit Tests
– BDD: Specflow
– Selenium (Web)
– QTP, UFT, TestComplete
– Bitmap eg Sikuli
Cheaper
To
Maintain
- How easy to Maintain?

4. 2) Get the Basics Right
• Stub the Interfaces!
• Lock the Data Down
• Sort Out the Environments!

5. 3) One Size Does Not Fit All

6. Different Approaches for Different
Sizes
Statement,
Method,
Class,
Component
System
Unit/Fluent/BDD
Message Replay,
Custom Framework,
BDD
GUI

7. Google Testing Sizes
Simon Stewart

8. Different Approaches for
Different Lines of Business

9. Different Approaches for
Different Lifecycle Stages
Cutting Edge
Proceed Cautiously
Stable
Move Forward
Legacy
Handle with Care
Focus on Changes or Wrap with Cheap Regression

10. Dev/
BA
BA/
QA
BA/
QA
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
BA
Dev/
BA
Dev/
BA
Dev in
Test
Classic BDD
Developer in Test
Dev/
BA
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
Envt/
Auto
Dev/
Envt/
Auto
BA
Dev/
Envt/
Auto
Developer Quality Mentality
Gherkins
Developers automate
Dev in
Test
Dedicated
automators help
keep pace
Automation focused
Dev lead drives team behaviour
4) Team Models that Work
Devs
write
frame-
work Dev in
Test
BA/
QA
QAQA checks coverage

11. 5) Coverage
Science or Art?

12. Science of Coverage
2*12*2*2*2*4*10 = 7680

13. Understand Risk
Perform Exploratory Testing
Summarise Coverage to Stakeholders
Art of Coverage

14. 6) Get Inside the Code!
Diff Code to understand risk of late changes
View Unit Test Code Coverage in Intellij
Find interesting edge cases in C# using PEX
Analyse Dependency Graph in Java
Analyse Performance with jvisualvm

15. 7) Pick your Arguments
• Teamcity vs Jenkins?
• Cucumber Ruby vs Cucumber JVM?
• Config Management in SVN or GIT/Stash?
• Certain level of unit test Coverage?
• Code review process?

16. 8) Incentivise Good Behaviour with
Metrics
• What are we trying
to achieve?
• How many test scenarios automated out of
the manual pack?
• Unit test coverage?
• Total time taken to test?
• Production defects?