EXTENT-2016: Software Testing & Trading Technology Trends 22 June, 2016, 10 Paternoster Square, London The Future of Software Testing Ingo Philipp, Evangelist & Product Manager, Tricentis Would like to know more? Visit our website: extentconf.com Follow us: https://www.linkedin.com/company/exactpro-systems-llc?trk=biz-companies-cym https://twitter.com/exactpro #extent2016 #exactpro

1. Ingo Philip Evangelist
Product Manager
Tricentis
The Future of
Software Testing

2. © 2016 by
Ingo Philipp
© 2016 by .© 2016 by .
The Future of Testing

3. © 2016 by
General Testing Relativity
Your future depends on what you do in your present.
Mahatma Gandhi

4. © 2016 by
80%
Manual Testing
20%
Automated Testing
UI
API
Testing
Present
The reliance on
manual testing is the
top technical challenge
in app development.
World Quality Report, 2015/16
Test automation
requires developers.
Testing Present

5. © 2016 by
Testing
Present
Testing
Future
Exploratory Tests
Automated UI Tests
API Tests
Orchestrated Service Virtualization
80%
Manual Testing
+85%
20%
Automated Testing - key enabler for high automation rates -
Testing Future
UI
API

6. © 2016 by
100%
0%
Sprint
Start
Sprint
End
API
UI
Start testing early.
Shift left test automation.
API Testing
Degree of
Completion
Time
Efficiency
Gain
+4x
Creation
+6x
Maintenance
+20x
Execution

7. © 2016 by
*voke, market snapshot report on service virtualization - 2012
On average, organizations require access to 33systems for development or testing.
18Average # of systems with
unrestricted access
96% of testers have
restricted test lab access
Orchestrated Service Virtualization
Testing
Present
Enterprise system landscapes are alike disease gene networks.
Welcome to the tester’s hell.
Identify 60% to 90% more defects at least
one step earlier in the development process.
*Service Virtualization – Window into Advanced DevOps - 2014

8. © 2016 by
Do the things right.
Automation is the dictate of today.
Automation doesn’t make testing easy, it makes testing possible.

9. © 2016 by
It’s me, a problem!
Start End
software delivery cycle
67%
average level of redundancy in
enterprise test portfolios
40%
average risk coverage achieved in
enterprise test portfolios
90%
of all test automation is
UI test automation
80%
overall testing effort goes
into manual testing
55%
of systems only partially
accessible by Dev/Test
56%
of overall test effort goes into
test case maintenance
30%
of bugs found in acceptance
& production stage
50%
of manual testing goes into test
data preparation & organization
Albert Einstein, 1921
“The first step to solve a problem is to accept that you have one.”
Is it just about speed?

10. © 2016 by
By compressing the delivery cycle,
do problems just move closer to each other?
Is it just about speed?
Start End
software delivery cycle

11. © 2016 by
Start End
software delivery cycle
No, they mutually reinforce each other!
The devil is in
the combination!
It’s about speed@quality.
Hence, the biggest strength of DevOps is not solving
problems, but rather exposing buried problems.

12. © 2016 by
Do the right things right.
Testing harder isn’t the answer, testing smarter is.
Wolfgang Platz

13. © 2016 by
Low
Risk
High
Risk
Medium
Risk
80%20%
Business Risk
Coverage
Test
Cases
The time needed for
testing is infinitely larger
than the time available.
Critical Limit
Risk-Coverage Optimization
Have the right test cases.
Do the right things.

14. © 2016 by
Risks
Your system under test.
Performance Issue
Usability Issue
Functional Issue
Stability Issue
Reliability IssueScalability Issue
Coherence Issue
Understandability Issue
Testability Issue
Convenience Issue
Security Issue
Accessibility Issue

15. © 2016 by
Specification
Based Testing
Exploratory Testing
Exploratory Testing
Analyze Potential Risks
Problem vs. No Problem
Monitor Known Risks
Pass Result vs. Fail Result
Effective testing results from the combination of both.
*That’s an illustration. Don’t confuse it with reality.
Increase your testing cross
section. Actively attack risk.
Go beyond the obvious.
Diversify your testing.
Enrich your test design.
Find more critical bugs.
Fast error detection.
Provide rapid feedback.
Make your testing
intellectually rich.
Core Benefits

16. © 2016 by
100%
Manual
Testing
0%
Automated
Testing
Testing
Present
Customer Case Study
There’s a way to do it
better. Find it!
Project Charter
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution

17. © 2016 by
Risk Coverage Optimization Test Data Management
48
Hours
Execution
Single Agent
8
Hours
Execution
Multiple Agents
Customer Case Study
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution
89%
Business Risk
Coverage
1193
Manual
Test Cases
37%
Business Risk
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
53%
Business Risk
Coverage
Smoke Testing
34
Minutes
Execution
Smoke Testing
Test Case Count 7%
Automation (UI & API) & OSV Continuous Integration & Distributed Execution
75% Redundancy 50% Effort Test Data
64% API Tests

18. © 2016 by
8
Hours
Execution
Multiple Agents
Customer Case Study
72%
Critical Defect Reduction
Production Defects
89%
Business Risk
Coverage
1193
Manual
Test Cases
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
34
Minutes
Execution
Smoke Testing
53%
Business Risk
Coverage
Smoke Testing
Test Case Count 7%
Test Case Design & Exploratory Testing
Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution
64% API Tests

19. © 2016 by
Bottom Line
The future starts today, not tomorrow.
Start remodeling your traditional approaches for greater agility today.