Digital transformation requires continuous testing to quickly adapt to changes and deliver value to customers. Traditional testing relies heavily on manual testing which is inefficient. Continuous testing uses test automation to test early and often through the development process. This shifts testing left and allows for more frequent testing of applications and APIs. Continuous testing leverages techniques like exploratory testing, test automation, and continuous integration to significantly increase testing efficiency and reduce risks. The goal is to close knowledge gaps through continuous and adaptive investigation.

1. Continuous Testing
Digital Transformation Requires Continuous Testing
Ingo Philipp

2. Businesses must
continuously exploit
digital technologies to both
create new sources
of customer value and
increase operational
agility in service of
customers.
© Forrester Research

3. Digital
Disruption
Across industries, companies face the
challenge of software-led transformation

4. Plan Code Version Build Provision Test Deploy Monitor
© Melissa Perri
The product is a solution.
If the problem isn't solved,
the product doesn't work.
The goal is to
accelerate this
value exchange
Identify
Needs
Receive
Money
Build
Solution
Your Company Your Customer

5. Requirements Design Implementation Testing Acceptance Deployment
Inception Elaboration Construction Transition
Development Operations
DevOps
Agile Horizon
Digital Singularity
Incremental
Rational Unified Process
4
Agile
Scrum, Kanban
2
Continuous
DevOps
1
6+
Big Bang
Waterfall
Development Response
Flexible Framework
Rigid Rules
Following a Plan
Contract Negotiation
Copious Documentation
Processes & Tools
Individuals & Interactions
Working Software
Customer Collaboration
Responding to Change

6. It’s not the strongest
that survive, nor the most
intelligent, but the one most
responsive to change
Charles Darwin
Because software is not written, it is rewritten

7. 80%
Manual Testing
20%
Test Automation
The reliance on
manual testing is the
top technical challenge
in app development
World Quality Report, 2015/16
Test automation
requires developers
UI LoadAPI
Present
Traditional Testing

8. Future
Continuous Testing
+85%
Test Automation
Exploratory Tests
Automated UI Tests
API Tests
Load Tests
80%
Manual Testing
20%
Test Automation
Present
Traditional Testing
Orchestrated Service Virtualization

9. 100%
0%
Sprint
Start
Sprint
End
API
UI
Start testing early,
shift left test automation Degree of
Completion
Time
Efficiency
Gain
+4x
Creation
+6x
Maintenance
+20x
Execution

10. *voke, market snapshot report on service virtualization - 2012
On average, organizations require access to 33systems for development or testing.
18Average # of systems with
unrestricted access
96% of testers have
restricted test lab access
Testing
Present
Enterprise system landscapes are alike disease gene networks.
Welcome to the tester’s hell

11. Automation doesn't make testing
easy, it makes testing possible
Lesson Learned
© Wolfgang Platz

12. A
B
Visiting London
~100 Billion Billion Billion Billion Billion Billion Billion Combinations
= 50

13. Testing harder isn’t the
answer, testing smarter is!
Lesson Learned
© Wolfgang Platz

14. Test Case

15. Passing
Test Case

16. Passing
Test Case
@iamdevloper

17. © BBC

18. Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan © BBC

19. Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan
That’s what we can know

20. Testing is exactly like
washing a pig. Because
it’s messy. It has no rules. No
clear beginning, middle, or end.
It’s kind of a pain in the ass,
and when you’re done you’re
not sure if the pig is really clean
or even why you were washing
a pig in the first place.
© Luke Sullivan
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
© Richard Bradshaw

21. © Cem Kaner
© Richard Bradshaw
The purpose of testing is to
close the knowledge gap
The goal is information,
not gratuitous automation
Testing is and always will be
a search for information
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap

22. That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
Checking
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process

23. » Here You Monitor Known Risks
That’s what we can know
That’s what we know
That’s
what we
check
Problem
Knowledge Gap
Confirmatory Testing
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process
MechanicalProcess

24. Confirmatory Testing
« Requires Processing »
© Rich Rogers
Verify through
Instructions
Pay attention to
Deviations
Create
Test Cases
Follow
Procedure
Examine
Requirements
Factory
Process
Exploratory Testing
« Requires Thinking »
Investigate through
Experiments
Pay attention to
Oracles
Create
Test Ideas
Follow
Clues
Examine
Risks
Adaptive
Investigation
» Here You Analyze Potential Risks
CognitiveProcess
MechanicalProcess » Here You Monitor Known Risks

25. Checking
Efficient Confirmatory Testing
Exploring
Effective Exploratory Testing
Testing
Thorough Testing
+
=
Agile
Testing Equation
© Elisabeth Hendrickson

26. If this doesn’t make an impact, then
its absence doesn’t make a difference
Our Proof
© Trey Smith

27. 100%
Manual
Testing
0%
Automated
Testing
Testing
Present
There’s a better
way to do it. Find it!
Project Charter
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution

28. Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution
48
Hours
Execution
Single Agent
8
Hours
Execution
Multiple Agents
11
Manual
Testers
4755
Manual
Test Cases
?
Unknown
Coverage
10
Weeks
Execution
89%
Business Risk
Coverage
1193
Manual
Test Cases
37%
Business Risk
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
53%
Business Risk
Coverage
Smoke Testing
34
Minutes
Execution
Smoke Testing
Test Case Count 7%
75% Redundancy 50% Effort Test Data
64% API Tests

29. 8
Hours
Execution
Multiple Agents
72%
Critical Defect Reduction
Production Defects
89%
Business Risk
Coverage
1193
Manual
Test Cases
5
Weeks
Execution
2.5
Weeks
Execution
92%
Automated
Test Cases
34
Minutes
Execution
Smoke Testing
53%
Business Risk
Coverage
Smoke Testing
Test Case Count 7%
Test Case Design & Exploratory Testing
64% API Tests
Risk Coverage Optimization Test Data Management UI/API Test Automation Continuous Integration & Distributed Execution

30. The future doesn’t just happen,
it gets happened, so make it happen
Conclusion
© Paul Tagliablue

31. Questions
The show is over. It’s your turn.
?