The IEEE 829 Test Documentation standard is thirty years old this year. Boris Beizer’s first book on software testing also turned thirty. Testing Computer Software, the best selling book on software testing, is twenty-five. During the last three decades, hardware platforms have evolved from mainframes to minis to desktops to laptops to tablets to smartphones. Development paradigms have shifted from waterfall to agile. Consumers expect more functionality, demand higher quality, and are less loyal to brands. The world has changed dramatically and testing must change to match it. Testing processes that helped us succeed in the past may prevent our success in the future. Lee Copeland shares his insights into the future of testing, sharing his Do’s and Don’ts in the areas of technology, organization, test processes, test plans, and automation. Join Lee for a thought provoking look at creating a better testing future.

1.
Session
Presented by:
Lee Copeland
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073
888‐2
W16
Concurrent
4/9/2014
3:15 PM
“Creating a Better Testing Future:
The World is Changing and
We Must Change With It”
SQE
300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Lee Copeland
SQE
With more than thirty years of experience as an information systems professional at
commercial and nonprofit organizations, Lee Copeland has held technical and
managerial positions in applications development, software testing, and software
process improvement. Lee has developed and taught numerous training courses on
software development and testing issues, and is a well-known speaker with Software
Quality Engineering. Lee presents at software conferences in the United States and
abroad. He is the author of the popular reference book, A Practitioner’s Guide to
Software Test Design.

3. 2/11/2014
1
Creating a Better
Testing Future
Lee Copeland
Testing Future
The World Is Changing and
Testing Must Change With It
1
Lee Copeland
Software Quality Engineering
lee@sqe.com
Topics
• Our World is Changingg g
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success
2
2

4. 2/11/2014
2
Technology Is Changing
Then
Now
33
Now
Customers Are Changing
Expect more
capability
Less tolerant
of low quality
44
Less loyal to
brands

5. 2/11/2014
3
Business Is Changing
55
• In the past, organizations controlled external
Communication is Changing
p , g
communication, both content and channels
• Today, with the rise of Social Media …
66

6. 2/11/2014
4
Communication is Changing
• Organizational structure controlled testerg
communication, both content and channels
• Today social media can connect testers world
77
• Today, social media can connect testers world-
wide
Development Process Is Changing
hThen
Now
88
Now

7. 2/11/2014
5
Development Process Is Changing
• COBOL
• CICS
• VSAM
• C++
• Java
• HTML 4/5
Then
99
HTML 4/5
• JavaScript
• XML
• SQL
• noSQL
Now
Testing Must Match Today’s Technology
IEEE 829
Testing
Standard
1983
1983 1987
1010

8. 2/11/2014
6
Testing Must Match Development
1111
Topics
• Our World is Changingg g
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success
1212

9. 2/11/2014
7
The Curse of Past Successes
1313
The Curse of Past Successes
• Processes that made us successful
in the past may prevent us from
being successful in the future
1414

10. 2/11/2014
8
The Curse of Past Successes
1515
Camera Film
The Curse of Past Successes
1616
Video Rental Stores

11. 2/11/2014
9
The Curse of Past Successes
1717
Newspapers
The Curse of Past Successes
1818
Brick and Mortar Stores

12. 2/11/2014
10
The Curse of Past Successes
1919
Getting Bills in the Mail
Topics
• Our World is Changingg g
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success
2020

13. 2/11/2014
11
The Don’ts and Do’s
Focus
Organization
TestProcessAutomation
2121
TechnologyPlanning
The Don’ts and Do’s
Organization
2222

14. 2/11/2014
12
Organization
• DON’T
– Maintain teams of “developers” and “testers”,
throwing products over the wall
2323
– This was supposed to foster “independence.”
Instead, it created “us vs. them” and “huh?”
Development Test
Organization
• DON’T
– Establish Testing Centers of Excellence
2424
– “It’s where the disengaged gather to not
produce”
– David Hussman

15. 2/11/2014
13
Organization
• DO
– Integrate developers and testers together
on the same team
2525
The Don’ts and Do’s
Test Process
2626

16. 2/11/2014
14
Test Process
• DON’T
– Cling to a Waterfall style when your
organization is moving toward Agile
2727
Test Process
• DON’T
– Try to test EVERYTHING
2828

17. 2/11/2014
15
Test Process
• DON’T
– Confuse Checking with Testing
2929
Michael Bolton
Test Process
• When we already believe something to be true,y g ,
we verify our belief by checking. Checking is a
process of confirmation, verification, and
validation
• Checking is a highly automatable process
3030

18. 2/11/2014
16
Test Process
• Testing is something that we do with theg g
motivation of finding new information
• It is a process of exploration, discovery,
investigation, and learning
• Testing is not automatable. It’s using our brain
in real time, and it’s really fun
3131
Test Process
• DO
– Adopt a Customer-focused approach to
testing
3232

19. 2/11/2014
17
Test Process
• DO
– Focus on “What Value Does Testing Add?”
3333
rather than “What Processes Does Testing
Execute?”
The Don’ts and Do’s
Technology
3434

20. 2/11/2014
18
Technology
• DON’T
– Use Stone Age techniques, tools, and
technologies
3535
Technology
• DO
– Embrace new technologies
3636

21. 2/11/2014
19
Technology
• DO
– Consider open source test tools
3737
The Don’ts and Do’s
Planning
3838

22. 2/11/2014
20
Planning
• DON’T
– Adopt rigid, overly detailed
plans
– Restrict testing to explicit
requirements
3939
– Remember: “No battle plan
survives contact with the enemy”
– Helmuth von Moltke
Planning
• DO
– Create plans that
Tell a story
Present essential data
Inspire and communicate
4040
Describe a reasoned plan of action

23. 2/11/2014
21
The Don’ts and Do’s
Automation
4141
Automation
• DON’T
– Rely on Capture/Replay
4242

24. 2/11/2014
22
Automation
• DON’T
– Try to automate every test
4343
Automation
• DON’T
– Try to replace humans in testing
4444

25. 2/11/2014
23
Automation
• DO
– Use Test-driven Development
4545
Automation
• DO
– Use Acceptance Test-driven Development
4646

26. 2/11/2014
24
Automation
• DO
– Use Exploratory Testing
“Simultaneous
learning, test design,
and test execution.”
4747
– James Bach
Automation
• DO
– and Session-based Exploratory Test
Management
Perform Exploratory
Testing in Sessions:
Charter
4848
Timebox
Reviewable result
Debriefing
Jon Bach

27. 2/11/2014
25
The Don’ts and Do’s
Focus
4949
Focus on Prevention Over Finding
• DO
– Remember, while finding defects is
important
– We should be focused on defect prevention
activities
5050

28. 2/11/2014
26
The Don’ts and Do’s
Focus
Organization
TestProcessAutomation
5151
TechnologyPlanning
Thanks for Attending
Shameless
commercial
message
lee@sqe.com
5252

29. 2/11/2014
27
My Thanks To
• Dorothy Graham
• David Hussman
• Jonathan Kohl
• Alon Linetzki
• Dan North
• Martin Pol
For their ideas,
review, and
comments
5353
• Rob Sabourin
• Alan Shalloway
• Adrian Zwingli