Ho Chi Minh City Software Testing Conference January 2015 Software Testing in the Agile World Website: www.hcmc-stc.org Author: Lee Copeland The IEEE 829 Test Documentation standard is thirty years old this year. Boris Beizer’s first book on software testing also turned thirty. Testing Computer Software, the best selling book on software testing, is twenty-five. During the last three decades, hardware platforms have evolved from mainframes to minis to desktops to laptops to tablets to smartphones. Development paradigms have shifted from waterfall to agile. Consumers expect more functionality, demand higher quality, and are less loyal to brands. The world has changed dramatically and testing must change to match it. Testing processes that helped us succeed in the past may prevent our success in the future. Lee Copeland shares his insights into the future of testing, sharing his Do’s and Don’ts in the areas of technology, organization, test processes, test plans, and automation. Join Lee for a thought provoking look at creating a better testing future.

1. 1
Lee Copeland
Software Quality Engineering
lee@sqe.com
@grandpacopeland
Creating a Better
Testing Future
The World Is Changing and
Testing Must Change With It

2. 2
Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

3. Technology Is Changing
Then
Now

4. Customers Are Changing
Less loyal to
brands
Expect more
capability
Less tolerant
of low quality

5. Business Is Changing

6. • In the past, organizations controlled external
communication, both content and channels
• Today, with the rise of Social Media …
Communication is Changing

7. • The Hotel Story
Communication is Changing

8. Communication is Changing
• Organizational structure controlled tester
communication, both content and channels
• Today, social media can immediately connect
testers world-wide

9. Development Process Is Changing
Then
Now

10. Development Process Is Changing
• COBOL
• CICS
• VSAM
• C#
• Java
• HTML 4/5
• JavaScript
• XML
• SQL
• noSQL
Then
Now

11. Testing Must Match Today’s Technology
IEEE 829
Testing
Standard
1983
1983 1987

12. Testing Must Match Development

13. Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

14. The Curse of Past Successes

15. The Curse of Past Successes
• Processes that made us successful
in the past may prevent us from
being successful in the future

16. The Curse of Past Successes
Camera Film

17. The Curse of Past Successes
Video Rental Stores

18. The Curse of Past Successes
Newspapers

19. The Curse of Past Successes
Brick and Mortar Stores

20. The Curse of Past Successes
Getting Bills in the Mail

21. Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

22. The Don’ts and Do’s
Focus
Organization
TestProcess
TechnologyPlanning
Automation

23. The Don’ts and Do’s
Organization

24. Organization
• DON’T
– Maintain teams of “developers” and “testers”,
throwing products over the wall
– This was supposed to foster “independence.”
Instead, it created “us vs. them” and “huh?”
Development Test

25. Organization
• DON’T
– Establish Testing Centers of Excellence
– “It’s where the disengaged gather to not
produce”
– David Hussman

26. Organization
• DO
– Integrate developers and testers together
on the same team

27. The Don’ts and Do’s
Test Process

28. Test Process
• DON’T
– Cling to a Waterfall style when your
organization is moving toward Agile

29. Test Process
• DON’T
– Try to test EVERYTHING

30. Test Process
• DON’T
– Confuse Checking with Testing
Michael Bolton

31. Test Process
• When we already believe something to be true,
we verify our belief by checking. Checking is a
process of confirmation, verification, and
validation
• Checking is a highly automatable process

32. Test Process
• Testing is something that we do with the
motivation of finding new information
• It is a process of exploration, discovery,
investigation, and learning
• Testing is not automatable. It’s using our brain
in real time, and it’s really fun

33. Test Process
• DO
– Adopt a Customer-focused approach to
testing

34. Test Process
• DO
– Focus on “What Value Does Testing Add?”
rather than “What Processes Does Testing
Execute?”

35. The Don’ts and Do’s
Technology

36. Technology
• DON’T
– Use Stone Age techniques, tools, and
technologies

37. Technology
• DO
– Embrace new technologies

38. Technology
• DO
– Consider open source test tools

39. The Don’ts and Do’s
Planning

40. Planning
• DON’T
– Adopt rigid, overly detailed
plans
– Restrict testing to explicit
requirements
– Remember: “No battle plan
survives contact with the enemy”
– Helmuth von Moltke

41. Planning
• DO
– Create plans that
 Tell a story
 Present essential data
 Inspire and communicate
 Describe a reasoned plan of action

42. The Don’ts and Do’s
Automation

43. Automation
• DON’T
– Rely on Capture/Replay

44. Automation
• DON’T
– Try to automate every test

45. Automation
• DON’T
– Try to replace humans in testing

46. Automation
• DO
– Use Test-driven Development

47. Automation
• DO
– Use Acceptance Test-driven Development

48. Automation
• DO
– Use Exploratory Testing
“Simultaneous
learning, test design,
and test execution.”
– James Bach

49. Automation
• DO
– and Session-based Exploratory Test
Management
Perform Exploratory
Testing in Sessions:
 Charter
 Timebox
 Reviewable result
 Debriefing
Jon Bach

50. The Don’ts and Do’s
Focus

51. Focus on Prevention Over Finding
• DO
– Remember, while finding defects is
important
– We should be focused on defect prevention
activities

52. The Don’ts and Do’s
Focus
Organization
TestProcess
TechnologyPlanning
Automation

53. Thanks for Attending
Shameless
commercial
message
lee@sqe.com

54. My Thanks To
• Dorothy Graham
• David Hussman
• Jonathan Kohl
• Alon Linetzki
• Dan North
• Martin Pol
• Rob Sabourin
• Alan Shalloway
• Adrian Zwingli
For their ideas,
review, and
comments