Exploratory Testing is something that is often misunderstood. It is actually an approach to testing that emphasizes testers ability to explore an unknown object or space through concurrent test design and test execution. While this may not say much to some of you, it is though something that all of us do when we are testing.
In this talk I will discuss the fundamental idea behind exploratory testing and briefly go through sources of information that have helped me develop my understanding of it. My objective is that after the talk you will have understanding about exploratory testing and you will also know where to find more information of it.
4. I want to thank James Bach and Michael Bolton for the
discussions we’ve had related to this topic. Much of my
current understanding has came from those discussions.
9. EXPLORATORY
TESTING
An approach to testing that emphasizes testers
ability to explore an unknown object or space
through concurrent test design and test execution.
Explore = travel through (an unfamiliar area) in order to
learn about it (Oxford Dictionaries)
10. FREEDOM &
RESPONSIBILITY
“A style of software testing that emphasizes the personal
freedom and responsibility of the individual tester
to optimize the quality of his or her work by treating
test design, test execution, test interpretation, and test-related
learning as mutually supportive activities that continue in parallel
throughout the project.” – Cem Kaner
11. At the end of each iteration, each team member should
be able to say why she is worth more money to her
employer than she was at the beginning. – (Brian
Marick)
12. CREATING BY
REACTING
“Likewise, most musicians don't compose entire
melodies in their head and then write them down;
instead, they noodle around on a instrument for a
while, playing with patterns and reacting to what
they hear, adjusting and sculpting.
An essential aspect of a painter's canvas and a
musical instrument is the immediacy with which
the artist gets something there to react to. A
canvas or sketchbook serves as an "external
imagination", where an artist can grow an idea
from birth to maturity by continuously reacting to
what's in front of him.” – Bret Victor (Learnable
Programming)
14. SUMMARIZING
• Most bugs are not obvious
• They are not out there in the open
• They are not found by following a script or walking the
happy path
• They hide all over the spaces that tend to be BIG
If we want to have chances in finding them
(bugs) – we need to become good at
exploring
15. SOURCES OF
INFORMATION
• Evolving Understanding of Exploratory Testing (by
Michael Bolton) http://www.developsense.com/resources.html#exploratory
• Exploratory Testing (by Cem Kaner) http://www.testingeducation.org/BBST/exploratory/
• Telling Your Exploratory Story (by Jon Bach) http://www.youtube.com/watch?v=uUdWm7ddgfU
• Exploratory Testing (by Jon Bach) http://www.youtube.com/watch?v=8Y4WCdJRfV4
• A Case Against Test Cases (by Jon Bach) http://www.quardev.com/blog/a_case_against_test_cases
16. SOURCES OF
INFORMATION
• The Leprechauns of Software Engineering (by Laurent
Bossavit) - https://leanpub.com/leprechauns
• Perfect Software: And Other Illusions about Testing (by
Jerry Weinberg) – You can find it from Amazon
• Explore It!: Reduce Risk and Increase Confidence with
Exploratory Testing (by Elisabeth Hendrickson) – You can
find it from Amazon
• Mindstorms: Children, Computers, and Powerful Ideas (by
Seymour Papert) – You can find it from Amazon
• Mindset: The New Psychology of Success (by Carol
Dweck) – You can find it from Amazon
• “The Idea Accelerator” (by Ken Hudson) – You can find it
from Amazon
17. SOURCES OF
INFORMATION
•
“What is Exploratory Testing?”
http://swtester.blogspot.ca/2012/05/what-is-exploratorytesting.html (by Paul Carvalho)
•
“The Little Black Book on Test Design”
http://thetesteye.com/blog/2011/09/the-little-black-book-ontest-design/ (by Rikard Edgren)
•
“To The New Tester”
http://www.satisfice.com/blog/archives/958 (by James Bach)
•
“Fractal Exploratory Testing” http://www.a-sisypheantask.com/2013/01/fractal-exploratorytesting.html#.UpxHGWRgY3Y (by Adam Knight)
•
“Learnable Programming”
http://worrydream.com/LearnableProgramming/ (by Bret
Victor)
18. SOURCES OF
INFORMATION
•
“Two Forgotten Agile Values: Discipline and Skill”
http://www.exampler.com/discipline-and-skill.html (by Brian
Marick)
•
“Practical Tools For Playing Well With Others”
http://oredev.org/2013/wed-fri-conference/practical-tools-forplaying-well-with-others (by J.B. Rainsberger)
•
“Patterns for The People” http://vimeo.com/19124478 (by
Kevlin Henney)
•
“The Satir Interaction Model” http://sstconline.org/2006/pdfs/JB1365Notes.pdf (by Virginia Satir)
•
“Test Heuristics Cheat Sheet” http://testobsessed.com/wpcontent/uploads/2011/04/testheuristicscheatsheetv1.pdf (by
Elisabeth Hendrickson)
-Understanding about Exploratory Testing based on my understanding about Testing-Not universal definition but helps you to understand what I mean when I talk about testing
Could be “Experimenting with a product” – Didn’t want to restrict your thinking only on a concrete product (e.g. software)Testing can have different missions:Common one being “Find important bugs”Can be also to see if first build of a web site is testable. Yes experimenting with a product in order to evaluate, but not necessarily to find important bugs – instead see if the build is testable. Or will it crash from every click.
In the end testing is much about asking “What if?”.What if I log in with a user that has no access rights? What if I send a message with customer that has leading zeros on its customer number? What if I operate on web portal over 8 hours with less than 15 minutes breaks? I explain that sometimes these questions can lead to me asking a question from someone else, operating with the software by myself or perhaps with the help of some specific tool. All start from a thought that aims at learning more about the product and therefore revealing quality related information.
When we evaluate a test process, we need to evaluate the mental process of testing. What happens on the keyboard, or the questions we make, is important, but still a result of our thinking. Remember this while we move into discussing about Exploratory Testing
Concentrate on the first part – open up the “concurrent test design & test execution” later with the help of a metaphorET is an approach, not a separate techniqueBy testers I mean the person that is testing, should not be confused to a role of tester.If testing is experimenting – then that requires exploration as far as I understand – which means that all testing is exploratory testing
If given freedom to think – need to honor that responsibilityExamples: not using tool when it is a good idea, not improving test data when it is a good idea
Product acts as a something there to react to“Concurrent test design and test execution” can be similar to continuously reacting to what’s in front of him.