Exploratory Testing in Practice Elisabeth Hendrickson Blog: testobsessed.com Twitter: @testobsessedSlides Last updated June 22, 2012
Two Sides of Testing Tested = Checked + Explored
That’s Nice… …But we Believe in ourcomprehensive, repeatable regression tests.
Incidental Correctness Given a series of 1’s and 0’s, Compute the corresponding decimal number Input Expected Result 000 0 0 111 7 7 101 5 5x 011 3 6
Missed RequirementsHey, Where’s the Motor?!? motor for this You neverremote control said anything car? about a motor!
Real World Users Happen You assume they’re reasonable people who will do reasonable things. They aren’t and they won’t.
So What Is Exploratory Testing?a systematic approach fordiscovering risksusing rigorousanalysis techniquescoupled withtesting heuristics.
OK…But What Do You Actually Do?Simultaneously……learning about the Target…While designing tests…And executing Them Immediatelyusing feedback from the last LittleExperiment to inform the next (Term coined by Cem Kaner. Definition refined byJames Bach. Others have contributed to ET including James Lyndsay, Jonathan Kohl, Michael Bolton.)
ExperimentSo it handled all Next, let’s try the variations interrupting theon disconnected transaction networks we here. could think of…
Use State Models Idle Time to No connect NetworkSystemUpdated Check Network No Updates Network Avail Update Get System Updates Content Downloaded 11
Use Data Models Customer VendorHas 0 or More Orders Has 0 or More Has 1 Or More Products
Use Deployment Diagrams Server FarmLocal Agent AWS
Use Heuristics See the Test Heuristics Cheatsheet for more http://bit.ly/testcheatsheetGoldilocks: too big, too small, just right Counts: 0, 1, ManySelection: Some, None, All Position: Beginning, Middle, EndViolate Domain-Specific Rules:Age -1 IP address 999.999.999.999Interrupt States Delete Dependent Data
In Flight Feedback at #AU2H Matt Barcomb (Coding)Jon Bach(Exploring)
Result: Feedback on Process AND Product We’ve automated I’ll add things the regression like that to thetest when we fixed acceptance the bug. criteria. We need to think about that kind of thing in future designs too.
Pitfall: Going Off in the Weeds it doesn’t work on15 year old laptopsrunning Windows 3.1! Uh… Why Would We Care?
Focus with ChartersExplore (Area/feature)With (Resources,conditions, Dependency,etc.)to discover (information)
Structure with Time Boxed Sessions Charters SessionsExplore login in combination Sessionwith bookmarks & history todiscover information about Sessioninteractions. SessionExplore the site with variousbrowsers and settings to Sessiondiscover risks related tounsupported configurations. SessionExplore error handling usingdata type attacks to find Sessionproblems related to datavalidation.