Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Exploratory testing

1,270 views

Published on

This is a presentation I made for Dev.bg

Published in: Software
  • Login to see the comments

Exploratory testing

  1. 1. 1 EXPLORATORY TESTING SCIENTIFIC AND CREATIVE THINKING IN REAL TIME Created by Viktor Slavchev Blog: Twitter:mrslavchev.com @Mr_Slavchev
  2. 2. 2 WHERE ET GAINED ITS BAD REPUTATION Three Digestible Diagrams to Describe Exploratory Testing
  3. 3. 3 THE PURPOSE OF TESTING?
  4. 4. 4 DEFINITION BY THE "OTHERS" Common answers: to find bugs to prove the product works to assure good quality to execute test cases
  5. 5. 5 BETTER DEFINITION BY CEM KANER'S BBST COURSE So ware testing: is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or the system under test
  6. 6. 6 THE FOUR PILLARS OF TESTING MYTHOLOGY:
  7. 7. 7 IT'S ALL ABOUT FINDING BUGS
  8. 8. 8 WRONG! Finding bugs is the result of our effort, not its purpose What if we find requirement mismatch? It's not a bug. O en the term bug is defined as a coding error only, but it can be much more: Usability issue - not an error, but terrible to use A bottle neck - terrible performance Security vulnerability What if we are looking for eventual risk to help prevention of bugs?
  9. 9. 9 TO GUARD/ASSURE/PROTECT THE QUALITY
  10. 10. 10 WRONG! How will you assure quality if you can't: Control developers' quality Control management quality Assure overall testing quality Assure your own quality - due to bias Quality is not a role, it's a mission A mission to be shared by all team members Quality is subjective Quality should be the greatest common divisor from the opinions of everyone involved in the project
  11. 11. 11 TESTING IS EASY Everyone can do it
  12. 12. 12 WRONG! "Playing" with the product is not testing Finding one obvious bug is not testing What anyone can do is not testing in the meaning of creative activity using the methods of modeling, questioning, exploratorion and experimentation performed by skilled experts aiming to expose vital information about the product and eventual risks so someone who matters can make an informed decission about its destiny
  13. 13. 13 TO EXECUTE SCRIPTS BASED ON REQUIREMENTS
  14. 14. 14 WHAT IF WE DON'T HAVE REQUIREMENTS? We can always quit
  15. 15. 15 WRONG! Representing testing as a script is early and mostly unnecessary formalisation. We do more than actually testing Might introduce tunneled vision - turn testing into confirmatory activity Please, don't "execute" the poor tests - testing is a performance, not a public execution Testing should be investigation with an open answer Test to prove that so ware doesn't work instead
  16. 16. 16 THEREFORE WHY NOT AUTOMATE IT?
  17. 17. 17 HOW DO PEOPLE SEE TESTING AUTOMATION? Chuck Norris of so ware testing, you run it and all bugs surrender.
  18. 18. 18 MESSAGE BY OUR SPONSOR: THE TESTING TROLL
  19. 19. 19 Next time your manager offers you to automate your job in order to save money, tell him: Why not automate your management, instead?
  20. 20. 20 HOW TO AUTOMATE MANAGEMENT?
  21. 21. 21 BUT, BUT, BUT... MANAGEMENT IS NOT THAT EASY Management includes taking responsibility for your actions, having knowledge on your context, taking risky decissions, so skills, technical knowledge, problem solving skills, good self management skills, knowledge in social sciences and etc. Now replace "management" with "testing"
  22. 22. You are not a manual tester, unless you are testing manuals. You are a tester. You DO use tools. You may not be writing automated checks. — Michael Bolton (@michaelbolton) January 17, 2017
  23. 23. 22 23 HOW AUTOMATION FITS INTO THE OVERALL TESTING PROCESS?
  24. 24. 24 THE DANGERS OF THE MANUAL/AUTOMATION DICHOTOMY Both definitions target only the interaction level of testing the product and ignore all other activities that we perform People thinking they can not/should not use tools to ease their job People being afraid to use their brains in testing People expecting Selenium/Cucumber/Ranorex or any other tool to do the job for them People preaching that human testing is absolutely irreplacable by automation People blaming automation for their poor test design, calling it "flaky tests"
  25. 25. 25 “Test design and test execution and learning mixed together in a mutually supportive way" James Bach WHAT IS EXPLORATORY TESTING Source: What is exploratory testing What is ET:
  26. 26. 26 NO BEST PRACTICES!
  27. 27. 27 "Best practices" are an empty container - they normally offer simple solution to a complex problem. There's no complex problem that has a fast and simple solution. They all require full involvment and in-depth analysis.
  28. 28. 28 KNOW YOUR CONTEXT We're not simply trying to catch all bugs, it's impossible We try to eliminate the risks that might cause loses to our business There is certain contexts where you will be expected the exactly opposite of what "the best practices advice you to do" Not considering your testing context might lead to blindly performing acitons that bring limited or no value to testing
  29. 29. 29 KNOWLEDGE OF COVERAGE How your testing effort "covers" the product? When we are done? How much testing is enough? Normally by testing coverage only requirements coverage is meant But there is more: Branch/Decision coverage Data coverage Error coverage Race condition coverage Variable coverage and more... in "On neglicence and testing coverage" Cem Kaner lists 101 types of coverage
  30. 30. 30 HEURISTICS AND ORACLES INSTEAD OF "BEST PRACTICES" Definition of the term "heuristic": "Approach to problem solving, learning, or discovery that employs a practical method not guaranteed to be optimal or perfect, but sufficient for the immediate goals" Definition of a testing oracle: "It's a heuristic device we use in order to tell if certain behaviour is an anomaly"
  31. 31. 31 TESTING ORACLES Product should be consistent with: History Image Comparable products Claims User's desires Product Purpose Statutes Source: Few HICCUPPS by M. Bolton
  32. 32. 32 DOING AUTOMATION FOR THE RIGHT PURPOSE Automation is a tool, just like any other, it can help you test, it can not test on its own. Try to think what benefits from automation you could get in your practical testing activities Find good automatable candidates - simple, fast and reliable No end to end testing is not a good candidate Awareness of what automation is good at and what it sucks at Awareness of the fallability of automated checks
  33. 33. 33 THE PROFILE OF AN EXPLORATORY TESTER Naturally curious Constantly suspicious to anything, mostly to him/herself Technical Able to produce documentation that describes his testing, not testing that describes his documentation Assisted by tools, not driven by them Investigative Scientific Creative
  34. 34. 34 THANK YOU VERY MUCH! If you liked it, you can find my blog here: http://mrslavchev.com Or let me know via email: mr.slavchev@gmail.com Any feedback is welcome! :)
  35. 35. 35 CREDITS: The title is a paraphrase of a quote by James Bach original: Exploratory testing: “scientific thinking in real- time” James Bach from the article "Exploratory testing explained" I don't take credit for any of the images, except the one that's mine, obviously, yet I am too lazy to link all of them, so - image credits - the internet Thanks to Irina for pointing out some dumb typos I made, because I am biased :)

×