Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TestIstanbul May 2013 Keynote Experiences With Exploratory Testing

Slides for Keynote: Experiences With Exploratory Testing

Delivered at TestIstanbul May 2013 by Alan Richardson

  • Be the first to comment

TestIstanbul May 2013 Keynote Experiences With Exploratory Testing

  1. 1. Experiences with Exploratory Testing Alan Richardson @eviltester
  2. 2. Future of Testing: New Techniques and Methodologies ● Exploratory Testing is not new, ● but people still don't do it. ● The future will be the same, as now, ● unless we take responsibility for it. ● Learn from experience, ● base experiments on observation.
  3. 3. Some things I've learned by doing and teaching Exploratory Testing ● Adopt a personal approach ● Exploration & experiment is natural ● Environment can kill exploratory testing ● Evidence based documentation ● Use Automation in your exploration
  4. 4. What does "Exploratory Testing" mean? "An Interactive Process of Simultaneous learning, test design, and test execution" James Bach
  5. 5. "If you master the source, but not the teachings, whenever you open your mouth you will speak at random; if you master the teachings but not the source, you will be like a one-eyed dragon." The story of Chinese Zen, by Nan Huai-Chin
  6. 6. Adopt a personal approach to Testing Take responsibility for your test approach: ● What does it mean to me? ● How do I do it? ● Why did I start? ● Why do I do it?
  7. 7. What do I mean by exploratory testing? ● Continual Evaluation & Re-evaluation ● Observation & Evidence ● Tool Augmented ● Low Overhead & Fast ● Skill Based ● Team Based ● Intent Characterized by:
  8. 8. Low Overhead & Fast ● Fast paced environments, becoming norm ● Success based on outcome, not compliance ● Maximize the time spend "doing" Techniques: ● Think through risks and target those. System risks, business risks. ● Chunk testing into 'charters' and 'sessions'. Log the testing against those. Raise defects.
  9. 9. Skill Based Rather than just relying on 'Techniques'... ● Assume the tester knows the techniques and applies them on an adhoc skill based basis ● Train people where risks and bugs exist and how to look for them ● Train people to be adaptable ● Trust people, give them responsibility
  10. 10. Team Based ● Use the whole team to review everyone else's work ● Write logs that others can read ● Review coverage of others testing and suggest new scope ● Pair to come up with new ideas ● Debrief rather than report ● The Manager tests as well
  11. 11. Continual Evaluation & Re-evaluation ● Start with minimal scope ● Test to learn what you need to test ● Write down what you know and don't know ● Continually revisit your approach and testing scope ● Revisit your use of tools ● Question your effectiveness ● Model the system and the process
  12. 12. Intent ● Know what you are testing for. ● Decide on what to test next. ● What risk are you concerned about? ● What requirement are you investigating? ● What data are you covering? ● Don't do it because you planned to. ● Do it because it is important now.
  13. 13. Observation & Evidence ● Focus / Defocus ● Go Meta ● Go Deeper ● Peripheral Observation ● Relate to existing evidence ● Produce Evidence ● screenshots, system logs, ● recordings, manual logs
  14. 14. Tool Augmented ● Tools augment our testing to help us test better ● Tools should never drive or dictate our process ● Exploratory Testing has heavy use of tools for: Observation, Data Creation, Logging, State Manipulation
  15. 15. My Exploratory Process ● Modelling ● Observing ● Reflective ● Intentional ● Manipulative
  16. 16. Why should you do exploratory testing? • Because you liked the definition? • Because you liked the characteristics? • Because you want to test more effectively? Personal Approach I don't know why you should do it. I only know why I do it, and why I started doing it.
  17. 17. Why did I start doing exploratory testing? "There is a risk that the test team will waste a lot of time and money by creating scripts that are wrong and will never be run" • Frustration Mainly • Heavily Scripted environment which wasn't working well
  18. 18. In order to be effective we had to Subvert the process Step # Description Expected Result 1 I've only added this because the tool made me I'll test the conditions in the case and write down my observations
  19. 19. Why do I keep doing Exploratory Testing? Because: ● It works ● I get better at it, the more I do it ● I can train people to do it ● It adapts to different environments, and processes
  20. 20. ● Stop Following ● Do and Learn ● Change What you do Take Responsibility to succeed
  21. 21. Develop Ability & enable Response Ability Response Able Adapt Able Adapt Ability Account Ability Account Able
  22. 22. What stops Exploratory Testing? ● Environment ● Personal Fear ● Existing Learning
  23. 23. Automation & Exploratory Testing ● Tool Augmented ● Exploratory Automated Testing ● Models & Checks ● Automation leads to Exploration
  24. 24. An example: Random Coverage of  Defined Scope ● New Registration Process ● Manual Testing kept finding problems, ● Every app change introduced new errors ● Randomize Coverage of a Model ● Model registration as a process of equivalent steps ● Fill Name, Fill Password, Fill PostCode etc. ● Model data as a set of templates to randomly fill ● {a-z,A-Z, }*3-25, {a-z,A-Z,0-9,!"£$%^&*()}*6-18 ● Leave to run, Automated Checks flag issues ● When run is clean, perform exploratory testing
  25. 25. When Automation Leads to exploration ● Sometimes you want to get somewhere known fast, ● so that you can explore unknowns ● Automate the 'getting to' ● Randomise to maximise chance that this shows a problem ● When there, engage brain and test
  26. 26. Explore and Experiment with your Testing ● The Future of Testing requires us to Unlearn. ● Change our existing beliefs based on evidence. ● Make it easy to experiment with new approaches. ● Evaluate Qualitatively as well as Quantitatively. ● Take responsibility, own your test approach.
  27. 27. 27 Blogs and Web Sites ● ● ● Testing Papers and Tools – @eviltester Online Training Courses ● Technical Web Testing 101 – ● Intro to Selenium – ● Selenium 2 WebDriver API – Videos Books Selenium Simplified Alan Richardson is an Independent Test Consultant based in the UK. He offers training and consultancy in Selenium WebDriver, exploratory and technical web testing. – Contact Alan for training and consultancy tailored to your needs: