Perhaps the biggest untold story in software testing is the untellable story of software testing skills. Let's talk about that. James Bach will attempt to describe and explain why most of what is important in software testing is routinely NOT described or explained. Tacit knowledge plays a huge role in our ability to test, and that's why so many people think that testing is simple and easy. Ask any experienced tester how to test and you are likely to get mumbles and platitudes-- which is exactly what you get from inexperienced testers. We need to get a handle on this problem, and we can do that in two ways: A) Learn how to make a little more of our knowledge explicit, and B) Learn how to develop and transmit tacit knowledge WITHOUT making it explicit. Come and hear about how to do that .

1. 7/14/2014
1
Testing Unexplained(The Importance of Tacit Knowledge in Testing and How to Develop It)
James Bach
@jamesmarcusbach
james@satisfice.com
Consulting Software Testing, Satisfice, Inc.
Tacit and Explicit Knowledge
EXPLICIT means it can be represented completely in the
form of a string of bits: words, pictures, even actions can be
explicit. (software is explicit)
TACIT means it is not manifested in a form that can be
equated to a string of bits: it is unspoken, unwritten,
unpictured.
• Relational Tacit Knowledge is tacit by convenience.
• Somatic Tacit Knowledge is tacit in your body.
• Social Tacit Knowledge is tacit in a community.
(see Collins, Tacit and Explicit Knowledge)

2. 7/14/2014
2
Examples of the Need for Tacit Knowledge
• You consider how users will interact with and adjust to a product.
• You consider what a specification and product was intended to say or
do, not just what they literally say and do.
• You notice what the specification and product strangely omits.
• You focus on business risks even when no one tells you what they are.
• You notice changes in the product’s behavior over time—none of which are
failures in and of themselves—you form conjectures of why that
happens, and you connect those conjectures with an evolving
understanding of plausible error and failure patterns.
• You use tools, you notice when those tools misbehave, and you make
adjustments or workarounds to get the job done.
• You report your test results based partly on how you think your clients
will react. You anticipate their questions based on your insight about
how they will understand you.
“2+2=“

3. 7/14/2014
3
“2+2=“
What is the Point?
You have MANY expectations about how a calculator
should work or might work…
(you acquire them automatically)
…but you aren’t aware of many of them...
(they are encoded or generated in your mind, but not
in words or pictures)
…and that means no explicit test procedure can ever
duplicate the value of tacit knowledge of a skilled
human tester.
We need a process that respects tacit knowledge.

4. 7/14/2014
4
Fact
Gathering
Result
Checking
Tester
Thinking
(theorizing
about meaning
of results)
Tester
Thinking
(failure
modeling and
test design)
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
Tester
Thinking
If testing is an intellectual process
that requires deep mental
engagement, personal motivation,
and constant learning to do well,
how do we manage the situation in
order to foster that?

5. 7/14/2014
5
Managing Tacit Knowledge
10
Heuristics bring useful structure
to problem-solving skill.
• adjective:
“serving to discover.”
• noun:
“a fallible method for solving a problem or
making a decision.”
“The engineering method is the use of heuristics
to cause the best change in a poorly understood situation
within the available resources.”
-- Billy Vaughan Koen, Discussion of The Method

6. 7/14/2014
6
11
-Idea
-Idea
…
Heuristics
They are applied, not followed.
1. Do this
2. Then do this
3. Then do this
4. Then do this
5. And then this…
This…
…not this.
The skilled tester
remains in control of
the process.
“2+2=“

7. 7/14/2014
7
Tacit Explicit
OtherPeopleTester
Your
Feelings &
Mental Models
Shared Artifacts
(specs, tools, etc.)
Stakeholders’
Feelings &
Mental Models
Inference
Observable
Consistencies
ReferenceConference
Experience
Oracles From the Inside Out
Consistency (“this agrees with that”)
an important theme in oracles
• Familiarity: The system is not consistentwith the pattern of any familiar problem.
• Explainability: The system is consistentwith our abilityto explain it.
• World: The systemis consistentwith objects and states,in the world, that it represents.
• History: The present version of the system is consistentwith past versions of it.
• Image: The system is consistentwith an image that the organization wants to project.
• Comparable Products: Thesystem is consistentwith comparable systems.
• Claims: The system is consistentwith what important people say it’s supposed to be.
• Users’ Expectations: The system is consistent with what users want.
• Product: Each element of the systemis consistentwith comparable elements in the same
system.
• Purpose: The system is consistentwith its purposes, both explicit and implicit.
• Statutes & Standards: The system is consistentwith applicable laws and standards.

8. 7/14/2014
8
A General Statement to
Protect Role of Tacit Knowledge/Skill
A system testing event in Malmo, Sweden.
Maria Kedemo, manager, is serving food.

9. 7/14/2014
9

10. 7/14/2014
10
Three Roles of Testing
Test Leads/Managers
Cultivate and support
Helpers
Cultivate and supervise
Heuristics and Tools
Use
Use
Clients and Context
Respondto
Respondto
“I am ABLE to make decisions that hurt people (or me).”
“I want to AVOID hurting people (including me).”
“I KNOW how my decisions might hurt people (or me).”

11. 7/14/2014
11
Excellent tacit knowledge can be
systematically cultivated.

12. 7/14/2014
12
The Tester’s Sociologist
Sociologist Harry Collins studies
how scientists actually work, and
what aspects of their work can
be made explicit.