A PRACTITIONER'S GUIDE TO SOFTWARE TEST DESIGN -- by Lee Copeland. This document lists the summary of each chapter , written in my own words .

1. A PRACTITIONER'S GUIDE TO SOFTWARE TEST DESIGN
AUTHOR ---- “LEE COPELAND”
SUMMARIZE BY ---- KHIZRA SAMAD

2. OVERVIEW
This book consists of following chapters
chapter 1 The Testing process
chapter 2 Case studies
chapter 3 Equivalence class testing
Chapter 4 –Boundary Value Testing
Chapter 5 –Decision Table Testing
Chapter 6 –Pairwise Testing
Chapter 7 – State-Transition Testing
Chapter 8 – Domain Analysis Testing
Chapter 9 – Use Case Testing
Chapter 10 – Control Flow Testing
Chapter 11 – Data Flow Testing
Chapter 12 – Scripted Testing
Chapter 13 – Exploratory Testing
Chapter 14 – Test Planning
Chapter 15 – Defect Taxonomies
Chapter 16 – When to Stop Testing
PURPOSE
This document lists the summary of each chapter , written in my own words

3. Chapter 1 – The Testing Process
 Testing is a concurrent life cycle process of engineering, using and maintaining test ware
in order to measure and improve the quality of software being tested.
 IEEE Standard Glossary of Software Engineering defines ‘’testing’ as : The process of
operating a system or a component under specified conditions , observing or recording
the results and making an evaluation of some aspect of the system or component.
 Boris Bezier describes five levels of testing maturity.
- Level 0 describe ‘There is no difference in testing and debugging’
- Level 1 describes ‘The purpose of testing is to show that software works’
- Level 2 describes ‘The purpose of testing is to show that software doesn’t works’
- Level 3 describes ‘The purpose of testing is not to prove anything, but to reduce the
perceived risk of not working to an acceptable value.’
- Level 4 describes ‘Testing is not an act. It is a mental discipline that results in low
risk software without much testing effort.’
 An effective and efficient test case must be designed, not just slapped together. Well-
designed test cases are composed of three parts : Inputs , outputs and order of execution.
 There are two main types of testing i.e. Black box testing and White box testing:
- Black box testing is a software testing technique that focuses on the analysis of
software functionality, versus internal system mechanisms. Black box testing was
developed as a method of analyzing client requirements, specifications and high-level
design strategies.
- White-box testing is a methodology used to ensure and validate the internal
framework, mechanisms, objects and components of a software application. White-
box testing verifies code according to design specifications and uncovers application
vulnerabilities.
 There are four different levels of testing typically

4. - Unit Testing: Unit testing is a software development process in which the smallest
testable parts of an application, called units, are individually and independently
scrutinized for proper operation.
- Integration Testing: Integration testing is a software testing methodology used to test
individual software components or units of code to verify interaction between various
software components and detect interface defects.
- System Testing: System Testing is a level of the software testing process where a
complete, integrated system/software is tested. The purpose of this test is to evaluate
the system’s compliance with the specified requirements.
- Acceptance Testing: is a level of the software testing process where a system is tested
for acceptability. The purpose of this test is to evaluate the system’s compliance with
the business requirements and assess whether it is acceptable for delivery.
Chapter 2 – Case Studies
Two case studies are presented in this chapter. First is ‘Brown & Donaldson’ and second is
‘Stateless University Registration system’ to illustrate the test case design techniques described
in this book.
 Brown & Donaldson is an online brokerage firm that can be use to practice the test design
techniques presented in this book.
 Stateless University Registration system ia an online student registration system of
stateless university.
Chapter 3 – Equivalence Class Testing
 Equivalence class portioning is techniques used to reduce the number of test cases to a
manageable level while still maintain reasonable test coverage. It can be applied at any
level of testing and is often a good technique to use first.
 In equivalence-partitioning technique we need to test only one condition from each
partition. This is because we are assuming that all the conditions in one partition will be
treated in the same way by the software. If one condition in a partition works, we assume
all of the conditions in that partition will work, and so there is little point in testing any of
these others. Similarly, if one of the conditions in a partition does not work, then we

5. assume that none of the conditions in that partition will work so again there is little point
in testing any more in that partition.
 An equivalence class consists of a set of data that is treated the same by the module or
that should produce the same result. Any data value with in a class is equivalent, in terms
of testing, to any other value.
- If one test case in an equivalence class detects a defect, all other test cases in the same
equivalence class are likely to defect the same defect.
- If one test case in an equivalence class does not detect a defect, no other test case in
the same equivalence class is likely to defect the defect.
 Equivalence class approach helps to reduce the number of test case from 100 (testing
each age) to four (one age in each equivalence class).
 Design by contract also known as contract programming, is an approach for designing
software. It prescribes that software designers should define formal, precise and
verifiable interface specifications for software components, which extend the ordinary
definition of abstract data types with preconditions, post conditions and invariants. These
specifications are referred to as "contracts", in accordance with a conceptual metaphor
with the conditions and obligations of business contracts.
 Contract is a legally binding agreement between two or more parties that describes what
each party promises to do or not do.
 Testing by contract is based on the design by contract philosophy. Its approach is to
create test cases only for the situations in which the pre-conditions are met.
 In the same way that design by contract can work to interfaces, with a new or existing
system testing by contract can work to the interfaces between subsystems. These
interfaces are not necessarily coding interface classes, but they are the externally exposed
APIs of the different subsystems.
 Defensive design approach is a different approach to design. In this case module is
designed to accept any input. If normal pre-conditions are met, the module will achieve
its normal post-conditions. If normal pre-conditions are not met, the module will notify
the caller by returning an error code or throwing an exception depending upon the
programing language used.

6.  Defensive testing is an approach that tests under both normal and abnormal pre-
conditions.
 Using equivalence partitioning method test cases can be divided into three sets of input
data called as classes. Each test case is a representative of respective class.
 Test cases for input box accepting numbers between 1 and 1000 using Equivalence
Partitioning:
1) One input data class with all valid inputs. Pick a single value from range 1 to 1000 as a
valid test case. If you select other values between 1 and 1000 then result is going to be
same. So one test case for valid input data should be sufficient.
2) Input data class with all values below lower limit. I.e. any value below 1, as a invalid
input data test case.
3) Input data with any value greater than 1000 to represent third invalid input class.
 So using equivalence partitioning we categorized all possible tests cases into three
classes. Test cases with other values from any class should give you the same result. We
have selected one representative from every input class to design our test cases. Test case
values are selected in such a way that largest number of attributes of equivalence class
can be exercised.
 Equivalence class is most suited to systems in which much of the input data takes values
with in ranges or within sets.Equivalence class testing is equally applicable at the unit,
integration, system and acceptance test levels. All it requires are inputs or outputs that
can be partitioned on the system’s requirements.
Chapter 4 –Boundary Value Testing
 Boundary value testing focuses on the boundaries simply because that is where so many
defects hide. Experienced testers may encounter this situation many times. Inexperienced
testers may have an intuitive feel that mistakes will occur most often at the boundaries.
 Boundary value analysis is a next part of Equivalence partitioning for designing test cases
where test cases are selected at the edges of the equivalence classes.
 Boundary value testing technique based on three steps. First, identify the equivalence
classes. Second, identify the boundaries of the equivalence class. Third, create test cases
for each boundary value by choosing one point on the boundary, one point just below the
boundary and one point just above the boundary.

7.  There is no hard-and-fast rule to test only one value from each equivalence class you
created for input domains. You can select multiple valid and invalid values from each
equivalence class according to your needs and previous judgments.
 Boundary value testing is applicable to the structure of input data as well as its
value.Boundary value testing can significantly reduce the number of test cases that must
be created and executed. It is most suited to systems in which much of input data takes on
values within ranges or within sets.
 Boundary value testing is equally applicable at the unit, integration, system and
acceptance test levels.
Chapter 5 –Decision Table Testing
 Decision tables are used to document complex business rules that a system must
implement. They also serve as a guide to creating test cases. It provides complete
coverage of test cases which help to reduce the rework on writing test scenarios & test
cases.
 Decision tables are important tool for tester. Unfortunately, many analysts, designers,
programmers and testers are not familiar with this technique.
 Any complex business flow can be easily converted into the test scenarios & test cases
using this technique.
 Such type of table are work iteratively, means the table created at the first iteration is
used as input table for next tables. Such iteration can be carried out only if the initial table
is unsatisfactory.
 The first task is to identify a suitable function or subsystem which reacts according to a
combination of inputs or events. The system should not contain too many inputs
otherwise the number of combinations will become unmanageable. It is better to deal
with large numbers of conditions by dividing them into subsets and dealing with the
subsets one at a time. Once you have identified the aspects that need to be combined, then
you put them into a table listing all the combinations of True and False for each of the
aspects.
 Condition represents various input conditions. Actions are the processes that should be
executed depending on the various combinations of input conditions. Each rule defines a

8. unique combination of conditions that result in the execution of the action associated with
the rule.
 At least one test case for each rule should be created. If the rule’s conditions are binary, a
single test for each combination is probably sufficient. If a condition is a range values,
consider testing at both the low and high end of the range.
 Decision table testing can be used whenever the system must implement complex
business rules when these rules can be represented as a combination of conditions and
when these conditions have discrete actions associated with them.
Chapter 6 –Pairwise Testing
 Pairwise testing is an effective test case generation technique that is based on the
observation that most faults are caused by interactions of at most two factors. Pairwise-
generated test suites cover all combinations of two therefore are much smaller than
exhaustive ones yet still very effective in finding defects.
 There is much evidence about the benefits of pairwise testing. Unfortunately, there are
only a few documented studies available.
 Pairwise testing may not choose combinations which the developers and testers know are
either frequently used or highly risk. If these combinations exist, use the pairwise test,
and then add additional test cases to minimize the risk of missing an important
combination.
 Pairwise testing defines a minimal subset that guides us to test for all single-mode and
double mode defects.
 Two different techniques are used in pairwise testing to identify all pairs for creating test
cases i.e.
- Orthogonal arrays
- Allpairs algorithms
 Pairwise testing is strongly influenced by a mathematical construct called an orthogonal
array, or an OA. Orthogonal arrays are used in a variety of disciplines, including medical
research, manufacturing, metallurgy, polling, and other fields that require testing and
statistical sampling.
 The process of using ‘orthogonal arrays’ to select pairwise subsets for testing is:

9. - Identify the variables
- Determine the number of choices for each variable.
- Locate an orthogonal array which has columns that correspond to the choices for
each variable.
- Map the test problem on to orthogonal array.
- Conduct the test case.
 Allpair algorithm is another way to identify all the pairs. This algorithm generates the
pairs directly without restoring to an external device like an orthogonal array.
 Combinations chosen by Orthogonal Array method may not be the same as those chosen
by Allpairs. It does not matter because what matter is that all the pair combinations of
parameters are chosen. Those are the combinations we want to test.
 Pairwise testing can significantly reduce the number of test cases that must be created and
executed. It is equally applicable at the unit, integration, system and acceptance test
levels. All it requires are combinations of inputs, each taking on various values, which
result in a combinatorial explosion, too many combinations to test.
 There is no underlying ‘software defect physics’ that guarantee pairwise testing will be of
benefit. There is only one way to know i.e. to Try it.
Chapter 7 – State-Transition Testing
 State-Transition diagram, like decision table, are another excellent tool to capture certain
types of system requirements and to document internal system design.State-Transition
diagrams document the events that come into and processed by a system as well system
responses.
 State transition technique is a dynamic testing technique, which is used when the system
is defined in terms of a finite number of states and the transitions between the states is
governed by the rules of the system.
 Or in other words, this technique is used when features of a system are represented as
states which transforms to other state. The transformations are determined by the rules of
the software.
 State transition model has four major parts.

10. 1. State is represented by circle. A state is a condition in which a system waiting for one or
more events
2. Transition is represented by an arrow. A transition represents a change from one state to
another caused by event.
3. Event is represented by a label on transition. An event is something that causes system to
change state. that origin a transition (closing a file or withdrawing money)
4. Action is represented by a command following a ‘/’. An action is an operation initiated
because of state changes.
 State-Transition Tables is another way to document system behavior. State Transition
diagram may be easier to comprehend, but state- transition tables are easier to use in a
complete and systematic manner.
 State-Transition Tables consists of four columns
- Current State
- Event
- Action
- Next State
 The advantage of State-Transition Table is that it lists all possible state combinations, not
just the valid ones.
 State-Transition diagrams can easily be used to create test cases. It defines four different
level of coverage.
1. Create a set of test cases such that all states are ‘visited’ at least once under test.
Generally this is a weak level of test coverage.
2. Create a set of test cases such that all events are ‘triggered’ at least once under test. Again
this is a weak level of coverage.
3. Create a set of test cases such that all paths are ‘executed’ at least once under test. This
level is a most preferred because of its level of coverage.
4. Create a set of test cases such that all transitions are ‘exercised’ at least once under test.
This level is generally the one recommended.
 State-Transition diagrams are excellent tool to capture certain system requirements those
that describe states and their associated transitions.
 State-Transition diagrams are not applicable to the system that has no state or does not
need to respond to real-time events from outside of the system.

11. Chapter 8 – Domain Analysis Testing
 Decision analysis is a technique that can be used to identify efficient an effective test cases
when multiple variables should be tested.
 Decision analysis technique builds on and generalizes equivalence class and boundary
value testing to n simultaneous dimensions.
 Two dimensional boundary defect can be following
- A shifted boundary in which boundary is displaced vertically or horizontally.
- A titled boundary in which the boundary is rotated at an incorrect angle.
- A missing boundary.
- An extra boundary.
 Choosing on and off points is more complicated that it may appear
- An ‘on’ point is a value that lies on a boundary.
- An ‘off’ point is a value that does not lie on a boundary.
- An ‘out’ point is a value that does not satisfy any boundary condition.
 In using the 1x1 domain analysis technique for each relational condition (>, >-, <- or <)
we choose one on point and one off point. For each strict equality condition (=) we choose
one on point and two off points, one slightly less than the conditional vale and one slightly
greater than the value.
 When multiple variables should be tested together either for efficiency or because of a
logical interaction Decision analysis will be used. Decision analysis technique is best
suited to numeric values; it can be generalized to Booleans, strings, enumerations.
Chapter 9 – Use Case Testing
 Use case testing is a technique that helps us identify test cases that exercise the whole
system on a transaction by transaction basis from start to finish. They are described by
Ivar Jacobson in his book Object-Oriented Software Engineering: A Use Case Driven
Approach [Jacobson, 1992].
 Use case is a description of a particular use of the system by an actor (a user of the
system). Each use case describes the interactions the actor has with the system in order to
achieve a specific task (or, at least, produce something of value to the user).

12.  Actors are generally people but they may also be other systems. Scenario is a sequence of
steps that describe the interactions between the actor and the system. Use cases are defined
in terms of the actor, not the system, describing what the actor does and what the actor
sees rather than what inputs the system expects and what the system’s outputs.
 Use case capture system’s functional requirements from the user’s perspective, not form
technical perspective, and irrespective of the development paradigm to be used.Use case
can be used to actively involve users in the requirements gathering and definition process.
 Use case provide basis for identifying a system’s key internal components, structure,
database and relationships. Use case serves as the foundation for developing test cases at
the system and acceptance level.
 Use cases describe the process flows through a system based on its most likely use. This
makes the test cases derived from use cases particularly good for finding defects in the
real-world use of the system (i.e. the defects that the users are most likely to come across
when first using the system).
 Each use case usually has a mainstream (or most likely) scenario and sometimes
additional alternative branches (covering, for example, special cases or exceptional
conditions). Each use case must specify any preconditions that need to be met for the use
case to work.
 Use cases must also specify post conditions that are observable results and a description of
the final state of the system after the use case has been executed successfully. Use case
testing technique should be used whenever system transactions are well defined.
Chapter 10 – Control Flow Testing
 Control flow graphs are the basis of control flow testing. Module’s control structure is
documented by these control flow graphs.
 Modules of code are converted to graphs, the paths through the graphs are analyzed and
test cases are created from the analysis.
 Different levels of coverage are defined in control flow graphs.
- Level 0 - Test whatever you want to test.
- Level 1 - Every statement within module is executed, under test, at least once.
- Level 2- Its 100% decision coverage and enough test cases are written so that each
decision that has a TRUE and FALSE outcome is evaluated at least once.

13. - Level 3- Not all statements are simple.
- Level 5 - Test cases are created to yield 100% multiple condition coverage.
- Level 6 – when number of paths are infinite , a significant and meaningful reduction
can be made by limiting loop execution to a small number of cases.
- Level 7- It’s the final and the highest level which gives 100% path coverage.
 Structure testing is part of control flow testing. Structure testing consists on following
steps.
- Drive the control flow graph from the software module.
- Compute the graph Cyclomatic Complexity( c ).
- Select a set of C basis path.
- Execute these tests.
 Control flow testing is very time consuming and for this tester must have sufficient
programming skills to understand the code and its control flow.
Chapter 11 – Data Flow Testing
 Data flow graph is similar to a control flow graph which based on selecting paths through
the program's control flow in order to explore sequences of events related to the status of
variables or data objects. Dataflow Testing focuses on the points at which variables
receive values and the points at which these values are used.
 Static data flow testing identifies potential defects and analyze source code infact it’s
done on source code without actually executing it.
 Dynamic data flow testing involves actual program execution and is based on
intermediate values that result from the program’s execution.
 Like control flow testing ,date flow testing also use all codes module that cannot e tested
thoroughly through reviews and inspection , for this tester must have sufficient
programming skills to understand the code which consider as a limitation of data flow
testing.
Chapter 12 – Scripted Testing

14. Chapter 13 – Exploratory Testing
 In Exploratory testing, the tester controls the design of test cases as they are performed
rather than days, weeks, or even months before. The information testers gains from
executing a set of tests then guides the tester in designing and executing the next set of
tests. Exploratory testing is about exploring, finding out about the software, what it does,
what it doesn’t do, what works and what doesn’t work. The tester is constantly making
decisions about what to test next and where to spend the (limited) time. This is an
approach that is most useful when there are no or poor specifications and when time is
severely limited.
 The process of exploratory testing consists following :
- Creating a conjecture of the proper functioning of the system.
- Designing one or more tests that would disapprove the conjecture.
- Executing these tests and observing the outcomes.
- Evaluating the outcomes against the conjecture.
- Repeating this process until the conjecture is proved or disapproves.
 Exploratory testing is used when you are asked to when defect is detected and you are
asked to provide rapid feedback about product quality on short notice, useful for scripted
testing.
Chapter 14 – Test Planning
 An on-going process throughout the project lifecycle with test plans being developed for
each phase of software development. Successful test planning enables the mapping of
tests to the software requirements and defines the entry and exit criteria for each phase
of testing.
 Classic planning and adaptive planning are different approaches. In Classic planning we
plan before the thing happens whereas in adaptive planning we plan as much as we can,
when we can but not before.
 The use of scripted testing does not preclude the use of exploratory testing and vice
versa.

15. Chapter 15 – Defect Taxonomies
 Defect taxonomies collect and organize the domain knowledge and project experience of
experts and are a valuable instrument of system testing for several reasons. They provide
systematic backup for the design of tests, support the decisions for the allocation of
testing resources and are a suitable basis for measuring the product and test quality.
 ‘Beizer’s Taxonomy’ was first taxonomy which was defined and contains 4 levels.
‘Kaner, Falk and Nguyen’s Taxonomy was a detailed taxonomy which contains over
400 types of defects. ‘Binder’s object oriented Taxonomy deals with encapsulation,
inheritance, polymorphism, message sequencing, and state transitions.
 Testing can be done without the use of taxonomies or with taxonomy to guide the design
of test cases.
 Taxonomies can be created at a number of levels: generic software system, development
paradigm, type of application and user interface metaphor.
Chapter 16 – When to Stop Testing
 When to stop testing decision is often made based on following five criteria
- You have met previously defined coverage goals
- The defect discovery rate has dropped below a previously defined threshold.
- The marginal cost of finding the next defect exceeds the expected loss from that
defect.
- The project team says team reaches consensus that it is appropriate to release the
product.
- The boss says, ‘Ship it’.
 Coverage is a measure of how much has been tested compared with how much is
available. Coverage level is defined at code level with metrics such as statement
coverage, branch coverage and path coverage. Defect Discovery rate is used as the
criteria for stopping testing. Marginal cost is the cost associated with one additional unit
of production. Team Consensus based on various factors including technical, financial,
political, and just the ‘gut feelings’. The last thing is ‘Ship it’ that is totally the decision
of boss and reason of that decision might be ‘first to market’ to win substantial market
share or to satisfy urgent needs of user and sometime to bring significant profit that
might be lost if we delayed shipment

16. Reference