2. Agenda
• Dynamic Analysis Overview
• [Generic] Test Steps of Performing Tests
• Back Box Testing Techniques
▫ Equivalence Class Partitioning
▫ Boundary Value Analysis
▫ State Transition Testing
▫ Cause-effect Graphing, Decision Tables
▫ Use Case Testing
▫ More Techniques
3. Dynamic Analysis Overview
• The test object (program) must be executable.
• In component and integration testing, test bed
(stubs and drivers) is used to make the test object
executable.
• Test bed generators can be used.
4. [Generic] Test Steps of Performing Tests
• Determine conditions and preconditions for the test
and the goals to be achieved.
• Specify the individual test cases (determine
input, expected output and post-conditions).
Writing test script (most often in programming
language or similar notation).
• Determine how to execute test cases (chaining
several test cases into test scenario / test
sequence, define priorities of test cases).
5. Black Box Test Techniques
• Test cases are designed only by using the
specification or the requirements of the test object.
• Point of Observation (PoO) is outside the test
object. Point of Control (PoC) is outside the test
object too (we only can choose adequate input test
data).
• Reasonable in component tests but usually applied
in higher test levels.
6. Black Box. Equivalence Class Partitioning.
• Equivalence Class – a group of values that are
supposed to be processed by test object the same
way.
• Equivalence classes for incorrect input must be
tested as well.
• For every single equivalence class, a representative
value (input value) should be chosen for testing. For
every input value, expected output or reaction
should be defined.
7. Black Box. Equivalence Class Partitioning #2
• Minimum test cases: every representative value of
an equivalence class appears in at least one test
case.
• Negative test cases: representatives of invalid
classes should not be combined with representatives
of other invalid classes.
EC-coverage = (number of tested EC /
total number of EC) * 100 %
The method is used in component, integration and
system testing.
8. Black Box. Boundary Value Analysis.
Checks the ‘border’ of the equivalence classes. On
every border, the exact boundary value and both
nearest adjacent values (inside and outside the
equivalence class) are tested.
It must be decided if it is sufficient to test a boundary
with two instead of pieces three test data.
BV-Coverage = (number of tested BV /
total number of BV) * 100 %
9. Black Box. Boundary Value Analysis #2
Rules for choosing boundary values:
• For ordered sets the first and last element is of special interest for
the test.
• If complex data structures are given as in- or output, for instance an
empty list or zero matrixes can be considered a boundary value.
• For numeric calculations, values that are close together as well as
values that are far apart should be taken into consideration as
boundary values.
• For invalid equivalence classes, boundary value analysis is only
useful when different exception handling for the test object is
expected depending on an equivalence class boundary.
• For lists and tables empty and full lists and the first and last
element are of interest, as they often show failures due to wrong
programming (Off-by-one problem).
10. Black Box. State Transition Testing.
• State diagrams illustrate the dependence of test
object on history. These diagrams are the basis for
designing tests.
• A finite-state machine consists of states
(nodes), transitions (links), inputs and outputs.
13. Black Box. State Transition Testing #4
• Test intensity levels / completion criteria: (1) every
state has been reached at least once, (2) every
transition has been executed at least once, (3) every
transition violating the specification has been
checked.
• For highly critical applications also: (4) all
combinations of transitions, (5) all transitions in any
order with all possible states, multiple instances in
succession.
14. Black Box. State Transition Testing #5
• State transition testing should be used where
functionality is influenced by the state of the test
object.
• Very useful in testing object-oriented systems.
• Is also a good technique for system test (e.g. GUI
test).
15. Black Box. Cause Effect Graphing.
The technique uses the dependencies between inputs
and their effects on outputs for identification of the
test cases.
17. Black Box. Cause Effect Graphing #3
Each cause and effect should occur at least once with
"yes" and "no" in the table.
An optimized decision table does not contain all
possible combinations, but the impossible or
unnecessary combinations are not entered.
19. Black Box. Cause Effect Graphing #5
Test completion criteria: execute every column in the
decision table by at least one test case.
The graph and the table may grow very fast. This
technique is not easily applicable without adequate
tool support.
20. Black Box. Use Case Testing.
• Use cases and use case diagrams are the basis for
determining test cases. Typical use of the system
(user-system interaction) is checked.
• The technique is relevant for customer, developer
and tester and is useful for system- and acceptance
testing.
22. Black Box. Use Case Testing #3
• Test cases:
▫ Each alternative (extension point) in the use case
diagram should be covered by a test case,
▫ Concrete input data and expected results cannot be
derived directly from use case – analysis needed to
define them.
• Test completion criteria: every use case (including
alternatives and extensions) and every possible
sequence of use cases is tested at least once.
23. Black Box. Use Case Testing #4
• The approach is supported by test specification
tools.
• However, there is no systematic method to
determine further test cases to test what is not
shown on the use case diagram (need to use other
techniques).
24. Black Box. More Techniques.
• Syntax Testing is used for testing interpreters, command
languages, compilers and protocol analyzers. May be
applied if a formal spec of input syntax is available.
• Random Testing is used for selection of test values based
on given statistical distribution of input values. Makes it
possible to use statistical models for predicting or
certifying system reliability.
• Smoke test is a ‘quick and dirty’ test which is primarily
aimed at verifying a minimum reliability of the test
object and concentrated on its main functions. Used to
decide if the test object is mature enough to be tested
further by more comprehensive test techniques. Used for
first and fast test of software updates.
25. More about Black Box.
• Faults in specification are not detected (when
techniques are used slavishly). If common sense is not
used in test design, wrong requirements will lead to
wrong test cases. Reviews must be used to find problems
in specifications.
• Not required functionality is not detected or tested not
enough because of absence of documentation and no
influence on coverage criteria. Such extra functionality is
often the cause of security problems.
• Black box methods verify the functionality. Correct
working of a software system has the highest priority so
black box techniques should always be applied.
![Agenda
• Dynamic Analysis Overview
• [Generic] Test Steps of Performing Tests
• Back Box Testing Techniques
▫ Equivalence Class Partitioning
▫ Boundary Value Analysis
▫ State Transition Testing
▫ Cause-effect Graphing, Decision Tables
▫ Use Case Testing
▫ More Techniques](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)