SOFTWARE ENGINEERING UNIT 6

1. 3/18/2009 S.Sreenivasa Rao 1
Chapter 13Chapter 13
Software Testing StrategiesSoftware Testing Strategies
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

2. 3/18/2009 S.Sreenivasa Rao 2
Software TestingSoftware Testing
Testing is the process of exercising aTesting is the process of exercising a
program with the specific intent of findingprogram with the specific intent of finding
errors prior to delivery to the end user.errors prior to delivery to the end user.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

3. 3/18/2009 S.Sreenivasa Rao
Test strategy provides a road map;Test strategy provides a road map;
Steps to be conductedSteps to be conducted
WhenWhen
How much effort, time, resourcesHow much effort, time, resources
Test strategyTest strategy
Test planningTest planning
Test case designTest case design
Test executionTest execution
Resultant data collection and evaluationResultant data collection and evaluation
3
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

4. 3/18/2009 S.Sreenivasa Rao
Verification and ValidationVerification and Validation
V&VV&V
Testing is part of V&VTesting is part of V&V
Verification: Are we building the product right?Verification: Are we building the product right?
Software correctly implements a specificSoftware correctly implements a specific
functionfunction
Validation: Are we building the right product?Validation: Are we building the right product?
Software is traceable to customerSoftware is traceable to customer
requirementsrequirements 4
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

5. 3/18/2009 S.Sreenivasa Rao 5
What Testing ShowsWhat Testing Shows
errorserrors
requirements conformancerequirements conformance
performanceperformance
an indicationan indication
of qualityof quality
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

6. 3/18/2009 S.Sreenivasa Rao 6
Who Tests the Software?Who Tests the Software?
developerdeveloper independent testerindependent tester
Understands the systemUnderstands the system
but, will test "gently"but, will test "gently"
and, is driven by "delivery"and, is driven by "delivery"
Must learn about the system,Must learn about the system,
but, will attempt to break itbut, will attempt to break it
and, is driven by qualityand, is driven by quality
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

7. 3/18/2009 S.Sreenivasa Rao 7
Testing StrategyTesting Strategy
unit testunit test integrationintegration
testtest
validationvalidation
testtest
systemsystem
testtest
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

8. 3/18/2009 S.Sreenivasa Rao
Levels of Testing
Client Needs
Requirements
Design
Code
Acceptance Testing
System Testing
Integration Testing
Unit testing
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

9. 3/18/2009 S.Sreenivasa Rao
RequirementRequirement --> high> high--order tests:order tests:
validation test, System testvalidation test, System test
DesignDesign --> Integration test> Integration test
CodeCode --> unit test> unit test
9
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

10. 3/18/2009 S.Sreenivasa Rao 10
Testing StrategyTesting Strategy
We begin byWe begin by ‘‘testingtesting--inin--thethe--smallsmall’’ and move towardand move toward
‘‘testingtesting--inin--thethe--largelarge’’
For conventional softwareFor conventional software
The module (component) is our initial focusThe module (component) is our initial focus
Integration of modules followsIntegration of modules follows
For OO softwareFor OO software
our focus whenour focus when ““testing in the smalltesting in the small”” changes from anchanges from an
individual module (the conventional view) to an OO classindividual module (the conventional view) to an OO class
that encompasses attributes and operations and impliesthat encompasses attributes and operations and implies
communication and collaborationcommunication and collaboration
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

11. 3/18/2009 S.Sreenivasa Rao 11
Strategic IssuesStrategic Issues
Specify product requirements in a quantifiable manner long befoSpecify product requirements in a quantifiable manner long beforere
testing.testing.
State testing objectives explicitly.State testing objectives explicitly.
Understand the users of the software and develop a profile for eUnderstand the users of the software and develop a profile for eachach
user category.user category.
Develop a testing plan that emphasizesDevelop a testing plan that emphasizes ““rapid cycle testing.rapid cycle testing.””
BuildBuild ““robustrobust”” software that is designed to test itselfsoftware that is designed to test itself
Use effective formal technical reviews as a filter prior to testUse effective formal technical reviews as a filter prior to testinging
Conduct formal technical reviews to assess the test strategy andConduct formal technical reviews to assess the test strategy and testtest
cases themselves.cases themselves.
Develop a continuous improvement approach for the testingDevelop a continuous improvement approach for the testing
process.process.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

12. 3/18/2009 S.Sreenivasa Rao 12
Unit TestingUnit Testing
modulemodule
to beto be
testedtested
test casestest cases
resultsresults
softwaresoftware
engineerengineer
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

13. 3/18/2009 S.Sreenivasa Rao 13
Unit TestingUnit Testing
interfaceinterface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
modulemodule
to beto be
testedtested
test casestest cases
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

14. 3/18/2009 S.Sreenivasa Rao 14
Unit Test EnvironmentUnit Test Environment
ModuleModule
stubstub stubstub
driverdriver
interfaceinterface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
RESULTSRESULTS
test casestest cases
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

15. 3/18/2009 S.Sreenivasa Rao 15
Integration Testing StrategiesIntegration Testing Strategies
Options:Options:
•• thethe ““big bangbig bang”” approachapproach
•• an incremental construction strategyan incremental construction strategy
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

16. 3/18/2009 S.Sreenivasa Rao 16
Top Down IntegrationTop Down Integration
top module is tested withtop module is tested with
stubsstubs
stubs are replaced one atstubs are replaced one at
a time, "depth first"a time, "depth first"
as new modules are integrated,as new modules are integrated,
some subset of tests is resome subset of tests is re--runrun
AA
BB
CC
DD EE
FF GG
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

17. 3/18/2009 S.Sreenivasa Rao 17
BottomBottom--Up IntegrationUp Integration
drivers are replaced one at adrivers are replaced one at a
time, "depth first"time, "depth first"
worker modules are grouped intoworker modules are grouped into
builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

18. 3/18/2009 S.Sreenivasa Rao 18
Sandwich TestingSandwich Testing
Top modules areTop modules are
tested with stubstested with stubs
Worker modules are grouped intoWorker modules are grouped into
builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

19. 3/18/2009 S.Sreenivasa Rao
Regression testingRegression testing
to ensure that changes do not introduceto ensure that changes do not introduce
unintended behavior or additional errors.unintended behavior or additional errors.
In context of an integration test: reIn context of an integration test: re--
execution of some subset of testsexecution of some subset of tests
Any correctionAny correction --> change of software> change of software
19
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

20. 3/18/2009 S.Sreenivasa Rao 20
Smoke TestingSmoke Testing
A common approach for creatingA common approach for creating ““daily buildsdaily builds”” for product softwarefor product software
Smoke testing steps:Smoke testing steps:
Software components that have been translated into code are inteSoftware components that have been translated into code are integratedgrated
into ainto a ““build.build.””
A build includes all data files, libraries, reusable modules, anA build includes all data files, libraries, reusable modules, andd
engineered components that are required to implement one or moreengineered components that are required to implement one or more
product functions.product functions.
A series of tests is designed to expose errors that will keep thA series of tests is designed to expose errors that will keep the builde build
from properly performing its function.from properly performing its function.
The intent should be to uncoverThe intent should be to uncover ““show stoppershow stopper”” errors that have theerrors that have the
highest likelihood of throwing the software project behind schedhighest likelihood of throwing the software project behind schedule.ule.
The build is integrated with other builds and the entire productThe build is integrated with other builds and the entire product (in its(in its
current form) is smoke tested daily.current form) is smoke tested daily.
The integration approach may be top down or bottomThe integration approach may be top down or bottom up.up.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

21. 3/18/2009 S.Sreenivasa Rao
Identify critical modulesIdentify critical modules
Address several software requirementsAddress several software requirements
Has a high level of control (high in the programHas a high level of control (high in the program
structure)structure)
Is complex or error proneIs complex or error prone
Has definite performance requirementsHas definite performance requirements
21
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

22. 3/18/2009 S.Sreenivasa Rao 22
ObjectObject--Oriented TestingOriented Testing
testing strategy changestesting strategy changes
the concept of thethe concept of the ‘‘unitunit’’ broadens due to encapsulationbroadens due to encapsulation
integration focuses on classes and their executionintegration focuses on classes and their execution
validation uses conventional black box methodsvalidation uses conventional black box methods
test case design draws on conventional methods, buttest case design draws on conventional methods, but
also encompasses special featuresalso encompasses special features
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

23. 3/18/2009 S.Sreenivasa Rao 23
OOT StrategyOOT Strategy
class testing is the equivalent of unit testingclass testing is the equivalent of unit testing
operations within the class are testedoperations within the class are tested
the state behavior of the class is examinedthe state behavior of the class is examined
integration applied three different strategiesintegration applied three different strategies
threadthread--based testingbased testing——integrates the set of classesintegrates the set of classes
required to respond to one input or eventrequired to respond to one input or event
useuse--based testingbased testing——integrates the set of classes requiredintegrates the set of classes required
to respond to one use caseto respond to one use case
cluster testingcluster testing——integrates the set of classes required tointegrates the set of classes required to
demonstrate one collaborationdemonstrate one collaboration
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

24. 3/18/2009 S.Sreenivasa Rao
Validation TestingValidation Testing
Validation succeeds when software functions in aValidation succeeds when software functions in a
manner that can be reasonably expected by themanner that can be reasonably expected by the
customer.customer.
Reasonable expectation are defined in the SoftwareReasonable expectation are defined in the Software
Requirement SpecificationRequirement Specification
FunctionalFunctional
BehavioralBehavioral
PerformancePerformance
DocumentationDocumentation
UsabilityUsability ……..
Alpha/Beta testingAlpha/Beta testing
Focus is on customer usageFocus is on customer usage
24
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

25. 3/18/2009 S.Sreenivasa Rao 25
High Order Testing:High Order Testing:
System testingSystem testingSystem testingSystem testing
Focus is on system integrationFocus is on system integration
Recovery testingRecovery testing
forces the software to fail in a variety of ways andforces the software to fail in a variety of ways and
verifies that recovery is properly performedverifies that recovery is properly performed
Automatic: reAutomatic: re--initialization, check pointing, datainitialization, check pointing, data
recovery, restartrecovery, restart
By human intervention ; meanBy human intervention ; mean--timetime--toto--repairrepair
Security testingSecurity testing
verifies that protection mechanisms built into a systemverifies that protection mechanisms built into a system
will, in fact, protect it from improper penetrationwill, in fact, protect it from improper penetration
Stress testingStress testing
executes a system in a manner that demands resourcesexecutes a system in a manner that demands resources
in abnormal quantity, frequency, or volumein abnormal quantity, frequency, or volume
Performance TestingPerformance Testing
test the runtest the run--time performance of software within thetime performance of software within the
context of an integrated systemcontext of an integrated system
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

26. 3/18/2009 S.Sreenivasa Rao 26
Debugging:Debugging:
A Diagnostic ProcessA Diagnostic Process
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

27. 3/18/2009 S.Sreenivasa Rao 27
The Debugging ProcessThe Debugging Process
test casestest cases
resultsresults
DebuggingDebugging
suspectedsuspected
causescauses
identifiedidentified
causescauses
correctionscorrections
regressionregression
teststests
new testnew test
casescases
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

28. 3/18/2009 S.Sreenivasa Rao 28
Debugging EffortDebugging Effort
time requiredtime required
to diagnose theto diagnose the
symptom andsymptom and
determine thedetermine the
causecause
time requiredtime required
to correct the errorto correct the error
and conductand conduct
regression testsregression tests
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

29. 3/18/2009 S.Sreenivasa Rao 29
Symptoms & CausesSymptoms & Causes
symptomsymptom
causecause
symptom and cause may besymptom and cause may be
geographically separatedgeographically separated
symptom may disappear whensymptom may disappear when
another problem is fixedanother problem is fixed
cause may be due to acause may be due to a
combination of noncombination of non--errorserrors
cause may be due to a systemcause may be due to a system
or compiler erroror compiler error
cause may be due tocause may be due to
assumptions that everyoneassumptions that everyone
believesbelieves
symptom may be intermittentsymptom may be intermittent
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

30. 3/18/2009 S.Sreenivasa Rao
Debugging tacticsDebugging tactics
Brute forceBrute force
Dump the memoryDump the memory
BacktrackingBacktracking
Start from presence of error, go backward in the code manuallyStart from presence of error, go backward in the code manually
Cause eliminationCause elimination
Cause hypothesisCause hypothesis ……
30
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

31. 3/18/2009 S.Sreenivasa Rao 31
Consequences of BugsConsequences of Bugs
damagedamage
mildmild
annoyingannoying
disturbingdisturbing
seriousserious
extremeextreme
catastrophiccatastrophic
infectiousinfectious
Bug TypeBug Type
Bug Categories:Bug Categories: functionfunction--related bugs,related bugs,
systemsystem--related bugs, data bugs, coding bugs,related bugs, data bugs, coding bugs,
design bugs, documentation bugs, standardsdesign bugs, documentation bugs, standards
violations, etc.violations, etc.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net

32. 3/18/2009 S.Sreenivasa Rao 32
Debugging: Final ThoughtsDebugging: Final Thoughts
thinkthink about theabout the
symptom you're seeing.symptom you're seeing.
Use toolsUse tools (e.g., dynamic debugger) to gain(e.g., dynamic debugger) to gain
more insight.more insight.
get helpget help from someone else.from someone else.
Be absolutely sure toBe absolutely sure to conduct regression testsconduct regression tests
when you do "fix" the bug.when you do "fix" the bug.
1.1.
2.2.
3.3.
4.4.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net