Software testing is an essential activity of the software development lifecycle. To ensure quality, applicability, and usefulness of a product, development teams must spend considerable time and resources testing, which makes the estimation of the software testing effort, a critical activity. This presentation presents a simple and useful method called qEstimation to estimate the size and effort of the software testing activities. The method measures the size of the test case in terms of test case points based on its checkpoints, preconditions and test data, as well as the type of testing. The testing effort is then computed using the size estimated in test case points. All calculations are embedded in a simple Excel tool, allowing estimators easily to estimate testing effort by providing test cases and their complexity.

1. Test Estimation in Practice
STC JULY 2014
VU NGUYEN
Faculty of Information Technology
University of Science, VNU-HCMC

2. Agenda
• Background and Motivation
• qEstimation Process
– Test Size Estimation
– Test Effort Estimation
• Conclusion
2

3. Background
• Software estimation
– process of determining cost, time, staff, and other attributes
of software projects
• Estimation is important for success or failure of
projects
• Methods and Metrics
– Source Lines of Code (SLOC)
– Function Points
– Use Case Points
– Story Points
– COCOMO
– Expert Judgment
3

4. Motivation
• Testing accounts for up to 50% of project effort
[1]
• Current problems
– estimates done for the whole project rather than
testing specific
– lack of reliable methods designed for estimating
software testing
• Our aim
– Introducing qEstimation for estimating size and
effort of testing projects
4

5. Agenda
• Background and Motivation
• qEstimation Process
– Test Size Estimation
– Test Effort Estimation
• Conclusion
5

6. Test Case
qEstimation Process
6
Test Case
Count TCPs
of all Test
Cases
Estimate
Testing
Effort
Parameters
Counted
Size
Estimated
Effort
[Test Cycle i]
Test Cycle Size Actual
Effort
Effort by
Activity
…. …. …. ….
Test cycle i …. …. ….
…. …. …. ….
Historical Data of this Project
Update
Historical
Data
Calibrate
Estimation
Model
Estimate size and effort of different test cycles of a same project:

7. qEstimation Principles
• Size measures complexity of each test cycle
• Test case’s complexity is based on
– Number of checkpoints
– Complexity of test setup or precondition
– Complexity of test data
• Test Case Point (TCP) is used as size unit
• Model refinement (calibration) is key to
estimating effort
– based on previous cycles of the current project or
similar projects
7

8. Count Size of Test Cycle
• Size of a test cycle: total TCPs of all test cases to be
executed in that test cycle
• Steps:
8
Test Case
Count
Checkpoints
Determine Set Up
Complexity
Determine Test
Data Complexity
Adjust based on
Test Type
(optional)
Unadjusted
TCPs
TCPs

9. Count Size of Test Cycle - 2
• Checkpoints
– Checkpoint is the condition in which the tester
verifies whether the result produced by the target
function matches the expected criterion
– One test case consists of one or many checkpoints
9
One checkpoint is counted as one TCP

10. Count Size of Test Cycle - 3
• Test Setup or Precondition
– Test setup specifies the condition to execute the test case
• Include setup steps to prepare environment for testing
– Four levels of Test Setup complexity
• Each is assigned a number of TCPs
10
Number
of TCP(*)
Complexity
Level
Description
0 None • The set up is not applicable or important to execute the test case
• Or, the set up is just reused from the previous test case to continue the current test case
1 Low • The condition for executing the test case is available with some simple modifications required
• Or, some simple set-up steps are needed
3 Medium • Some explicit preparation is needed to execute the test case
• Or, The condition for executing is available with some additional modifications required
• Or, some additional set-up steps are needed
5 High • Heavy hardware and/or software configurations are needed to execute the test case

11. Count Size of Test Cycle - 4
• Test Data
– Test Data is used to execute the test case
• It can be generated at the test case execution time, sourced from previous
tests, or generated by test scripts
– Four levels of Test Data complexity
• Each is assigned a number of TCPs
Number of TCP
(*)
Complexity
Level
Description
0 None • No test data preparation is needed
1 Low • Simple test data is needed and can be created during the test case execution time
• Or, the test case uses a slightly modified version of existing test data and requires
little or no effort to modify the test data
3 Medium • Test data is deliberately prepared in advance with extra effort to ensure its
completeness, comprehensiveness, and consistency
6 High • Test data is prepared in advance with considerable effort to ensure its completeness,
comprehensiveness, and consistency
• This could include using support tools to generate data and a database to store and
manage test data
• Scripts may be required to generate test data

12. Count Size of Test Cycle - 5
• Adjust TCPs based on Type of Test
– This is an OPTIONAL step
– Adjustment is based on types of test cases
• Each type of test case is assigned a weight
• Adjusted TCP of the test case = Counted TCP x Weight(*)
12

13. Agenda
• Background and Motivation
• qEstimation Process
– Test Size Estimation
– Test Effort Estimation
• Conclusion
13

14. Estimate Effort of Test Cycle
• Overview
– Two estimation methods
• Based on Test Velocity
• Regression analysis of Size and Effort of completed test
cycles
– Effort distributed by activity
• Test Planning
• Test Analysis and Design
• Test Execution
• Test Tracking and Reporting
14
Each of these activities may
be performed multiple times

15. Estimate Effort of Test Cycle - 2
• Estimate Effort based on Test Velocity
Effort(person-hour) = Size(TCP) / Test Velocity(TCP per person-hour)
– Test Velocity is measured as TCP/person-hour
• Test Velocity is dependent on project
• It should be calculated based on data from completed test cycles
of the same project.
15

16. Estimate Effort of Test Cycle - 3
• Estimate effort using Linear Regression
Analysis
– Find out the equation of effort and size using similar
completed test cycles of a project
16
y = 0.0729x + 1.6408
0
10
20
30
40
50
60
70
80
90
100
0 100 200 300 400 500 600 700 800 900 1000
Effort(PM)
Adjusted TCP
Equation of
Size and Effort
The data analysis tool like
Excel can be used to find
out the equation

17. Calibrate qEstimation
• Calibration: a process adjusting parameters for
a model using historical data or experiences
• With qEstimation, you can calibrate:
(1) TCP assigned to each complexity level of Test Setup
(2) TCP assigned to each complexity level of Test Data
(3) Test Velocity
(4) Effort distribution
(5) Weights of test case types
• Process can be done with the help of tools
17

18. 18
Tool Demo

19. Conclusion
• qEstimation is an agile approach to estimating
tests
– Estimate Size in TCP by measuring the complexity
of test cases
– Estimate Effort using Test Velocity or Regression
– An Excel toolkit to simplify the approach
• Advantages
– Easy to implement
– Reflecting real complexity of test cases
– Independent with the level of details of test cases
– Useful for estimating testing effort
19

20. References
• [1] Y. Yang, Q. Li, M. Li, Q. Wang, An empirical analysis on distribution patterns of
software maintenance effort, International Conference on Software Maintenance,
2008, pp. 456-459
• [2] N. Patel, M. Govindrajan, S. Maharana, S. Ramdas, “Test Case Point Analysis”,
Cognizant Technology Solutions, White Paper, 2001
• [3] QASymphony: www.qasymphony.com
• [4] Vu Nguyen, Vu Pham, and Vu Lam. “qEstimation: a process for estimating size
and effort of software testing.” Proceedings of the 2013 International Conference
on Software and System Process. ACM, 2013

21. © 2014 HCMC Software Testing Club
Thank You