Testing metrics provide objective measurements of software quality and the testing process. They measure attributes like test coverage, defect detection rates, and requirement changes. There are base metrics that directly capture raw data like test cases run and results, and calculated metrics that analyze the base metrics, like first run failure rates and defect slippage. Tracking these metrics throughout testing provides visibility into project readiness, informs management decisions, and identifies areas for improvement. Regular review and interpretation of the metrics is needed to understand their implications and make changes to the development lifecycle.

1. SOFTWARE TESTING
METRICS
Presenter : P.M.Venkatesh Babu

2. 2
What is a METRIC
• Metrics can be defined as “STANDARDS OF
MEASUREMENT
• Metric is a unit used for describing or
measuring an attribute
• Test metrics are the means by which the
software quality can be measured
• Test metrics provides the visibility into the
readiness of the product, and gives clear
measurement of the quality and
completeness of the product

3. 3
Why we need Metrics ?
• You cannot improve what you cannot measure
• You cannot Control what you cannot measure
 Without measurement it is impossible to tell whether the
process implemented is improving or not
 Metrics helps in taking decisions for next phase of
activities
 Metrics helps in understanding the type of improvement
required and helps in taking decisions on process or
technology change

4. 4
Why Metrics in Software Testing?
There will be certain questions during and after testing such as :
 How long would it take to test ?
 How Bad / Good is the product?
 How many bugs still remain in the product?
 Will testing be completed on time?
 Was the testing done effectively?
 How much effort went into testing the product?
To Answer these questions properly we need some type of
measurements and record keeping to justify the answers.
This is where the testing metrics comes into picture.

5. Testing Metrics Life Cycle
 Analysis Phase:
• Identify the Metrics which has to be generated
• Define the identified Metrics
 Communication Phase:
• Explain the need of the Metrics to the stakeholders
• Educate the testing team about the data points for generating the Metrics
 Evaluation Phase:
• Capture and verify the data used for generating the Metrics
• Calculate the Metrics based on the data captured
 Reporting Phase :
• Develop the Metrics report and distribute to the stakeholders
• Take feedback from the stakeholders for any improvements
5

6. 6
Types Of Metrics
Base metrics (Direct Measure)
• The Base metrics constitute the raw data gathered by the
test Engineers throughout the testing effort
• The Base metrics are used to provide project status reports
to the Test lead and to the project manager
• The Base metrics provide the input data to feed into the
formulas used to derive Calculated metrics
• Examples of Base metrics are:
# of test cases
# of test cases executed

7. 7
Types Of Metrics (contd..)
Calculated Metrics (Indirect Measure)
• The Calculated Metrics convert the Base metrics data into
more useful information
• The Calculated Metrics are generally prepared by the Test
lead and is used to track the progress of the project at
different levels like at Module level, at Tester level and for
the project as a whole
• The Calculated Metrics provide valuable information that
when used and implemented often times leads to significant
improvements in the Overall SDLC

8. 8
Base Metrics and Testing Phases
TEST METRIC TESTING PHASE
Number of test cases Test Development Phase
Number of Test cases Passed Test Execution Phase
Number of Test cases Executed Test Execution Phase
Number of Test cases Failed Test Execution Phase
Number of Test cases under
Investigation
Test Development Phase
Number of Test cases Blocked Test Dev / Execution Phase
Number of Test cases Re-
executed
Regression Phase
Number of First run Failures Test execution Phase
Total Executions Test Reporting Phase
Total Passes and Failures Test Reporting Phase
Test case Execution time Test Reporting Phase
Test Execution time Test Reporting Phase

9. 9
Calculated Metrics and Phases
The Following Calculated metrics are created at Test Reporting
Phase or Post Test Analysis Phase:
 % of Test cases Passed
 % of Test Coverage
 % of Defects corrected
 % of Test cases Blocked
 % of Rework
 % of Test Effectiveness
 1st
Run Fail Rate
 Defect discovery rate
 Overall Fail rate

10. 10
Test case Defect Density
The number of errors found in test cases v/s test cases developed and
executed
• ( Defective Test cases / Total Test cases ) * 100
Example : Total no of test cases developed is 1360, total test cases executed
is 1280, total no of test cases passed is 1065, total no of test scripts failed
is 215
So Test case Defect Density is :
215 X 100
------------------------------- = 16.8 %
1280
The 16.8 % value can also be called as Test Case Efficiency % which
depends upon the total number of Test cases which found defects

11. 11
Defect Slippage Ratio
No of bugs reported from Production V/S No of defects reported during
execution
No of Defects slipped / ( Number of Defects Raised – Number Defects
Withdrawn) * 100
Example : Customer reported 21 defects, total no of defects found
while testing are, total no of Invalid defects are 17
So Slippage ratio is : [ 21 / (267 – 17) ] X 100 = 8.4%

12. 12
Requirement Volatility Metric
This metric ensures that the requirements are normalized or defined
properly while estimating
No of requirements agreed V/S No of requirements changed
• (No of requirements Added + Deleted + Modified) * 100 / No of
original requirements
Example : SVN 1.3 release has 67 requirements initially, later 7 new
requirements are added, 3 requirements are deleted from initial
requirements and modified 11 requirements
Hence Requirement volatility is calculated as :
(7 + 3 + 11 ) X 100 / 67 = 31.34 %
This means that almost 1/3 of the requirements changed after the initial
identification of requirements

13. 13
Conclusion
• The Test metrics should be reviewed & interpreted on regular basis
throughout the test effort and particularly after the application is
released into production
• There are several key factors in implementing and using the metrics
in the organization, beginning with determining the goal for
developing the metrics, followed by the identification of metrics to be
tracked and ending with sufficient analysis of the resulting data to be
able to make changes to the software development lifecycle
• - So finally Metrics themselves so not create improvements, they do
provide the objective information necessary to understand what
changes are necessary to be implemented

14. 14
Questions ???