SlideShare a Scribd company logo
1 of 35
Defect Management Process
Yolanda Williams
ybowdrywilliams@gmail.com
http://www.slideshare.net/ybowdrywillia
ms
A practical guide for handling opportunities in IT
Quality Assurance
AGENDA
Today's session is divided into two main
Sections
2
What is a Defect
Management Process?
• Knowledge sharing
• Discussion
Defect Analysis and
recommendations
• Critical Analysis
• Issue
• Change Control
Coordination
• UAT an dry run Defects
• Recommendations
• Discussion
Process Improvement : What
are they thinking?
Developers
“While the defect itself may not be a big
deal, the fact that there was a defect is
a big deal.”
Testers
“If this defect could have gotten this far
into the process before it was
captured, what other defects may be
present that have not been discovered.”
Customers
“A defect is anything that causes
customer dissatisfaction, whether in the
requirements, design, code or testing or
not”
Program
Leadership‟s
advice:
Efforts should be made to analyze the
process that originated the defect to
understand what caused the defect.
3
Management Reporting
What Leaders need?
To make more
informed
decisions -- e.g.,
redesign of error
prone modules,
the need for
more testing,
etc.
To provide
strategic
information and
metrics to senior
management --
defect
trends, problem
systems, etc.
To provide
insight into
areas where the
process could
be improved to
either prevent
defects or
minimize their
impact.
To provide
insight into the
likelihood that
target dates and
cost estimates
will be achieved
or overran.
4
Goals of DMP
Agreement on the Goals of the Defect Management
Process
• The foremost goals are to
• Prevent the Defect
• Early detection
• Minimize the impact
• Decrease in change requests
• The process should be Risk driven.
• Integrated with Software Development
• Continuous effort to improve the process
• Most defects are caused by due process
5
Defect Management Process
Defect
Prevention
Defect
Discovery
Defect
Resolution
Process
Improvement
6
Objectives
To enable participants understand and apply defect
prevention concepts
Defect Prevention Objectives
 Identify and analyze the causes of defects
 Reduction in number of defect categories
 Reduction in frequency of common defects
 Reduction in the extent of defect escape between test phases and
releases
7
Defining the Defect Prevention
Defect Prevention is a process of
improving quality and productivity by
preventing the injection of defects into a
software work product.
Definition: “…an activity of continuous
institutionalized learning during which
common causes of errors in work products
are systematically identified and process
changes eliminating those causes are
made.”
[Eickelmann]
8
Defect Prevention Activities
9
Establish practice of Root Cause Analysis within
projects for Analysis of Identified Defects
Identify critical processes as part of root cause
analysis
Set goals for improving critical processes with
team-level accountability
Reduce most frequent type of defects such as
“ not following coding guidelines” or Ambiguity
within Requirements and Specifications
Analyze opportunities for improvement by
conducting escape analysis.
Defect Prevention
 It is virtually impossible
to eliminate the defects
altogether.
 Testers/Developers
should collaborate for
quick detection of
defects and minimize
the risk.
 Assess the critical risks
associated with the
system and identify the
defects
10
Minimize expected Impact
Prioritize
based on
Impact
Estimate
Expected
Impact
Identify
Critical
Risks
Deliverable Baselines
When should it occur in the Development
timeline?
 When a Predefined milestone is
reached then Product is Baselined
 Further development work continues
from one stage to another.
 A product should be considered
baselined when developers pass it on
for integration testing.
11
Defect Discovery
 A defect is said to be discovered when it is
brought to the attention of the developers
and acknowledged (i.e., “Accepted”) to be
valid one.
 A defect discovery team should comprise of
respectable and knowledgeable individuals
and should be lead by a Facilitator.
12
Find/Discover Defect
• Discover defects
before they
become major
problems.
Report Defect
• Report defects to
developers so that
they can be
resolved.
Acknowledge/Accept
Defect
• Agreed that the reported
defect is valid by the
developers
DEFECT RESOLUTION PROCESS
A resolution process needs to be
established for use in the event there
is a dispute regarding a defect. For
example, if the group uncovering the
defect believes it is a defect but the
developers do not, a quick-
resolution process must be in
place. The 2 recommended
processes:
• Arbitration by the software owner -
- the customer using the software
determines whether or not the
problem is a defect.
• Arbitration by a software
development manager -- a senior
manager of the software
development department will be
selected to resolve the dispute.
Prioritize
Risk
Schedule
Fix
Fixing
Defects
Report the
Resolution
13
Process Improvement : General
Feedback
 Senior Management must understand, support, and be a part of the
Defect Management Program.
 The Defect Management process should be integrated into the overall
software development process.
 To the extent practical, the process should be repeatable and
automated.
 Specific development approaches
(e.g., testing, inspections, reviews, exit criteria etc.) should be chosen
based on project objectives and risks that must be addressed.
 Process improvement efforts should be driven by the measurement
process: metrics, reporting & decision making.
 It should be noted, the process does not require a significant
investment, yet will likely result in significant payback
14
Implementing Process
Improvements
Continuous
Improvement
through
Collaboration
Define Critical
Metrics
Identify
Critical Risks
Identify
Process
Improvements
Develop
Project
Management
Plan
Pilot
“Improved”
Defect
Management
Process
15
Define Critical Metrics
 Critical Metrics Joint Definition Session:
◦ What can we do to ensure that more of these are found earlier
in the cycle?
◦ Have the defects from these „last minute‟ discoveries been
analyzed to determine why they were missed in our normal
SIT test cycle?
◦ When these issues are found late in the cycle, what is done to
ensure that we catch them next time?
◦ This is stemming from the way our business team is doing
their testing (ad-hoc testing coming late, late in the process).
 Correction of Errors (COE) analysis
◦ the number of defects which were found in the final days of
UAT and all of the ones we found during the dry runs.
◦ Why were they not found earlier?
◦ What phase of testing that uncovers the majority of them?
16
Calculated Metrics & Phases
 % Complete
 % Defects Corrected
 % Test Coverage
 % Rework
 % Test Cases Passed
 % Test Effectiveness
 % Test Cases Blocked
 % Test Efficiency
 1st Run Fail Rate
 Defect Discovery Rate
 Overall Fail Rate
17
Identify Critical Risks
 Risks which negatively impact the Critical Metrics
set
◦ Some of the few Risks may be as follows:
 Stable mature technology vs. new technology
 Technical platform
 Large integrated system vs. small stand-alone
system
 Tight target dates
 Limited budget
 Bottlenecks within the Development & Testing
cycles
 Resource constraints: Overview of all resources
within Dev and testing to decrease:
 Resource access request/on-boarding bottlenecks
 Resource need competition between projects
18
Identify Process
Improvements
Escapes
This is done by driving the defect discovery as far
back into the software development process as
possible.
 Escape Analysis process benefits:
◦ improve product quality and customer satisfaction
◦ lower costs spent on resolving customer-found defects
◦ Reduce defect escape between test phases and
releases
 Test scenario – test case reviews
 Exit Criteria for Developer testing
 QA acceptance of Development
 Environment/Non-functional Requirements
 Technical requirements
 Environment baselines
 GUI checklists prior to test case execution
 No Ad-hoc testing: document all variants of testing and their 19
Types of Errors
20
Human Errors
Omission
• Knowledge
• Ignorance
• Informational
• Process-related
Translational
• Miscommunication
• Mismatch in solution
and requirement
• Mismatch with
requirement and test
case
Design and Coding
• Affect Data Integrity
• Alter Correctly stored
data
• Affect downstream
system dependencies
• Affect testing outcomes
Testing
• Failure to notice a
problem
• Misreading the screen.
• Failure to execute a
planned test.
• Criteria disagreements
• CRs, Rejections
Lessons Learned – Feedback
Testing
Integration Testing should be
scheduled so that the core
modules are initially tested.
Rigorous unit testing to be
done to avoid logical errors.
Basic level review and testing
should be done at developer's
level before handing over the
code to the testers and
reviewers.
Communication
Client responsibilities should
be clearly communicated in
the beginning of the project.
Design documents should
contain all the necessary
validations to avoid validation
error.
QA should create an
accessible communication
model to increase
transparency, collaboration
and trust
Process
Checklist should be used to
avoid GUI errors
Test cases should be formed
with test data.
Developer team should accept
defect while researching to
move defect through pipeline.
HPQC should be utilized at all
test phases :
DIT, FIT, SIT, UAT, Prod to
maintain visibility of all
program defect and for trend
assessment
21
SAMPLE ISSUE # 1
Corrections include
• Increased downstream
communication for Tier 1
software owners
• Create a GUI Checklist to
baseline what is acceptable
viewing per page type
• Increase Dev visibility into high
impact change control at pre/post
go live Checkpoints
• Include Change Coordinator in
install planning that can increase
risk to prod environment
• Escalation method:
• IT Change coordinator for release management
team to Program Manager, RM Manager and QA
Managment
• Concerns
• Increased workload
• the multiple installs of the OM700 program that
needed review and approvals.
• Issues were found during a dry run last week,
• Issues found in a load test from past weekend
• the change for next week was found last night.
• Not enough planning and test cases performed on
the screen that this program runs in.
• Reactive response
• The change for Monday went to CAB meeting to
ensure proper testing and planning have been
done.
• Why?
• To prevent any impacting issues going forward as
occurred with the go live in Production installs.
22
SAMPLE DEFECT ANALYSIS #1
DEFECTS
• 13 Open Defects.
• 60% defects with Source Code and
Design as the root cause.
• 107 Rejected defects for this triage
• 30% of all reported defects were
rejected.
OPPORUNITIES
• Implement a Rejection Resolution
process to understand why so
many defects are reported and
rejected.
• Increase visibility to Design
Traceability and Unit
testing/Development Integration
testing for increased quality in
upstream Program processes.
23
DEFECT CAUSE /
DEFECT STATUS
Status: IT Release Testing As Of 21-Oct-2011
11.90 17.53
Sev 2 MTTR "Days" Sev 3 MTTR "Days"Sev 1 MTTR "Days"
1.74
Sev 4 MTTR "Days"
0.00
Defect Mean Time To Repair for System Integration Test
AVG MTTR "Days"
10.39
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
22/Aug/11
29/Aug/11
05/Sep/11
12/Sep/11
19/Sep/11
26/Sep/11
03/Oct/11
10/Oct/11
17/Oct/11
24/Oct/11
31/Oct/11
07/Nov/11
Days
Defect "MTTR" Mean Time To Repair
Sev 1 MTTR Sev 2 MTTR Sev 3 MTTR Sev 4 MTTR
Status: IT Release Testing As Of 21-Oct-2011
SIT 30-Aug-11 SIT 21-Oct-11 SIT 30-Aug-11 SIT 4-Nov-11
Regression 24-Oct-11 Regression 4-Nov-11 Regression 24-Oct-11 Regression
Planned Start Planned Finish Actual Start Trend Finish
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150 22/Aug/11
29/Aug/11
05/Sep/11
12/Sep/11
19/Sep/11
26/Sep/11
03/Oct/11
10/Oct/11
17/Oct/11
24/Oct/11
31/Oct/11
07/Nov/11
TestCases
Test Case "Pass" Powercurve
SIT Actual Pass SIT Projection SIT Trend
Regression Actual Passed Regression Projection Regression Trend
Test Case defect density
Total number of errors found in test scripts vs. developed and
executed.
 (Defective Test Scripts /Total Test Scripts) * 100
Example: Total test script developed 1360, total test script executed
1280, total test script passed 1065, total test script failed 215
So, test case defect density is
215 X 100
---------------------------- = 16.8%
1280
This 16.8% value can also be called as test case efficiency %, which
is depends upon total number of test cases which uncovered
defects.
26
Smaller
test case
density %
Increased
test case
efficiency
High test
case
execution
rates
Status: IT Release Testing
Sev 4 DensitySev 2 Density
18.18% 0.83%
Defects Density for System Integration Test
Sev 1 Density
2.48%
Total Density
68.60%47.11%
Sev 3 Density
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
22/Aug/11
29/Aug/11
05/Sep/11
12/Sep/11
19/Sep/11
26/Sep/11
03/Oct/11
10/Oct/11
17/Oct/11
24/Oct/11
31/Oct/11
07/Nov/11
Defect Density (Defects/Pass TCs)
Overall Defect Density Defect Density (Industry Avg)
Sev 1-2 Defect Density Sev 3- 4 Defect Density
Status: IT Release Testing
3 74
Sev 3-4 Defect Backlog
Defect Discovery and Severity 1-2 Backlog for System Integration Test
Sev 1-2 Defect Backlog
0
5
10
15
20
25
30
0
10
20
30
40
50
60
70
80
90
22/Aug/11
29/Aug/11
05/Sep/11
12/Sep/11
19/Sep/11
26/Sep/11
03/Oct/11
10/Oct/11
17/Oct/11
24/Oct/11
31/Oct/11
07/Nov/11
DefectBacklog
Defects
Defect Discovery and Severity 1-2 Backlog
Defects (Sev 1-2) Defects (Sev 3-4) Defect Backlog (Sev 1-2)
IT TESTING ITERATION 2 UAT AND DRY RUN ANALYSIS
DEFECTS
OPPORUNITIES
• Implement a Rejection
Resolution process to
understand why so many
defects are reported and
rejected.
• Increase visibility to Design
Traceability and Unit
testing/Development
Integration testing for
increased quality in
upstream Program
processes.
29
Status: Sample Release Testing Pass Trend Analysis
SIT 19-Sep-11 SIT 28-Nov-11 SIT 21-Sep-11 SIT 30-Jan-12
Regression 29-Nov-11 Regression 5-Dec-11 Regression 29-Nov-11 Regression
Planned Start Planned Finish Actual Start Trend Finish
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1000
1050
1100
1150
1200
1250
1300
15-Sep-11
22-Sep-11
29-Sep-11
06-Oct-11
13-Oct-11
20-Oct-11
27-Oct-11
03-Nov-11
10-Nov-11
17-Nov-11
24-Nov-11
01-Dec-11
08-Dec-11
15-Dec-11
22-Dec-11
29-Dec-11
05-Jan-12
12-Jan-12
19-Jan-12
26-Jan-12
02-Feb-12
09-Feb-12
TestCases
Test Case "Pass" Powercurve
SIT Actual Pass SIT Projection SIT Trend
Regression Actual Passed Regression Projection Regression Trend
Status: Sample Mean Time To Repair Data
Sev 4 MTTR "Days"
16.70
Sev 1 MTTR "Days"
17.45 17.99
Sev 2 MTTR "Days" Sev 3 MTTR "Days"
0.69
Defect Mean Time To Repair for System Integration Test
AVG MTTR "Days"
13.21
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0 15/Sep/11
22/Sep/11
29/Sep/11
06/Oct/11
13/Oct/11
20/Oct/11
27/Oct/11
03/Nov/11
10/Nov/11
17/Nov/11
24/Nov/11
01/Dec/11
08/Dec/11
15/Dec/11
22/Dec/11
29/Dec/11
05/Jan/12
12/Jan/12
19/Jan/12
26/Jan/12
02/Feb/12
09/Feb/12
Days
Defect "MTTR" Mean Time To Repair
Sev 1 MTTR Sev 2 MTTR Sev 3 MTTR Sev 4 MTTR
Status: Sample Defect Disovery and Resolution
Backlog
32 118
Sev 3-4 Defect BacklogSev 1-2 Defect Backlog
Defect Discovery and Severity 1-2 Backllog for System Integration Test
0
10
20
30
40
50
60
70
80
90
100
0
50
100
150
200
250 15-Sep-11
22-Sep-11
29-Sep-11
06-Oct-11
13-Oct-11
20-Oct-11
27-Oct-11
03-Nov-11
10-Nov-11
17-Nov-11
24-Nov-11
01-Dec-11
08-Dec-11
15-Dec-11
22-Dec-11
29-Dec-11
05-Jan-12
12-Jan-12
19-Jan-12
26-Jan-12
02-Feb-12
09-Feb-12
DefectBacklog
Defects
Defect Discovery and Severity 1-2 Backlog
Defects (Sev 1-2) Defects (Sev 3-4) Defect Backlog (Sev 1-2)
Defect Prevention Support Criteria and Actions
33
• Management Approval and Support: Management must support the project team and the
objectives of the defect management effort if it is to be successful.
• Receptivity of Project Team: The project team should wholeheartedly embrace the
program. Project management, in particular, must take ownership for the successful
implementation of the program.
• Project Duration: The project should not be so long that results will not be known for a long
period of time. A project duration of three to six months is probably ideal.
• Project Risk: The project should be sufficiently large and complex that the results will be taken
seriously, but not so large and complex that success of the pilot project is unlikely.
• Early in the Life Cycle: The defect management program should be integrated into project
plans and not retrofitted into a project whose approach is well established.
Criteria for implementing Defect Prevention efforts
• Conduct Training: The project team should be trained in the Defect Management concepts, if
necessary. Example: on-boarding new team members.
• Incorporate Defect Management Into Project Plan: Each pilot project should adapt the Defect
Management approach to its particular circumstances. Emphasis should be placed on
automating as much of the process as practical, and measuring the process. Data collected
from the pilot projects should form the baseline for the Critical Metrics Set.
• Review Results and Refine Approach: Progress implementing the Defect Management
Program should be monitored and the approach refined as appropriate.
• Report Results: Periodic status reports, which report pilot project results, should be issued.
Defect Prevention Actions
Problem Closure & Congratulate Team
• Acknowledge significance and value of problem
solution
• Recognize team‟s collective efforts in solving the
problem as well as individual contributions
• Document what was learned in solving the
problem; lessons learned not only about the
problem which was solved but also about the
problem solving process
• Consider investigating other potential causes as
preventive actions
• Create case study/problem solving reports
34
Thank You
35
If you have any questions or concerns about this presentation
Contact
Yolanda Williams ybowdrywilliams@gmail.com

More Related Content

What's hot

Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional TestingNishant Worah
 
Requierement traceability matrix
Requierement traceability matrixRequierement traceability matrix
Requierement traceability matrixLuthfia Ulinnuha
 
Software Quality Assurance
Software Quality AssuranceSoftware Quality Assurance
Software Quality AssuranceSachithra Gayan
 
SOFTWARE QUALITY ASSURANCE.ppt
SOFTWARE QUALITY ASSURANCE.pptSOFTWARE QUALITY ASSURANCE.ppt
SOFTWARE QUALITY ASSURANCE.pptDrTThendralCompSci
 
Software Inspection And Defect Management
Software Inspection And Defect ManagementSoftware Inspection And Defect Management
Software Inspection And Defect ManagementAjay K
 
An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)Usersnap
 
Agile QA presentation
Agile QA presentationAgile QA presentation
Agile QA presentationCarl Bruiners
 
Software quality assurance activites
Software quality assurance activitesSoftware quality assurance activites
Software quality assurance activitesGolu Gupta
 
Infographic: Importance of Performance Testing
Infographic: Importance of Performance TestingInfographic: Importance of Performance Testing
Infographic: Importance of Performance TestingKiwiQA
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8a34sharm
 
Manual testing interview questions and answers
Manual testing interview questions and answersManual testing interview questions and answers
Manual testing interview questions and answersTestbytes
 
Change management in Software Engineering
Change management in Software EngineeringChange management in Software Engineering
Change management in Software EngineeringHiba Ghannam
 
Performance testing presentation
Performance testing presentationPerformance testing presentation
Performance testing presentationBelatrix Software
 
Load and performance testing
Load and performance testingLoad and performance testing
Load and performance testingQualitest
 
Software Configuration Management (SCM)
Software Configuration Management (SCM)Software Configuration Management (SCM)
Software Configuration Management (SCM)Er. Shiva K. Shrestha
 
Performance and load testing
Performance and load testingPerformance and load testing
Performance and load testingsonukalpana
 
Difference between functional testing and non functional testing
Difference between functional testing and non functional testingDifference between functional testing and non functional testing
Difference between functional testing and non functional testingpooja deshmukh
 

What's hot (20)

Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional Testing
 
Chapter 4 - Defect Management
Chapter 4 - Defect ManagementChapter 4 - Defect Management
Chapter 4 - Defect Management
 
Requierement traceability matrix
Requierement traceability matrixRequierement traceability matrix
Requierement traceability matrix
 
Software Quality Assurance
Software Quality AssuranceSoftware Quality Assurance
Software Quality Assurance
 
Test Strategy
Test StrategyTest Strategy
Test Strategy
 
SOFTWARE QUALITY ASSURANCE.ppt
SOFTWARE QUALITY ASSURANCE.pptSOFTWARE QUALITY ASSURANCE.ppt
SOFTWARE QUALITY ASSURANCE.ppt
 
Software Inspection And Defect Management
Software Inspection And Defect ManagementSoftware Inspection And Defect Management
Software Inspection And Defect Management
 
An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)An Overview of User Acceptance Testing (UAT)
An Overview of User Acceptance Testing (UAT)
 
Agile QA presentation
Agile QA presentationAgile QA presentation
Agile QA presentation
 
Software quality assurance activites
Software quality assurance activitesSoftware quality assurance activites
Software quality assurance activites
 
Infographic: Importance of Performance Testing
Infographic: Importance of Performance TestingInfographic: Importance of Performance Testing
Infographic: Importance of Performance Testing
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8
 
Manual testing interview questions and answers
Manual testing interview questions and answersManual testing interview questions and answers
Manual testing interview questions and answers
 
Change management in Software Engineering
Change management in Software EngineeringChange management in Software Engineering
Change management in Software Engineering
 
Performance testing presentation
Performance testing presentationPerformance testing presentation
Performance testing presentation
 
Load and performance testing
Load and performance testingLoad and performance testing
Load and performance testing
 
Software Configuration Management (SCM)
Software Configuration Management (SCM)Software Configuration Management (SCM)
Software Configuration Management (SCM)
 
Testing methodology
Testing methodologyTesting methodology
Testing methodology
 
Performance and load testing
Performance and load testingPerformance and load testing
Performance and load testing
 
Difference between functional testing and non functional testing
Difference between functional testing and non functional testingDifference between functional testing and non functional testing
Difference between functional testing and non functional testing
 

Similar to IT Quality Testing and the Defect Management Process

9.process improvement chapter 9
9.process improvement chapter 99.process improvement chapter 9
9.process improvement chapter 9Warui Maina
 
EFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEWEFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEWJournal For Research
 
Lesson 8...Question Part 2
Lesson 8...Question Part 2Lesson 8...Question Part 2
Lesson 8...Question Part 2bhushan Nehete
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycleDiUS
 
A Research Study on importance of Testing and Quality Assurance in Software D...
A Research Study on importance of Testing and Quality Assurance in Software D...A Research Study on importance of Testing and Quality Assurance in Software D...
A Research Study on importance of Testing and Quality Assurance in Software D...Sehrish Asif
 
Test Team Responsibilities
Test Team ResponsibilitiesTest Team Responsibilities
Test Team ResponsibilitiesANKUR-BA
 
Recent and-future-trends spm
Recent and-future-trends spmRecent and-future-trends spm
Recent and-future-trends spmPrakash Poudel
 
Aim (A).pptx
Aim (A).pptxAim (A).pptx
Aim (A).pptx14941
 
SOFTWARE TESTING
SOFTWARE TESTINGSOFTWARE TESTING
SOFTWARE TESTINGacemindia
 
New Approaches To Software Testing Essay
New Approaches To Software Testing EssayNew Approaches To Software Testing Essay
New Approaches To Software Testing EssayNicole Gethers
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven TestingJorge Boria
 
Introduction to Software Testing - Part 2
Introduction to Software Testing - Part 2Introduction to Software Testing - Part 2
Introduction to Software Testing - Part 2Sachin-QA
 
Introduction to Quality Assurance Part 2
Introduction to Quality Assurance Part 2Introduction to Quality Assurance Part 2
Introduction to Quality Assurance Part 2Vidya-QA
 
Introduction to Software Testing Part 2
Introduction to Software Testing Part 2Introduction to Software Testing Part 2
Introduction to Software Testing Part 2Rajesh-QA
 
Fundamentals of testing
Fundamentals of testingFundamentals of testing
Fundamentals of testingBugRaptors
 

Similar to IT Quality Testing and the Defect Management Process (20)

Quality management
Quality managementQuality management
Quality management
 
9.process improvement chapter 9
9.process improvement chapter 99.process improvement chapter 9
9.process improvement chapter 9
 
EFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEWEFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEW
 
Lesson 8...Question Part 2
Lesson 8...Question Part 2Lesson 8...Question Part 2
Lesson 8...Question Part 2
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
 
A Research Study on importance of Testing and Quality Assurance in Software D...
A Research Study on importance of Testing and Quality Assurance in Software D...A Research Study on importance of Testing and Quality Assurance in Software D...
A Research Study on importance of Testing and Quality Assurance in Software D...
 
Softwaretesting
SoftwaretestingSoftwaretesting
Softwaretesting
 
SQA_Class
SQA_ClassSQA_Class
SQA_Class
 
Test Team Responsibilities
Test Team ResponsibilitiesTest Team Responsibilities
Test Team Responsibilities
 
Recent and-future-trends spm
Recent and-future-trends spmRecent and-future-trends spm
Recent and-future-trends spm
 
Aim (A).pptx
Aim (A).pptxAim (A).pptx
Aim (A).pptx
 
CTFL chapter 05
CTFL chapter 05CTFL chapter 05
CTFL chapter 05
 
Software testing introduction
Software testing  introductionSoftware testing  introduction
Software testing introduction
 
SOFTWARE TESTING
SOFTWARE TESTINGSOFTWARE TESTING
SOFTWARE TESTING
 
New Approaches To Software Testing Essay
New Approaches To Software Testing EssayNew Approaches To Software Testing Essay
New Approaches To Software Testing Essay
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven Testing
 
Introduction to Software Testing - Part 2
Introduction to Software Testing - Part 2Introduction to Software Testing - Part 2
Introduction to Software Testing - Part 2
 
Introduction to Quality Assurance Part 2
Introduction to Quality Assurance Part 2Introduction to Quality Assurance Part 2
Introduction to Quality Assurance Part 2
 
Introduction to Software Testing Part 2
Introduction to Software Testing Part 2Introduction to Software Testing Part 2
Introduction to Software Testing Part 2
 
Fundamentals of testing
Fundamentals of testingFundamentals of testing
Fundamentals of testing
 

More from Yolanda Williams

Program Management for Business Transformations
Program Management for Business TransformationsProgram Management for Business Transformations
Program Management for Business TransformationsYolanda Williams
 
Clarity MS Project and Open Workbench
Clarity MS Project and Open WorkbenchClarity MS Project and Open Workbench
Clarity MS Project and Open WorkbenchYolanda Williams
 
What is Program Management - An Overview
What is Program Management - An OverviewWhat is Program Management - An Overview
What is Program Management - An OverviewYolanda Williams
 
Ansoff Matrix Analysis - Yolanda Williams
Ansoff Matrix Analysis - Yolanda WilliamsAnsoff Matrix Analysis - Yolanda Williams
Ansoff Matrix Analysis - Yolanda WilliamsYolanda Williams
 
Carnival Cruises Marketing plan and Business Case - Yolanda Williams
Carnival Cruises Marketing plan and Business Case - Yolanda WilliamsCarnival Cruises Marketing plan and Business Case - Yolanda Williams
Carnival Cruises Marketing plan and Business Case - Yolanda WilliamsYolanda Williams
 
Acquisition and restructuring strategies - Yolanda Williams
Acquisition and restructuring strategies - Yolanda WilliamsAcquisition and restructuring strategies - Yolanda Williams
Acquisition and restructuring strategies - Yolanda WilliamsYolanda Williams
 
Teleflex Canada: A Strategy Business Case - Yolanda Williams
Teleflex Canada: A Strategy Business Case - Yolanda WilliamsTeleflex Canada: A Strategy Business Case - Yolanda Williams
Teleflex Canada: A Strategy Business Case - Yolanda WilliamsYolanda Williams
 
Shutterfly Marketing Case - Yolanda Williams
Shutterfly Marketing Case - Yolanda WilliamsShutterfly Marketing Case - Yolanda Williams
Shutterfly Marketing Case - Yolanda WilliamsYolanda Williams
 
BlueFly.com Financial Analysis 2007-2010 - Yolanda Williams
BlueFly.com Financial Analysis 2007-2010 - Yolanda WilliamsBlueFly.com Financial Analysis 2007-2010 - Yolanda Williams
BlueFly.com Financial Analysis 2007-2010 - Yolanda WilliamsYolanda Williams
 
Get started with social networking - Yolanda Williams
Get started with social networking - Yolanda WilliamsGet started with social networking - Yolanda Williams
Get started with social networking - Yolanda WilliamsYolanda Williams
 
Customers Waiting in Lines - Service Operations - Yolanda Williams
Customers Waiting in Lines - Service Operations - Yolanda WilliamsCustomers Waiting in Lines - Service Operations - Yolanda Williams
Customers Waiting in Lines - Service Operations - Yolanda WilliamsYolanda Williams
 

More from Yolanda Williams (12)

PI5_InspectAdapt
PI5_InspectAdaptPI5_InspectAdapt
PI5_InspectAdapt
 
Program Management for Business Transformations
Program Management for Business TransformationsProgram Management for Business Transformations
Program Management for Business Transformations
 
Clarity MS Project and Open Workbench
Clarity MS Project and Open WorkbenchClarity MS Project and Open Workbench
Clarity MS Project and Open Workbench
 
What is Program Management - An Overview
What is Program Management - An OverviewWhat is Program Management - An Overview
What is Program Management - An Overview
 
Ansoff Matrix Analysis - Yolanda Williams
Ansoff Matrix Analysis - Yolanda WilliamsAnsoff Matrix Analysis - Yolanda Williams
Ansoff Matrix Analysis - Yolanda Williams
 
Carnival Cruises Marketing plan and Business Case - Yolanda Williams
Carnival Cruises Marketing plan and Business Case - Yolanda WilliamsCarnival Cruises Marketing plan and Business Case - Yolanda Williams
Carnival Cruises Marketing plan and Business Case - Yolanda Williams
 
Acquisition and restructuring strategies - Yolanda Williams
Acquisition and restructuring strategies - Yolanda WilliamsAcquisition and restructuring strategies - Yolanda Williams
Acquisition and restructuring strategies - Yolanda Williams
 
Teleflex Canada: A Strategy Business Case - Yolanda Williams
Teleflex Canada: A Strategy Business Case - Yolanda WilliamsTeleflex Canada: A Strategy Business Case - Yolanda Williams
Teleflex Canada: A Strategy Business Case - Yolanda Williams
 
Shutterfly Marketing Case - Yolanda Williams
Shutterfly Marketing Case - Yolanda WilliamsShutterfly Marketing Case - Yolanda Williams
Shutterfly Marketing Case - Yolanda Williams
 
BlueFly.com Financial Analysis 2007-2010 - Yolanda Williams
BlueFly.com Financial Analysis 2007-2010 - Yolanda WilliamsBlueFly.com Financial Analysis 2007-2010 - Yolanda Williams
BlueFly.com Financial Analysis 2007-2010 - Yolanda Williams
 
Get started with social networking - Yolanda Williams
Get started with social networking - Yolanda WilliamsGet started with social networking - Yolanda Williams
Get started with social networking - Yolanda Williams
 
Customers Waiting in Lines - Service Operations - Yolanda Williams
Customers Waiting in Lines - Service Operations - Yolanda WilliamsCustomers Waiting in Lines - Service Operations - Yolanda Williams
Customers Waiting in Lines - Service Operations - Yolanda Williams
 

Recently uploaded

20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 

Recently uploaded (20)

20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 

IT Quality Testing and the Defect Management Process

  • 1. Defect Management Process Yolanda Williams ybowdrywilliams@gmail.com http://www.slideshare.net/ybowdrywillia ms A practical guide for handling opportunities in IT Quality Assurance
  • 2. AGENDA Today's session is divided into two main Sections 2 What is a Defect Management Process? • Knowledge sharing • Discussion Defect Analysis and recommendations • Critical Analysis • Issue • Change Control Coordination • UAT an dry run Defects • Recommendations • Discussion
  • 3. Process Improvement : What are they thinking? Developers “While the defect itself may not be a big deal, the fact that there was a defect is a big deal.” Testers “If this defect could have gotten this far into the process before it was captured, what other defects may be present that have not been discovered.” Customers “A defect is anything that causes customer dissatisfaction, whether in the requirements, design, code or testing or not” Program Leadership‟s advice: Efforts should be made to analyze the process that originated the defect to understand what caused the defect. 3
  • 4. Management Reporting What Leaders need? To make more informed decisions -- e.g., redesign of error prone modules, the need for more testing, etc. To provide strategic information and metrics to senior management -- defect trends, problem systems, etc. To provide insight into areas where the process could be improved to either prevent defects or minimize their impact. To provide insight into the likelihood that target dates and cost estimates will be achieved or overran. 4
  • 5. Goals of DMP Agreement on the Goals of the Defect Management Process • The foremost goals are to • Prevent the Defect • Early detection • Minimize the impact • Decrease in change requests • The process should be Risk driven. • Integrated with Software Development • Continuous effort to improve the process • Most defects are caused by due process 5
  • 7. Objectives To enable participants understand and apply defect prevention concepts Defect Prevention Objectives  Identify and analyze the causes of defects  Reduction in number of defect categories  Reduction in frequency of common defects  Reduction in the extent of defect escape between test phases and releases 7
  • 8. Defining the Defect Prevention Defect Prevention is a process of improving quality and productivity by preventing the injection of defects into a software work product. Definition: “…an activity of continuous institutionalized learning during which common causes of errors in work products are systematically identified and process changes eliminating those causes are made.” [Eickelmann] 8
  • 9. Defect Prevention Activities 9 Establish practice of Root Cause Analysis within projects for Analysis of Identified Defects Identify critical processes as part of root cause analysis Set goals for improving critical processes with team-level accountability Reduce most frequent type of defects such as “ not following coding guidelines” or Ambiguity within Requirements and Specifications Analyze opportunities for improvement by conducting escape analysis.
  • 10. Defect Prevention  It is virtually impossible to eliminate the defects altogether.  Testers/Developers should collaborate for quick detection of defects and minimize the risk.  Assess the critical risks associated with the system and identify the defects 10 Minimize expected Impact Prioritize based on Impact Estimate Expected Impact Identify Critical Risks
  • 11. Deliverable Baselines When should it occur in the Development timeline?  When a Predefined milestone is reached then Product is Baselined  Further development work continues from one stage to another.  A product should be considered baselined when developers pass it on for integration testing. 11
  • 12. Defect Discovery  A defect is said to be discovered when it is brought to the attention of the developers and acknowledged (i.e., “Accepted”) to be valid one.  A defect discovery team should comprise of respectable and knowledgeable individuals and should be lead by a Facilitator. 12 Find/Discover Defect • Discover defects before they become major problems. Report Defect • Report defects to developers so that they can be resolved. Acknowledge/Accept Defect • Agreed that the reported defect is valid by the developers
  • 13. DEFECT RESOLUTION PROCESS A resolution process needs to be established for use in the event there is a dispute regarding a defect. For example, if the group uncovering the defect believes it is a defect but the developers do not, a quick- resolution process must be in place. The 2 recommended processes: • Arbitration by the software owner - - the customer using the software determines whether or not the problem is a defect. • Arbitration by a software development manager -- a senior manager of the software development department will be selected to resolve the dispute. Prioritize Risk Schedule Fix Fixing Defects Report the Resolution 13
  • 14. Process Improvement : General Feedback  Senior Management must understand, support, and be a part of the Defect Management Program.  The Defect Management process should be integrated into the overall software development process.  To the extent practical, the process should be repeatable and automated.  Specific development approaches (e.g., testing, inspections, reviews, exit criteria etc.) should be chosen based on project objectives and risks that must be addressed.  Process improvement efforts should be driven by the measurement process: metrics, reporting & decision making.  It should be noted, the process does not require a significant investment, yet will likely result in significant payback 14
  • 15. Implementing Process Improvements Continuous Improvement through Collaboration Define Critical Metrics Identify Critical Risks Identify Process Improvements Develop Project Management Plan Pilot “Improved” Defect Management Process 15
  • 16. Define Critical Metrics  Critical Metrics Joint Definition Session: ◦ What can we do to ensure that more of these are found earlier in the cycle? ◦ Have the defects from these „last minute‟ discoveries been analyzed to determine why they were missed in our normal SIT test cycle? ◦ When these issues are found late in the cycle, what is done to ensure that we catch them next time? ◦ This is stemming from the way our business team is doing their testing (ad-hoc testing coming late, late in the process).  Correction of Errors (COE) analysis ◦ the number of defects which were found in the final days of UAT and all of the ones we found during the dry runs. ◦ Why were they not found earlier? ◦ What phase of testing that uncovers the majority of them? 16
  • 17. Calculated Metrics & Phases  % Complete  % Defects Corrected  % Test Coverage  % Rework  % Test Cases Passed  % Test Effectiveness  % Test Cases Blocked  % Test Efficiency  1st Run Fail Rate  Defect Discovery Rate  Overall Fail Rate 17
  • 18. Identify Critical Risks  Risks which negatively impact the Critical Metrics set ◦ Some of the few Risks may be as follows:  Stable mature technology vs. new technology  Technical platform  Large integrated system vs. small stand-alone system  Tight target dates  Limited budget  Bottlenecks within the Development & Testing cycles  Resource constraints: Overview of all resources within Dev and testing to decrease:  Resource access request/on-boarding bottlenecks  Resource need competition between projects 18
  • 19. Identify Process Improvements Escapes This is done by driving the defect discovery as far back into the software development process as possible.  Escape Analysis process benefits: ◦ improve product quality and customer satisfaction ◦ lower costs spent on resolving customer-found defects ◦ Reduce defect escape between test phases and releases  Test scenario – test case reviews  Exit Criteria for Developer testing  QA acceptance of Development  Environment/Non-functional Requirements  Technical requirements  Environment baselines  GUI checklists prior to test case execution  No Ad-hoc testing: document all variants of testing and their 19
  • 20. Types of Errors 20 Human Errors Omission • Knowledge • Ignorance • Informational • Process-related Translational • Miscommunication • Mismatch in solution and requirement • Mismatch with requirement and test case Design and Coding • Affect Data Integrity • Alter Correctly stored data • Affect downstream system dependencies • Affect testing outcomes Testing • Failure to notice a problem • Misreading the screen. • Failure to execute a planned test. • Criteria disagreements • CRs, Rejections
  • 21. Lessons Learned – Feedback Testing Integration Testing should be scheduled so that the core modules are initially tested. Rigorous unit testing to be done to avoid logical errors. Basic level review and testing should be done at developer's level before handing over the code to the testers and reviewers. Communication Client responsibilities should be clearly communicated in the beginning of the project. Design documents should contain all the necessary validations to avoid validation error. QA should create an accessible communication model to increase transparency, collaboration and trust Process Checklist should be used to avoid GUI errors Test cases should be formed with test data. Developer team should accept defect while researching to move defect through pipeline. HPQC should be utilized at all test phases : DIT, FIT, SIT, UAT, Prod to maintain visibility of all program defect and for trend assessment 21
  • 22. SAMPLE ISSUE # 1 Corrections include • Increased downstream communication for Tier 1 software owners • Create a GUI Checklist to baseline what is acceptable viewing per page type • Increase Dev visibility into high impact change control at pre/post go live Checkpoints • Include Change Coordinator in install planning that can increase risk to prod environment • Escalation method: • IT Change coordinator for release management team to Program Manager, RM Manager and QA Managment • Concerns • Increased workload • the multiple installs of the OM700 program that needed review and approvals. • Issues were found during a dry run last week, • Issues found in a load test from past weekend • the change for next week was found last night. • Not enough planning and test cases performed on the screen that this program runs in. • Reactive response • The change for Monday went to CAB meeting to ensure proper testing and planning have been done. • Why? • To prevent any impacting issues going forward as occurred with the go live in Production installs. 22
  • 23. SAMPLE DEFECT ANALYSIS #1 DEFECTS • 13 Open Defects. • 60% defects with Source Code and Design as the root cause. • 107 Rejected defects for this triage • 30% of all reported defects were rejected. OPPORUNITIES • Implement a Rejection Resolution process to understand why so many defects are reported and rejected. • Increase visibility to Design Traceability and Unit testing/Development Integration testing for increased quality in upstream Program processes. 23 DEFECT CAUSE / DEFECT STATUS
  • 24. Status: IT Release Testing As Of 21-Oct-2011 11.90 17.53 Sev 2 MTTR "Days" Sev 3 MTTR "Days"Sev 1 MTTR "Days" 1.74 Sev 4 MTTR "Days" 0.00 Defect Mean Time To Repair for System Integration Test AVG MTTR "Days" 10.39 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 22/Aug/11 29/Aug/11 05/Sep/11 12/Sep/11 19/Sep/11 26/Sep/11 03/Oct/11 10/Oct/11 17/Oct/11 24/Oct/11 31/Oct/11 07/Nov/11 Days Defect "MTTR" Mean Time To Repair Sev 1 MTTR Sev 2 MTTR Sev 3 MTTR Sev 4 MTTR
  • 25. Status: IT Release Testing As Of 21-Oct-2011 SIT 30-Aug-11 SIT 21-Oct-11 SIT 30-Aug-11 SIT 4-Nov-11 Regression 24-Oct-11 Regression 4-Nov-11 Regression 24-Oct-11 Regression Planned Start Planned Finish Actual Start Trend Finish 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 22/Aug/11 29/Aug/11 05/Sep/11 12/Sep/11 19/Sep/11 26/Sep/11 03/Oct/11 10/Oct/11 17/Oct/11 24/Oct/11 31/Oct/11 07/Nov/11 TestCases Test Case "Pass" Powercurve SIT Actual Pass SIT Projection SIT Trend Regression Actual Passed Regression Projection Regression Trend
  • 26. Test Case defect density Total number of errors found in test scripts vs. developed and executed.  (Defective Test Scripts /Total Test Scripts) * 100 Example: Total test script developed 1360, total test script executed 1280, total test script passed 1065, total test script failed 215 So, test case defect density is 215 X 100 ---------------------------- = 16.8% 1280 This 16.8% value can also be called as test case efficiency %, which is depends upon total number of test cases which uncovered defects. 26 Smaller test case density % Increased test case efficiency High test case execution rates
  • 27. Status: IT Release Testing Sev 4 DensitySev 2 Density 18.18% 0.83% Defects Density for System Integration Test Sev 1 Density 2.48% Total Density 68.60%47.11% Sev 3 Density 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% 22/Aug/11 29/Aug/11 05/Sep/11 12/Sep/11 19/Sep/11 26/Sep/11 03/Oct/11 10/Oct/11 17/Oct/11 24/Oct/11 31/Oct/11 07/Nov/11 Defect Density (Defects/Pass TCs) Overall Defect Density Defect Density (Industry Avg) Sev 1-2 Defect Density Sev 3- 4 Defect Density
  • 28. Status: IT Release Testing 3 74 Sev 3-4 Defect Backlog Defect Discovery and Severity 1-2 Backlog for System Integration Test Sev 1-2 Defect Backlog 0 5 10 15 20 25 30 0 10 20 30 40 50 60 70 80 90 22/Aug/11 29/Aug/11 05/Sep/11 12/Sep/11 19/Sep/11 26/Sep/11 03/Oct/11 10/Oct/11 17/Oct/11 24/Oct/11 31/Oct/11 07/Nov/11 DefectBacklog Defects Defect Discovery and Severity 1-2 Backlog Defects (Sev 1-2) Defects (Sev 3-4) Defect Backlog (Sev 1-2)
  • 29. IT TESTING ITERATION 2 UAT AND DRY RUN ANALYSIS DEFECTS OPPORUNITIES • Implement a Rejection Resolution process to understand why so many defects are reported and rejected. • Increase visibility to Design Traceability and Unit testing/Development Integration testing for increased quality in upstream Program processes. 29
  • 30. Status: Sample Release Testing Pass Trend Analysis SIT 19-Sep-11 SIT 28-Nov-11 SIT 21-Sep-11 SIT 30-Jan-12 Regression 29-Nov-11 Regression 5-Dec-11 Regression 29-Nov-11 Regression Planned Start Planned Finish Actual Start Trend Finish 0 50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 1150 1200 1250 1300 15-Sep-11 22-Sep-11 29-Sep-11 06-Oct-11 13-Oct-11 20-Oct-11 27-Oct-11 03-Nov-11 10-Nov-11 17-Nov-11 24-Nov-11 01-Dec-11 08-Dec-11 15-Dec-11 22-Dec-11 29-Dec-11 05-Jan-12 12-Jan-12 19-Jan-12 26-Jan-12 02-Feb-12 09-Feb-12 TestCases Test Case "Pass" Powercurve SIT Actual Pass SIT Projection SIT Trend Regression Actual Passed Regression Projection Regression Trend
  • 31. Status: Sample Mean Time To Repair Data Sev 4 MTTR "Days" 16.70 Sev 1 MTTR "Days" 17.45 17.99 Sev 2 MTTR "Days" Sev 3 MTTR "Days" 0.69 Defect Mean Time To Repair for System Integration Test AVG MTTR "Days" 13.21 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 15/Sep/11 22/Sep/11 29/Sep/11 06/Oct/11 13/Oct/11 20/Oct/11 27/Oct/11 03/Nov/11 10/Nov/11 17/Nov/11 24/Nov/11 01/Dec/11 08/Dec/11 15/Dec/11 22/Dec/11 29/Dec/11 05/Jan/12 12/Jan/12 19/Jan/12 26/Jan/12 02/Feb/12 09/Feb/12 Days Defect "MTTR" Mean Time To Repair Sev 1 MTTR Sev 2 MTTR Sev 3 MTTR Sev 4 MTTR
  • 32. Status: Sample Defect Disovery and Resolution Backlog 32 118 Sev 3-4 Defect BacklogSev 1-2 Defect Backlog Defect Discovery and Severity 1-2 Backllog for System Integration Test 0 10 20 30 40 50 60 70 80 90 100 0 50 100 150 200 250 15-Sep-11 22-Sep-11 29-Sep-11 06-Oct-11 13-Oct-11 20-Oct-11 27-Oct-11 03-Nov-11 10-Nov-11 17-Nov-11 24-Nov-11 01-Dec-11 08-Dec-11 15-Dec-11 22-Dec-11 29-Dec-11 05-Jan-12 12-Jan-12 19-Jan-12 26-Jan-12 02-Feb-12 09-Feb-12 DefectBacklog Defects Defect Discovery and Severity 1-2 Backlog Defects (Sev 1-2) Defects (Sev 3-4) Defect Backlog (Sev 1-2)
  • 33. Defect Prevention Support Criteria and Actions 33 • Management Approval and Support: Management must support the project team and the objectives of the defect management effort if it is to be successful. • Receptivity of Project Team: The project team should wholeheartedly embrace the program. Project management, in particular, must take ownership for the successful implementation of the program. • Project Duration: The project should not be so long that results will not be known for a long period of time. A project duration of three to six months is probably ideal. • Project Risk: The project should be sufficiently large and complex that the results will be taken seriously, but not so large and complex that success of the pilot project is unlikely. • Early in the Life Cycle: The defect management program should be integrated into project plans and not retrofitted into a project whose approach is well established. Criteria for implementing Defect Prevention efforts • Conduct Training: The project team should be trained in the Defect Management concepts, if necessary. Example: on-boarding new team members. • Incorporate Defect Management Into Project Plan: Each pilot project should adapt the Defect Management approach to its particular circumstances. Emphasis should be placed on automating as much of the process as practical, and measuring the process. Data collected from the pilot projects should form the baseline for the Critical Metrics Set. • Review Results and Refine Approach: Progress implementing the Defect Management Program should be monitored and the approach refined as appropriate. • Report Results: Periodic status reports, which report pilot project results, should be issued. Defect Prevention Actions
  • 34. Problem Closure & Congratulate Team • Acknowledge significance and value of problem solution • Recognize team‟s collective efforts in solving the problem as well as individual contributions • Document what was learned in solving the problem; lessons learned not only about the problem which was solved but also about the problem solving process • Consider investigating other potential causes as preventive actions • Create case study/problem solving reports 34
  • 35. Thank You 35 If you have any questions or concerns about this presentation Contact Yolanda Williams ybowdrywilliams@gmail.com

Editor's Notes

  1. Audience: Can you think of any additional goals for the DMP?
  2. It should also be noted that there are some human factors/cultural issues involved with the defect discovery process.  When a defect is initially uncovered, it may be very unclear whether it is a defect, a change, user error, or a misunderstanding.  Developers may resist calling something a defect because that implies "bad work" and may not reflect well on the development team.  Users may resist calling something a "change" because that implies that the developers can charge them more money.  Some organizations have skirted this issue by initially labeling everything by a different name -- e.g.,  "incidents" or "issues."  From a defect management perspective, what they are called is not an important issue.  What is important is that the defect be quickly brought to the developers' attention and formally controlled.
  3. To report on the status of individual defects.To provide tactical information and metrics to help project management make more informed decisions -- e.g., redesign of error prone modules, the need for more testing, etc.To provide strategic information and metrics to senior management -- defect trends, problem systems, etc.To provide insight into areas where the process could be improved to either prevent defects or minimize their impact.To provide insight into the likelihood that target dates and cost estimates will be achieved.
  4. Functional team – Ad hocPerformanceSITUATPost Go-live
  5. Name other risks that would affect the DMP?
  6. Human errors:Types of Errors Omission,Ignorance,Commission,Typography,Knowledge,Information,External