Submit Search
Upload
Characterizing and Predicting Which Bugs Get Reopened
•
1 like
•
2,002 views
Thomas Zimmermann
Follow
Presented at ICSE 2012 in Zurich.
Read less
Read more
Technology
Business
Report
Share
Report
Share
1 of 32
Recommended
Predicting Method Crashes with Bytecode Operations
Predicting Method Crashes with Bytecode Operations
Thomas Zimmermann
Analytics for smarter software development
Analytics for smarter software development
Thomas Zimmermann
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Web Application Remediation - OWASP San Antonio March 2007
Web Application Remediation - OWASP San Antonio March 2007
Denim Group
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
TEST Huddle
Software Design for Testability
Software Design for Testability
amr0mt
Defect effort prediction models in software
Defect effort prediction models in software
IAEME Publication
Recommended
Predicting Method Crashes with Bytecode Operations
Predicting Method Crashes with Bytecode Operations
Thomas Zimmermann
Analytics for smarter software development
Analytics for smarter software development
Thomas Zimmermann
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Design for Testability: A Tutorial for Devs and Testers
Design for Testability: A Tutorial for Devs and Testers
TechWell
Web Application Remediation - OWASP San Antonio March 2007
Web Application Remediation - OWASP San Antonio March 2007
Denim Group
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
Peter Zimmerer - Evolve Design For Testability To The Next Level - EuroSTAR 2012
TEST Huddle
Software Design for Testability
Software Design for Testability
amr0mt
Defect effort prediction models in software
Defect effort prediction models in software
IAEME Publication
Software testability slide share
Software testability slide share
BeBo Technology
AV-Comparatives Performance Test
AV-Comparatives Performance Test
Herbert Rodriguez
SW Engineering Management
SW Engineering Management
Robert Sayegh
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Комсс Файквэе
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
ESET Middle East
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Cristiano Caetano
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Clemens Reijnen
Online exa-syste
Online exa-syste
Anand Bavarava
Soffront Defect tracking
Soffront Defect tracking
Soffront Software
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Bill Duncan
Cross-project defect prediction
Cross-project defect prediction
Thomas Zimmermann
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Aaron Humerickhouse
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Denim Group
Software Testing Basics
Software Testing Basics
sachinmistry786
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Dante Briones
Real Cost of Software Remediation
Real Cost of Software Remediation
Denim Group
Attacking antivirus
Attacking antivirus
UltraUploader
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Marc Miquel
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Thomas Zimmermann
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Thomas Zimmermann
Bug Advocacy
Bug Advocacy
nazeer pasha
More Related Content
What's hot
Software testability slide share
Software testability slide share
BeBo Technology
AV-Comparatives Performance Test
AV-Comparatives Performance Test
Herbert Rodriguez
SW Engineering Management
SW Engineering Management
Robert Sayegh
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Комсс Файквэе
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
ESET Middle East
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Cristiano Caetano
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Clemens Reijnen
Online exa-syste
Online exa-syste
Anand Bavarava
Soffront Defect tracking
Soffront Defect tracking
Soffront Software
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Bill Duncan
Cross-project defect prediction
Cross-project defect prediction
Thomas Zimmermann
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Aaron Humerickhouse
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Denim Group
Software Testing Basics
Software Testing Basics
sachinmistry786
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
suwalki24.pl
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Dante Briones
Real Cost of Software Remediation
Real Cost of Software Remediation
Denim Group
Attacking antivirus
Attacking antivirus
UltraUploader
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Marc Miquel
What's hot
(19)
Software testability slide share
Software testability slide share
AV-Comparatives Performance Test
AV-Comparatives Performance Test
SW Engineering Management
SW Engineering Management
Dtl 2012 kl-app_ctl1.2
Dtl 2012 kl-app_ctl1.2
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Protecting Enterprise - An examination of bugs, major vulnerabilities and exp...
Bill of-rights-white-paper-final-012312
Bill of-rights-white-paper-final-012312
Alm 4 Azure with screenshots
Alm 4 Azure with screenshots
Online exa-syste
Online exa-syste
Soffront Defect tracking
Soffront Defect tracking
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Automatic Proactive Troubleshooting with IBM Rational Build Forge
Cross-project defect prediction
Cross-project defect prediction
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
Magenic-White-Paper-Continuous-Inegration-for-QA-Teams
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
RSA 2015 Blending the Automated and the Manual: Making Application Vulnerabil...
Software Testing Basics
Software Testing Basics
Open Source tools in Continuous Integration environment (case study for agil...
Open Source tools in Continuous Integration environment (case study for agil...
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
San Jose Selenium Meetup 22 Mar 2012: The Restless Are Getting Native
Real Cost of Software Remediation
Real Cost of Software Remediation
Attacking antivirus
Attacking antivirus
Quality Assurance 1: Why Quality Matters
Quality Assurance 1: Why Quality Matters
Similar to Characterizing and Predicting Which Bugs Get Reopened
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Thomas Zimmermann
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Thomas Zimmermann
Bug Advocacy
Bug Advocacy
nazeer pasha
Bug best practice
Bug best practice
gaoliang641
Classic Testing Mistakes 0226
Classic Testing Mistakes 0226
MBA_Community
Improving developer tester collaboration with microsoft visual studio 2010
Improving developer tester collaboration with microsoft visual studio 2010
Mohamed Samy
10 Reasons Why You Fix Bugs As Soon As You Find Them
10 Reasons Why You Fix Bugs As Soon As You Find Them
Rosie Sherry
DevOps - Continuous Integration, Continuous Delivery - let's talk
DevOps - Continuous Integration, Continuous Delivery - let's talk
D Z
bug-advocacy
bug-advocacy
KALYAN Chakravarthy
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
Mary Clemons
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
michael.labriola
Debugging
Debugging
Jonathan Holloway
Continuous integration
Continuous integration
Boris Dominic
How to Reproduce a Non Reproducible Bug
How to Reproduce a Non Reproducible Bug
BugRaptors
midterm_fa08.pdf
midterm_fa08.pdf
RavinderKSingla
Quality of Bug Reports in Open Source
Quality of Bug Reports in Open Source
Thomas Zimmermann
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
Vaibhav Khanna
179 black-box-software-testing-copyright-2003-cem-kaner1652
179 black-box-software-testing-copyright-2003-cem-kaner1652
ngothanhtungth
Agile testing overview
Agile testing overview
raianup
Agile testingoverview
Agile testingoverview
Rangabashyam S
Similar to Characterizing and Predicting Which Bugs Get Reopened
(20)
Not my bug! Reasons for software bug report reassignments
Not my bug! Reasons for software bug report reassignments
Characterizing and predicting which bugs get fixed
Characterizing and predicting which bugs get fixed
Bug Advocacy
Bug Advocacy
Bug best practice
Bug best practice
Classic Testing Mistakes 0226
Classic Testing Mistakes 0226
Improving developer tester collaboration with microsoft visual studio 2010
Improving developer tester collaboration with microsoft visual studio 2010
10 Reasons Why You Fix Bugs As Soon As You Find Them
10 Reasons Why You Fix Bugs As Soon As You Find Them
DevOps - Continuous Integration, Continuous Delivery - let's talk
DevOps - Continuous Integration, Continuous Delivery - let's talk
bug-advocacy
bug-advocacy
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
Debugging
Debugging
Continuous integration
Continuous integration
How to Reproduce a Non Reproducible Bug
How to Reproduce a Non Reproducible Bug
midterm_fa08.pdf
midterm_fa08.pdf
Quality of Bug Reports in Open Source
Quality of Bug Reports in Open Source
Software engineering 22 error detection and debugging
Software engineering 22 error detection and debugging
179 black-box-software-testing-copyright-2003-cem-kaner1652
179 black-box-software-testing-copyright-2003-cem-kaner1652
Agile testing overview
Agile testing overview
Agile testingoverview
Agile testingoverview
More from Thomas Zimmermann
Software Analytics = Sharing Information
Software Analytics = Sharing Information
Thomas Zimmermann
MSR 2013 Preview
MSR 2013 Preview
Thomas Zimmermann
Klingon Countdown Timer
Klingon Countdown Timer
Thomas Zimmermann
Data driven games user research
Data driven games user research
Thomas Zimmermann
Empirical Software Engineering at Microsoft Research
Empirical Software Engineering at Microsoft Research
Thomas Zimmermann
Security trend analysis with CVE topic models
Security trend analysis with CVE topic models
Thomas Zimmermann
Analytics for software development
Analytics for software development
Thomas Zimmermann
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Predicting Defects using Network Analysis on Dependency Graphs
Predicting Defects using Network Analysis on Dependency Graphs
Thomas Zimmermann
Meet Tom and his Fish
Meet Tom and his Fish
Thomas Zimmermann
Predicting Subsystem Defects using Dependency Graph Complexities
Predicting Subsystem Defects using Dependency Graph Complexities
Thomas Zimmermann
Got Myth? Myths in Software Engineering
Got Myth? Myths in Software Engineering
Thomas Zimmermann
Mining Workspace Updates in CVS
Mining Workspace Updates in CVS
Thomas Zimmermann
Mining Software Archives to Support Software Development
Mining Software Archives to Support Software Development
Thomas Zimmermann
Unit testing with JUnit
Unit testing with JUnit
Thomas Zimmermann
esolang: Esoterische Programmiersprachen
esolang: Esoterische Programmiersprachen
Thomas Zimmermann
TA-RE: An Exchange Language for Mining Software Repositories
TA-RE: An Exchange Language for Mining Software Repositories
Thomas Zimmermann
Fine-grained Processing of CVS Archives with APFEL
Fine-grained Processing of CVS Archives with APFEL
Thomas Zimmermann
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
Thomas Zimmermann
More from Thomas Zimmermann
(20)
Software Analytics = Sharing Information
Software Analytics = Sharing Information
MSR 2013 Preview
MSR 2013 Preview
Klingon Countdown Timer
Klingon Countdown Timer
Data driven games user research
Data driven games user research
Empirical Software Engineering at Microsoft Research
Empirical Software Engineering at Microsoft Research
Security trend analysis with CVE topic models
Security trend analysis with CVE topic models
Analytics for software development
Analytics for software development
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Predicting Defects using Network Analysis on Dependency Graphs
Predicting Defects using Network Analysis on Dependency Graphs
Meet Tom and his Fish
Meet Tom and his Fish
Predicting Subsystem Defects using Dependency Graph Complexities
Predicting Subsystem Defects using Dependency Graph Complexities
Got Myth? Myths in Software Engineering
Got Myth? Myths in Software Engineering
Mining Workspace Updates in CVS
Mining Workspace Updates in CVS
Mining Software Archives to Support Software Development
Mining Software Archives to Support Software Development
Unit testing with JUnit
Unit testing with JUnit
esolang: Esoterische Programmiersprachen
esolang: Esoterische Programmiersprachen
TA-RE: An Exchange Language for Mining Software Repositories
TA-RE: An Exchange Language for Mining Software Repositories
Fine-grained Processing of CVS Archives with APFEL
Fine-grained Processing of CVS Archives with APFEL
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
DynaMine: Finding Common Error Patterns by Mining Software Revision Histories
Recently uploaded
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
MadyBayot
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
Nanddeep Nachan
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
Overkill Security
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
wesley chun
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
MIND CTI
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
The Digital Insurer
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
apidays
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel Araújo
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
apidays
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Juan lago vázquez
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
apidays
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
Khushali Kathiriya
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
jfdjdjcjdnsjd
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
lior mazor
Recently uploaded
(20)
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
Characterizing and Predicting Which Bugs Get Reopened
1.
Characterizing and Predicting
Which Bugs Get Reopened Thomas Zimmermann Nachiappan Nagappan Microsoft Research Philip J. Guo Stanford University Brendan Murphy Microsoft Research © Microsoft Corporation
2.
A bug’s life Picture
on the right via http://www.bugzilla.org/docs/2.18/html/lifecycle.html © Microsoft Corporation
3.
Final part of
a trilogy… Which bugs are fixed? Bug reassignments Bug reopens (this paper) ICSE 2010 CSCW 2011 ICSE 2012 SEIP © Microsoft Corporation
4.
…and partly a
remake Emad Shihab, Akinori Ihara, Yasutaka Kamei, Walid M. Ibrahim, Masao Ohira, Bram Adams, Ahmed E. Hassan, Ken-ichi Matsumoto: Predicting Re-opened Bugs: A Case Study on the Eclipse Project. WCRE 2010: 249-258 © Microsoft Corporation
5.
Shihab et. al:
Predicting Reopened Bugs This paper: Characterizing Reopened Bugs (WCRE 2010) Four dimensions: work habits, bug report, bug fix and team Predicted reopened bugs with a precision of 62.9% and a recall of 84.5% (d-trees) Top node analysis found that bug report dimension was most influential | © Microsoft Corporation
6.
Shihab et. al:
Predicting Reopened Bugs This paper: Characterizing Reopened Bugs (WCRE 2010) Four dimensions: work habits, bug report, Partial replication of Shihab et al. bug fix and team New measurements: organizational and Predicted reopened bugs with a precision geographic distance, reputation, how found of 62.9% and a recall of 84.5% (d-trees) Qualitative component on the causes of Top node analysis found that bug report bug reopens (identified with a survey) dimension was most influential Descriptive models (logistic regression) | | © Microsoft Corporation
7.
Methodology
Qualitative • “In your experience, what are reasons why a bug would be reopened multiple times” survey • 358 out of 1,773 responded. Card sort. Quantitative • All bug reports for Windows Vista and Windows 7 analysis • Logistic regression model for reopened bugs Manual • Random sample of reopened bugs inspection • 20 bug reports © Microsoft Corporation
8.
What are reasons
for bug report reopens? © Microsoft Corporation
9.
Causes of bug
reopens Not FIXED Related to Root Cause Bugs difficult to reproduce Developers misunderstood root cause Bug had insufficient information Related to Priority Priority of the bug increased FIXED Regression bugs Process-related Process © Microsoft Corporation
10.
#1: Difficult to
reproduce “The bug is hard to reproduce and so the fix was made without being able to fully verify it. A good example is a customer who reports something. We think we see the issue in house and fix that. It turns out we saw something different…” “Bugs which are difficult to reproduce generally get re-activated multiple times. At first, developers will give a simple repro attempt before resolving bugs 'Not repro'. But if the bug opener is able to reproduce the issue again, or perhaps comes up with better repro instructions, then the developer will pay more attention the second time the bug is activated.” “Heisenbugs” © Microsoft Corporation
11.
#2: Misunderstood root
cause “The bug is tracking an unidentified symptom and it takes a while to fully root cause. This comes up a lot with memory leaks: there will be an unknown memory leak in a component and the owning team plays whack-a-mole with the code defects to remove memory issues one-by-one.” “Not fixing the root cause and only addressing symptoms. Without root cause understood for the bug a patch/hack can often be done that will then be reactivated.” © Microsoft Corporation
12.
#3: Insufficient information
“Poor bug quality. If the bug wasn't described well enough, or not enough diagnostic info was there, the dev will guess and fix *something* in order to make the bug go away. What they fix isn't always what the person who filed the bug ran into.” “If a bug report does not accurately convey enough information about what is actually wrong (i.e. it describes incorrect behavior but neglects to mention data loss) or if the bug does not convey a dependency (such as another team relying on a fix), a bug may be de-prioritized and resolved without fixing.” © Microsoft Corporation
13.
#4: Increased priority
“Bugs are closed because one person or triage team believes the bug is not worthy of fixing (i.e. too risky, don't care, etc.), but then a few days later a VP or external customer reports the same issue, then the bug has a higher priority.” “Other reason is lack of business justification or too late in product cycle; reopened when sufficient justification exists or new cycle begins.” “One team may feel an issue is critical while the other does not see it as important enough, and instead of carrying a discussion, the bug is bounced around.” © Microsoft Corporation
14.
#5: Regression bugs
“First attempt at fix was flawed in some way, and wasn't caught because of lack of testing or unknown related scenario regression.” “I've seen cases in the past where it was thought that a bug was fixed only to find that a corner case had been missed.” “I've also seen cases where the bug was only being hit due to a timing issue and something changed that affected the timing and the bug disappeared again.” © Microsoft Corporation
15.
#6: Process-related bugs
“Sometimes bugs are reopened due to a misunderstanding of process. e.g. dev resolves bug when fix is submitted, but tester reactivates because bug still repros (because fix has not yet reached tester).” “Bug is verified fixed in a feature of developer’s branch and the fix takes too long to hit the main branch.” © Microsoft Corporation
16.
#6: Process-related bugs
“First of all, I don’t like the model where we reactivate bugs that were Fixed but the issue was not resolved. Logically it makes sense, but tracking the thread of the issue through multiple checkins & reactivates can be hell if it happens more than once or twice. I would prefer a model where once a checkin has been made for a bug, that bug is done! New issues, or issues that linger despite a previous fix, should/ would be tracked in a new bug.” © Microsoft Corporation
17.
What factors
correlate with bug reopens? © Microsoft Corporation
18.
Does the source
of a bug (how it was found) influence the likelihood of bug reopens? Bug Sources Vista Win7 Reopen rate for all bugs P Q Code analysis tools 0.52P 0.73Q less likely to be reopened Human review 0.85P 0.66Q Ad-hoc testing 0.87P 0.99Q Internal user 1.12P 0.97Q Component testing 1.13P 0.81Q System testing 1.21P 1.46Q more likely to be reopened Customer 1.33P 1.12Q © Microsoft Corporation
19.
Does opener reputation
influence the likelihood of bug reopens? For each bug, calculate opener’s reputation by aggregating over all bugs in the past. Hooimeijer and Weimer: Modeling bug report quality. ASE 2007. © Microsoft Corporation
20.
Does opener reputation
influence the likelihood of bug reopens? more likely to be reopened less likely to be reopened © Microsoft Corporation
21.
Does organizational and
geographic distance influence the likelihood of bug reopens? Vista Win7 Organizational Opened by and initially assigned to … distance … the same person X R … someone with the same manager 1.13X 0.96R … someone with a different manager 1.37X 1.07R Geographic Opened by and initially assigned to … distance … the same person X R … someone in the same building 1.27X 0.93R … someone in a different building but in the same country 1.45X 1.00R … someone in a different country 1.52X 1.14R © Microsoft Corporation
22.
Does organizational and
geographic distance influence the likelihood of bug reopens? Vista Win7 Assigned to opener at some point in time Y S Never assigned to opener, but assigned to 0.54Y 0.39S someone with the same manager as opener Never assigned to anyone with same manager 0.27Y 0.34S Never assigned to opener, but assigned to 0.41Y 0.37S someone in the same building Never assigned to anyone in same building, but 0.31Y 0.43S assigned to someone in the same country Never assigned to anyone in the same country 0.20Y 0.20S © Microsoft Corporation
23.
Descriptive statistical analysis
• All pre- and post-release bug reports for Windows Vista and Windows 7 until July 2009 • Logistic regression model to characterize – Probability that a bug will be reopened • Logistic regression model to characterize – Probability that a bug will be fixed after the bug has been reopened – Probability that a bug will be fixed (Guo et al., ICSE 2010) • Same factors as in Guo et al., ICSE 2010 © Microsoft Corporation
24.
Factor
Reopen (Vista) Human review not significant Code analysis tool -0.503 Bug source: Component testing 0.238 (categorical) Ad-hoc testing (baseline) System testing 0.204 Customer 0.239 Internal user not significant Reputation of bug opener -0.266 Reputation of 1st assignee not significant Opened by temporary employee 0.178 Initial severity level 0.127 Severity upgraded? 0.331 Opener / any assignee same manager? 0.721 Opener / any assignee same building? 0.468 Num. editors 0.236 Num. assignee building 0.090 Num. component path changes -0.160 © Microsoft Corporation
25.
Factor
Reopen (Vista) Human review not significant Code analysis tool decrease (-0.503) Bug source: Component testing increase (0.238) (categorical) Ad-hoc testing (baseline) System testing increase (0.204) Customer increase (0.239) Internal user not significant Reputation of bug opener -0.266 Reputation of 1st assignee not significant Opened by temporary employee 0.178 Initial severity level 0.127 Severity upgraded? 0.331 Opener / any assignee same manager? 0.721 Opener / any assignee same building? 0.468 Num. editors 0.236 Num. assignee building 0.090 Num. component path changes -0.160 © Microsoft Corporation
26.
Factor
Reopen (Vista) Human review not significant Code analysis tool decrease (-0.503) Bug source: Component testing increase (0.238) (categorical) Ad-hoc testing (baseline) System testing increase (0.204) Customer increase (0.239) Internal user not significant Reputation of bug opener decrease Reputation of 1st assignee not significant Opened by temporary employee increase Initial severity level increase Severity upgraded? increase Opener / any assignee same manager? increase Opener / any assignee same building? increase Num. editors increase Num. assignee building increase Num. component path changes decrease © Microsoft Corporation
27.
Which *reopened*
bugs get fixed? vs. Which bugs get fixed? © Microsoft Corporation
28.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
29.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
30.
Fixed When
Fixed (Vista) Factor Reopened (Vista) [Guo, ICSE 2010] Human review 0.377 0.511 Code analysis tool not significant 0.357 Bug source: Component testing -0.160 0.065 (categorical) Ad-hoc testing System testing not significant -0.129 Customer -0.498 -0.347 Internal user -0.465 -0.454 Reputation of bug opener 1.632 2.193 Reputation of 1st assignee 1.651 2.463 Opened by temporary employee -0.144 -0.125 Initial severity level not significant 0.033 Severity upgraded? not significant 0.256 Opener / any assignee same manager? not significant 0.676 Opener / any assignee same building? not significant 0.270 Num. editors 0.127 0.240 Num. assignee building -0.213 -0.257 Num. component path changes -0.162 -0.232 Num. re-opens n/a -0.135 © Microsoft Corporation
31.
Lessons learned
• Improve reproducibility of bug reports • Provide better tools to identify root cause • Better estimate initial priorities • Reduce the complexity of branching (bugs were “verified” in the wrong branch) © Microsoft Corporation
32.
Thank you!
Partial replication of Shihab et al. New measurements: organizational and geographic distance, reputation, how found Qualitative component on the causes of bug reopens (survey): root cause, priority, process Descriptive models based on logistic regression http://research.microsoft.com/ese © Microsoft Corporation