Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quality of Bug Reports in Open Source

Presented at Dagstuhl Seminar 07491, "Mining Programs and Processes", 2007.

Quality of Bug Reports in Open Source

  1. 1. Quality of Bug Reports in Open Source Nicolas Bettenburg Adrian Schröter Rahul Premraj Saarland University Saarland University Saarland University Sascha Just Cathrin Weiss Tom Zimmermann Saarland University University of Zürich University of Calgary
  2. 2. Developers
  3. 3. Developers 175222 I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please...
  4. 4. Developers 175222 I20030205 Run the following example. Double click on a tree item and I wand to create a new plugin in Eclipse using CDT. Shall it notice that it does not expand. possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to Comment out the Selection listener and now double click create a new plugin ( user defined plugin ) using CDT. After on any tree item and notice that it expands. that I wand to impliment it in my programe. If it possible?. Any one can help me please... public static void main(String[] args) { Display display = new Display(); ...
  5. 5. Researchers
  6. 6. Researchers
  7. 7. What makes a good bug report?
  8. 8. Ask Google
  9. 9. Ask Google
  10. 10. Ask developers Experts
  11. 11. Survey Five minutes
  12. 12. Participants 809 contacts 156 responses
  13. 13. Information Question 1 Which of the following items have you previously used when fixing bugs? (select as many items as you wish) Question 2 Which three items helped you the most? (select at most three items)
  14. 14. Results Helped most Used when fixing a bug
  15. 15. Results steps to reproduce Helped most stack traces screenshots observed behavior test cases/examples expected behavior version hardware severity Used when fixing a bug
  16. 16. Problems Question 3 Which of the following problems have you encountered when fixing bugs? (select as many items as you wish) Question 4 Which three problems caused you most delay in fixing bugs? (select at most three items)
  17. 17. Results Most delay Encountered when fixing a bug
  18. 18. Results incomplete information Most delay erroneous steps to reproduce wrong observed behaviour wrong expected behaviour wrong product information bad grammar errors in test cases duplicates wrong system information viruses/spam Encountered when fixing a bug
  19. 19. bug bugzilla case checkbox behavior biggest based comments common component crucial delay clearly determine developer error etc expected filed fixing information issue missing helpful include incomplete mozilla number people problem product project question reports reproduce reproduction scope sometimes status steps system test testcase understand used user usually version wrong
  20. 20. bug bugzilla case checkbox behavior biggest based comments common component crucial delay clearly determine developer error etc expected filed fixing information issue missing helpful include incomplete mozilla number people problem product project question reports reproduce reproduction scope sometimes status steps system test testcase understand used user usually version wrong
  21. 21. Good vs. poor reports Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please...
  22. 22. Good vs. poor reports Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... Poor Average Good
  23. 23. Good vs. poor reports +++ APACHE: 229 +++ ECLIPSE: 397 +++ MOZILLA: 560 +++ Create a new plugin using CDT. I wand to create a new plugin in Eclipse using CDT. Shall it possible. I had made a R&D in eclipse documentation. I had get an idea about create a plugin using Java. But i wand to create a new plugin ( user defined plugin ) using CDT. After that I wand to impliment it in my programe. If it possible?. Any one can help me please... Poor Average Good
  24. 24. How can we measure quality?
  25. 25. Measure quality Report
  26. 26. Measure quality Quality Score Report CUEzilla
  27. 27. Measure quality Quality Score Report CUEzilla Set of Features Keywords Stack Trace Code Examples Readability Repro Steps Screenshots
  28. 28. Leave-one out validation Training – build a model Testing
  29. 29. Results Predicted by CUEzilla 0 0 1 0 0 0 2 0 0 0 Rated by developers 4 11 29 17 4 0 1 6 12 5 0 1 2 4 1
  30. 30. Results Predicted by CUEzilla 0 0 1 0 0 0 2 0 0 0 Rated by developers 4 11 29 17 4 0 1 6 12 5 0 1 2 4 1 44% agreement
  31. 31. Results Predicted by CUEzilla 0 0 1 0 0 0 2 0 0 0 Rated by developers 4 11 29 17 4 0 1 6 12 5 0 1 2 4 1 44% agreement 87% close by one
  32. 32. Recommendations Add a stack trace to improve your bug report by 20%.
  33. 33. Recommendations Add a stack trace to improve your bug report by 20%.
  34. 34. Recommendations Why should I care? Add a stack trace to improve your bug report by 20%.
  35. 35. Creating incentive
  36. 36. Creating incentive Bug reports with stack traces get fixed sooner.
  37. 37. Mining facts Keywords Readability Stack Trace Repro Steps Code Examples Screenshots Set of Features
  38. 38. Mining facts Keywords Readability Stack Trace Time to fix Repro Steps Code Examples Screenshots Set of Features Fixed or not
  39. 39. Mining facts Keywords Readability Stack Trace Correlate Time to fix Repro Steps Code Examples Screenshots Set of Features Fixed or not
  40. 40. Did you know? Bug reports containing stack traces get fixed sooner. (APACHE/ECLIPSE/MOZILLA) Bug reports that are easier to read have lower lifetimes. (APACHE/ECLIPSE/MOZILLA) Including code samples in your bug report increases the chances of it getting fixed. (MOZILLA)
  41. 41. Future work
  42. 42. Reporter survey Question 1 What information have you previously provided in bug reports? (select as many items as you wish) Question 2 What of the above information was most difficult to provide? (select at most three items) Question 3 In your opinion, what information is most relevant to developers? (select at most three items) 310 responses
  43. 43. Comments Developers 48 Reporters 127 Comments 175
  44. 44. Better tool support
  45. 45. Better tool support Add a stack trace to improve your bug report by 20%. Bug reports with stack traces get fixed sooner.
  46. 46. bug cases cdt code create cuezilla defined delay behaviour developers duplicates eclipse documentation fixing encountered examples helped impliment expected information items java mozilla observed please plugin poor possible problems programe quality report results question researchers saarland stack select test screenshots steps system shall thanks trace university user viruses wand wrong

×