We think FindBugs is a great tool for finding coding style and best practice types of issues, so we conducted a little experiment a few months ago. We analyzed Jenkins core code with both FindBugs and Coverity. These slides provide a high-level summary of our results.

1. OSS Java Analysis
What You Might Be Missing

2. Overview
• Open Source Static Analysis Options for Java
• Case Study: Analysis of Jenkins Project
• Detailed Findings
2 Copyright 2013, Coverity, Inc.

3. Consistent Use = More Maintainable Code
Numerous Static Analysis Options for Java
Copyright 2013, Coverity, Inc.3
Checkstyle
Defines and enforces coding standards
Finds common programming flaws
and duplicated code
Finds bugs
Finds coding errors, ensures proper
testing coverage and more

4. • Will you use it?
• Does it find issues that you care about? Does it fit into your team’s
workflow? Is it worth the effort?
• Is it actionable?
• Does it just find issues, or does it help you understand how and
where to fix them?
• Is it accurate?
• Will you waste significant time on false positives? Are results
consistent across builds?
• Is it integrated?
• Do you have to go out of your way to use it, or does it bring the
problems to you?
Key Considerations
Copyright 2013, Coverity, Inc.4

5. Case Study: Jenkins
Coverity Development Testing Platform and FindBugs
5 Copyright 2013, Coverity, Inc.

6. Background
• Analyzed Jenkins 1.496 core code using up-to-date
Coverity and FindBugs (as of Dec 2012)
• Both solutions focus on finding defects, have IDE and
command line interfaces
Copyright 2013, Coverity, Inc.6
FindBugs
• Often used automatically in Eclipse
• Extensible tool
• Management capabilities focused
on individual developers
Coverity Platform
• Most often run in Eclipse or via
automated build
• Extensible tool, open platform
• Sophisticated management
capabilities addressing the needs of
developers, teams & organizations

7. Little Overlap
7 Copyright 2013, Coverity, Inc.
Only 28 relevant issues found by
both solutions

8. Defect Summary (Real Bugs Only)
8
Type Coverity FindBugs Both
Resource leaks 86 12 13
Security problems 1 0 0
Concurrency problems 22 10 9
High Severity Subtotal 109 22 22
Coding Standards, Best
Practices, Other
88 605 6
Total Bugs 197 627 28
Coverity
FindBug
s
Impact Distribution
High Med-Low
Copyright 2013, Coverity, Inc.
Each solution found
unique issues (neither is
sufficient to replace the
other)

9. Here is a resource leak identified by FindBugs:
• Identifies the problem and general location
• But there’s no indication of the affected object/variable, the
relevant code path or events
• Developers need to figure it out on their own
• Not a big deal in this simple code
• Complex code will be a pain!
Example: FindBugs Resource Leak
Copyright 2013, Coverity, Inc.9
A leak might affect something on this line

10. Here is a leak identified by Coverity:
The object will continue to hold resources until the garbage collector
eventually finalizes it—which may never happen.
Example: Coverity Resource Leak
Copyright 2013, Coverity, Inc.10
“listener” allocated. Implements
Closeable, so it needs to be closed
“listener” used, but still needs to be closed
Leak: no longer
usable, and never closed

11. Example: Coverity Race Condition
11 Copyright 2013, Coverity, Inc.
This is an HTTP request handler
“o” is associated with session
Another thread may modify the session
Session attribute is deleted. The right one?
Erroneous code path identified
You may delete the wrong key! Good luck finding this manually!

12. Example: Coverity Null Dereference
Copyright 2013, Coverity, Inc.12
Passing null to function
Dereferenced two layers deep
You should check for null before or during the call to rebuild()

13. Conclusion
• Different analysis tools often find different (important and
complementary) issues
• Use multiple tools that find issues important to you
• Avoid tools that take too long to analyze, require too much
manual work or have too many false positives
13 Copyright 2013, Coverity, Inc.

14. Copyright 2013 Coverity, Inc.