Eclipse JDT Java null analysis to prevent NullPointerException presentation given at EclipseCon Europe 2016 in Ludwigsburg Germany

1. The End of the world as we know it?
AKA your last NullPointerException $1B bugs!
Michael @Vorburger.ch
EclipseCon Europe 2016, October 25, 2016

2. EclipseCon NullPointerException2
Agenda
1. Topic Intro
2. “How to” (incl. mvn build & IDE config)
3. Some experiences from ongoing work
4. Call to community action++

3. EclipseCon NullPointerException3
null analysis
is not at all by Red Hat!
AFAIK contributions mostly
by Stephan Herrmann; as
volunteer! #ACK
is a Linux Foundation hosted
platform, leading the future of open
Software Defined Networking
(SDN), based on OSGi (Karaf),
backed by a strong multi vendor
community with 100s of Devs. Red
Hat contributes to ODL, for which
I’ve started to use Eclipse null
analysis @work, and extended
some tooling, and will share some
experiences from #dayjob.
BTW: Red Hat is a
great company to work
for…
redhat.com/jobs
#LifeAtRedHat

4. EclipseCon NullPointerException4
History
C. A. R. Hoare (Mr. QuickSort et al) jokingly apologized in 2009 for
“adding null reference to ALGOL [=> C => Java], which probably caused
a billion dollars of pain and damage in the last 40 years” (AKA $1B
mistake)
NB: There actually are alt. (JVM) langs with types systems without any
null at all. (Try, for fun.) Meanwhile in the real world, Java has null - and
that ain’t never gonna change anymore…. but still not all hope is lost! ;-)

5. EclipseCon NullPointerException5
Why
NPE
Blog
Post
most
viewed?

6. EclipseCon NullPointerException6
What?
Static (compile-time)
code analysis re. po.
null-ness of method
parameters, local
variables, fields.
Why?
Aim to prove that your
code is null safe, so
NullPointerException
will never occur anymore.
How?
Let’s see together...

7. EclipseCon NullPointerException7
Get started

8. EclipseCon NullPointerException
<dependency>
<groupId>org.eclipse.jdt</groupId>
<artifactId>org.eclipse.jdt.annotation</artifactId>
<version>2.0.0</version>
</dependency>
8
@ which?
Let’s get back to more about this (alternatives) in a moment…

9. EclipseCon NullPointerException
package-info.java, in EACH package, src/main before /test:
@org.eclipse.jdt.annotation.NonNullByDefault
package yourpackage;
9
@NonNullByDefault (Recommendation)
Activating null analysis via Project Properties alone won’t do much yet.
You’ll need to express what you want analysed with Java annotations.
And occasionally, where you must, use @Nullable in (existing..) code.

10. EclipseCon NullPointerException10

11. EclipseCon NullPointerException11
Enjoy your extended coffee break… ;-)
init() close()
Done!
Shortest presentation ever.

12. EclipseCon NullPointerException12
Done?
1. How will a (Maven) build enforce this, same as IDE?
2. How will each project get configured for this in-IDE?
3. Libraries, huh? (#existingcode #nogreenfield)
3.1 Library EEA & the build...

13. EclipseCon NullPointerException13
This was the most useful I was able to find, and (significantly...) extended and built upon:
● https://github.com/sylvainlaurent/eclipse-external-annotations-m2e-plugin
● https://github.com/sylvainlaurent/null-pointer-analysis-examples
● https://github.com/sylvainlaurent/eclipse-external-annotations
Curious: Sylvain lives literally next door, and we have common friends - what are the odds??
#FLOSS
Research existing attempts to address this
#ACK: Others have explored the same space before me - but I hadn’t found a full solution...

14. EclipseCon NullPointerException
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
...
<compilerId>jdt</compilerId>
<compilerArguments>
<properties>.settings/org.eclipse.jdt.core.prefs</properties>
? ^ ${basedir}/ ?
<dependencies>
<dependency> <!-- of maven-compiler-plugin, not project -->
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-compiler-jdt</artifactId>
14
Build
1. https://wiki.eclipse.org/Tycho/FAQ#Can_I_use_the_Tycho_compiler_support_in_non-OSGi_projects.2C_too.3F
2. https://github.com/jbosstools/m2e-jdt-compiler
3. https://issues.apache.org/jira/browse/MCOMPILER-123

15. EclipseCon NullPointerException
https://github.com/vorburger/eclipse-null-eea-augments illustrates 3. in full example project.
https://github.com/vorburger/eclipse-external-annotations-m2e-plugin/commits/individualEEAs is
a M2E AbstractProjectConfigurator which copies JDT compilerArguments properties from artifact
in compiler-plugin dependency, or compilerArguments properties, into IDE.
(TBD ⇒ ? #helpwanted)
15
Project Configuration
Depends on where you sit on having .settings/** in SCM… ;-) Basically you can either:
1. use workspace wide default preferences (recommended: Oomph!); NOK for gradually..
2. save .settings/*jdt.core.prefs project preferences properties per project in SCM
3. .settings/ in .gitignore & org.eclipse.jdt.core.prefs in Maven artifact, and:
a. use maven-dependency-plugin to unpack it into target/ for build
point maven-compiler-plugin to use that instead of .settings/
b. declare <dependency> and use M2E extension to set into each project (JDT API)

16. EclipseCon NullPointerException
java.util java.util
Optional.ofNullable(null) /* @Nullable */ V Map.get(Object key)
Null type mismatch: required ...
'@NonNull String' but the provided
value is null
16
The underlying problem is interfacing with existing
library code which you cannot modify... or... can we?!
Libraries...
You always use external 3rd-party libraries in all of your projects.
Even if you think you don’t, you do - JDK! Look at 1st of many you’ll hit:

17. EclipseCon NullPointerException17

18. EclipseCon NullPointerException18
External *.eea annotation files “augment” (not in the
Star Trek sense..) Eclipse Java Build Path entries with
additional @NonNull VS @Nullablable information.
Useful when working with 3rd-party library JARs you
cannot easily change - including the JDK RT.
Eclipse External null Annotations (EEA)

19. EclipseCon NullPointerException19

20. EclipseCon NullPointerException20

21. EclipseCon NullPointerException21
https://github.com/vorburger/eclipse-null-eea-augments (TBD ⇒ ? #helpwanted #449653)
Producing EEA JARs as Maven artifacts
slf4j-api-eea ch.vorburger.null:eea-parent
maven-resources-plugin config.
maven-compiler-plugin disabled
maven-surefire-plugin disabled
/pom.xml
/org/slf4j/LoggerFactory.eea
/eea-for-gav
NOT using Maven classifier - just naming convention of artifactId ending in *-eea (for filtering
list of dependencies for performance in M2E plugin). Maven Classifiers don’t work great in
M2E, and aren’t a fit here anyway - the EEA isn’t really an additional artifact for one (≉ GAV).

22. EclipseCon NullPointerException
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerArguments>
<annotationpath>CLASSPATH</annotationpath>
<dependencies> <!-- of project, not maven-compiler-plugin -->
<dependency>
<groupId>ch.vorburger.null.eea</groupId>
<artifactId>slf4j-api-eea</artifactId>
<version>1.0.0-SNAPSHOT</version>
<scope>provided</scope>
22
EEA & the build
1. Granularity not identical to in-IDE, but this works just fine… #pragmatic
2. Not REALLY needed on CP, but this approach avoids rebuilding infra. for DL, dist, etc.
#itworks

23. EclipseCon NullPointerException
External Annotations & M2E Maven in IDE
23
I’ve extended M2E Project Configurator
(same which sets Compiler Properties) to
set External annotations for Maven Dependencies classpath container.
Problem: How to map JAR (or Project) with *.eea to Maven GAVs / JRE?
I’ve used a file in JAR (eea-for-gav: org.slf4j:slf4j-api). It has
N lines; useful e.g. for org.mockito:mockito-all & :mockito-core & “fat” J.
Supports: Open/Close (=> binary) project - standard Maven Workspace
Dependency Resolution (TBD ⇒ ? #helpwanted #457792)

24. EclipseCon NullPointerException24
In an ideal world, we should all have a common repo
of EEA artifacts where we share null annotations...
How-to new Git repo. @ eclipse.org? GitHub PRs
how?
Or simply a GitHub Org?? (Soft on IP CLA?)
I’m willing to kick something off… Is there anyone
here who would be willing to contribute to this effort?
External Annotations repository #helpwanted
Eclipse.org Bug 449653 and Bug 457792

25. Some Experiences
&
Wrapping Up

26. EclipseCon NullPointerException26
How-to move existing product to be NullPointerException free?
1. Set up your product’s org.eclipse.jdt.core.prefs & IDE Setup infra etc.
It has to “just work” (OOB)… for Maven-based projects, see prez. part 1! ;)
2. Iterate submitting small incremental changes, project by project, src/main
before src/test; @NonNullByDefault package by package:
Add package-info.java => lots red, adapt, some @Nullable, etc.
3. Contribute to (my) shared EEA library manually as required as you go
4. If push back, can start with IDE only, enforcement in build later
(but do target build enforcement, else quickly regressions from others)
Process (ODL: 60+ .git * 10s .project * 10s .java)

27. EclipseCon NullPointerException27
1. FindBugs: Oldest, but (supposedly) more limited null analysis.
Still great for other static code analysis (just being ext. intro. in ODL)
2. Checker Framework: Interesting… actually. Extensive external
annotations for JDK (*.jaif); IDE integration (TBC; must have IMHO)
3. JetBrains.com IntelliJ IDEA: Some inference by bytecode analysis,
also external annotations (XML); but no Java 8 generics, and how-to
in build?
Alternatives?

28. EclipseCon NullPointerException
● org.eclipse.jdt.annotation
■ Android’s, Lombok’s…
● javax.annotation
from com.google.code.findbugs:annotation (NOT :jsr305 BTW...)
● org.jetbrains:annotations
■ org.checkerframework
● com.sun.istack.internal (rt.jar)
28
They’re NOT all the same… JDT’s & Checker’s are null type annotations (Java 8 @Target
TYPE_USE), for generics! FB/JSR-305 is NOT, others IDK. The semantics differ, too; FB’s
@Nullable :( You could also create your own. But simplest to get started is to just use JDT’s.
Yeah, there are several “competing” annotations for (non) nullability…
Accept it, pick one as primary for your project with your tool, move on! ;)
@ which @Nullable & @NonNull (NotNull)

29. EclipseCon NullPointerException29
Gotcha! (Quiz Time)
@Nullable File file() {
return null;
}
@Nullable java.io.File file() {
return null;
}
java.io.@Nullable File file() {
return null;
}

30. EclipseCon NullPointerException30
Gotcha! (Quiz Time)
@Nullable File file() {
return null;
}
@javax.annotation.Nullable java.io.File file() {
return null;
}
java.io.@org.eclipse.jdt.annotation.Nullable File file() {
return null;
}
https://bugs.eclipse.org/bugs/show_bug.cgi?id=506376

31. EclipseCon NullPointerException31
FindBugs annotations interop?
● Configure javax.annotation.Nullable & NonNull
(used e.g. in Google Guava! and ODL...) as secondary
annotations in “Annotations for Null Specifications”
preference intro. Since Neon (4.6) M5, save EEA. I’m still
gaining experience with InterOp… diff. only evaluated when reading class files external
to the project? That’s inconsistent… see Hello.java prob with fileMayBeNull2.
● Use Eclipse’s null analysis instead of FindBugs’, in both
build and IDE (use FB for other code analysis; exclude)

32. EclipseCon NullPointerException32
As of 2016.10.12:
441 closed VS 135 open bugs in CDT with “[null] …” in summary.
(As usual: Not all 135 are true “bugs”; some are unclear, some lingering old “discussions”.)
Sense of “not ideal solution, we had to compromise” in some areas (e.g. Bug #505828)
Filed / found during evaluation:
1. Bug #443146 Code Completion unnecessary @NonNull when @NonNullByDefault
2. Bug #506375 Allow NonNullByDefault in external library annotation
3. Bug #505828 Inherit stronger than @NonNullByDefault
4. Bug #500885 NullPointerException at jdt.internal.compiler.lookup.BinaryTypeBinding
5. Bug #489900 annotation 2.1.0 in Maven central
Status & Bugs

33. EclipseCon NullPointerException33
Annotations & @Override (with EEA…)
@Override
public boolean equals(Object obj)
Illegal redefinition of parameter obj,
inherited method from Object does not
constrain this parameter
@Override
public boolean equals(@Nullable Object obj)
Can be worked around by adding @Nullable - at every use! (Bug #505828 …)

34. EclipseCon NullPointerException34
● ODL: (0) APPLY.. (1) OMG build on ecj… TBC! ;-) (2)
@eclipse & @javax & (3) IntelliJ cohabitation? #me
● EEA: Repo with more contribs, releasing #me&you?
Maybe conversion tool of Checker Framework JDK (*.jaif)
metadata to EEA ?? Gen. EEA by automatic analysis?
● Plug-in Dependencies CP container? Gradle?? #you?
● Try it all out, contribute eea, provide feedback! #you
Future Ideas (YOU? #helpwanted)

35. Q & A
and YOUR feedback / experiences?

37. THANK YOU
plus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNews

38. EXTRA SLIDES

39. EclipseCon NullPointerException
@NonNullByDefault({}) // deactivate it :-(
public class Bean {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
39
Zoom: Java Beans - null by default?

40. EclipseCon NullPointerException40
Zoom: Java Beans - Collection properties
public class Bean {
private final List<String> names = new ArrayList<>();
public List<String> getNames() {
return names;
}
// no setNames()
}

41. EclipseCon NullPointerException
public class Bean {
private Optional<String> name =
Optional.empty();
public Optional<String> getName() {
return name;
}
public void setName(String name) {
this.name = Optional.of(name);
}
// void unsetName() ?
}
41
Zoom: Java Beans - Optional<> properties
public class Bean {
private @Nullable String name;
public Optional<String> getName() {
return Optional.ofNullable(name);
}
public void setName(String name) {
this.name = name;
}
// void unsetName() ?
}

42. EclipseCon NullPointerException42
Zoom: Java Beans - mandatory properties
Traditionally we don’t think about mandatory (required) bean properties.. if (some) clearly
always are (beware..), use final and constructor or Builder pattern which throws Exception.
public class Bean {
private final String name;
public Bean(String name) {
this.name = name;
}
public String getName() {
return name;
}
}
public class Bean {
private final String name;
public static Builder builder() { …
public static class Builder {
public Bean build() { …
}

43. EclipseCon NullPointerException43
Zoom: Java Beans - mandatory properties (II)
If you would like static compile-time checking of mandatory bean properties instead of
(checked or unchecked) exceptions at run-time from the Builder, you can… It’s a PITA to
hand-write, but ideal for code generators! Recommendation: Use http://Immutables.org
For an example Builder of a Bean with some mandatory and some optional properties which
enforces setting all required at compile time, in fixed order, by returning a particular (inner
static) Build class for each (!) mandatory property, AKA “staged Builder”, follow
https://github.com/google/auto/issues/370 (originally in Person.java in
vorburger/eclipse-null-eea-augments)

44. EclipseCon NullPointerException44
Zoom: Fields
TODO huh, seriously?! WTF. This is a significant obstacle to migrating existing code! Options?
public int hashCode() {
(...) + (field == null ? 0 : field.hashCode());
Potential null pointer access: this expression has a '@Nullable' type
Quick Fix “Extract to local variable (all occurrences)”
public int hashCode() {
Type field2 = field;
(...) + (field2 == null ? 0 : field2.hashCode());
PS: For hashCode() in particular, use Objects.hash(...) - but problem in general remains.

45. EclipseCon NullPointerException45
Zoom: Beans’ Fields: hashCode() & equals()
In Beans’ hashCode() and equals() implementations, instead of introducing field indirection,
use Java 7’s Objects.hash(...) and equalsHelper (* by yours truly). Or, again, consider using
some bean code generator, such as e.g. Google’s AutoValue or http://immutables.github.io,
etc. (BTW https://github.com/google/guava/issues/2521)
public int hashCode() {
Type field2 = field;
(...) + (field2 == null ? 0 : field2.hashCode());

46. EclipseCon NullPointerException46
● https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-computer-science/
● https://wiki.eclipse.org/JDT_Core/Null_Analysis/Options
● ...
Recommended reading