Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564]

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Everything You Wanted to Know About JIT
Compilation but Were Afraid to Ask
[CON3564]
David Buck
Principal Member of Technical Staff
Java Platform Group
October 3rd, 2017
Confidential – Oracle Internal/Restricted/Highly Restricted 3

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

• JVM Sustaining Engineer
• OpenJDK 8 Update Project
Maintainer
• JavaOne Rock Star
• Co-author of Oracle WebLogic
Server 11g 構築・運用ガイド
• @DavidBuckJP
• https://blogs.oracle.com/buck/
Who am I?

Who are you?
• Just about any Java programmer who uses HotSpot

What?
• Crash course on HotSpot's JIT compilation

Why?
• Focus on what matters (no premature optimization)
• Better tuning / troubleshooting
• Fun

Why Now?
• Still an area of frequent development
– Tiered compilation
– Interaction with new features
• Still an area of frequent misunderstanding / confusion / interest
– Premature optimization

Program Agenda
Introduction
Implementation
Monitoring
Control
1
2
3
4

Introduction

What is JIT?
Just-in-Time compilation

Classic JIT
• All methods or functions
compiled immediately before use
• No interpreter
• Example: JRockit

Classic JIT Example: JRockit

JIT is very old
• LISP (John McCarthy, 1960)
• QED text editor regular expressions (Ken Thompson, 1968)
• Smalltalk (1983)
• Self (1990s?)

JIT for Java is very old
• Microsoft and Symantec had their own JIT compilers for browsers
• Symantec deal with Sun (project java activator)
• JIT added to "Classic" JVM in JDK 1.2
• HotSpot available as an optional addition in 1.2
• HotSpot shipped with 1.3

Static Compilation
• Pros:
– No runtime overhead
• Cons:
– Portability issues
– No ability to adapt
• To hardware
• To application behavior
Interpreter
• Pros:
– Portable
– Adaptable
• Cons:
– Interpretation overhead
Static Compilation vs. Interpreter

Classic JIT compared to both compilation / interpreter
• Pros:
– No interpretation runtime overhead
– Portable
– Adaptable
• Cons:
– Runtime JIT compilation overhead
– Slower startup time

Static Initializers
The poster boy for wasteful JIT overhead
static {
// important class-level
// initialization goes here!
}

HotSpot JIT Compilation
• Only JIT compile hot stuff (hotspots)
• Interpret the rest

Why is this important?
• Don’t JIT stuff that is not hot
• No need to wait for a JIT to finish (asynchronous JIT)

HotSpot JIT compared to Classic JIT
• Pros
– Portable
– Adaptable
– Fast (startup and peak)
• Cons
– Complicated to implement

A Note on Nomenclature…
• HotSpot == Runtime Profiling used to identify candidates for JIT
compilation
• HotSpot == JVM
– Profiling technique
– JIT policy
• HotSpot does use JIT

Implementation

JIT Compiler Design Goals
• Lightweight
– Fast startup
– Small memory footprint
– Low compilation overhead
• Generate high-performance code
– Many complex optimizations
– Based on heavy profiling of code
– Output peak performance often bests C/C++

Client
• Short-lived processes
• Run on client machines with low
resources (desktops / laptops)
Server
(Application servers,
Batch processing)
• Long-lived processes
• Run on dedicated machines with
plentiful resources
Two Use Cases

Client Compiler (C1):
• Only simple optimizations
• Quick compilation
• Low memory requirements
• Not dependent on profiling data
• Reasonable peak performance
Server Compiler (C2):
• Highly optimizing
• Slow compilation
• Heavy memory usage
• Dependent on profiling data
• Great peak performance
Two JIT Compilers

Profiling
Interpreter collets profiling information, basically free
• Invocation counter
• Backwards jumps
• Invokevirtual targets
• Branch prediction

Client JVM
• Only C1 used
• Many tuning defaults / ergonomics
/ heuristics tuned for client usage
• Reaches peak performance quickly
Server JVM
• Traditionally only C2 used (more to
come)
/ heuristics tuned for server usage
• May take a while to reach peak
performance
• The only JVM to support 64-bit
A Tail of Two JVMs

Java Launcher
bin/java
jre/lib/<arch>/server/libjvm.so
jre/lib/<arch>/client/libjvm.so

Client JVM
• Only C1 used
/ heuristics tuned for client usage
• Reaches peak performance quickly
Server JVM
• Traditionally only C2 used (more to
come)
/ heuristics tuned for server usage
• May take a while to reach peak
performance
• The only JVM to support 64-bit
A Tail of Two JVMs

class A {
B b;
public void foo() {
y = b.get();
...do stuff...
z = b.get();
sum = y + z;
}
}
class B {
int value;
final int get() {
return value;
}
}
Optimization Example

class A {
B b;
public void foo() {
y = b.get();
...do stuff...
z = b.get();
sum = y + z;
}
}
class B {
int value;
final int get() {
return value;
}
}

class A {
B b;
public void foo() {
y = b.value;
...do stuff...
z = b.value;
sum = y + z;
}
}
class B {
int value;
final int get() {
return value;
}
}

class A {
B b;
public void foo() {
y = b.value;
...do stuff...
z = y;
sum = y + z;
}
}
class B {
int value;
final int get() {
return value;
}
}

class A {
B b;
public void foo() {
y = b.value;
...do stuff...
y = y;
sum = y + y;
}
}
class B {
int value;
final int get() {
return value;
}
}

class A {
B b;
public void foo() {
y = b.value;
...do stuff...
sum = y + y;
}
}
class B {
int value;
final int get() {
return value;
}
}

class A {
B b;
public void foo() {
y = b.get();
...do stuff...
z = b.get();
sum = y + z;
}
}
class A {
B b;
public void foo() {
y = b.value;
...do stuff...
sum = y + y;
}
}
Optimization Example (Before and After)

Tiered Compilation
• Interpretation -> C1 -> C2
(very simplified version)
• Quick warmup of C1, but
with the higher peak
performance of C2

Tiered Compilation
• Existed since early JDK 6 releases (not officially supported)
• Officially supported in JDK 7 (use not recommended)
• Enabled by default from JDK 8

Tiered Compilation
• Level 0 - interpreter
• Level 1 - C1 with full optimization (no profiling)
• Level 2 - C1 with invocation and backedge counters
• Level 3 - C1 with full profiling (level 2 + MDO)
• Level 4 - C2

Code Cache
• Contains:
– JITed code
– JNI wrapper / stubs
– HotSpot template interpreter
• Cleaned up by the “sweeper thread”
• Compiler disabled if you run out of space

Monitoring

public static void main(String[] args) {
long i = 0;
while (true) {
i = addStuff(i, 1);
if (i % Long.MAX_VALUE == 0)
System.out.println(i);
}
}
private static long addStuff(long a, long b) {
return a+b;
}
A Rather Contrived Example…

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 50
$ java -XX:+PrintCompilation HotMethodDemo
156 24 3 HotMethodDemo::addStuff (4 bytes)
161 24 3 HotMethodDemo::addStuff (4 bytes) made not entrant
Some Visibility

Timestamp Compilation ID Compilation Tier Method Size Other Information
Some Visibility

100 1 3 java.lang.String::hashCode (55 bytes)
101 2 3 java.lang.String::equals (81 bytes)
104 3 3 java.lang.String::indexOf (70 bytes)
105 4 3 java.lang.String::charAt (29 bytes)
106 5 3 java.lang.String::length (6 bytes)
107 6 3 java.lang.Object::<init> (1 bytes)
111 7 n 0 java.lang.System::arraycopy (native) (static)
113 8 3 java.lang.Math::min (11 bytes)
113 9 3 java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
114 10 1 java.lang.Object::<init> (1 bytes)
114 6 3 java.lang.Object::<init> (1 bytes) made not entrant
115 11 1 java.lang.ref.Reference::get (5 bytes)
118 12 3 java.util.HashMap::hash (20 bytes)
118 13 3 java.util.Arrays::copyOfRange (63 bytes)
120 14 3 java.util.HashMap::getNode (148 bytes)
121 15 4 java.lang.String::equals (81 bytes)
124 16 3 java.util.LinkedList::indexOf (73 bytes)
125 17 3 java.util.HashMap::get (23 bytes)
133 18 ! 3 java.io.BufferedReader::readLine (304 bytes)
148 19 3 sun.misc.JarIndex::addToList (59 bytes)
149 20 3 java.util.LinkedList::add (7 bytes)
150 2 3 java.lang.String::equals (81 bytes) made not entrant
150 21 3 java.lang.String::<init> (82 bytes)
150 22 1 java.lang.ThreadLocal::access$400 (5 bytes)
151 23 3 java.lang.String::getChars (62 bytes)
163 26 % 3 HotMethodDemo::main @ 2 (28 bytes)
168 27 3 HotMethodDemo::main (28 bytes)
Some Visibility

Compilation Flags
b Blocking compile
% On stack replacement
! Method has exception handlers
s Method declared as synchronized
n Method declared as native

Compilation Flags
b Blocking compile

Blocking Compilation
• Methods are compiled asynchronously by default
• Can force compilation to block with -Xbatch

Compilation Flags
b Blocking compile

long i = 0;
while (true) {
i = addStuff(i, 1);
}
}
return a+b;
}

On Stack Replacement
• JIT compile a method while it is being run
• ARs for Interpreter and JIT frames look very different
– Must convert interpreter frame to JIT frame
• JIT compiled method entry is in the middle of method

Some More Visibility

private static long addStuff(long, long);
descriptor: (JJ)J
flags: ACC_PRIVATE, ACC_STATIC
Code:
stack=4, locals=4, args_size=2
0: lload_0
1: lload_2
2: ladd
3: lreturn
Bytecode - addStuff

public static void main(java.lang.String[]);
descriptor: ([Ljava/lang/String;)V
flags: ACC_PUBLIC, ACC_STATIC
Code:
stack=4, locals=3, args_size=1
0: lconst_0
1: lstore_1
2: lload_1
3: lconst_1
4: invokestatic #2 // Method addStuff:(JJ)J
7: lstore_1
8: lload_1
9: ldc2_w #4 // long 9223372036854775807l
12: lrem
13: lconst_0
14: lcmp
15: ifne 2
18: getstatic #6 // Field
java/lang/System.out:Ljava/io/PrintStream;
21: lload_1
22: invokevirtual #7 // Method java/io/PrintStream.println:(J)V
25: goto 2
Bytecode - main

Some More Visibility

$ java -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining HotMethodDemo
@ 4 HotMethodDemo::addStuff (4 bytes)
@ 22 java/io/PrintStream::println (not loaded) not inlineable
@ 4 HotMethodDemo::addStuff (4 bytes)
@ 22 java/io/PrintStream::println (not loaded) not inlineable
@ 4 HotMethodDemo::addStuff (4 bytes) inline (hot)
227 23 % 3 HotMethodDemo::main @ -2 (28 bytes) made not entrant
A Lot More Visibility

Control

A Word of Warning…
• Performance tuning JIT compilation usually a waste of time
• Being able to troubleshoot / workaround bugs may be useful

Most Important Options (All or Nothing)
• -Xint
– Interpreter only (disables all JIT)
– Good for trying to simplify test cases
• -Xcomp
– JIT only (disables interpreter)
– Good for trying to reproduce JIT issues / behavior

Most Important Options (JVM Selection)
• -client
– Only available on 32-bit
– C1 compiler only
– Default compile threshold: 1,500
• -server
– Only option for 64-bit JVMs
– C2 compiler (and C1 w/ Tiered Compilation)
– Default compile threshold: 10,000

Most Important Options (JIT Timing)
• XX:CompileThreshold
– Number of invocations before method is considered “hot”
– Default varies based on JVM (client / server)
• Xbatch
– Disables asynchronous compilation
– Makes compilation much more deterministic

Most Important Options (Tiered Compilation)
• -XX:-TieredCompilation
– Effectively disables C1 on server JVMs
• -XX:TieredStopAtLevel
– TieredStopAtLevel=1 can be used to get “client like” behavior

Most Important Options (Fine-grained Control)
• XX:CompileCommand
– help (does not list all commands)
– option
– log
– compileonly
– exclude
– print
– inline
– dontinline

Compiler Control
• New in JDK 9 (JEP 165)
• Inspired by JRockit
• JSON-based configuration file
• Can be specified with XX:CompilerDirectivesFile

Compiler Control: Example 1
{ // start of another directives block
// match ant method whose class end with 'Concurrent'
match: ["*Concurrent.*"],
c2: {
// disable compilation
Exclude:true,
}
// with the c1 directive unspecified the options remains default.
}

Compiler Control: Example 2
{
match: ["java*.*", "oracle*.*"],
c1: {
// A bool option. Extra trailing comma should not cause a parse error
PrintAssembly:true,
},
c2: {
// force inline patters prepended with +, prevent with -
inline: ["+vm*.*","-*.*" ]
},
},

Compiler Control: diagnostic commands
• jcmd <pid> Compiler.add_directives <file>
Add additional directives from the file. The new directives will be added on top of the old, with the first directive in the
file ending up on the top of the directives stack.
• jcmd <pid> Compiler.list_directives
List all directives on the directives stack from top to bottom.
• jcmd <pid> Compiler.clear_directives
Clear the directives stack
• jcmd <pid> Compiler.remove_directives
Remove the top element from the directives stack

Conclusions

• JIT Compilation can do optimizations static compilers can not do
• You should avoid premature optimization
• JIT (especially C2) behavior can be very non-deterministic

Thank You!

• The Java HotSpot Performance Engine Architecture
(A little out-of-date, but still a good primer)
http://www.oracle.com/technetwork/java/whitepaper-
135217.html
• PrintAssembly (HSDIS)
https://wiki.openjdk.java.net/display/HotSpot/PrintAsse
mbly
• It's All Relative - CompileCommand JVM option
http://jpbempel.blogspot.com/2016/03/compilecomman
d-jvm-option.html
• Kris Mok's PrintCompilation Notes
https://gist.github.com/rednaxelafx/1165804#file_notes.
md
• Java Performance, Hunt & John
https://www.pearson.com/us/higher-
education/program/Hunt-Java-
Performance/PGM182574.html
• Java Performance Companion, Hunt & Beckwith &
Parhar & Rutisson
https://www.pearson.com/us/higher-
education/program/Hunt-Java-Performance-
Companion/PGM168439.html
Resources

Stay connected
• Join us: DevOps Corner (Developer Lounge – Moscone West)
• Learn more: openjdk.java.net | wercker.com/java
• Follow: @OpenJDK, @wercker #JavaOne #DevOps
80

Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564]

Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564]

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564]

Similar to Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564] (20)

More from David Buck

More from David Buck (20)

Recently uploaded

Recently uploaded (20)

Everything You Wanted to Know About JIT Compilation but Were Afraid to Ask [JavaOne 2017 CON3564]