With the recent open-sourcing of JDK Mission Control, there is even more interest in Java Mission Control and Java Flight Recorder than ever. A lot has changed, but these tools still remain on the forefront of debugging and troubleshooting Java issues, both during development and in production. This session covers these recent changes, migrating from earlier versions, and what the JDK Mission Control community’s plans are for the future.
4. JVM Sustaining Engineer
OpenJDK 8 Update Project
Maintainer
JavaOne Rock Star
Co-author of Oracle WebLogic
Server 11g 構築・運用ガイド
@DavidBuckJP
https://blogs.oracle.com/buck/
Who am I? David Buck (left)
6. Introduction to JDK Mission Control and JDK Flight Recorder [DEV2316]
Marcus Hirt and Klara Ward
Monday 16:00 Moscone South Room 202
Improving Observability in Your Application with JFR and JMC [DEV3460]
Marcus Hirt and Mario Torre
Tuesday 11:30 Moscone South Room 201
Java Flight Recorder: Black Box of Java Applications[DEV3957]
Poonam Parhar
Wednesday 12:30 Moscone South Room 203
Robotics on JDK 11? With Modules? Are You… [DEV2329]
Marcus Hirt, Miro Wengner, and Robert Savage
Wednesday 16:00 Moscone South Room 313
Four Productive Ways to Use Open Source JFR and JMC Revisited [DEV3118]
Sven Reimers and Martin Klähn
Thursday 11:15 Moscone South Room 304
Enhanced Java Flight Recorder at Alibaba [DEV3667]
Sanhong Li, Fangxi Yin, and Guangyu Zhu
Thursday 12:15 Moscone South Room 203
Performance Monitoring with Java Flight Recorder on OpenJDK [DEV2406]
Hirofumi Iwasaki and Hiroaki Nakada
Thursday 13:15 Moscone South Room 201
7. Introduction to JDK Mission Control and JDK Flight Recorder [DEV2316]
Marcus Hirt and Klara Ward
Monday 16:00 Moscone South Room 202
Improving Observability in Your Application with JFR and JMC [DEV3460]
Marcus Hirt and Mario Torre
Tuesday 11:30 Moscone South Room 201
Java Flight Recorder: Black Box of Java Applications[DEV3957]
Poonam Parhar
Wednesday 12:30 Moscone South Room 203
Robotics on JDK 11? With Modules? Are You… [DEV2329]
Marcus Hirt, Miro Wengner, and Robert Savage
Wednesday 16:00 Moscone South Room 313
Four Productive Ways to Use Open Source JFR and JMC Revisited [DEV3118]
Sven Reimers and Martin Klähn
Thursday 11:15 Moscone South Room 304
Enhanced Java Flight Recorder at Alibaba [DEV3667]
Sanhong Li, Fangxi Yin, and Guangyu Zhu
Thursday 12:15 Moscone South Room 203
Performance Monitoring with Java Flight Recorder on OpenJDK [DEV2406]
Hirofumi Iwasaki and Hiroaki Nakada
Thursday 13:15 Moscone South Room 201
9. Several Closely Related Technologies
(JRockit / Java / JDK) Mission Control
Flight Recording
Oracle JDK: JDK 7
OpenJDK: JDK 11
JEP 167: Event-Based JVM Tracing (JDK 7)
JEP 328: Flight Recorder (JDK 11)
10. Flight Recorder
Technology to generate and persist JVM event data
Low-overhead
Intended for use in production
Viable for “always on” use-case
11. Mission Control
Eclipse-based (RCP) GUI tool
Visualization of Flight Recording data
Automated Flight Recording analysis
JMX console for real-time monitoring (and control) of target
Plug-in framework for additional functionality
Support for integration into Eclipse IDE
12. Mission Control
Eclipse-based (RCP) GUI tool
Visualization of Flight Recording data
Automated Flight Recording analysis
JMX console for real-time monitoring (and control) of target
Plug-in framework for additional functionality
Support for integration into Eclipse IDE
16. Value Add
JRockit was originally sold as a “Virtual Machine for Java”
Sun required each Java licensee to have a “value add”
“Superior performance” didn’t go over well...
JRockit’s monitoring and management tools were presented as its
value add to differentiate itself from Sun’s offerings
29. A Combined Legacy
JRockit’s best features ported over to HotSpot
Diagnostic commands (JDK 7)
Compiler control (JDK 9)
Native memory tracking (JDK 7)
Flight Recorder (JDK 7)
Build system (behind the scenes) (JDK 8)
JRockit Mission Control -> Java Mission Control
31. JRockit Runtime Analyzer
Based on experience gained while consulting
“Cast a big net” to avoid multiple support iterations
Most data collection piggy-backed onto internal JVM operations
(low overhead)
Data persisted at zipped XML
34. JRockit Mission Control 4
Post Oracle acquisition (JRockit R28)
Flight Recorder replaces JRockit Runtime Analyzer
XML data -> binary data
Much lower overhead to collect and persist data
39. JDK Mission Control 7
Open source
No longer bundled with JDK (as of JDK 11)
Flight Recorder implementation open sourced (as of JDK 11)
40. Universal Permissive License
Clear patent protection
Clear & simplified relicensing
Reduced overhead in source files
It can be used as a contributor agreement
See FAQ (link at end of slide deck) for details
41. Universal Permissive License
Clear patent protection
Clear & simplified relicensing
Reduced overhead in source files
It can be used as a contributor agreement
See FAQ (link at end of slide deck) for details
42. Oracle Contributor Agreement
Must be signed and sent to Oracle to contribute code to Mission
Control
Same requirement as other OpenJDK projects
Only needs to be done once
43. “...you're doing an awesome job in building a real community and a
*very* open source project.“
- Red Hat Contributor to Marcus Hirt
jmc-dev alias, November 8, 2018
44. Compatibility
Flight Recording format changes between JDK versions
Mission Control has always been backwards compatible with flight
recordings from older HotSpot releases.
JDK 11 requires JMC 7
JMC 7 is currently the only version that supports all HotSpot based
flight recordings
48. GitHub Mirror
Only mainline development branch
Not in full sync at the moment
Recommend using hg until project Skara migration *
* Maybe...
49. A Tale of Two Shells...
Shell 1
cd releng/third-party
mvn p2:site
mvn jetty:run
Shell 2
cd core
mvn install
cd ..
mvn clean package
50. A Tale of Two Shells...
Equinox p2: provisioning platform for OSGi artifacts and meta-data
Shell 1 sets up a p2 repository that hosts dependencies needed by
the build process run in shell 2
51. Cross Compilation?!
Regardless of build platform, binaries for all three target platforms
are generated
Platform-dependent bits are downloaded as dependencies
52. Building
Quick and easy to do
Builds virtually out of the box on most Linux distros
Build platform does not need to match target platform
54. Safepoints
Stop-the-World implementation
Guarantees Java state of thread is known
All threads executing Java code must stop
JNI code can continue
Cooperative suspension
Polling based
By YPLeroux at English Wikipedia, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=17734859
55. Safepoints
Stop-the-World implementation
Guarantees Java state of thread is known
All threads executing Java code must stop
JNI code can continue
Cooperative suspension
Polling based
By YPLeroux at English Wikipedia, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=17734859
56. Safepoint Polling
Interpreter can stop instantly
template dispatch table replaced
“polling” is essentially free
Explicit poll inserted into C1/C2 JIT output
Global safepoint polling page in address space
Poll implemented by attempted read of this address
Safepoint triggered by change in page permissions
57. JIT Safepoint Polling
On return from method call
Backend branches in most loops
Any transition to native code (JVM / JNI)
58. Collatz Conjecture
Simple rules:
If number is even, divide by 2
If number is odd, multiply by 3 and add 1
Conjecture: Start with any positive integer, and you’ll eventually
end up at 1
An open problem in math
63. Let’s test the Collatz Conjecture...
the stupidest way imaginable!
64. Testing the Collatz Conjecture
private static long next(long num) {
return (num%2 == 0) ? num / 2 : 3 * num + 1;
}
65. Testing the Collatz Conjecture
public static void main(String[] args) {
long num;
for (int i=2; i < Integer.MAX_VALUE; i++) {
num = i;
for (int j=0; j < Integer.MAX_VALUE; j++) {
num = next(num);
if (num == 1) break;
}
if (num != 1) {
System.out.println(i + " did not end at 1.");
System.exit(-1);
} else if (i % 1000000 == 0) {
System.out.println("done with " + i);
}
}
}
66. Counted Loop
Loop that increments / decrements an int index
Worst case scenario, we loop 4 billion times (very short)
HotSpot may not bother adding a safepoint pool to the loop
67. Testing the Collatz Conjecture
public static void main(String[] args) {
long num;
for (int i=2; i < Integer.MAX_VALUE; i++) {
num = i;
for (int j=0; j < Integer.MAX_VALUE; j++) {
num = next(num);
if (num == 1) break;
}
if (num != 1) {
System.out.println(i + " did not end at 1.");
System.exit(-1);
} else if (i % 1000000 == 0) {
System.out.println("done with " + i);
}
}
}
68. Testing the Collatz Conjecture
public static void main(String[] args) {
long num;
for (int i=2; i < Integer.MAX_VALUE; i++) {
num = i;
for (int j=0; j < Integer.MAX_VALUE; j++) {
num = next(num);
if (num == 1) break;
}
if (num != 1) {
System.out.println(i + " did not end at 1.");
System.exit(-1);
} else if (i % 1000000 == 0) {
System.out.println("done with " + i);
}
}
}
69. Profiler Types
Sampling Profiler
Process sampled periodically
Invocation count is not exact
Observer effect should be
negligible
Invocation Counter Profiler
Code is instrumented
Exact number of invocations
Observer effect often distorts
results
70. Profiler Types
Sampling Profiler
Process sampled periodically
Invocation count is not exact
Observer effect should be
negligible
Invocation Counter Profiler
Code is instrumented
Exact number of invocations
Observer effect often distorts
results
71. Traditional Sampler
JVMTI / Bytecode instrumentation can only stop thread at a
safepoint poll
What happens between safpoint polls is invisible to the profiler
74. Flight Recording
Can asynchronously stop execution thread at any time
No dependence on safe points for profiling
HotSpot internal API (AsyncGetCallTrace)
Honest Profiler
Oracle Developer Studio
76. What happened?
Safepoints have corresponding metadata
Byte Code Index (bci)
Liveness data (what stack positions point to Java heap)
77. What happened?
By default, metadata is normally only generated for safepoints
DebugNonSafepoints flag forces metadata generation for all code
(obvious performance cost)
80. Takeaways
Open sourcing of JMC has been a great success
JMC 7 and higher are the ideal MC to use regardless of JDK version
Binary builds from Oracle should be available in the near future
JMC is simple to build on your own today
Flight Recording’s AsyncGetCallTrace-based profiling can provide a
level of accuracy unachievable by most other tools.
