Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Bytecode Verification
The Hero That Java Needs
David Buck
Principal Member of Technical Staff
Java SE Sustaining Engineering
September, 2016

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
4

About Me
David Buck
• Java SE Sustaining Engineering
• Mostly JRockit fixes
• OpenJDK 8 Updates
Project Maintainer
• Hobbies: Programming
5

Program Agenda
Introduction
Dangers
Demo
Implementation
Importance
Usage
Conclusions
1
2
3
4
5
6
6
7

Introduction
7

What Is It?
• Analysis of bytecode
• Syntax check
• Symantec check
• Ensures stability / security of runtime
8

When does it happen?
• Analysis done during class loading
• Sometimes delayed until right before method execution
• But only done at most once per loaded method
9

Traditional Interpreted Language
Source Code Interpreter
10

Traditional Interpreted Language
Source Code Interpreter
11

Traditional Compiled Language
Source Code ExecutableCompile
12

Traditional Compiled Language
Source Code ExecutableCompile
13

Java
Source Code BytecodeCompile
14
JVM

Java
15
JVM

Java
16
JVM

Java
17
JVM

What Does It Do?
18

What Does It Do?
• Protects runtime from bad people
"Why the verifier is so important…. write once and crack anywhere“
-Keith McGuigan
19

What Does It Do?
• Protects runtime from bad people
"Why the verifier is so important…. write once and crack anywhere“
-Keith McGuigan
• Protects runtime from you
20

Why learn about it?
• The best technologies are invisible…
• Victim of its own success
21

Dangers
22

Class Metadata
23

Class Metadata
• Has a direct superclass
24

Class Metadata
• Superclass is not marked final
25

Class Metadata
• No final methods are overridden
26

Class Metadata
• No final methods are overridden
27

Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
28

0: iconst_0
1: iconst_1
2: iconst_2
29

0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
30

0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
31

0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
2
32

0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
2
33

Operand Stack Underflow
0: iadd
1: iadd
2: iadd
0
LIMIT
1
2
START
34

0: iadd
1: iadd
2: iadd
0
LIMIT
3
START
35

0: iadd
1: iadd
2: iadd
3
LIMIT
START
36

0: iadd
1: iadd
2: iadd
?
LIMIT
START
37

0: iadd
1: iadd
2: iadd
?
LIMIT
START
38

Type Checking
• Each operation is checked
– Correct types on the stack
– Correct types in local variable “slots”
• Specification uses Prolog to define requirements
39

Prolog?!
• Predicate logic of type system are described by Prolog well
• Java is probably the first of this kind of use by a mainstream programming
language
40

Prolog?!
Facts:
cat(tom).
41

Prolog?!
Facts:
parent_child(sally, bob).
42

Prolog?!
Rules:
Head :- Body.
43

Prolog?!
Rules:
sibling(X, Y) :- parent_child(Z, X),
parent_child(Z, Y).
44

The Specification
45

Java Bytecode
Expressive Power
Java Language
46

Demo
47

ClassA
public class ClassA {
public int doSomething(int i1, int i2, int i3)
{
return i1+i2+i3;
}
}
48

ClassB
public class ClassB {
public Integer doSomethingElse(int i1, int i2, int i3)
{
return new Integer(i1+i2+i3);
}
}
49

ClassC
public class ClassC extends ClassA {}
50

Demo
public class Demo {
public static void main(String[] args) {
ClassA obj = new ClassC();
System.out.println(obj.doSomething(1,2,3));
}
}
51

Object
ClassA ClassB
ClassC
Demo
52

It works…
$ java Demo
6
$
53

Lets do something bad…
public class ClassC extends ClassB {}
54

Lets do something bad…
public class ClassC extends ClassB {}
55

Object
ClassA ClassB
ClassC
Demo
56

Object
ClassA ClassB
ClassC
Demo
57

Object
ClassA ClassB
ClassC
Demo
58

Demo
public class Demo {
public static void main(String[] args) {
ClassA obj = new ClassC();
System.out.println(obj.doSomething(1,2,3));
}
}
59

$ java Demo
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
Demo.main([Ljava/lang/String;)V @15: invokevirtual
Reason:
Type 'ClassC' (current frame, stack[1]) is not assignable to 'ClassA'
Current Frame:
bci: @15
flags: { }
locals: { '[Ljava/lang/String;', 'ClassC' }
stack: { 'java/io/PrintStream', 'ClassC', integer, integer, integer }
Bytecode:
0x0000000: bb00 0259 b700 034c b200 042b 0405 06b6
0x0000010: 0005 b600 06b1
60

As expected, the verifier protects us from ourselves.
61

As expected, the verifier protects us from ourselves.
What if we disable it…
62

We reap what we sow…
$ java -Xverify:none Demo
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fa93be7991c, pid=22925, tid=140364857087744
#
# JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 1.8.0_91-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed
oops)
# Problematic frame:
# V [libjvm.so+0x46391c]
63

Demo Takeaways
• No obvious evidence that bad bytecode was root cause of crash
• A class is only valid in the context of previously loaded classes
• No malicious intent / 3rd party tools used
64

Optional 2nd Demo
$ java -Xverify:none Crack
=============== DEBUG MESSAGE: illegal bytecode sequence -
method not verified ================
Exception in thread "Thread-0"
java.lang.NullPointerException
at Pointer.deref(Pointer.jasm)
at Crack.breakLock(Crack.java:13)
at Crack$1.run(Crack.java:29)
Thread Thread[main,5,main] leaving monitor
$
65

Implementation
66

Type Inference Verifier
– AKA the Old Verifier
Type Checking Verifier
– AKA Split Verifier
– AKA The New Hotness
67
A Tale of Two Verifiers…

– AKA the Old Verifier
Type Checking Verifier
– AKA Split Verifier
– AKA The New Hotness
68
A Tale of Two Verifiers…

• Class files <= 49 (JDK 5)
• Requires CFG construction
• Worst case scenario can require
many passes
Diagram by JMP EAX - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=34222288
69

When We Do Syntactic / Semantic checking
70
JVM

Type Checking Verifier (AKA Split Verifier)
• Class files >= 50 (JDK 6)
• Depends on StackMapTable Attribute
• Transfers much of the responsibility to javac
Source Code BytecodeCompile JVM
StackMap
Tables
71

StackMapTable
• Identifies the type of each stack position / local variable
• One needed for every instruction that is the target of a jump
– Methods without branches will not have them
• Are stored as deltas to save space
• Allow single pass verification
72

Importance
73

3rd Party Tools
• Non-Java languages
• Bytecode obfuscators
• Bytecode optimizers
• 3rd party Java compilers
• Bytecode assemblers
– Oolong
– Jasmin
– JASM
74

Runtime Mischief
• Runtime Code Generation
• Runtime Code Modification
• Usual suspects:
– BCEL
– ASM
– AOP
– Instrumentation tools / agents
75

Compatibility Issues
• A serious limitation for bytecode manipulation
• Tools like instrumentation agents may not know the rules of more recent
classfile versions
76

r/programminghorror
try {
new OraclePKIProvider();
} catch (Throwable t) { ; }
77

r/programminghorror
• Verification enabled
– VerifyError silently eaten by catch clause
– Application runs fine
• Verification disabled
– Broken bytecode loaded, environment breaks
78

Usage
79

options
• -Xverify:
– none
• disables all verification. Only use for debugging!
– remote
• default. Verifies all classes not loaded by boot class path.
– all
• Verifies everything.
• -noverify
• Same as –Xverify:none
80

Xverify:remote
• Has nothing to do with remote / local
• Horribly named
• Our own documentation was wrong for well over a decade
81

Cost of Verification
• Classloading could be CPU-bound in the 90s
• Skipping verification could speed up class loading, giving a faster startup
82

83

• On modern hardware, class loading is no longer CPU-bound, it is IO-bound
– Even on SSD hardware
• Verification is more or less free
84

Development Usage
• Verification is just as important in Development as in Production (if not
more!)
• Some products explicitly disable verification by default in “Developer”
configurations!
• Previously unseen verify errors thrown when code is moved into
production
85

Verification Support by Class File Version
• <= class file version 49 (JDK 5)
– Only type inference supported
• class file version 50 (JDK 6)
– Type checking w/ fallback to type inference
• >= class file version 51 (JDK 7)
– only type checking supported
– (JDK 7 only) force use type inference w/ -XX:-UseSplitVerifier
86

Conclusions
87

Summary
• Always use verification
– Even in development
– Even with trusted code
– Even when startup time is important
• Verification depends on already loaded classes
• Split Verifier is here to stay
88

References
[ Cracking the Hotspot JVM ]
https://blogs.oracle.com/kamg/entry/cracking_the_hotspot_jvm
[ 4.10. Verification of class Files ]
https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10
89

Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
90

Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

Similar to Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500] (20)

More from David Buck

More from David Buck (20)

Recently uploaded

Recently uploaded (20)

Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]