Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

Often the true mark of a successful technology is when something works so well that those who benefit the most from it are not even aware of its existence. Java’s bytecode verification undoubtedly falls into this category, but it is as vital as ever to keeping Java secure and safe. This session takes a deep dive into the safeguards that bytecode verification provides for us and shows how it continues to protect us from not only malicious code but also our own mistakes.

  • Login to see the comments

  • Be the first to like this

Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]

  1. 1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Bytecode Verification The Hero That Java Needs David Buck Principal Member of Technical Staff Java SE Sustaining Engineering September, 2016
  2. 2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 4
  3. 3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | About Me David Buck • Java SE Sustaining Engineering • Mostly JRockit fixes • OpenJDK 8 Updates Project Maintainer • Hobbies: Programming 5
  4. 4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Program Agenda Introduction Dangers Demo Implementation Importance Usage Conclusions 1 2 3 4 5 6 6 7
  5. 5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Introduction 7
  6. 6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What Is It? • Analysis of bytecode • Syntax check • Symantec check • Ensures stability / security of runtime 8
  7. 7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | When does it happen? • Analysis done during class loading • Sometimes delayed until right before method execution • But only done at most once per loaded method 9
  8. 8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Traditional Interpreted Language Source Code Interpreter 10
  9. 9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Traditional Interpreted Language Source Code Interpreter 11
  10. 10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Traditional Compiled Language Source Code ExecutableCompile 12
  11. 11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Traditional Compiled Language Source Code ExecutableCompile 13
  12. 12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Java Source Code BytecodeCompile 14 JVM
  13. 13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Java Source Code BytecodeCompile 15 JVM
  14. 14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Java Source Code BytecodeCompile 16 JVM
  15. 15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Java Source Code BytecodeCompile 17 JVM
  16. 16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What Does It Do? 18
  17. 17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What Does It Do? • Protects runtime from bad people "Why the verifier is so important…. write once and crack anywhere“ -Keith McGuigan 19
  18. 18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | What Does It Do? • Protects runtime from bad people "Why the verifier is so important…. write once and crack anywhere“ -Keith McGuigan • Protects runtime from you 20
  19. 19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Why learn about it? • The best technologies are invisible… • Victim of its own success 21
  20. 20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Dangers 22
  21. 21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Class Metadata 23
  22. 22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Class Metadata • Has a direct superclass 24
  23. 23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Class Metadata • Has a direct superclass • Superclass is not marked final 25
  24. 24. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Class Metadata • Has a direct superclass • Superclass is not marked final • No final methods are overridden 26
  25. 25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Class Metadata • Has a direct superclass • Superclass is not marked final • No final methods are overridden 27
  26. 26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 28
  27. 27. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 29
  28. 28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 0 LIMIT 30
  29. 29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 0 LIMIT 1 31
  30. 30. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 0 LIMIT 1 2 32
  31. 31. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Overflow stack=2, locals=1, args_size=1 0: iconst_0 1: iconst_1 2: iconst_2 0 LIMIT 1 2 33
  32. 32. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Underflow stack=3, locals=1, args_size=1 0: iadd 1: iadd 2: iadd 0 LIMIT 1 2 START 34
  33. 33. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Underflow stack=3, locals=1, args_size=1 0: iadd 1: iadd 2: iadd 0 LIMIT 3 START 35
  34. 34. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Underflow stack=3, locals=1, args_size=1 0: iadd 1: iadd 2: iadd 3 LIMIT START 36
  35. 35. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Underflow stack=3, locals=1, args_size=1 0: iadd 1: iadd 2: iadd ? LIMIT START 37
  36. 36. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Operand Stack Underflow stack=3, locals=1, args_size=1 0: iadd 1: iadd 2: iadd ? LIMIT START 38
  37. 37. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Type Checking • Each operation is checked – Correct types on the stack – Correct types in local variable “slots” • Specification uses Prolog to define requirements 39
  38. 38. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Prolog?! • Predicate logic of type system are described by Prolog well • Java is probably the first of this kind of use by a mainstream programming language 40
  39. 39. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Prolog?! Facts: cat(tom). 41
  40. 40. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Prolog?! Facts: parent_child(sally, bob). 42
  41. 41. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Prolog?! Rules: Head :- Body. 43
  42. 42. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Prolog?! Rules: sibling(X, Y) :- parent_child(Z, X), parent_child(Z, Y). 44
  43. 43. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | The Specification 45
  44. 44. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Java Bytecode Expressive Power Java Language 46
  45. 45. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Demo 47
  46. 46. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | ClassA public class ClassA { public int doSomething(int i1, int i2, int i3) { return i1+i2+i3; } } 48
  47. 47. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | ClassB public class ClassB { public Integer doSomethingElse(int i1, int i2, int i3) { return new Integer(i1+i2+i3); } } 49
  48. 48. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | ClassC public class ClassC extends ClassA {} 50
  49. 49. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Demo public class Demo { public static void main(String[] args) { ClassA obj = new ClassC(); System.out.println(obj.doSomething(1,2,3)); } } 51
  50. 50. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Object ClassA ClassB ClassC Demo 52
  51. 51. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | It works… $ java Demo 6 $ 53
  52. 52. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Lets do something bad… public class ClassC extends ClassB {} 54
  53. 53. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Lets do something bad… public class ClassC extends ClassB {} 55
  54. 54. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Object ClassA ClassB ClassC Demo 56
  55. 55. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Object ClassA ClassB ClassC Demo 57
  56. 56. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Object ClassA ClassB ClassC Demo 58
  57. 57. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Demo public class Demo { public static void main(String[] args) { ClassA obj = new ClassC(); System.out.println(obj.doSomething(1,2,3)); } } 59
  58. 58. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | $ java Demo Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.VerifyError: Bad type on operand stack Exception Details: Location: Demo.main([Ljava/lang/String;)V @15: invokevirtual Reason: Type 'ClassC' (current frame, stack[1]) is not assignable to 'ClassA' Current Frame: bci: @15 flags: { } locals: { '[Ljava/lang/String;', 'ClassC' } stack: { 'java/io/PrintStream', 'ClassC', integer, integer, integer } Bytecode: 0x0000000: bb00 0259 b700 034c b200 042b 0405 06b6 0x0000010: 0005 b600 06b1 60
  59. 59. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | As expected, the verifier protects us from ourselves. 61
  60. 60. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | As expected, the verifier protects us from ourselves. What if we disable it… 62
  61. 61. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | We reap what we sow… $ java -Xverify:none Demo # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007fa93be7991c, pid=22925, tid=140364857087744 # # JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 1.8.0_91-b14) # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x46391c] 63
  62. 62. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Demo Takeaways • No obvious evidence that bad bytecode was root cause of crash • A class is only valid in the context of previously loaded classes • No malicious intent / 3rd party tools used 64
  63. 63. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Optional 2nd Demo $ java -Xverify:none Crack =============== DEBUG MESSAGE: illegal bytecode sequence - method not verified ================ Exception in thread "Thread-0" java.lang.NullPointerException at Pointer.deref(Pointer.jasm) at Crack.breakLock(Crack.java:13) at Crack$1.run(Crack.java:29) Thread Thread[main,5,main] leaving monitor $ 65
  64. 64. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Implementation 66
  65. 65. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Type Inference Verifier – AKA the Old Verifier Type Checking Verifier – AKA Split Verifier – AKA The New Hotness 67 A Tale of Two Verifiers…
  66. 66. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Type Inference Verifier – AKA the Old Verifier Type Checking Verifier – AKA Split Verifier – AKA The New Hotness 68 A Tale of Two Verifiers…
  67. 67. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Type Inference Verifier • Class files <= 49 (JDK 5) • Requires CFG construction • Worst case scenario can require many passes Diagram by JMP EAX - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=34222288 69
  68. 68. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | When We Do Syntactic / Semantic checking Source Code BytecodeCompile 70 JVM
  69. 69. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Type Checking Verifier (AKA Split Verifier) • Class files >= 50 (JDK 6) • Depends on StackMapTable Attribute • Transfers much of the responsibility to javac Source Code BytecodeCompile JVM StackMap Tables 71
  70. 70. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | StackMapTable • Identifies the type of each stack position / local variable • One needed for every instruction that is the target of a jump – Methods without branches will not have them • Are stored as deltas to save space • Allow single pass verification 72
  71. 71. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Importance 73
  72. 72. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 3rd Party Tools • Non-Java languages • Bytecode obfuscators • Bytecode optimizers • 3rd party Java compilers • Bytecode assemblers – Oolong – Jasmin – JASM 74
  73. 73. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Runtime Mischief • Runtime Code Generation • Runtime Code Modification • Usual suspects: – BCEL – ASM – AOP – Instrumentation tools / agents 75
  74. 74. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Compatibility Issues • A serious limitation for bytecode manipulation • Tools like instrumentation agents may not know the rules of more recent classfile versions 76
  75. 75. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | r/programminghorror try { new OraclePKIProvider(); } catch (Throwable t) { ; } 77
  76. 76. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | r/programminghorror • Verification enabled – VerifyError silently eaten by catch clause – Application runs fine • Verification disabled – Broken bytecode loaded, environment breaks 78
  77. 77. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Usage 79
  78. 78. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | options • -Xverify: – none • disables all verification. Only use for debugging! – remote • default. Verifies all classes not loaded by boot class path. – all • Verifies everything. • -noverify • Same as –Xverify:none 80
  79. 79. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Xverify:remote • Has nothing to do with remote / local • Horribly named • Our own documentation was wrong for well over a decade 81
  80. 80. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Cost of Verification • Classloading could be CPU-bound in the 90s • Skipping verification could speed up class loading, giving a faster startup 82
  81. 81. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Cost of Verification 83
  82. 82. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Cost of Verification • On modern hardware, class loading is no longer CPU-bound, it is IO-bound – Even on SSD hardware • Verification is more or less free 84
  83. 83. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Development Usage • Verification is just as important in Development as in Production (if not more!) • Some products explicitly disable verification by default in “Developer” configurations! • Previously unseen verify errors thrown when code is moved into production 85
  84. 84. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Verification Support by Class File Version • <= class file version 49 (JDK 5) – Only type inference supported • class file version 50 (JDK 6) – Type checking w/ fallback to type inference • >= class file version 51 (JDK 7) – only type checking supported – (JDK 7 only) force use type inference w/ -XX:-UseSplitVerifier 86
  85. 85. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Conclusions 87
  86. 86. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Summary • Always use verification – Even in development – Even with trusted code – Even when startup time is important • Verification depends on already loaded classes • Split Verifier is here to stay 88
  87. 87. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | References [ Cracking the Hotspot JVM ] https://blogs.oracle.com/kamg/entry/cracking_the_hotspot_jvm [ 4.10. Verification of class Files ] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10 89
  88. 88. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 90
  89. 89. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 91

×