Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scale Up with Lock-Free Algorithms @ JavaOne

785 views

Published on

Scale Up with Lock-Free Algorithms, presented at JavaOne 2017

Published in: Technology
  • Login to see the comments

Scale Up with Lock-Free Algorithms @ JavaOne

  1. 1. Scale Up with Lock-Free Algorithms Non-blocking concurrency on JVM Presented at JavaOne 2017 /Roman Elizarov @ JetBrains
  2. 2. Speaker: Roman Elizarov • 16+ years experience • Previously developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ACM ICPC • Now work on Kotlin @ JetBrains
  3. 3. Shared
  4. 4. Shared Mutable
  5. 5. Shared Mutable State
  6. 6. Shared Mutable State Why?
  7. 7. Big Big Data
  8. 8. Data 1 Data 2 Data N
  9. 9. Data 1 Data 2 Data N map map map
  10. 10. Data 1 Data 2 Data N map map map reduce answer
  11. 11. Embarrassingly parallel Data 1 Data 2 Data N map map map reduce answer
  12. 12. Big Big Data
  13. 13. Big Big Data Real-time
  14. 14. Big Big Data Real-time Concurrent requests / processing
  15. 15. Big Big Data Real-time Concurrent requests / processing Performance Scalability
  16. 16. A toy problem
  17. 17. A toy problem – stack
  18. 18. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T)
  19. 19. public final class Node<T> { private final Node<T> next; private final T value; public Node(Node<T> next, T value) { this.next = next; this.value = value; } public Node<T> getNext() { return next; } public T getValue() { return value; } } A toy problem – stack
  20. 20. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T)
  21. 21. A toy problem – empty stack class Node<T>(val next: Node<T>?, val value: T) top
  22. 22. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А
  23. 23. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
  24. 24. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
  25. 25. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A
  26. 26. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
  27. 27. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
  28. 28. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
  29. 29. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A
  30. 30. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur
  31. 31. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur
  32. 32. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur result = 2
  33. 33. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
  34. 34. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
  35. 35. Does it work?
  36. 36. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
  37. 37. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B next = A value = 3 C
  38. 38. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B next = A value = 3 C
  39. 39. A toy problem – synchronized stack class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { top = Node(top, value) } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
  40. 40. Does it scale?
  41. 41. Benchmark @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
  42. 42. Benchmark @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
  43. 43. Benchmark results 0 5 10 15 20 25 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LinkedStack Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz; 32 HW threads; Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
  44. 44. Contention P Q pop1
  45. 45. Contention P Q pop1
  46. 46. Contention P Q pop1 pop2 wait
  47. 47. Contention P Q work pop1 pop2 wait
  48. 48. Deadlocks
  49. 49. Lock-free?
  50. 50. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B expect
  51. 51. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B update
  52. 52. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B update expect
  53. 53. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
  54. 54. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
  55. 55. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
  56. 56. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
  57. 57. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
  58. 58. Using AtomicReference - push class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1
  59. 59. Using AtomicReference - push class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1 2
  60. 60. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1 2 3
  61. 61. Using AtomicReference - push 1 2 3 class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
  62. 62. Powerful we have become!
  63. 63. Using AtomicReference - pop class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { while (true) { val cur = top.get() ?: return null if (top.compareAndSet(cur, cur.next)) return cur.value } } }
  64. 64. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { … } }
  65. 65. It’s a trap
  66. 66. Using volatile variable class LinkedStackLF<T> { @Volatile private var top: Node<T>? = null fun push(value: T) { // ... } }
  67. 67. Using AtomicReferenceFieldUpdater package java.util.concurrent.atomic; /** @since 1.5 */ public abstract class AtomicReferenceFieldUpdater<T,V> { public static <U,W> AtomicReferenceFieldUpdater<U,W> newUpdater( Class<U> tclass, Class<W> vclass, String fieldName; public abstract boolean compareAndSet(T obj, V expect, V update; }
  68. 68. Using AtomicReferenceFieldUpdater private volatile Node<T> top;
  69. 69. Using AtomicReferenceFieldUpdater private volatile Node<T> top; private static final AtomicReferenceFieldUpdater<LockFree, Node> TOP = AtomicReferenceFieldUpdater .newUpdater(LockFree.class, Node.class, "top");
  70. 70. Using AtomicReferenceFieldUpdater private volatile Node<T> top; private static final AtomicReferenceFieldUpdater<LockFree, Node> TOP = AtomicReferenceFieldUpdater .newUpdater(LockFree.class, Node.class, "top"); if (TOP.compareAndSet(this, cur, upd)) return;
  71. 71. Using VarHandle package java.lang.invoke; /** @since 9 */ public abstract class VarHandle { @MethodHandle.PolymorphicSignature public native boolean compareAndSet(Object... args); }
  72. 72. Using VarHandle private volatile Node<T> top; private static final VarHandle TOP; static { try { TOP = MethodHandles.lookup() .findVarHandle(LockFree.class, "top", Node.class); } catch (NoSuchFieldException | IllegalAccessException e) { throw new InternalError(e); } }
  73. 73. Using VarHandle private volatile Node<T> top; private static final VarHandle TOP; static { try { TOP = MethodHandles.lookup() .findVarHandle(LockFree.class, "top", Node.class); } catch (NoSuchFieldException | IllegalAccessException e) { throw new InternalError(e); } } if (TOP.compareAndSet(this, cur, upd) return;
  74. 74. Using AtomicFU J private val top = atomic<Node<T>?>(null)
  75. 75. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return
  76. 76. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Code like AtomicReference
  77. 77. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference compile
  78. 78. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference AtomicReferenceFUcompile atomicFU
  79. 79. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference VarHandlecompile atomicFU
  80. 80. Was it worth it?
  81. 81. Benchmark results 0 5 10 15 20 25 30 35 40 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
  82. 82. 0 5 10 15 20 25 30 35 40 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack Benchmark results Yeh! Nay…
  83. 83. Contention P Q retry pop1 pop2 try update
  84. 84. Too toy of a problem? class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { … } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
  85. 85. Too toy of a problem – make it more real? class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { … } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next Blackhole.consumeCPU(100L) return cur.value } }
  86. 86. Too toy of a problem – make it more real? class LockFree<T> { private val top = atomic<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { while (true) { val cur = top.value ?: return null Blackhole.consumeCPU(100L) if (top.compareAndSet(cur, cur.next)) return cur.value } } }
  87. 87. Benchmark results 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
  88. 88. Workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
  89. 89. Read-dominated workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } }
  90. 90. Read-dominated workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } } class LinkedStack<T> { @Synchronized fun peek() = top?.value }
  91. 91. Read-dominated workload @State(Scope.Benchmark) open class LockFreeBenchmark { private val stack = LockFree<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } } class LockFree<T> { fun peek() = top.value?.value }
  92. 92. Benchmark results – x10 reads 0 5 10 15 20 25 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
  93. 93. Benchmark results – x100 reads 0 1 2 3 4 5 6 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
  94. 94. But … scalability?
  95. 95. Real-world workload @Benchmark fun benchmarkReadWorld() { stack.push(1) repeat(10) { check(stack.peek() == 1) Blackhole.consumeCPU(100L) } check(stack.pop() == 1) Blackhole.consumeCPU(100L) }
  96. 96. Benchmark results – real world 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
  97. 97. Learn to ask the right questions You shall, young Padawan.
  98. 98. Links • JMH http://openjdk.java.net/projects/code-tools/jmh/ • Kotlin https://kotlinlang.org • AtomicFU https://github.com/Kotlin/kotlinx.atomicfu
  99. 99. Thank you Any questions? Slides are available at www.slideshare.net/elizarov email me to elizarov at gmail relizarov
  100. 100. Appendix
  101. 101. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur1 cur2
  102. 102. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur1 cur2
  103. 103. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A result1 = 2 cur1 cur2 result2 = 2

×