Successfully reported this slideshow.
Upcoming SlideShare
×

# Scale Up with Lock-Free Algorithms @ JavaOne

785 views

Published on

Scale Up with Lock-Free Algorithms, presented at JavaOne 2017

Published in: Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No

### Scale Up with Lock-Free Algorithms @ JavaOne

1. 1. Scale Up with Lock-Free Algorithms Non-blocking concurrency on JVM Presented at JavaOne 2017 /Roman Elizarov @ JetBrains
2. 2. Speaker: Roman Elizarov • 16+ years experience • Previously developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ACM ICPC • Now work on Kotlin @ JetBrains
3. 3. Shared
4. 4. Shared Mutable
5. 5. Shared Mutable State
6. 6. Shared Mutable State Why?
7. 7. Big Big Data
8. 8. Data 1 Data 2 Data N
9. 9. Data 1 Data 2 Data N map map map
10. 10. Data 1 Data 2 Data N map map map reduce answer
11. 11. Embarrassingly parallel Data 1 Data 2 Data N map map map reduce answer
12. 12. Big Big Data
13. 13. Big Big Data Real-time
14. 14. Big Big Data Real-time Concurrent requests / processing
15. 15. Big Big Data Real-time Concurrent requests / processing Performance Scalability
16. 16. A toy problem
17. 17. A toy problem – stack
18. 18. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T)
19. 19. public final class Node<T> { private final Node<T> next; private final T value; public Node(Node<T> next, T value) { this.next = next; this.value = value; } public Node<T> getNext() { return next; } public T getValue() { return value; } } A toy problem – stack
20. 20. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T)
21. 21. A toy problem – empty stack class Node<T>(val next: Node<T>?, val value: T) top
22. 22. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А
23. 23. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
24. 24. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
25. 25. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A
26. 26. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
27. 27. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
28. 28. A toy problem – stack push class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } }
29. 29. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A
30. 30. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur
31. 31. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur
32. 32. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur result = 2
33. 33. A toy problem – stack pop class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
34. 34. A toy problem – stack class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null fun push(value: T) { top = Node(top, value) } fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
35. 35. Does it work?
36. 36. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B
37. 37. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B next = A value = 3 C
38. 38. A toy problem – concurrent push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B next = A value = 3 C
39. 39. A toy problem – synchronized stack class Node<T>(val next: Node<T>?, val value: T) class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { top = Node(top, value) } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
40. 40. Does it scale?
41. 41. Benchmark @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
42. 42. Benchmark @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
43. 43. Benchmark results 0 5 10 15 20 25 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LinkedStack Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz; 32 HW threads; Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
44. 44. Contention P Q pop1
45. 45. Contention P Q pop1
46. 46. Contention P Q pop1 pop2 wait
47. 47. Contention P Q work pop1 pop2 wait
49. 49. Lock-free?
50. 50. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B expect
51. 51. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B update
52. 52. Lock-free push class Node<T>(val next: Node<T>?, val value: T) next = null value = 1 top А next = A value = 2 B update expect
53. 53. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
54. 54. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
55. 55. AtomicReference package java.util.concurrent.atomic; /** @since 1.5 */ public class AtomicReference<V> { private volatile V value; public V get() { return value; } public boolean compareAndSet(V expect, V update) { // … } }
56. 56. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
57. 57. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
58. 58. Using AtomicReference - push class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1
59. 59. Using AtomicReference - push class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1 2
60. 60. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } } 1 2 3
61. 61. Using AtomicReference - push 1 2 3 class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { while (true) { val cur = top.get() val upd = Node(cur, value) if (top.compareAndSet(cur, upd)) return } } }
62. 62. Powerful we have become!
63. 63. Using AtomicReference - pop class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { while (true) { val cur = top.get() ?: return null if (top.compareAndSet(cur, cur.next)) return cur.value } } }
64. 64. Using AtomicReference class LockFree<T> { private val top = AtomicReference<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { … } }
65. 65. It’s a trap
66. 66. Using volatile variable class LinkedStackLF<T> { @Volatile private var top: Node<T>? = null fun push(value: T) { // ... } }
67. 67. Using AtomicReferenceFieldUpdater package java.util.concurrent.atomic; /** @since 1.5 */ public abstract class AtomicReferenceFieldUpdater<T,V> { public static <U,W> AtomicReferenceFieldUpdater<U,W> newUpdater( Class<U> tclass, Class<W> vclass, String fieldName; public abstract boolean compareAndSet(T obj, V expect, V update; }
68. 68. Using AtomicReferenceFieldUpdater private volatile Node<T> top;
69. 69. Using AtomicReferenceFieldUpdater private volatile Node<T> top; private static final AtomicReferenceFieldUpdater<LockFree, Node> TOP = AtomicReferenceFieldUpdater .newUpdater(LockFree.class, Node.class, "top");
70. 70. Using AtomicReferenceFieldUpdater private volatile Node<T> top; private static final AtomicReferenceFieldUpdater<LockFree, Node> TOP = AtomicReferenceFieldUpdater .newUpdater(LockFree.class, Node.class, "top"); if (TOP.compareAndSet(this, cur, upd)) return;
71. 71. Using VarHandle package java.lang.invoke; /** @since 9 */ public abstract class VarHandle { @MethodHandle.PolymorphicSignature public native boolean compareAndSet(Object... args); }
72. 72. Using VarHandle private volatile Node<T> top; private static final VarHandle TOP; static { try { TOP = MethodHandles.lookup() .findVarHandle(LockFree.class, "top", Node.class); } catch (NoSuchFieldException | IllegalAccessException e) { throw new InternalError(e); } }
73. 73. Using VarHandle private volatile Node<T> top; private static final VarHandle TOP; static { try { TOP = MethodHandles.lookup() .findVarHandle(LockFree.class, "top", Node.class); } catch (NoSuchFieldException | IllegalAccessException e) { throw new InternalError(e); } } if (TOP.compareAndSet(this, cur, upd) return;
74. 74. Using AtomicFU J private val top = atomic<Node<T>?>(null)
75. 75. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return
76. 76. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Code like AtomicReference
77. 77. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference compile
78. 78. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference AtomicReferenceFUcompile atomicFU
79. 79. Using AtomicFU J private val top = atomic<Node<T>?>(null) if (top.compareAndSet(cur, upd)) return Bytecode Code like AtomicReference VarHandlecompile atomicFU
80. 80. Was it worth it?
81. 81. Benchmark results 0 5 10 15 20 25 30 35 40 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
82. 82. 0 5 10 15 20 25 30 35 40 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack Benchmark results Yeh! Nay…
83. 83. Contention P Q retry pop1 pop2 try update
84. 84. Too toy of a problem? class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { … } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next return cur.value } }
85. 85. Too toy of a problem – make it more real? class LinkedStack<T> { private var top: Node<T>? = null @Synchronized fun push(value: T) { … } @Synchronized fun pop(): T? { val cur = top ?: return null top = cur.next Blackhole.consumeCPU(100L) return cur.value } }
86. 86. Too toy of a problem – make it more real? class LockFree<T> { private val top = atomic<Node<T>?>(null) fun push(value: T) { … } fun pop(): T? { while (true) { val cur = top.value ?: return null Blackhole.consumeCPU(100L) if (top.compareAndSet(cur, cur.next)) return cur.value } } }
87. 87. Benchmark results 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
88. 88. Workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmark() { stack.push(1) check(stack.pop() == 1) } }
89. 89. Read-dominated workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } }
90. 90. Read-dominated workload @State(Scope.Benchmark) open class LinkedStackBenchmark { private val stack = LinkedStack<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } } class LinkedStack<T> { @Synchronized fun peek() = top?.value }
91. 91. Read-dominated workload @State(Scope.Benchmark) open class LockFreeBenchmark { private val stack = LockFree<Int>() @Benchmark fun benchmarkReadDominated() { stack.push(1) repeat(10) { check(stack.peek() == 1) } check(stack.pop() == 1) } } class LockFree<T> { fun peek() = top.value?.value }
92. 92. Benchmark results – x10 reads 0 5 10 15 20 25 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
93. 93. Benchmark results – x100 reads 0 1 2 3 4 5 6 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
94. 94. But … scalability?
95. 95. Real-world workload @Benchmark fun benchmarkReadWorld() { stack.push(1) repeat(10) { check(stack.peek() == 1) Blackhole.consumeCPU(100L) } check(stack.pop() == 1) Blackhole.consumeCPU(100L) }
96. 96. Benchmark results – real world 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 1 2 4 8 16 32 64 128 Millions Number of threads Throughput (ops/s) LockFree LinkedStack
97. 97. Learn to ask the right questions You shall, young Padawan.
98. 98. Links • JMH http://openjdk.java.net/projects/code-tools/jmh/ • Kotlin https://kotlinlang.org • AtomicFU https://github.com/Kotlin/kotlinx.atomicfu
99. 99. Thank you Any questions? Slides are available at www.slideshare.net/elizarov email me to elizarov at gmail relizarov
100. 100. Appendix
101. 101. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur1 cur2
102. 102. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A cur1 cur2
103. 103. A toy problem – concurrent pop class Node<T>(val next: Node<T>?, val value: T) next = A value = 2 top next = null value = 1 B A result1 = 2 cur1 cur2 result2 = 2