While compute becomes faster and cheaper we are tempted to abandon sanity and shield ourselves from reality and laws of physics. The resulting mess of monstrous Slack instances rampaging across our RAM should makes us stop (because our computers did it already) and wonder where did we go wrong? Rising developer salaries and time to market pace are tempting us to abandon all hope for optimising our code and understanding our systems. Contrary to what casual reader could think this is a deeply technical presentation. We will gaze into hardware counters, NUMA nodes, vector registers and that darkness will stare back at us. All this to get a taste of what is possible on current hardware, to learn the COST of scalability and forever change how you feel when accessing invoice list in your local utilities provider UI so that after 20s of waiting all 12 elements will be displayed (surely Cthulhu must be eating their compute because it is NOT possible Tauron hosts it’s billing services on FIRST GEN IPHONE).

1. IS WRITING PERFORMANT
CODE TOO EXPENSIVE?

2. WHOIS
TOMEK KOWALCZEWSKI
▸ Works @ Codewise
▸ Failed tech lead
▸ Spent about half of his carrier
optimising allocations :(

4. SPRINT

5. 10 000$ / MONTH

6. 20 000$ / MONTH

7. ©DRESDEN DATABASE SYSTEMS GROUP

8. ISSUES
ISSUES SUMMARY
▸ Compute gets cheaper and faster
▸ Cost of development goes up
▸ Technology is only small part of the equation

9. VERY LARGE DATABASES CONFERENCE 2018
source: vldb

10. VERY LARGE DATABASES CONFERENCE 2017
source: wikipedia

11. VERY LARGE DATABASES CONFERENCE
2015 2016 2017
20ms
80ms
150ms

12. VERY LARGE DATABASES CONFERENCE
2015 2016 2017
20ms
80ms
150ms
250MS
HUMAN PERCEPTION

13. VERY LARGE DATABASES CONFERENCE
source: vldb

14. EFFICIENTLY COMPILING EFFICIENT QUERY PLANS FOR MODERN HARDWARE
DBMS "X" MYSQL 4.1 MONETDB HAND-CODED C PROGRAM
220ms
3s 700ms
27s28s

15. SCALABILITY! BUT AT WHAT COST?

16. NOW, WE ARE GOING TO INTRODUCE SOME
NEW CANDIDATES FOR "STATE-OF-THE-ART"
ALGORITHMS, BY WRITING SOME FOR-
LOOPS AND RUNNING THEM ON MY LAPTOP.
Frank McSherry
SCALABILITY! BUT AT WHAT COST?

17. SCALABILITY! BUT AT WHAT COST?

18. ©DRESDEN DATABASE SYSTEMS GROUP

19. ©DRESDEN DATABASE SYSTEMS GROUP

20. WHERE DID WE GO WRONG

21. THERE ARE NO PROBLEMS ONLY OPPORTUNITIES

22. PERFORMANCE?
PERFORMANCE!
‣ Intuition about what is possible
‣ Impact on real life Java code

23. PERFORMANCE
WHAT AFFECTS PERFORMANCE?
‣ Stuff, lot’s of stuff
‣ Branch prediction
‣ Prefetching (vs. random access)
‣ NUMA
‣ GC, etc…

24. CPU FUNCTIONAL BLOCKS

25. CPU FUNCTIONAL BLOCKS
NUMA domain 0 NUMA domain 1

26. AMD EPYC DIE

27. AMD EPYC DIAGRAM
./lstopo --physical --output-format png /tmp/lstopo.png

28. AMD EPYC DIAGRAM

29. AMD EPYC DIE

30. AMD EPYC CORE PING LATENCY (JAVA-JMH)
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>1.21</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>1.21</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.xerial</groupId>
<artifactId>jnuma</artifactId>
<version>0.1.3</version>
</dependency>

31. AMD EPYC CORE PING LATENCY (JAVA-JMH)
@Setup
public void setUp(CoreSequence coreSequence) {
Numa.runOnNode(node);
ByteBuffer buffer = Numa.allocOnNode(capacity, node);
long address = address(buffer);
}

32. AMD EPYC CORE PING LATENCY (JAVA-JMH)
@Benchmark
@Group("pingpong")
public void ping(Control cnt, GroupState groupState, CoreAssigner coreAssigner)
{
while (!cnt.stopMeasurement && !UNSAFE.compareAndSwapInt(null,
groupState.flagAddress, 0, 1)) {
// this body is intentionally left blank
}
}
@Benchmark
@Group("pingpong")
public void pong(Control cnt, GroupState groupState, CoreAssigner coreAssigner)
{
while (!cnt.stopMeasurement && !UNSAFE.compareAndSwapInt(null,
groupState.flagAddress, 1, 0)) {
// this body is intentionally left blank
}
}

33. AMD EPYC CORE PING LATENCY (JAVA-JMH)

34. AMD EPYC CORE PING LATENCY (JAVA-JMH)

35. INTEL XEON PLATINUM PING LATENCY (JAVA-JMH)

36. NUMA
RAM READ THROUGHPUT
A. <500 MB/s
B. <2GB/s
C. <20GB/s
D. ~100GB/s

37. NUMA
RAM READ THROUGHPUT - PREDICTABLE PATTERN
@Benchmark
public int readByteBufferInOrder() {
Numa.runOnNode(threadNumaNode);
int result = 0;
for (int i = 0; i < byteBuffer.limit(); i += 8) {
result += byteBuffer.getLong(i);
}
return result;
}

38. AMD MEMORY BANDWIDTH

39. JMH RESULTS
AVERAGE TIME
Time(μs)
BASE
RANDOM
INARRAY
PRECOMPUTEDARRAY
SORTEDRANDOM
848
1,701
2,717
2,964

40. JMH RESULTS
L1-DCACHE-LOAD-MISSES
Events
BASE
RANDOM
INARRAY
PRECOMPUTEDARRAY
SORTEDRANDOM
23,702
91,26594,47489,169

41. NUMA
RAM THROUGHPUT - READ 400MB OF DATA
A. ~1s
B. 0.348s
C. 0.150s
D. It depends

42. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
Selectivity (%)
0 10 20 30 40 50 60 70 80 90 100
533ms
43ms 43ms

43. LESLIE LAMPORT

44. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
MASK = 0111 1111
~MASK = 1000 0000
42 = 0010 1010
13 = 0000 1101
13 ^ MASK = 0111 0010
+ 42 = 1001 1100
&~MASK = 1000 0000

45. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainIf() {
int result = 0;
for (int i = 0; i < inputSize; i++) {
if (onHeapValues.getInt(i * 4) < predicate) {
result++;
}
}
return result;
}

46. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainArithmetic() {
int result = 0;
int mask = 0b0111_1111;
for (int i = 0; i < inputSize; i++) {
int x = onHeapValues.getInt(i * 4) ^ mask;
int z = (predicate + x) & ~mask;
result += Integer.bitCount(z);
}
return result;
}

47. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
0 10 20 30 40 50 60 70 80 90 100
CONDITIONAL ARITHMETIC
97ms
533ms
43ms 43ms

48. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
0 10 20 30 40 50 60 70 80 90 100
CONDITIONAL ARITHMETIC
PACKED
11ms
97ms
533ms
43ms 43ms

49. BRANCHLESS DOOM

50. BRANCHLESS DOOM
“The mov-only DOOM renders approximately one
frame every 7 hours, so playing this version requires
somewhat increased patience.”

51. CONCLUSION
‣ COMPUTE IS GETTING FASTER
‣ DEVELOPERS ARE GETTING MORE EXPENSIVE
‣ (GOOD :))
‣ SOFTWARE & LIBRARIES ARE GETTING SLOWER
‣ OPPORTUNITIES AHEAD

52. CONCLUSION
‣ O(NLOGN) ALGORITHM IS 50 000 TIMES FASTER
THAN O(N2)
‣ DO WE WANT TO ADD 50 000 TIMES MORE
SERVERS OR OPTIMISE?

53. PAPERS
REFERENCES
‣Efficiently Compiling Efficient Query Plans for Modern
Hardware
‣Constant-Time Query Processing
‣https://danluu.com/branch-prediction/
‣StarCraft II as an AI research environment
‣Multiple Byte Processing with FullWord Instructions
‣http://www.vldb.org/2017/keynotes.php

54. TEXT
COPYRIGHTS
▸ Houston opportunities by XKCD.com
▸ A/B Testing example: https://en.wikipedia.org/wiki/A/
B_testing#/media/File:A-B_testing_example.png by
Maxime Lorant
▸ AMD for EPYC diagrams