Companies want to validate products early, with little time for good engineering and performance work. Yet good code can provide 10-100x speed up which brings tremendous value to clients. We get help from modern hardware and algorithms but we need to know what are its strengths and limitations so we can consciously decide when to invest in engineering and what added value to expect.

1. IS WRITING PERFORMANT
CODE TOO EXPENSIVE?
SLIDO.COM #GEECON2019
TOMASZ KOWALCZEWSKI

2. WHOIS
TOMASZ KOWALCZEWSKI
▸ Codewise/Tornimo.io
▸ Twitter: @tkowalcz
▸ Failed tech lead
▸ Spent about half of his carrier optimising allocations :(

3. FAILED
ASPIRATIONS
IN
DATABASE
SYSTEMS

4. SPRINT

5. 10 000$ / MONTH

6. 20 000$ / MONTH

7. ©DRESDEN DATABASE SYSTEMS GROUP

8. WHERE DID WE GO WRONG

9. 0.1X DEVELOPER NEEDED
THE 0.1X DEVELOPER
▸ Let’s not add that feature
▸ Let’s not adopt this new technology
▸ Let’s not automate this
▸ Can we approximate the result so no one notices?
▸ Can we use a small lookup table?
▸ Benji Weber: https://benjiweber.co.uk/blog/2016/01/25/why-i-strive-to-be-a-0-1x-
engineer/
▸ Mike Acton: https://twitter.com/mike_acton/status/989001065893801984

10. VERY LARGE DATABASES CONFERENCE 2017
source: wikipedia

11. VERY LARGE DATABASES CONFERENCE
2015 2016 2017
20ms
80ms
150ms

12. VERY LARGE DATABASES CONFERENCE
2015 2016 2017
20ms
80ms
150ms
250MS
HUMAN PERCEPTION

13. EFFICIENTLY COMPILING EFFICIENT QUERY PLANS FOR MODERN HARDWARE
DBMS "X" MYSQL 4.1 MONETDB HAND-CODED C
220ms
3s 700ms
27s28s

14. SCALABILITY! BUT AT WHAT COST?

15. SCALABILITY! BUT AT WHAT COST?

16. NOW, WE ARE GOING TO INTRODUCE SOME
NEW CANDIDATES FOR "STATE-OF-THE-ART"
ALGORITHMS, BY WRITING SOME FOR-
LOOPS AND RUNNING THEM ON MY LAPTOP.
Frank McSherry
SCALABILITY! BUT AT WHAT COST?

17. SCALABILITY! BUT AT WHAT COST?

18. SCALABILITY! BUT AT WHAT COST?

19. SCALABILITY! BUT AT WHAT COST?

20. THERE ARE NO PROBLEMS ONLY OPPORTUNITIES
source: xkcd

21. PERFORMANCE?
PERFORMANCE!
‣ Intuition about what is possible
‣ Impact on real life Java code

22. CPU FUNCTIONAL BLOCKS
source: Martin Thompson’s blog

23. 2 SOCKET SERVER

24. CPU FUNCTIONAL BLOCKS
NUMA domain 0 NUMA domain 1

25. AMD EPYC DIE

26. AMD EPYC DIE

27. AMD EPYC DIAGRAM
./lstopo --physical --output-format png /tmp/lstopo.png

28. LATENCY EXPERIMENT
CORE 0
CORE 0 “LOCAL” RAM
CORE 1
CORE N
CAS
CAS
CAS
CAS = UNSAFE.compareAndSwapInt
…

29. AMD EPYC CORE PING LATENCY (JAVA-JMH)
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>1.21</version>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>1.21</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.xerial</groupId>
<artifactId>jnuma</artifactId>
<version>0.1.3</version>
</dependency>

30. AMD EPYC CORE PING LATENCY (JAVA-JMH)
@Setup
public void setUp(CoreSequence coreSequence) {
Numa.runOnNode(node);
ByteBuffer buffer = Numa.allocOnNode(capacity, node);
long address = address(buffer);
}

31. AMD EPYC CORE PING LATENCY (JAVA-JMH)
@Benchmark
@Group("pingpong")
public void ping(Control cnt, GroupState groupState, CoreAssigner coreAssigner) {
while (
!cnt.stopMeasurement &&
!UNSAFE.compareAndSwapInt(null, groupState.flagAddress, 0, 1)
) {
// this body is intentionally left blank
}
}
@Benchmark
@Group("pingpong")
public void pong(Control cnt, GroupState groupState, CoreAssigner coreAssigner) {
while (
!cnt.stopMeasurement &&
!UNSAFE.compareAndSwapInt(null, groupState.flagAddress, 1, 0)
) {
// this body is intentionally left blank
}
}

32. AMD EPYC CORE PING LATENCY (JAVA-JMH)

33. AMD EPYC CORE PING LATENCY (JAVA-JMH)

34. INTEL XEON PLATINUM PING LATENCY (JAVA-JMH)

35. NUMA
RAM READ THROUGHPUT
A. <500 MB/s
B. <2GB/s
C. <20GB/s
D. ~100GB/s

36. NUMA
RAM READ THROUGHPUT - PREDICTABLE PATTERN
@Benchmark
public int readByteBufferInOrder() {
Numa.runOnNode(threadNumaNode);
int result = 0;
for (int i = 0; i < byteBuffer.limit(); i += 8) {
result += byteBuffer.getLong(i);
}
return result;
}

37. AMD MEMORY BANDWIDTH

38. BRANCH PREDICTION
RAM THROUGHPUT - READ 400MB OF DATA
A. ~1s
B. 0.348s
C. 0.150s
D. It depends

39. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
Selectivity (%)
0 10 20 30 40 50 60 70 80 90 100
533ms
43ms 43ms

40. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainIf() {
int result = 0;
for (int i = 0; i < inputSize; i++) {
if (onHeapValues.getInt(i * 4) < predicate) {
result++;
}
}
return result;
}
SIMPLE PREDICATE

41. LESLIE LAMPORT

42. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
MASK = 0111 1111
~MASK = 1000 0000
42 = 0010 1010
13 = 0000 1101
13 ^ MASK = 0111 0010
+ 42 = 1001 1100
&~MASK = 1000 0000

43. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainIf() {
int result = 0;
for (int i = 0; i < inputSize; i++) {
if (onHeapValues.getInt(i * 4) < predicate) {
result++;
}
}
return result;
}
SIMPLE PREDICATE

44. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainIf() {
int result = 0;
for (int i = 0; i < inputSize; i++) {
result += RETURN_1_IF_MATCH(
onHeapValues.getInt(i * 4),
predicate
)
}
return result;
}
BRANCHLESS (PSEUDOCODE)

45. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
@Benchmark
public int plainArithmetic() {
int result = 0;
int mask = 0b0111_1111;
for (int i = 0; i < inputSize; i++) {
int x = onHeapValues.getInt(i * 4) ^ mask;
int z = (predicate + x) & ~mask;
result += Integer.bitCount(z);
}
return result;
}
BRANCHLESS

46. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
0 10 20 30 40 50 60 70 80 90 100
CONDITIONAL ARITHMETIC
97ms
533ms
43ms 43ms

47. MULTI BYTE PROCESSING WITH FULL WORD INSTRUCTIONS
0
150MS
300MS
450MS
600MS
0 10 20 30 40 50 60 70 80 90 100
CONDITIONAL ARITHMETIC
PACKED
11ms
97ms
533ms
43ms 43ms

48. BRANCHLESS DOOM

49. BRANCHLESS DOOM
“The mov-only DOOM renders approximately
one frame every 7 hours,
so playing this version requires somewhat increased patience.”

50. CONCLUSION
‣ THERE ARE TWO HATS YOU CAN WEAR
‣ 0.1X CONCENTRATES ON USER VALUE
‣ 10X ON TECHNOLOGY
‣ THESE ARE NOT EXCLUSIVE
‣ LEARN ALGORITHMS AND DATA STRUCTURES (LINDY’S!)
‣ EXPERIMENT
‣ DEVELOP INTUITION OF WHAT IS POSSIBLE

51. PAPERS
REFERENCES
‣Efficiently Compiling Efficient Query Plans for Modern
Hardware
‣Constant-Time Query Processing
‣https://danluu.com/branch-prediction/
‣StarCraft II as an AI research environment
‣Multiple Byte Processing with FullWord Instructions
‣http://www.vldb.org/2017/keynotes.php

52. COPYRIGHTS
COPYRIGHTS
▸ Houston opportunities by XKCD.com
▸ A/B Testing example: https://en.wikipedia.org/wiki/A/
B_testing#/media/File:A-B_testing_example.png by Maxime
Lorant
▸ AMD for EPYC diagrams
▸ Martin Thompson (https://mechanical-
sympathy.blogspot.com/2013/02/cpu-cache-flushing-
fallacy.html) for 2 socket CPU diagram