The Case for Learned Index Structures paper reading & sharing

1. The Case for
Learned Index Structures
Eric Fu

2. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

3. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

4. Introduction
• Index: map key to position efficiently
• B-Tree
• Self-balanced binary search tree
• Store on disk
• Lookup in O(log n)

5. B-Tree vs. Models
• Task: Predict the offset of value given a key
• Input: key
• Output:
• B-Tree: [pos, pos + pagesize]
• Model: [pos - min_err, pos + max_err]

6. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

7. What is Machine Learning?
• Machine learning is a field of computer science that
gives computers the ability to learn without being explicitly
programmed.
• Statistics: collect data  build model  predict

8. Problems
• Regression 回归
• Classification 分类
• Clustering 聚类 Clustering

9. Algorithms
• Linear Regression 线性回归
• Decision Tree 决策树
• Neural Network 神经网络
• Support Vector Machine (SVM) 支持向量机
• Bayes Classifier 贝叶斯分类器
• K-means
.......

10. Linear Regression 线性回归

11. Decision Tree 决策树

12. Neuron 神经元
Activation Function

13. Neural Network 神经网络
neuron

14. Deep Neural Network 深度神经网络
GoogLeNet, 22 layers

15. Machine Learning
• A regular process
• Feature extraction
• Train model
• Test model
• Objective function
• minimum error (e.g. MSE)

16. Machine Learning
• The biggest challenge - overfitting

17. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

18. B-Tree vs. Models
• Task: Predict the offset of value given a key
• Input: key
• Output:
• B-Tree: [pos, pos + pagesize]
• Model: [pos - min_err, pos + max_err]
How to bound
min_err, max_err? No test dataset!

19. Index as a Function
• B-Tree or ML model are fitting this curve in different approach.

21. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

22. Recursive Model Index (RMI)
Root and middle nodes
• Pick a model for next stage
Leaf nodes
• Predict position

23. Recursive Model Index (RMI)
Solved last-mile dilemma!
• 100M - 1M - 10K – 100
• Not a tree

24. Hybrid Index
Parameter: model structures

25. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

26. Test datasets
• weblogs：访问时间 timestamp -> log entry (~200M)
• maps：纬度 longitude -> locations (~200M)
• web-documents：documents (strings) -> document-id (~10M)
• lognormal：

27. Test models
• B-Tree with different page sizes
• very competitive performance
• RMI with 2-stage models using simple grid-search
• 0 to 2 hidden layers
• layer-width ranging from 4 to 32 nodes
• Total time = lookup time + search time

31. Conclusion
• Up to 3x faster
• An order-of-magnitude smaller
• Data distribution dependent

32. Inserts and Updates
• Achilles heel of learned indexes because of the potentially high cost
for learning models
• Introduce additional space in sorted dataset, similar to a B-Tree
• Assume that the inserts follow roughly a similar pattern as the
learned CDF
• What happens if the distribution changes?
• Retrain model. Stage 2 -> Stage 1
• Delta-index

33. Agenda
• Introduction
• Background
• Range Index
• RM-Index
• Performance
• Point Index & Existence Index

34. Point Index
• Hash collisions
• Probing (e.g. linked list)
• Trade-off between time and space
• Learned Hash-map
• more uniform hash function
• more uniquely mapping

35. Point Index
• Baseline Hash-map
• only uses 2 multiplications,
3 bitshifts and 3 XORs
• 2-stage RMI models
• 100k models on the 2nd stage
• without any hidden layers.
• available slots from 75% to
125% of the data

36. Existence Index
• most importantly Bloom-Filters
• Guarantee no false negative
• Potential false positive (FP)
• Targeted FPR = 0.1% then ~14x bits
• Targeted FPR = 0.01% then ~18x bits

37. Bloom-filters with learned hash-functions
• We denote the set of keys by K and the set of non-keys by U
• Dataset of non-keys U
• randomly generated keys
• based on logs of previous queries
• generated by another ML model
• A binary classification task
• NN with Sigmoid activation function

38. Bloom-filters with learned hash-functions
• non-zero FPR and FNR
• as the FPR goes down, the FNR will go up
• How to preserve FPR = 0 constraint?
• an overflow Bloom filter

39. Bloom-filters with learned hash-functions
• Task: blacklisted phishing URLs (~1.7M)
• a character-level RNN (GRU)
• Bloom filter
• desired 1% FPR requires 2.04MB space
• Our approach
• GRU model: 0.0259MB
• With spill-over Bloom filter: 1.07MB (save ~47%)

40. Thank You
Q&A

41. Furthermore...
• Machine Learning by Zhou Zhi-hua
• TensorFlow and deep learning without a PhD