1. VELOX:
MODELS IN ACTION
Presented by Dan Crankshaw
crankshaw@cs.berkeley.edu
Henry Milner, Joseph Gonzalez, Peter Bailis, Haoyuan Li, Tomer Kaftan,
Zhao Zhang, Ali Ghodsi, Michael Franklin, Michael Jordan, and Ion Stoica
https://amplab.cs.berkeley.edu/projects/velox/
2. MODELS AT REST
Data
Well
Studied
Train
Observe
Predictions Model Predict
24. What’s wrong?
1. Built from scratch for each
application
2. Different systems
25. What’s wrong?
1. Built from scratch for each
application
2. Different systems
3. Space inefficient
26. What’s wrong?
1. Built from scratch for each
application
2. Different systems
3. Space inefficient
4. Stale predictions
27. What’s wrong?
1. Built from scratch for each
application
2. Different systems
3. Space inefficient
4. Stale predictions
5. The T-Swift effect Sample Bias
28. Catify: Music for Cats
Pipeline
Tachyon + HDFS
NGINX
Node.js App Server
MongoDB
Training Data
New Model
34. BENEFITS
1. Low-latency and scalable
predictions as a service
2. Integrated approach leads to
fresher, better predictions
35. BENEFITS
1. Low-latency and scalable
predictions as a service
2. Integrated approach leads to
fresher, better predictions
3. Easy translation to production
predictions
36. BENEFITS
1. Low-latency and scalable
predictions as a service
2. Integrated approach leads to
fresher, better predictions
3. Easy translation to production
predictions
4. Eases operational pain
65. ACTIVE LEARNING: LinUCB
Rating
Songs
Uncertainty
Prediction
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10:
Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
66. ACTIVE LEARNING: LinUCB
Rating
Songs
Look at upper
confidence bound
Uncertainty
Prediction
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10:
Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
67. ACTIVE LEARNING: LinUCB
Rating
Songs
Look at upper
confidence bound
Uncertainty
Prediction
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. WWW '10:
Proceedings of the 19th international conference on World wide web, New York, New York, USA: ACM. doi:10.1145/1772690.1772758
72. Catify: Music for Cats
Pipeline
Tachyon + HDFS
NGINX
Node.js App Server
MongoDB
Training Data
New Model
73. USER-FACING API
GET
/velox/catify/predict?userid=22&song=27632
GET
/velox/catify/predict_top_k?userid=22&k=100
74. USER-FACING API
GET
/velox/catify/predict?userid=22&song=27632
GET
/velox/catify/predict_top_k?userid=22&k=100
POST
/velox/catify/observe?userid=22&song=27632?score=3.7
82. The future of research in scalable learning systems will be in the
integration of the learning lifecycle:
Data
Training
Feedback
Predictions Model Serving
84. SUMMARY
•Model training and predictions rely on ad-hoc,
manual processes spread across multiple systems
85. SUMMARY
•Model training and predictions rely on ad-hoc,
manual processes spread across multiple systems
•The Velox system automatically maintains multiple
models while providing low latency, scalable, and
personalized predictions
86. SUMMARY
•Model training and predictions rely on ad-hoc,
manual processes spread across multiple systems
•The Velox system automatically maintains multiple
models while providing low latency, scalable, and
personalized predictions
•Velox is part of BDAS, is coming soon…
87. SUMMARY
•Model training and predictions rely on ad-hoc,
manual processes spread across multiple systems
•The Velox system automatically maintains multiple
models while providing low latency, scalable, and
personalized predictions
•Velox is part of BDAS, is coming soon…
•https://amplab.cs.berkeley.edu/projects/velox/