Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Deploying scikit-learn
Models in Production
Rajat Arya (@rajatarya)
Product Manager, Dato Inc.
1
2
Dato provides a platform for building intelligent
apps
Data
Engineering
Data
Intelligence
Deployment
• Fast & scalable
•...
3
DATA
ML
Algorithm
How Everyone Starts with ML
• Running experiments
• Plots are the results
• Not clear how to get this ...
4
DATA
ML
Algorithm
Deployment?
• Write a spec for other team to
implement in ‘production’ language
• Translate code in 6-...
5
Current Challenges
• Machine Learning Models
are opaque objects
• Export format like PMML
don’t support many
models
• Fo...
6
Starting from the Beginning
GOAL: Handle live production traffic directly served from
the trained machine learning model...
One: Easy to Integrate
• REST APIs for both querying
and management
• Have client libraries in other
languages (no Python ...
Two: High Performance
• Utilize Load Balancer for
distributing request load
• Integrated distributed cache
so repeated que...
Three: Fault Tolerant
• Model running on many
machines
• System operational during
node failure
9
App
A
P
I
C
A
C
H
E
A
P
...
Four: Scalable
• Elastic scale nodes in cluster
up and down
• Easy to configure, cache
automatically updates with
cluster ...
Five: Maintainable
• Zero downtime during model
deployment
• Metrics & logs
• Model management
11
App
A
P
I
C
A
C
H
E
A
P
...
Six: Extensible
• Arbitrary Python
• Use any set of Python
packages
• Model ensembling
12
App
A
P
I
C
A
C
H
E
A
P
I
C
A
C
...
13
Requirements Recap
1. Easy to Integrate
2. High Performance
3. Fault Tolerant
4. Scalable
5. Maintainable
6. Extensible...
14
Do-It-Yourself
• Web Service layer:
- Tornado, Flask, Keen, Django, etc
• Caching layer:
- Redis, Cassandra, Memcached,...
15
… or use Dato Predictive Services
We set out with this goal, and used these requirements
… and now I'd like to show it ...
DEMO: Deploying a scikit-learn model using
Dato Predictive Services
16
17
Models as Services
• Deploy models as low-latency REST services
• Elastically scale up or out with one command
• Monito...
Upcoming SlideShare
Loading in …5
×

Py data scikit-production

  • Be the first to comment

Py data scikit-production

  1. 1. Deploying scikit-learn Models in Production Rajat Arya (@rajatarya) Product Manager, Dato Inc. 1
  2. 2. 2 Dato provides a platform for building intelligent apps Data Engineering Data Intelligence Deployment • Fast & scalable • Rich data type support • Visualization • App-oriented ML • Supporting utils • Extensibility • Batch & always-on • RESTful interface • Elastic & robust Build, deploy, & manage your intelligent apps with Dato.
  3. 3. 3 DATA ML Algorithm How Everyone Starts with ML • Running experiments • Plots are the results • Not clear how to get this deployed
  4. 4. 4 DATA ML Algorithm Deployment? • Write a spec for other team to implement in ‘production’ language • Translate code in 6-12 months • Stale / irrelevant model implemented • Two teams maintaining two systems Custom Model Data Engineers, Data Architects, DevOps, App Developers App A P I Data Scientist
  5. 5. 5 Current Challenges • Machine Learning Models are opaque objects • Export format like PMML don’t support many models • Focus on training, not prediction
  6. 6. 6 Starting from the Beginning GOAL: Handle live production traffic directly served from the trained machine learning model What are the requirements if we wanted to build a similar architecture for ML Models?
  7. 7. One: Easy to Integrate • REST APIs for both querying and management • Have client libraries in other languages (no Python lock-in) 7 App A P I
  8. 8. Two: High Performance • Utilize Load Balancer for distributing request load • Integrated distributed cache so repeated queries are only answered once 8 App A P I C A C H E A P I C A C H E Engine A P I C A C H E LB
  9. 9. Three: Fault Tolerant • Model running on many machines • System operational during node failure 9 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Engine Engine Engine
  10. 10. Four: Scalable • Elastic scale nodes in cluster up and down • Easy to configure, cache automatically updates with cluster changes 10 App A P I C A C H E A P I C A C H E LB GLC Model GLC Model Engine Engine A P I C A C H E Engine A P I C A C H E Engine
  11. 11. Five: Maintainable • Zero downtime during model deployment • Metrics & logs • Model management 11 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Engine Engine Engine
  12. 12. Six: Extensible • Arbitrary Python • Use any set of Python packages • Model ensembling 12 App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Python Python Python
  13. 13. 13 Requirements Recap 1. Easy to Integrate 2. High Performance 3. Fault Tolerant 4. Scalable 5. Maintainable 6. Extensible App A P I C A C H E A P I C A C H E A P I C A C H E LB GLC Model GLC Model GLC Model Python Python Python
  14. 14. 14 Do-It-Yourself • Web Service layer: - Tornado, Flask, Keen, Django, etc • Caching layer: - Redis, Cassandra, Memcached, DynamoDb, BerkeleyDb, MySQL, etc • Logs: - Logback, LogStash, Splunk, Loggly • Metrics: - AWS CloudWatch, Mixpanel, Librato, etc
  15. 15. 15 … or use Dato Predictive Services We set out with this goal, and used these requirements … and now I'd like to show it to you.
  16. 16. DEMO: Deploying a scikit-learn model using Dato Predictive Services 16
  17. 17. 17 Models as Services • Deploy models as low-latency REST services • Elastically scale up or out with one command • Monitoring & Model Management • Deploy existing Python models • Run on AWS EC2 or Hadoop YARN Dato Predictive Services Predictive Engine REST Client Direct Model Mgmt

×