Developing and experimenting with machine learning models in Python is easy and well supported by robust and agile libraries such as scikit-learn, although efficiently deploying multi-model systems at scale is still a challenge in the data science field.
This talk will focus on the main issues related to deploying machine learning models and how to make scikit-learn production-ready with minimal operational efforts, by means of Cloud Computing services, in particular Amazon Web Services.
Prerequisites: basic Machine Learning understanding (modeling and training), minimal knowledge about scikit-learn and Python utilities such as Pandas and boto.
How to deploy machine learning models in the Cloud
1. How to deploy Machine Learning
models in the Cloud
Alex Casalboni
PYCON SETTE
@ Firenze
clda.co/pycon7-alex
2. About Me
PYCON SETTE @ FIRENZE
@alex_casalboni
clda.co/pycon7-alex
Bachelor in Computer Science
Master in Sound & Music Engineering
6+ years of Web Development
3+ years of Python
3. Have you ever met a Data Scien@st?
PYCON SETTE @ FIRENZE
Bob
Bob leads a skilled team of data scienCsts
Bob has built a great ML model
Bob loves Python
Bob likes user-‐centered ML
clda.co/pycon7-alex
4. How did Bob build his model in Python?
PYCON SETTE @ FIRENZE
clda.co/pycon7-code
clda.co/pycon7-alex
Bob
5. PYCON SETTE @ FIRENZE
GPU availability
distributed compuCng
infrastructure-‐aware code
1. Bob needs Elas@city
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
6. PYCON SETTE @ FIRENZE
MulC-‐model architecture
A/B tesCng & versioning
RESTful API
2. Bob needs plenty of Features
Auth layer
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
7. PYCON SETTE @ FIRENZE
infrastructure maintenance
conCnuous delivery & deployments
automaCon, performance & monitoring
3. Data Scien@sts ain’t DevOps
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps
8. PYCON SETTE @ FIRENZE
model design, training & tesCng
code reviews & debugging
data exploraCon
4. DevOps ain’t Data Scien@sts
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps
9. Deployment Strategies
PYCON SETTE @ FIRENZE
1. Web-‐app controller
clda.co/pycon7-alex
how will Bob update his model(s)?
same website auth layer?
shared (limited) uWSGI processes?
It’s disarmingly simple, but…
Bob
10. Deployment Strategies
PYCON SETTE @ FIRENZE
2. Fleet of servers
clda.co/pycon7-alex
same problems as before
Bob has many more machines to maintain
sCll no elasCcity (over-‐provisioning)
Bigger capacity and no code changes, but…
Bob Bob2 Bob3
Load Balancing
11. Deployment Strategies
PYCON SETTE @ FIRENZE
3. Auto Scaling
clda.co/pycon7-alex
<Bob>
sCll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elasCcity and availability , but…
AWS ELB + Auto Scaling
(or maybe ElasCc Beanstalk?)
12. Deployment Strategies
PYCON SETTE @ FIRENZE
4. Serverless
clda.co/pycon7-alex
+
Amazon API
Gateway
AWS
Lambda
Happy
Bob
Infinite scalability, global CDN and caching
Transparent elasCcity (and PAYG)
RESTful & auth layer + versioning (for free!)
Bob doesn’t need to be a DevOps anymore!
13. The Serverless Cloud
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
There is always a server somewhere,
you just don't have to worry about it :)
14. Does Bob’s code change a lot?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Basically, no change!
Bob
15. How does Bob keep the model updated?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Bob
Hint: use gzip!
16. PYCON SETTE @ FIRENZE
clda.co/pycon7-lambda
clda.co/pycon7-alex
17. Serverless Cloud Domina@on
PYCON SETTE @ FIRENZE
Amazon Web Services (Lambda)
Microso` Azure (FuncCons)
Google Cloud Plaborm (FuncCons)
IBM Bluemix (OpenWhisk)
hook.io
Cloud PlaSorms Hybrid & Microservices
stackhut.com
webtask.io
iron.io
clda.co/pycon7-alex