Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production
•
1 like•1,742 views
Empowering the Data Scientist with "1-Click" Production Deployment and Canary Testing of High-Performance and Highly-Scalable Spark ML and TensorFlow Models directly from Jupyter/iPython Notebooks using Docker, Kubernetes, Netflix OSS, Microservices, and Spinnaker. With proper tooling and metrics, Data Scientists can directly deploy, analyze, A/B test, rollback, and scale out their Spark ML and TensorFlow model into live production serving with zero friction. We will show you the open source tools that we've built based on Docker, Kubernetes, Netflix Open Source, Microservices, Spinnaker - and even Chaos Monkey! Speaker: Chris Fregly @ PipelineIO, formerly Databricks and Netflix
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production
Similar to Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production (20)
More from Chris Fregly
More from Chris Fregly (20)
Recently uploaded
Recently uploaded (20)
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production
- 1. ADVANCED SPARK AND TENSORFLOW MEETUP Updates and Announcements August 4, 2016 advancedspark.com
- 2. MEETUP METRICS • 4,000 Members in 1Year!? • Github Repo: 600 Stars, 230 Forks • DockerHub Repo: 5,400 Pulls!!
- 3. MEETUP AGENDA • Deploying and Scaling ML Models with PipelineIO Open Source (Chris Fregly) • Fundamental Algorithms of Neural Networks (Sam Abrahams)
- 4. PIPELINE.IO ExtendingYour ML Pipelines into Production 100% Open Source! http://github.com/fluxcapacitor/pipeline
- 5. BRAINSTORMING AND VALIDATING • Major Gaming Company • Large Ride Sharing Service • Popular Q & A Site • Online Clothing Retailer • DominantVideo Streaming
- 6. PIPELINE.IO FOCUS • Model Deploying andTesting • Model Scaling and Serving • Online ModelTraining • Dynamic Model Optimizing
- 7. MODEL DEPLOYING AND TESTING Continuously Test and Deploy Models in Production!
- 8. DEMO! Deploy Spark ML DecisionTree to Production Deploy to Cloud or On-Premise!
- 9. MODEL SCALING AND SERVING
- 10. DEMO! Circuit Breakers and Request Batching
- 11. ONLINE MODEL TRAINING • Continuous, Incremental, and Partial Training • Kafka + Spark Streaming + Spark ML • Real-time, Dynamic Recommendations
- 13. DYNAMIC MODEL OPTIMIZING Generate Optimized Code from Spark ML!
- 14. DEMO! Dynamic Code Generation of DecisionTree
- 16. WE’RE HIRING!! • Kafka, Spark ML, and TensorFlow Contributors • Systems Engineers • GPU/CUDA Engineers • C++, Java, Scala, Python ONLY NICE PEOPLE!!
- 17. SANTA CLARA WORKSHOP, AUG 6, 2016