Smokey and the Multi-Armed Bandit Featuring BERT Reynolds Updated

•

1 like•1,943 views

Chris Fregly

Software

Chris Fregly
Developer Advocate
AI and Machine Learning
@AWS
Smokey and the Multi-Armed Bandit
Featuring BERT Reynolds

Abstract
First, I will train and deploy multiple natural language understanding
(NLU) models and compare them in live production using reinforcement
learning to dynamically shift traffic to the winning model.
Second, I will describe the differences between A/B and multi-armed
bandit tests including exploration-exploitation, reward-maximization, and
regret-minimization.
Third, I will dive deep into the details of building and scaling a multi-
armed bandit deployment on AWS using a real-time, stream-based text
classifier with TensorFlow, PyTorch, and BERT on 150+ million reviews
from the Amazon Customer Reviews Dataset.

Me Developer Advocate
AI and Machine Learning @ AWS
(Based in San Francisco)
Co-Author of the O'Reilly Book,
"Data Science on AWS."
Founder of the Advanced
Kubeflow Meetup (Global)
https://www.datascienceonaws.com
data-science-on-aws
@cfregly
linkedin.com/in/cfregly
https://meetup.com/Data-Science-on-AWS

Data Science on AWS – Book and Workshop Outline
https://www.datascienceonaws.com/

Agenda
• Compare A/B Tests vs. Multi-Armed Bandit Tests
• Optimize Bandits with Reinforcement Learning
• Train 2 BERT Languge Models with TensorFlow
• Train a Multi-Armed Bandit Model with Vowpal Wabbit
• Test 2 BERT Models with a Bandit
• DEMO: Scale Multi-Armed Bandits on AWS

Traditional A/B Tests
• Static
• Cannot Add New Models After Test Begins
• Static Traffic Split Between Models A and B
• May Negatively Impact Business Metrics
• Must Run Experiment to Completion
• No Concept of Reward for Winning Model

Multi-Armed Bandit Tests
• Add New Models
• Dynamically Shift Traffic
• Explore-Exploit Strategy
• Finish Experiment Early - or Run Longer!
• Minimize Regret (Business Impact)
• Maximize Reward

Train 2 BERT Models with TensorFlow (Models A & B)
• BERT Mania!
• Fine-Tuning BERT

Train a Bandit Model with Reinforcement Learning (RL)
• Popular Reinforcement Learning Strategies
• Epsilon Greedy
• Thompson’s Sampling
• Online Cover
• Bagging
• Implemented in Vowpal Wabbit (VW)!
• Try Our Open Source RL Containers
• https://github.com/aws/sagemaker-rl-container

Test 2 BERT Models with a Multi-Armed Bandit Model

DEMO: Scale Multi-Armed Bandits on AWS
• BERT Model 1: TensorFlow
• BERT Model 2: PyTorch

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DEMO!

More Resources
• O’Reilly Book - Data Science on AWS – Early Release Available!
• https://datascienceonaws.com
• GitHub Repo
• https://github.com/data-science-on-aws/workshop
• AWS Blog Post on Multi-Armed Bandits
• https://aws.amazon.com/blogs/machine-learning/power-contextual-bandits-using-continual-learning-
with-amazon-sagemaker-rl/
• Bandit Algorithms
• https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Contextual-Bandit-algorithms
• Open Source SageMaker Reinforcement Learning Containers
• https://github.com/aws/sagemaker-rl-container

Thank you!
© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Chris Fregly
data-science-on-aws
@cfregly
linkedin.com/in/cfregly

What's hot

Tensorflow in production with AWS Lambda

Fabian Dubois

PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...

Chris Fregly

Yannis Zarkadas. Enterprise data science workflows on kubeflow

MarynaHoldaieva

With the breadth of sheer functionalities which need to be addressed in the Machine Learning world around building, training, serving and managing models, getting it done in a consistent, composable, portable, and scalable manner is hard. The Kubernetes framework is well suited to address these issues, which is why it's a great foundation for deploying ML workloads. Kubeflow is designed to take advantage of these benefits. In this talk, we are going to address how to make it easy for everyone to develop, deploy, and manage portable, scalable ML everywhere and support the full lifecycle Machine Learning using open source technologies like Kubeflow, Tensorflow, PyTorch,Tekton, Knative, Istio and others. We are going to discuss how to enable distributed training of models, model serving, canary rollouts, drift detection, model explainability, metadata management, pipelines and others. Additionally we will discuss Watson productization in progress based on Kubeflow Pipelines and Tekton, and point to Kubeflow Dojo materials and follow-on workshops.

End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage

Animesh Singh

H2O è la piattaforma di analisi dati distribuita più interessante degli ultimi anni, inserita da Gartner come Visionary nella Data Science. Permette di eseguire elaborazioni massivamente parallele, e si collega facilmente a piattaforme essenziali per il calcolo distribuito ed l'analisi predittiva, incluse Spark, TensorFlow e MxNet. Nel talk conosceremo l'architettura di questa libreria, le sue funzionalità e potenzialità, tramite demo interattive con Spark, TensorFlow, ed alcuni algoritmi particolarmente in voga.

Gabriele Nocco - Massive distributed processing with H2O - Codemotion Milan 2017

Codemotion

Developing and experimenting with machine learning models in Python is easy and well supported by robust and agile libraries such as scikit-learn, although efficiently deploying multi-model systems at scale is still a challenge in the data science field. This talk will focus on the main issues related to deploying machine learning models and how to make scikit-learn production-ready with minimal operational efforts, by means of Cloud Computing services, in particular Amazon Web Services. Prerequisites: basic Machine Learning understanding (modeling and training), minimal knowledge about scikit-learn and Python utilities such as Pandas and boto.

How to deploy machine learning models in the Cloud

Alex Casalboni

What do the terms serverless, containers, and virtual machines mean? Which should I use to build my app? The answer (as always) is "it depends." In this session learn the tradeoffs between these different approaches, whether you're building your app from scratch or want to move an existing web or mobile application to the cloud. We'll discuss open source tools such as Kubernetes, Istio, and Knative, and we'll discuss Google Cloud Platform tools like Compute Engine, Google Kubernetes Engine (GKE), App Engine, and Cloud Functions.

Where should I run my code? Serverless, Containers, Virtual Machines and more

Bret McGowen - NYC Google Developer Advocate

From Functions-as-a-Service to Backend-as-a-Service, even Big Data-as-a-Service, Serverless is taking many different shapes. Learn what these mean and how Google Cloud Platform is building technology to make sure there's nothing standing between you and running your code. You'll see live demos of integration between Firebase, Cloud Functions, Cloud Pub/Sub (and even machine learning) to build autoscaling apps in record time - all without managing servers or application runtimes. Bret is on the Google Cloud Platform team at Google, focusing on serverless products like Google Cloud Functions, App Engine, Firebase, machine learning APIs, and more. He's often on the running trail, volleyball court or kickball field.

Serverless with Google Cloud

Bret McGowen - NYC Google Developer Advocate

Whizlabs webinar - Deploying Portfolio Site with AWS Serverless

Dhaval Nagar

Infrastructure as Code and AWS CDK

SupratipBanerjee

The slides used during the mlops.community meetup on KFServing. We looked inside some popular model formats like the SavedModel of Tensorflow and the Model Archiver of PyTorch, to understand how the weights of the NN are saved there, the graph and the signature concepts. We discussed the relevant resources of the deployment stack of Istio (the ingress gateway, the sidecar and the virtual service) and Knative (the service and revisions), as well as Kubeflow and KFServing. Then we got into the design details of KFServing, its custom resources, its controller. Then we spent some time to discuss the monitoring stack, the metrics of the servable as well as the model metrics which end up to Prometheus. We looked at the inference payload and prediction logging to observe drifts and trigger the retraining of the pipeline. Finally, a few words about the awesome community and the roadmap of the project on multi-model serving and inference routing graph.

Serving models using KFServing

Theofilos Papapanagiotou

Dip into prometheus

Zaar Hai

Speaker: Ben Hall Title: Machine Learning Workloads with Kubernetes and Kubeflow Abstract: Kubeflow, a Machine Learning Toolkit, supports developers and data scientists when developing and training Tensorflow-based workloads. In this talk, Ben will demonstrate how the recent Kubeflow 0.1 release allows everyone to capture the power of Kubernetes, allowing models to be developed, trained across 1000s of machines before being served to consumers. Bio: Ben is the founder of Katacoda and Ocelot Uproar, a company focused on training and building products loved by users. Ben enjoys looking for the next challenges to solve. He tweets at @Ben_Hall and blogs at blog.benhall.me.uk Thanks to all TensorFlow London meetup organisers and supporters: Seldon.io Altoros Rewired Google Developers Rise London

TensorFlow London 14: Ben Hall 'Machine Learning Workloads with Kubernetes an...

Seldon

IaC on AWS Cloud

Bhuvaneswari Subramani

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2rm4hFD. Yevgeniy Brikman talks about how to write automated tests for infrastructure code, including the code written for use with tools such as Terraform, Docker, Packer, and Kubernetes. Topics covered include: unit tests, integration tests, end-to-end tests, dependency injection, test parallelism, retries and error handling, static analysis, property testing and CI / CD for infrastructure code. Filmed at qconsf.com. Yevgeniy Brikman is the co-founder of Gruntwork, a company that provides DevOps as a Service. He is the author of two books published by O'Reilly Media: Hello, Startup and Terraform: Up & Running. Previously, he worked as a software engineer at LinkedIn, TripAdvisor, Cisco Systems, and Thomson Financial.

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

C4Media

Data Science, Machine Learning, and Artificial Intelligence has exploded in popularity in the last five years, but the nagging question remains, “How to put models into production?” Engineers are typically tasked to build one-off systems to serve predictions which must be maintained amid a quickly evolving back-end serving space which has evolved from single-machine, to custom clusters, to “serverless”, to Docker, to Kubernetes. In this talk, we present KubeFlow- an open source project which makes it easy for users to move models from laptop to ML Rig to training cluster to deployment. In this talk we will discuss, “What is KubeFlow?”, “why scalability is so critical for training and model deployment?”, and other topics. Users can deploy models written in Python’s skearn, R, Tensorflow, Spark, and many more. The magic of Kubernetes allows data scientists to write models on their laptop, deploy to an ML-Rig, and then devOps can move that model into production with all of the bells and whistles such as monitoring, A/B tests, multi-arm bandits, and security.

Introducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)

DataWorks Summit

Title Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + Kubernetes + PyTorch + XGBoost + Airflow + MLflow + Spark + Jupyter + TPU Video https://youtu.be/vaB4IM6ySD0 Description In this workshop, we build real-world machine learning pipelines using TensorFlow Extended (TFX), KubeFlow, and Airflow. Described in the 2017 paper, TFX is used internally by thousands of Google data scientists and engineers across every major product line within Google. KubeFlow is a modern, end-to-end pipeline orchestration framework that embraces the latest AI best practices including hyper-parameter tuning, distributed model training, and model tracking. Airflow is the most-widely used pipeline orchestration framework in machine learning. Pre-requisites Modern browser - and that's it! Every attendee will receive a cloud instance Nothing will be installed on your local laptop Everything can be downloaded at the end of the workshop Location Online Workshop Agenda 1. Create a Kubernetes cluster 2. Install KubeFlow, Airflow, TFX, and Jupyter 3. Setup ML Training Pipelines with KubeFlow and Airflow 4. Transform Data with TFX Transform 5. Validate Training Data with TFX Data Validation 6. Train Models with Jupyter, Keras/TensorFlow 2.0, PyTorch, XGBoost, and KubeFlow 7. Run a Notebook Directly on Kubernetes Cluster with KubeFlow 8. Analyze Models using TFX Model Analysis and Jupyter 9. Perform Hyper-Parameter Tuning with KubeFlow 10. Select the Best Model using KubeFlow Experiment Tracking 11. Reproduce Model Training with TFX Metadata Store and Pachyderm 12. Deploy the Model to Production with TensorFlow Serving and Istio 13. Save and Download your Workspace Key Takeaways Attendees will gain experience training, analyzing, and serving real-world Keras/TensorFlow 2.0 models in production using model frameworks and open-source tools. Related Links 1. PipelineAI Home: https://pipeline.ai 2. PipelineAI Community Edition: http://community.pipeline.ai 3. PipelineAI GitHub: https://github.com/PipelineAI/pipeline 4. Advanced Spark and TensorFlow Meetup (SF-based, Global Reach): https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup 5. YouTube Videos: https://youtube.pipeline.ai 6. SlideShare Presentations: https://slideshare.pipeline.ai 7. Slack Support: https://joinslack.pipeline.ai 8. Web Support and Knowledge Base: https://support.pipeline.ai 9. Email Support: support@pipeline.ai

Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...

Chris Fregly

Kubeflow Distributed Training and HPO

Animesh Singh

Rene Groeschke

CodeFest

100% Puppet Cloud Deployment of Legacy Software

Puppet

What's hot (20)

Tensorflow in production with AWS Lambda

PipelineAI Optimizes Your Enterprise AI Pipeline from Distributed Training to...

Yannis Zarkadas. Enterprise data science workflows on kubeflow

End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage

Gabriele Nocco - Massive distributed processing with H2O - Codemotion Milan 2017

How to deploy machine learning models in the Cloud

Where should I run my code? Serverless, Containers, Virtual Machines and more

Serverless with Google Cloud

Whizlabs webinar - Deploying Portfolio Site with AWS Serverless

Infrastructure as Code and AWS CDK

Serving models using KFServing

Dip into prometheus

TensorFlow London 14: Ben Hall 'Machine Learning Workloads with Kubernetes an...

IaC on AWS Cloud

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

Introducing Kubeflow (w. Special Guests Tensorflow and Apache Spark)

Hands-on Learning with KubeFlow + Keras/TensorFlow 2.0 + TF Extended (TFX) + ...

Kubeflow Distributed Training and HPO

Rene Groeschke

100% Puppet Cloud Deployment of Legacy Software

Similar to Smokey and the Multi-Armed Bandit Featuring BERT Reynolds Updated

Speaker: Seth Vargo Language: English Although not officially coined until 2009, DevOps ideals have been explicitly discussed since at least 2006. Recently, however, the term "DevOps" has gained increasing popularity across a variety of fields and industries. DevOps is not a development methodology or technology; DevOps is an ideology. It is a way to facilitate organizational prosperity and growth while increasing each individual employee's happiness along the way. As DevOps has gained in prominence, a gap has been created between the original definition of DevOps and this new "enterprise-ready" buzzword. For organizations beginning DevOps practices, this talk will provide a 10,000ft view of DevOps and how you can properly implement DevOps practices in your organization. For organizations that are currently practicing DevOps, this talk will cover common pitfalls, ways to sustain a happy culture, and new tips to foster organizational prosperity. Visit our website: http://atmosphere-conference.com/

Atmosphere Conference 2015: The 10 Myths of DevOps

PROIDEA

MongoDB & Chirp

Antonio Di Motta

Building an MLOps Stack for Companies at Reasonable Scale

Merelda

Amazon SageMaker (December 2018)

Julien SIMON

Julien Simon, Principal Technical Evangelist at Amazon - Machine Learning: Fr...

Codiax

Micro services may not be the best idea

Samuel ROZE

From Notebook to production with Amazon SageMaker

Amazon Web Services

LLMOps for Your Data: Best Practices to Ensure Safety, Quality, and Cost

Aggregage

Enterprise Architectures with Ruby (and Rails)

Konstantin Gredeskoul

Machine Learning involves many different tools. Installing and setting them up properly is a time-consuming task, especially when working at scale. To solve this problem, AWS has built a collection of Amazon Machine Images (AMI) which package all the popular Machine Learning and Python tools: TensorFlow, PyTorch, conda and many more. In this session, we'll take you through a tour of these Deep Learning AMIs and we'll show you how to use them to speed up and simplify your projects.

Simplify Machine Learning with the Deep Learning AMI | AWS Floor28

Amazon Web Services

APIdays Paris 2019 - Maintain & Evolve a Public GraphQL API by Aurélien Davi...

apidays

deep_autoviml is a powerful new deep learning library with a very simple design goal: Make it as easy as possible for novices and experts alike to experiment with and build tensorflow.keras preprocessing pipelines and models in as few lines of code as possible. deep_autoviml will enable data scientists, ML engineers and data engineers to fast prototype tensorflow models and data pipelines for MLOps workflows using the latest TF 2.4+ and keras preprocessing layers. You can now upload your saved model to any Cloud provider and make predictions out of the box since all the data preprocessing layers are attached to the model itself! In this webinar, we will discuss the problems that deep_AutoViML can solve, its architecture design and demo how to build powerful TF.Keras models on structured data, NLP and Image data domains. https://www.aicamp.ai/event/eventdetails/W2021080918

Deep AutoViML For Tensorflow Models and MLOps Workflows

Bill Liu

Looking to implement MLOps using AWS services and Kubeflow? Come and learn about machine learning from the experts of Provectus and Amazon Web Services (AWS)! Businesses recognize that machine learning projects are important but go beyond just building and deploying models, which is mostly done by organizations. Successful ML projects entail a complete lifecycle involving ML, DevOps, and data engineering and are built on top of ML infrastructure. AWS and Amazon SageMaker provide a foundation for building infrastructure for machine learning while Kubeflow is a great open source project, which is not given enough credit in the AWS community. In this webinar, we show how to design and build an end-to-end ML infrastructure on AWS. Agenda - Introductions - Case Study: GoCheck Kids - Overview of AWS Infrastructure for Machine Learning - Provectus ML Infrastructure on AWS - Experimentation - MLOps - Feature Store Intended Audience Technology executives & decision makers, manager-level tech roles, data engineers & data scientists, ML practitioners & ML engineers, and developers Presenters - Stepan Pushkarev, Chief Technology Officer, Provectus - Qingwei Li, ML Specialist Solutions Architect, AWS Feel free to share this presentation with your colleagues and don't hesitate to reach out to us at info@provectus.com if you have any questions! REQUEST WEBINAR: https://provectus.com/webinar-mlops-and-reproducible-ml-on-aws-with-kubeflow-and-sagemaker-aug-2020/

MLOps and Reproducible ML on AWS with Kubeflow and SageMaker

Provectus

1) Learn about Myplanet's Headless CMS solution using Gatsby Preview and Contentful’s UI Extensions (https://www.contentful.com/resources/serverless/) 2) their Serverless project with IBM - using Apache OpenWhisk (https://www.ibm.com/cloud/functions) 3) how Myplanet got involved with AWS DeepRacer - a fun way to get started with Reinforcement Learning (RL), and their racing experience at re:Invent DeepRacer League (https://reinvent.awsevents.com/learn/deepracer/) 4) their Machine Learning (ML) research related to finding DeepRacer’s ideal line (https://medium.com/myplanet-musings/the-best-path-a-deepracer-can-learn-2a468a3f6d64). BONUS: Two TED Talks referenced in the intro 5) When ideas have sex | Matt Ridley | Jul 14, 2010 https://www.ted.com/talks/matt_ridley_when_ideas_have_sex 6) Why The Best Leaders Make Love The Top Priority | Matt Tenney | Dec 5, 2019 https://www.youtube.com/watch?v=qCVoohdyI6I VIDEO: https://youtu.be/ZH1xxmBNx5k

Serverless projects at Myplanet

Daniel Zivkovic

Scaling Machine Learning from zero to millions of users (May 2019)

Julien SIMON

In this session, we cover best practices for enterprises that want to use powerful open-source technologies to simplify and scale their machine learning (ML) efforts. Learn how to use Apache Spark, the data processing and analytics engine commonly used at enterprises today, for data preparation as it unifies data at massive scale across various sources. We train models using TensorFlow, and we use MLflow to track experiment runs between multiple users within a reproducible environment. We then manage the deployment of models to production. We show you how MLflow can be used with any existing ML library and incrementally incorporated into an existing ML development process. This session is brought to you by AWS partner, Databricks.

ML Best Practices: Prepare Data, Build Models, and Manage Lifecycle (AIM396-S...

Amazon Web Services

Build 2019 Recap

Eran Stiller

Slides for Agile Testers Conference 2018 Technology Based Testing by Alan Richardson What do you learn if you want to test 'beyond the acceptance criteria'? Technical risk based testing can help. In this case I'm going to use the phrase Technical Testing to cover: "identify technology based risks to drive testing". This thought process can help us make informed decisions about the scope of exploratory testing we will carry out. It also helps focus your studies on the technical knowledge appropriate for the project you are testing. ## Blurb This requires: - understanding of the technology - risk identification - tools applicable to the technology This presentation will use a simple example to demonstrate that: - Even simple technology can pose risk - Combining simple technology can increase risk - Understanding technology allows us to evaluate risk * http://www.eviltester.com * http://www.compendiumdev.co.uk * https://twitter.com/eviltester

Technology Based Testing

Alan Richardson

An Introduction to Amazon SageMaker (October 2018)

Julien SIMON

Use Case Patterns for LLM Applications (1).pdf

M Waleed Kadous

Similar to Smokey and the Multi-Armed Bandit Featuring BERT Reynolds Updated (20)

Atmosphere Conference 2015: The 10 Myths of DevOps

MongoDB & Chirp

Building an MLOps Stack for Companies at Reasonable Scale

Amazon SageMaker (December 2018)

Julien Simon, Principal Technical Evangelist at Amazon - Machine Learning: Fr...

Micro services may not be the best idea

From Notebook to production with Amazon SageMaker

LLMOps for Your Data: Best Practices to Ensure Safety, Quality, and Cost

Enterprise Architectures with Ruby (and Rails)

Simplify Machine Learning with the Deep Learning AMI | AWS Floor28

APIdays Paris 2019 - Maintain & Evolve a Public GraphQL API by Aurélien Davi...

Deep AutoViML For Tensorflow Models and MLOps Workflows

MLOps and Reproducible ML on AWS with Kubeflow and SageMaker

Serverless projects at Myplanet

Scaling Machine Learning from zero to millions of users (May 2019)

ML Best Practices: Prepare Data, Build Models, and Manage Lifecycle (AIM396-S...

Build 2019 Recap

Technology Based Testing

An Introduction to Amazon SageMaker (October 2018)

Use Case Patterns for LLM Applications (1).pdf

More from Chris Fregly

AWS reInvent 2022 reCap AI/ML and Data

Chris Fregly

Chris Fregly (Principal Solution Architect, AI and machine learning at AWS) will give a brief presentation on the various ways to perform scalable Pandas, Modin, and Ray on AWS. He will then answer questions from the audience and moderator, Alejandro Herrera (whatever he is) at Ponder. Chris Fregly is a Principal Solution Architect for AI and Machine Learning at Amazon Web Services (AWS) based in San Francisco, California. He is the organizer of the Global Data Science on AWS meetup. He is co-author of the O'Reilly Book, "Data Science on AWS." Related Links O'Reilly Book: https://www.amazon.com/dp/1492079391/ Website: https://datascienceonaws.com Meetup: https://meetup.datascienceonaws.com GitHub Repo: https://github.com/data-science-on-aws/ YouTube: https://youtube.datascienceonaws.com Slideshare: https://slideshare.datascienceonaws.com

Pandas on AWS - Let me count the ways.pdf

Chris Fregly

RSVP Webinar: https://www.eventbrite.com/e/webinarkubeflow-tensorflow-tfx-pytorch-gpu-spark-ml-amazonsagemaker-tickets-45852865154 Talk #0: Introductions and Meetup Announcements By Chris Fregly and Antje Barth Talk #1: Ray Overview, Ray AI Runtime on AWS using Amazon SageMaker, EC2, EMR, EKS by Chris Fregly, Principal Specialist Solution Architect, AI and Machine Learning @ AWS Talk #2: Deep-dive Blueprints for Amazon Elastic Kubernetes Service (EKS) including Ray and Spark by Apoorva Kulkarni, Sr. Specialist Solution Architect, Containers and Kubernetes @ AWS RSVP Webinar: https://www.eventbrite.com/e/webinarkubeflow-tensorflow-tfx-pytorch-gpu-spark-ml-amazonsagemaker-tickets-45852865154 Zoom link: https://us02web.zoom.us/j/82308186562 Related Links O'Reilly Book: https://www.amazon.com/dp/1492079391/ Website: https://datascienceonaws.com Meetup: https://meetup.datascienceonaws.com GitHub Repo: https://github.com/data-science-on-aws/ YouTube: https://youtube.datascienceonaws.com Slideshare: https://slideshare.datascienceonaws.com

Ray AI Runtime (AIR) on AWS - Data Science On AWS Meetup

Chris Fregly

Amazon reInvent 2020 Recap: AI and Machine Learning Video here: https://youtu.be/YSXe02Y5pHM NEW RELEASE! Build, Automate, Manage, and Scale ML Workflows with the NEW Amazon SageMaker Pipelines by Hallie Crosby Weishahn. Description of Talk and Demo AWS recently announced Amazon SageMaker Pipelines (https://aws.amazon.com/sagemaker/pipelines/), the first purpose-built, easy-to-use Continuous Integration and Continuous Delivery (CI/CD) service for machine learning. SageMaker Pipelines has three main components which improve the operational resilience and reproducibility of your workflows: 1) pipelines, 2) model registry, and 3) projects. In this talk and demo, Hallie will walk us through the new Amazon SageMaker Pipelines feature including MLOps support. Date/Time 9-10am US Pacific Time (Third Monday of Every Month) RSVP: https://www.eventbrite.com/e/1-hr-free-workshop-pipelineai-gpu-tpu-spark-ml-tensorflow-ai-kubernetes-kafka-scikit-tickets-45852865154 Meetup: https://www.meetup.com/Data-Science-on-AWS/ Zoom: https://zoom.us/j/690414331 Webinar ID: 690 414 331 Phone: +1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll) Related Links Meetup: https://meetup.datascienceonaws.com GitHub Repo: https://github.com/data-science-on-aws/ O'Reilly Book: https://datascienceonaws.com YouTube: https://youtube.datascienceonaws.com Slideshare: https://slideshare.datascienceonaws.com Support: https://support.pipeline.ai Monthly Workshop: https://www.eventbrite.com/e/full-day-workshop-kubeflow-gpu-kerastensorflow-20-tf-extended-tfx-kubernetes-pytorch-xgboost-tickets-63362929227 RSVP: https://www.eventbrite.com/e/1-hr-free-workshop-pipelineai-gpu-tpu-spark-ml-tensorflow-ai-kubernetes-kafka-scikit-tickets-45852865154

Amazon reInvent 2020 Recap: AI and Machine Learning

Chris Fregly

Waking the Data Scientist at 2am: Detect Model Degradation on Production Models with Amazon SageMaker Endpoints & Model Monitor In this talk, I describe how to deploy a model into production and monitor its performance using SageMaker Model Monitor. With Model Monitor, I can detect if a model's predictive performance has degraded - and alert an on-call data scientist to take action and improve the model at 2am while the DevOps folks sleep soundly through the night. Topics: AI and Machine Learning, Model Deployment, Anomaly Detection, Amazon SageMaker Endpoints, and Model Monitor

Waking the Data Scientist at 2am: Detect Model Degradation on Production Mod...

Chris Fregly

Quantum Computing with Amazon Braket

Chris Fregly

15 Tips to Scale a Large AI/ML Workshop - Both Online and In-Person

Chris Fregly

AWS Re:Invent 2019 Re:Cap

Chris Fregly

Title TensorFlow + Swift + OpenAI's Unsupervised Sentiment Neuron, KubeFlow, TFX, Kubernetes, Kafka, Airflow, Jupyter, Scikit Agenda 1. TensorFlow + Swift + OpenAI's Unsupervised Sentiment Neuron (45 mins) Speaker: Tanmay Bakshi (https://www.linkedin.com/in/tanmay-bakshi-b15012a1) (More details coming soon...) 2. KubeFlow, TFX, Kubernetes, Kafka, Airflow, Jupyter, Scikit, GPU/TPU, Kafka, Scikit-Learn and JupyterLab (15 mins) (More details coming soon...) ** RSVP & LOGIN HERE ** Eventbrite: https://www.eventbrite.com/e/1-hr-free-pipelineai-gpu-tpu-spark-tensorflow-kubernetes-kafka-scikit-tickets-45852865154 Meetup: https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup/events/kczhrpyxhbcc/ Zoom: https://zoom.us/j/690414331 Webinar ID: 690 414 331 Phone: +1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll) Related Links PipelineAI Home: https://pipeline.ai PipelineAI Community Edition: http://community.pipeline.ai PipelineAI GitHub: https://github.com/PipelineAI/pipeline Advanced Spark and TensorFlow Meetup (SF-based, Global Reach): https://www.meetup.com/Advanced-Spark-and-TensorFlow-Meetup YouTube Videos: https://youtube.pipeline.ai SlideShare Presentations: https://slideshare.pipeline.ai Slack Support: https://joinslack.pipeline.ai Web Support and Knowledge Base: https://support.pipeline.ai Email Support: help@pipeline.ai

Swift for TensorFlow - Tanmay Bakshi - Advanced Spark and TensorFlow Meetup -...

Chris Fregly

Chris Fregly, Founder @ PipelineAI, will walk you through a real-world, complete end-to-end Pipeline-optimization example. We highlight hyper-parameters - and model pipeline phases - that have never been exposed until now. While most Hyperparameter Optimizers stop at the training phase (ie. learning rate, tree depth, ec2 instance type, etc), we extend model validation and tuning into a new post-training optimization phase including 8-bit reduced precision weight quantization and neural network layer fusing - among many other framework and hardware-specific optimizations. Next, we introduce hyperparameters at the prediction phase including request-batch sizing and chipset (CPU v. GPU v. TPU). Lastly, we determine a PipelineAI Efficiency Score of our overall Pipeline including Cost, Accuracy, and Time. We show techniques to maximize this PipelineAI Efficiency Score using our massive PipelineDB along with the Pipeline-wide hyper-parameter tuning techniques mentioned in this talk. Bio Chris Fregly is Founder and Applied AI Engineer at PipelineAI, a Real-Time Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production with Kubernetes and GPUs." Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.

Hyper-Parameter Tuning Across the Entire AI Pipeline GPU Tech Conference San ...

Chris Fregly

Advanced Spark and TensorFlow Meetup - Dec 12 2017 - Dong Meng, MapR + Kubern...

Chris Fregly

Online Workshop Note: A GPU-based cloud instance will be provided to each attendee for the duration of this event!! At 8am PT on the morning of this workshop, we will email the Webinar details to your email address registered with Eventbrite. If this email address is not up to date - or you do not get the email by 8am PT - please email your Eventbrite confirmation to help@pipeline.ai and we'll send you the details. http://pipeline.ai Title PipelineAI Distributed Spark ML + Tensorflow AI + GPU Workshop Time Start: 9am PT Time End: 1pm PT Time Highlights We will each build an end-to-end, continuous Tensorflow AI model training and deployment pipeline on our own GPU-based cloud instance. At the end, we will combine our cloud instances to create the LARGEST Distributed Tensorflow AI Training and Serving Cluster in the WORLD! Pre-requisites Just a modern browser, internet connection, and a good night's sleep! We'll provide the rest. Agenda Spark ML TensorFlow AI Storing and Serving Models with HDFS Trade-offs of CPU vs. *GPU, Scale Up vs. Scale Out CUDA + cuDNN GPU Development Overview TensorFlow Model Checkpointing, Saving, Exporting, and Importing Distributed TensorFlow AI Model Training (Distributed Tensorflow) TensorFlow's Accelerated Linear Algebra Framework (XLA) TensorFlow's Just-in-Time (JIT) Compiler, Ahead of Time (AOT) Compiler Centralized Logging and Visualizing of Distributed TensorFlow Training (Tensorboard) Distributed Tensorflow AI Model Serving/Predicting (TensorFlow Serving) Centralized Logging and Metrics Collection (Prometheus, Grafana) Continuous TensorFlow AI Model Deployment (TensorFlow, Airflow) Hybrid Cross-Cloud and On-Premise Deployments (Kubernetes) High-Performance and Fault-Tolerant Micro-services (NetflixOSS) More Info including GitHub and Docker Repos http://pipeline.ai

High Performance Distributed TensorFlow in Production with GPUs - NIPS 2017 -...

Chris Fregly

PipelineAI + TensorFlow AI + Spark ML + Kuberenetes + Istio + AWS SageMaker +...

Chris Fregly

High Performance TensorFlow in Production - Big Data Spain - Madrid - Nov 15 ...

Chris Fregly

http://pipeline.ai Using the latest advancements from TensorFlow including the Accelerated Linear Algebra (XLA) Framework, JIT/AOT Compiler, and Graph Transform Tool, I’ll demonstrate how to optimize, profile, and deploy TensorFlow Models - and the TensorFlow Runtime - in GPU-based production environment. This talk is 100% demo based on open source tools and completely reproducible through Docker on your own GPU cluster. Bio Chris Fregly is Founder and Research Engineer at PipelineAI, a Streaming Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High-Performance TensorFlow in Production." Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco. http://pipeline.ai

Optimizing, Profiling, and Deploying TensorFlow AI Models with GPUs - San Fra...

Chris Fregly

Applying my Netflix experience to a real-world problem in the ML and AI world, I will demonstrate a full-featured, open-source, end-to-end TensorFlow Model Training and Deployment System using the latest advancements from Kubernetes, Istio, and TensorFlow. In addition to training and hyper-parameter tuning, our model deployment pipeline will include continuous canary deployments of our TensorFlow Models into a live, hybrid-cloud production environment. This is the holy grail of data science - rapid and safe experiments of ML / AI models directly in production. Following the Successful Netflix Culture that I lived and breathed (https://www.slideshare.net/reed2001/culture-1798664/2-Netflix_CultureFreedom_Responsibility2), I give Data Scientists the Freedom and Responsibility to extend their ML / AI pipelines and experiments safely into production. Offline, batch training and validation is for the slow and weak. Online, real-time training and validation on live production data is for the fast and strong. Learn to be fast and strong by attending this talk. Bio: Chris Fregly is Founder and Research Engineer at PipelineAI, a Streaming Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production." Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco. http://pipeline.ai

Building Google's ML Engine from Scratch on AWS with GPUs, Kubernetes, Istio,...

Chris Fregly

Using the latest advancements from TensorFlow including the Accelerated Linear Algebra (XLA) Framework, JIT/AOT Compiler, and Graph Transform Tool , I’ll demonstrate how to optimize, profile, and deploy TensorFlow Models in GPU-based production environment. This talk contains many demos based on open source tools. You can completely reproduce all demos through Docker on your own GPU cluster. See http://pipeline.ai for links to the GitHub Repo.

Nvidia GPU Tech Conference - Optimizing, Profiling, and Deploying TensorFlow...

Chris Fregly

http://pipeline.ai Applying my Netflix experience to a real-world problem in the ML and AI world, I will demonstrate a full-featured, open-source, end-to-end TensorFlow Model Training and Deployment System using the latest advancements from Kubernetes, Istio, and TensorFlow. In addition to training and hyper-parameter tuning, our model deployment pipeline will include continuous canary deployments of our TensorFlow Models into a live, hybrid-cloud production environment. This is the holy grail of data science - rapid and safe experiments of ML / AI models directly in production. Following the Successful Netflix Culture that I lived and breathed (https://www.slideshare.net/reed2001/culture-1798664/2-Netflix_CultureFreedom_Responsibility2), I give Data Scientists the Freedom and Responsibility to extend their ML / AI pipelines and experiments safely into production. Offline, batch training and validation is for the slow and weak. Online, real-time training and validation on live production data is for the fast and strong. Learn to be fast and strong by attending this talk. http://pipeline.ai

Building Google Cloud ML Engine From Scratch on AWS with PipelineAI - ODSC Lo...

Chris Fregly

Using the latest advancements from TensorFlow including the Accelerated Linear Algebra (XLA) Framework, JIT AOT Compiler, and Graph Transform Tool , I’ll demonstrate how to optimize, profile, and deploy TensorFlow Models in GPU-based production environment. This talk is 100% demo based with open source tools and completely reproducible through Docker on your own GPU cluster. In addition, I spin up a GPU cloud instance for every attendee in the audience. We go through the notebooks together as I demonstrate the process of continuously training, optimizing, deploying, and serving a TensorFlow model on a large, distributed cluster of Nvidia GPUs managed by the attendees. http://pipeline.ai

Optimizing, Profiling, and Deploying TensorFlow AI Models in Production with ...

Chris Fregly

http://pipeline.ai Title PipelineAI Distributed Spark ML + Tensorflow AI + GPU Workshop *A GPU-based cloud instance will be provided to each attendee as part of this event Highlights We will each build an end-to-end, continuous Tensorflow AI model training and deployment pipeline on our own GPU-based cloud instance. At the end, we will combine our cloud instances to create the LARGEST Distributed Tensorflow AI Training and Serving Cluster in the WORLD! Agenda Spark ML Tensorflow AI Storing and Serving Models with HDFS Trade-offs of CPU vs. *GPU, Scale Up vs. Scale Out CUDA + cuDNN GPU Development Overview Tensorflow Model Checkpointing, Saving, Exporting, and Importing Distributed Tensorflow AI Model Training (Distributed Tensorflow) Centralized Logging and Visualizing of Distributed Tensorflow Training (Tensorboard) Distributed Tensorflow AI Model Serving/Predicting (Tensorflow Serving) Centralized Logging and Metrics Collection (Prometheus, Grafana) Continuous Tensorflow AI Model Deployment (Tensorflow, Airflow) Hybrid Cross-Cloud and On-Premise Deployments (Kubernetes) High-Performance and Fault-Tolerant Microsservices using Request Batching and Circuit Breakers (NetflixOSS) Github Repo https://github.com/fluxcapacitor/pipeline

High Performance TensorFlow in Production -- Sydney ML / AI Train Workshop @ ...

Chris Fregly

More from Chris Fregly (20)