AWS SageMaker: Deploy ML Models from Notebook to Production
•Download as PPTX, PDF•
2 likes•2,301 views
Talk @ AWS Builders Day London / Dublin
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to AWS SageMaker: Deploy ML Models from Notebook to Production
Similar to AWS SageMaker: Deploy ML Models from Notebook to Production (20)
More from Julien SIMON
More from Julien SIMON (20)
Recently uploaded
Recently uploaded (20)
AWS SageMaker: Deploy ML Models from Notebook to Production
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Julien Simon Principal Technical Evangelist, AI & Machine Learning, AWS @julsimon From Notebook to Production with Amazon SageMaker
- 2. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Put Machine Learning in the hands of every developer and data scientist Our mission
- 3. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Application Services Platform Services Frameworks & Infrastructure API-driven services:Vision, Language & Speech Services, Chatbots AWS ML Stack Deploy machine learning models with high-performance machine learning algorithms, broad framework support, and one-click training, tuning, and inference. Develop sophisticated models with any framework, create managed, auto-scaling clusters of GPUs for large scale training, or run prediction on trained models.
- 4. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Application Services Platform Services Frameworks & Infrastructure API-driven services:Vision, Language & Speech Services, Chatbots AWS ML Stack Deploy machine learning models with high-performance machine learning algorithms, broad framework support, and one-click training, tuning, and inference. Develop sophisticated models with any framework, create managed, auto-scaling clusters of GPUs for large scale training, or run prediction on trained models.
- 5. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Data Visualization & Analysis Business Problem ML problem framing Data Collection Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging YesNo DataAugmentation Feature Augmentation The Machine Learning Process Re-training Predictions
- 6. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Amazon SageMaker Pre-built notebooks for common problems K-MeansClustering Principal Component Analysis Neural TopicModelling FactorizationMachines Linear Learner XGBoost Latent Dirichlet Allocation ImageClassification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Built-in, high- performance algorithms Build Git integration Elastic inference
- 7. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Git integration
- 8. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Elastic Inference https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/
- 9. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Amazon SageMaker Pre-built notebooks for common problems K-MeansClustering Principal Component Analysis Neural TopicModelling FactorizationMachines Linear Learner XGBoost Latent Dirichlet Allocation ImageClassification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Built-in, high- performance algorithms Build New built-in algorithms scikit-learn environment Model marketplace Search
- 10. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Search training jobs
- 11. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Machine Learning Marketplace
- 12. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Amazon SageMaker Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Train Deploy model in production Scale and manage the production environment P3DN, C5N TensorFlow on 256 GPUs Resume HPO tuning job Build
- 13. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Amazon SageMaker Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Deploy Model compilation Elastic inference Inference pipelines TrainBuild
- 14. Working with Amazon SageMaker
- 15. TheAmazon SageMakerAPI • Python SDK orchestrating all Amazon SageMaker activity • High-level objects for algorithm selection, training, deploying, automatic model tuning, etc. • Spark SDK (Python & Scala) • AWS CLI: ‘aws sagemaker’ • AWS SDK: boto3, etc.
- 16. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Model Training (on EC2) Model Hosting (on EC2) Trainingdata Modelartifacts Training code Helper code Helper codeInference code GroundTruth Client application Inference code Training code Inference requestInference response Inference Endpoint
- 17. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Training code Factorization Machines Linear Learner PrincipalComponent Analysis K-Means Clustering XGBoost And more Built-inAlgorithms BringYour Own ContainerBringYour Own Script Model options
- 18. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Built-in algorithms orange: supervised, yellow: unsupervised Linear Learner: regression, classification Image Classification: Deep Learning (ResNet) Factorization Machines: regression, classification, recommendation Object Detection (SSD): Deep Learning (VGG or ResNet) K-Nearest Neighbors: non-parametric regression and classification NeuralTopic Model: topic modeling XGBoost: regression, classification, ranking https://github.com/dmlc/xgboost Latent DirichletAllocation: topic modeling (mostly) K-Means: clustering BlazingText:GPU-basedWord2Vec, and text classification Principal ComponentAnalysis: dimensionality reduction Sequence to Sequence: machine translation, speech speech to text and more RandomCut Forest: anomaly detection DeepAR: time-series forecasting (RNN) Object2Vec: general-purpose embedding IP Insights: usage patterns for IP addresses Semantic Segmentation: Deep Learning
- 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blazing Text https://dl.acm.org/citation.cfm?id=3146354
- 20. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Demo: Text Classification with BlazingText https://github.com/awslabs/amazon-sagemaker- examples/tree/master/introduction_to_amazon_algorithms/blazingtext_text_classification_dbpedia
- 21. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. OptimizingTensorFlow https://aws.amazon.com/blogs/machine-learning/faster-training-with- optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3-instances/ (March 2018) Training a ResNet-50 benchmark with the synthetic ImageNet dataset using our optimized build ofTensorFlow 1.11 on a c5.18xlarge instance type is 11x faster than training on the stock binaries. https://aws.amazon.com/about-aws/whats- new/2018/10/chainer4-4_theano_1-0- 2_launch_deep_learning_ami/ (October 2018)
- 22. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Demo: Elastic Inference withTensorFlow https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_iris_dnn_classifier_using_estimators/
- 24. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Automatic Model Tuning Finding the optimal set of hyper parameters 1. Manual Search (”I know what I’m doing”) 2. Grid Search (“X marks the spot”) • Typically training hundreds of models • Slow and expensive 3. Random Search (“Spray and pray”) • Works better and faster than Grid Search • But… but… but… it’s random! 4. HPO: use Machine Learning • Training fewer models • Gaussian Process Regression and Bayesian Optimization • You can now resume from a previous tuning job
- 25. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Demo: HPO withTensorFlow https://github.com/awslabs/amazon-sagemaker- examples/tree/master/hyperparameter_tuning/tensorflow_mnist CONV1 POOL1 CONV2 POOL2 DENSE1 DROPOUT1 OUTPUT Class probabilities INPUT
- 27. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Optimizing for the underlying hardware https://aws.amazon.com/blogs/aws/amazon-sagemaker-neo-train-your-machine-learning-models-once-run-them-anywhere/ • Train once, run anywhere • Frameworks and algorithms • TensorFlow, Apache MXNet, PyTorch, ONNX, and XGBoost • Hardware architectures • ARM, Intel, and NVIDIA starting today • Cadence, Qualcomm, and Xilinx hardware coming soon • Amazon SageMaker Neo will be released as open source enabling hardware vendors to customize it for their processors and devices.
- 28. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Demo: Compiling ResNet-50 for the Raspberry Pi Configure the compilation job { "RoleArn":$ROLE_ARN, "InputConfig": { "S3Uri":"s3://jsimon-neo/model.tar.gz", "DataInputConfig": "{"data": [1, 3, 224, 224]}", "Framework": "MXNET" }, "OutputConfig": { "S3OutputLocation": "s3://jsimon-neo/", "TargetDevice": "rasp3b" }, "StoppingCondition": { "MaxRuntimeInSeconds": 300 } } Compile the model $ aws sagemaker create-compilation-job --cli-input-json file://config.json --compilation-job-name resnet50-mxnet-pi $ aws s3 cp s3://jsimon-neo/model- rasp3b.tar.gz . $ gtar tfz model-rasp3b.tar.gz compiled.params compiled_model.json compiled.so Predict with the compiled model from dlr import DLRModel model = DLRModel('resnet50', input_shape, output_shape, device) out = model.run(input_data)
- 30. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Inference Pipelines • Linear sequence of 2-5 containers that process inference requests • Feature engineering with scikit-learn or SparkML (on AWS Glue or Amazon EMR) • Predict with built-in or custom containers • The sequence is deployed as a a single model • Useful to preprocess, predict, and post-process • Available for real-time prediction and batch transform
- 31. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Demo: Inference with scikit-learn and linear learner https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/scikit_learn_inference_pipeline/
- 32. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Amazon SageMaker Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Build Train Deploy
- 33. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Selected Amazon SageMaker customers
- 35. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Resources http://aws.amazon.com/free https://ml.aws https://aws.amazon.com/sagemaker https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark https://github.com/awslabs/amazon-sagemaker-examples https://gitlab.com/juliensimon/ent321 SageMaker workshop at AWS re:Invent 2018 https://medium.com/@julsimon https://gitlab.com/juliensimon/dlnotebooks
- 36. Thank you! Julien Simon Principal Technical Evangelist, AI and Machine Learning @julsimon
Editor's Notes
- SageMaker makes it easy to build ML models and get them ready for training by providing everything you need to quickly connect to your training data, and to select and optimize the best algorithm and framework for your application. Amazon SageMaker includes hosted Jupyter notebooks that make it is easy to explore and visualize your training data stored in Amazon S3. You can connect directly to data in S3, or use AWS Glue to move data from Amazon RDS, Amazon DynamoDB, and Amazon Redshift into S3 for analysis in your notebook. To help you select your algorithm, Amazon SageMaker includes the 10 most common machine learning algorithms which have been pre-installed and optimized to deliver up to 10 times the performance you’ll find running these algorithms anywhere else. Amazon SageMaker also comes pre-configured to run TensorFlow and Apache MXNet, two of the most popular open source frameworks, or you have the option of using your own framework.
- SageMaker makes it easy to build ML models and get them ready for training by providing everything you need to quickly connect to your training data, and to select and optimize the best algorithm and framework for your application. Amazon SageMaker includes hosted Jupyter notebooks that make it is easy to explore and visualize your training data stored in Amazon S3. You can connect directly to data in S3, or use AWS Glue to move data from Amazon RDS, Amazon DynamoDB, and Amazon Redshift into S3 for analysis in your notebook. To help you select your algorithm, Amazon SageMaker includes the 10 most common machine learning algorithms which have been pre-installed and optimized to deliver up to 10 times the performance you’ll find running these algorithms anywhere else. Amazon SageMaker also comes pre-configured to run TensorFlow and Apache MXNet, two of the most popular open source frameworks, or you have the option of using your own framework.
- You can begin training your model with a single click in the Amazon SageMaker console. The service manages all of the underlying infrastructure for you and can easily scale to train models at petabyte scale. To make the training process even faster and easier, Amazon SageMaker can automatically tune your model to achieve the highest possible accuracy.
- Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results. For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.
- Seq2Seq: used by Amazon Translate and AWS Sockeye LDA: used by Amazon Comprehend
- Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results. For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.