AI & Machine Learning Web Day | Einführung in Amazon SageMaker, eine Werkbank für ML-Algorithmen und Deep-Learning

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
Constantin Gonzalez
Principal Solutions Architect, Amazon Web Services
Eine komplette Werkbank für maschinelles Lernen (ML)

Bei Amazon investieren wir seit über
20 Jahren in maschinelles Lernen
Suchen &
Entdecken
Lieferung &
Logistik
Aktuelle
Produkte
Zukünftige
Initiativen

Maschinelles Lernen
bei Amazon (1995)

Maschinelles Lernen für Entwickler und Data Scientists
einfach nutzbar machen
ML @ AWS: Unser Ziel

Kunden, die heute ML auf AWS nutzen

Frameworks &
Infrastructure
AWS Deep Learning AMI
GPU
(P3 Instances)
MobileCPU
IoT
(Greengrass)
Vision:
Rekognition Image
Rekognition Video
Speech:
Polly
Transcribe
Language:
Lex Translate
Comprehend
Apache
MXNet
PyTorch
Cognitive
Toolkit
Keras
Caffe2
& Caffe
TensorFlow Gluon
Application
AI Services
Managed
Platform
Services
Amazon Machine
Learning
Mechanical
Turk
Spark &
EMR
Amazon
SageMaker
AWS
DeepLens
AWS ML Stack

Schauen wir uns den
Machine Learning (ML) Prozess
genauer an

Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Der Machine Learning Prozess
Re-training

Ein vollständig automatisierter Service,
der Data Scientists und Entwicklern hilft,
schnell und einfach Machine-Learning-Modelle zu bauen,
und diese als intelligente Applikationen in Produktion zu
bringen.
Amazon SageMaker

Highly-optimized
machine learning
algorithms
Amazon SageMaker
BauenPre-built
notebook
instances

Highly-optimized
machine learning
algorithms
One-click training
for ML, DL, and
custom algorithms
BauenPre-built
notebook
instances
Easier training with
hyperparameter
optimization
Trainieren
Amazon SageMaker

Highly-optimized
machine learning
algorithms
Deployment
without
engineering effort
Fully-managed
hosting at scale
BauenPre-built
notebook
instances
Ausrollen
Amazon SageMaker
Trainieren
One-click training
for ML, DL, and
custom algorithms
Easier training with
hyperparameter
optimization

Amazon ECR
Model Training (on EC2)
Amazon SageMaker
Client application
Training code

Amazon ECR
Trainingdata
Training code Helper code
Client application
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Helper codeInference code
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
GroundTruth
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

Intuit nutzt Amazon SageMaker
“With Amazon SageMaker, we can accelerate our Artificial
Intelligence initiatives at scale by building and deploying our
algorithms on the platform. We will create novel large-scale
machine learning and AI algorithms and deploy them on this
platform to solve complex problems that can power prosperity for
our customers.
"
- Ashok Srivastava, Chief Data Officer, Intuit

Vorteile von SageMaker für Intuit
Ad-hoc Aufbau und
Management von Notebook-
Umgebungen
Limitierte Auswahl für das
Ausrollen von Modellen
Konflikte zwischen Teams um
limitierte Ressourcen
Einfache Daten-Exploration
in SageMaker Notebooks
Virtualisierung als Grundlage für
Flexibilität
Automatisch skalierende
Umgebung für Modell-Hosting
Vorher Nachher

Model Hosting
(SageMaker)
N a h e E c h t z e i t -B e t r u g s e r k e n n u n g i n A W S m i t S a g e M a k e r
Calculate
Features
Reader
Cleanser
Processor
Data
Lookup
Training
Feature Store Model Training
(SageMaker)
Model
Client Service
Amazon
EMR
Amazon
SageMaker
Amazon
SageMaker

Amazon SageMaker
1 2 3 4
I I I I
Notebook-Instanzen Algorithmen ML Training Service ML Hosting Service

1
I
Notebook-Instanzen
Explorative Daten-Analyse Ohne Schmerzen
Authoring &
Notebooks
ETL-Zugriff auf AWS
Datenbank-Services
Zugriff auf
S3 Data Lake
• Empfehlung/Personalisierung
• Betrugs-Erkennung
• Forecasting
• Bild-Klassifizierung
• Abwanderungs-Prognosen
• Marketing-Zielgruppenoptimierung
• Log-Verarbeirung und
Anomalie-Erkennung
• Sprache zu Text
• … und vieles mehr!
“Just add data”

Streaming-Daten für
nidrigere Trainings-
Kosten
Schneller, in nur
einem Satz trainieren
Höhere
Zuverlässigkeit bei
extrem großen
Datenmengen
Auswahl
verschiedener ML-
Algorithmen
Amazon SageMaker: 10x bessere Algorithmen
2
I
Algorithmen

Kosten vs. Zeit
$$$$
$$$
$$
$
Minuten Stunden Tage Wochen Monate
Einzelner
Rechner
Verteilt, mit
großen Rechnern

Streaming
GPU Zustände

Streaming
Datenmenge
Speicher
Datenmenge
Zeit/Kosten

Verteilt
GPU Zustand
GPU Zustand
GPU Zustand

Verteilt
GPU Lokaler
Zustand
GPU Lokaler
Zustand
GPU Lokaler
Zustand
Gemeinsamer
Zustand

Kosten vs. Zeit
$$$$
$$$
$$
$
Beste Alternative
Amazon SageMaker
Minuten Stunden Tage Wochen Monate

Unendlich skalierbare ML Algorithmen

Linear Learner
Regression (mean squared error)
SageMaker Other
1.02 1.06
1.09 1.02
0.332 0.183
0.086 0.129
83.3 84.5
Classification (F1 Score)
SageMaker Other
0.980 0.981
0.870 0.930
0.997 0.997
0.978 0.964
0.914 0.859
0.470 0.472
0.903 0.908
0.508 0.508
30 GB datasets for web-spam and web-url classification
0
0,2
0,4
0,6
0,8
1
1,2
0 5 10 15 20 25 30
CostinDollars
Billable time in Minutes
sagemaker-url sagemaker-spam other-url other-spam

Factorization Machines
˜y = w0 + hw1, xi +
X
i,j>i
xixj · hvi, vji
Log_loss F1 Score Seconds
SageMaker 0.494 0.277 820
Other (10 Iter) 0.516 0.190 650
Other (20 Iter) 0.507 0.254 1300
Other (50 Iter) 0.481 0.313 3250
Click Prediction 1 TB advertising dataset,
m4.4xlarge machines, perfect scaling.
$-
$20,00
$40,00
$60,00
$80,00
$100,00
$120,00
$140,00
$160,00
$180,00
$200,00
1 2 3 4 5 6 7 8CostinDollars
Billable Time in Hours
10
machines
20
machines
30
machines
4050

0
1
2
3
4
5
6
7
8
10 100 500
BillableTimeinMinutes Number of Clusters
sagemaker other
K-Means Clustering
k SageMaker Other
Text
1.2GB
10 1.18E3 1.18E3
100 1.00E3 9.77E2
500 9.18.E2 9.03E2
Images
9GB
10 3.29E2 3.28E2
100 2.72E2 2.71E2
500 2.17E2 Failed
Videos
27GB
10 2.19E2 2.18E2
100 2.03E2 2.02E2
500 1.86E2 1.85E2
Advertising
127GB
10 1.72E7 Failed
100 1.30E7 Failed
500 1.03E7 Failed
Synthetic
1100GB
10 3.81E7 Failed
100 3.51E7 Failed
500 2.81E7 Failed
Running Time vs. Number of Clusters
~10x Faster!

Principal Component Analysis (PCA)
More than 10x faster
at a fraction the cost!
0,00
20,00
40,00
60,00
80,00
100,00
120,00
8 10 20
Mb/Sec/Machine
Number of Machines
other sagemaker-deterministic sagemaker-randomized
Cost vs. Time Throughput and Scalability
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
0 10 20 30 40 50
CostinDollars
Billable time in Minutes
other sagemaker-deterministic sagemaker-randomized

Neural Topic Modeling
Perplexity vs. Number of Topic
Encoder: feedforward net
Input term counts vector
µ
z
Document
Posterior
Sampled Document
Representation
Decoder:
Softmax
Output term counts vector
0
2000
4000
6000
8000
10000
12000
0 50 100 150 200
Perplexity
Number of Topics
NTM Other

Time Series Forecasting (DeepAR)
Mean absolute
percentage error
P90 Loss
DeepAR R DeepAR R
traffic
Hourly occupancy rate of 963
bay area freeways
0.14 0.27 0.13 0.24
electricity
Electricity use of 370
homes over time
0.07 0.11 0.08 0.09
pageviews
Page view hits
of websites
10k 0.32 0.32 0.44 0.31
180k 0.32 0.34 0.29 NA
One hour on p2.xlarge, $1
zi,t 2, xi,t 1 zi,t 1, xi,t zi,t, xi,t+1
hi,t 1 hi,t hi,t+1
`(zi,t 1|✓i,t 1) `(zi,t|✓i,t) `(zi,t+1|✓i,t+1)
zi,t 1 zi,t zi,t+1
Input
Network

Weitere ML Algorithmen

Spectral LDA
Training Time vs. Number of Topics
0
50
100
150
200
250
0 20 40 60 80 100TrainingTimeinMinutes
Number of Topics
lda-data-a lda-data-b other-data-a other-data-b

Boosted Decision Trees
Throughput vs. Number of MachinesXGBoost is one of the most
commonly used
implementations of boosted
decision trees in the world.
It is now available in Amazon
SageMaker!
0
200
400
600
800
1000
1200
1400
0 10 20 30 40 50 60 70
ThroughputinMB/Sec
Number of Machines (C4.8xLarge)

Sequence to Sequence
English-German Translation
0
5
10
15
20
25
0 5 10 15 20 25
BLEUScore
Billable Time in Hours
P2.16x P2.8x P2.x
Best known result!
Based on Sockeye and Apache
incubated MxNet, Multi-GPU,
and can be used for Neural
Machine Translation.
Supports both RNN/CNN
as encoder/decoder

Image Classification
Implementation in MxNet of
ResNet.
Other networks such as
DenseNet and Inception will
be added in the future.
Transfer learning: begin with
a model already trained on
ImageNet!
0
0,5
1
1,5
2
2,5
3
3,5
0 1 2 3 4 5
Speedup
Number of Machine (P2)
Speedup with Horizontal Scaling

2
I
Algorithmen
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
IM Estimators in
Apache Spark Bring Your Own Algorithm (You build the Container)
Amazon SageMaker: 10x bessere Algorithmen

Automatisch verteiltes Training mit hoher Flexibilität
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
Bring Your Own Algorithm (You build the Container)
3
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
Secured–
Amazon ECR
Save Inference Image
IM Estimators in
Apache Spark
CPU GPU HPO

4
I
ML Hosting Service
Amazon ECR
Amazon SageMaker
Einfaches Model Deployment mit Amazon SageMaker
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!

4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Prod is the primary
Create a Model
ModelName: prod
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Model versions
Prod is the primary
Create versions of a Model
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
InitialInstanceCount: 3
ModelName: prod
VariantName: primary
InitialVariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Prod is the primary
Create weighted
ProductionVariants
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Prod is the primary
Create an
EndpointConfiguration
from one or many
ProductionVariant(s)EndpointConfiguration
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Prod is the primary
must be served there! Create an Endpoint from
one EndpointConfiguration
Inference Endpoint
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
Prod is the primary
One-Click!
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
ü Auto-Scaling Inference
APIs
ü A/B Testing (more to
come)
ü Low Latency & High
Throughput
ü Bring Your Own Model
ü Python SDK
Amazon SageMaker

Mehr Informationen
Infinitely Scalable
Machine Learning
Algorithms with Amazon
SageMaker
https://www.youtube.com
/watch?v=VT4tM0-7L80
Deep Learning with
Apache MXNet and
Gluon
/watch?v=me1qOzSg8M
U
Introducing Amazon
SageMaker
/watch?v=4pbXdsjZx_k

Detaillierte Dokumentation
https://aws.amazon.com/documentation/sagemaker/

Amazon SageMaker – Jetzt sind Sie dran!
• Getting started with Amazon SageMaker:
https://aws.amazon.com/sagemaker/
• Amazon SageMaker SDK:
• Für Python: https://github.com/aws/sagemaker-python-sdk
• Für Spark: https://github.com/aws/sagemaker-spark
• Amazon SageMaker Beispiele:
https://github.com/awslabs/amazon-sagemaker-examples
• AWS ML Online Demos:
https://s3.amazonaws.com/aiml-demo-site/index.html
• Zeigen Sie uns, was Sie gebaut haben!

Amazon SageMaker
GO BUILD!
End-to-End Managed ML Platform

AI & Machine Learning Web Day | Einführung in Amazon SageMaker, eine Werkbank für ML-Algorithmen und Deep-Learning

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AI & Machine Learning Web Day | Einführung in Amazon SageMaker, eine Werkbank für ML-Algorithmen und Deep-Learning

Similar to AI & Machine Learning Web Day | Einführung in Amazon SageMaker, eine Werkbank für ML-Algorithmen und Deep-Learning (20)

More from AWS Germany

More from AWS Germany (20)

Recently uploaded

Recently uploaded (17)

AI & Machine Learning Web Day | Einführung in Amazon SageMaker, eine Werkbank für ML-Algorithmen und Deep-Learning