2018/07/03 に実施した Amazon SageMaker ハンズオンの資料です． https://pages.awscloud.com/AmazonSageMaker20180703-jp.html

1. 2018.07.03
Amazon SageMaker

2. 7/3/2018 Amazon SageMaker
16:00-17:00
Amazon SageMaker
-
- Amazon SageMaker 3
- Amazon SageMaker
- Build-in Algorithm
- Native Framework (TF/MXNet/Chainer)
- Bring your own Algorithm
- Amazon SageMaker
- Amazon SageMaker
- Greengrass
17:30-19:00
Amazon SageMaker
- Build-in Algorithm
- Chainer
17:00- Q&A and Break
19:00-19:30
Q&A | Close

3. •
•
•
⎼ Amazon Athena
⎼ AWS Glue
⎼ Amazon SageMaker

4. Agenda
SageMaker
SageMaker

9. •
• 1
1
• 1
•
•

11. SageMaker

12. SageMaker
5

13. SageMaker
•
•
•
•
•
•
•
•
•

14. Jupyter Notebook
•
•
• 4 ml.t2, ml.m4,
ml.p2, ml.p3
• VPC
ENI VPC

15. CreateTrainingJob API Docker
•
•
2
• m4, m5, c4, c5,
p2, p3
•
S3
https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html

16. :
CloudWatch Logs
CloudWatch Logs

17. :
Estimater hyperparameters
SageMaker
https://github.com/awslabs/amazon-sagemaker-examples/tree/master/hyperparameter_tuning
https://github.com/aws/sagemaker-python-sdk#sagemaker-automatic-model-tuning

18. / :
SageMaker Tensorflow / MXNet / Chainer
github
• SageMaker pull /
•
instance_type ’local’
https://github.com /aw s/sagem aker-python-sdk#local-m ode
https://github.com /aw s/sagem aker-tensorflow -containers
https://github.com /aw s/sagem aker-m xnet-containers
https://github.com /aw s/sagem aker-chainer-container

19. CreateEndpoint API Docker
• AB
•
•
https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpointConfig.htm l
https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpoint.htm l
https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpoint.htm l
https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpointW eightsAndCapacities.htm l

20. :
AWS Greengrass SageMaker
/ . .- . / - ./ .- .
• SageMaker
S3
• DeepLens

21. SageMaker

22. SageMaker 2
AWS SDK
•
•
SageMaker SDK
•
• AWS SDK SageMaker SDK AWS SDK
scikit-learn
• Python Spark
• Jupyter Notebook
https://github.com/aws/sagemaker-python-sdk
https://github.com/aws/sagemaker-spark

23. SageMaker 2
AWS SDK
•
•
SageMaker SDK
•
• AWS SDK SageMaker SDK AWS SDK
scikit-learn
• Python Spark
• Jupyter Notebook
https://github.com/aws/sagemaker-python-sdk
https://github.com/aws/sagemaker-spark

24. create-endpoint
create-notebook-instance
create-training-job
delete-endpoint
delete-notebook-instance
describe-endpoint
describe-notebook-instance
…
estimator = TensorFlow(…)
estimator.set_hyperparameters(…)
estimator.fit(…)
predictor = estimator.deploy(…)
Predictor.predict(…)

25. SageMaker
1. SageMaker
2. Tensorflow/MXNet/Chainer
3.

26. SageMaker
1. SageMaker
→
2. Tensorflow/MXNet/ Chainer
→
3.
→

27. SageMaker

28. • Linear Learner
• Factorization Machines
• XGBoost
• Image Classification
• seq2seq
• K-means
• PCA
• LDA
• Neural Topic Model
• DeepAR Forecasting
• BlazingText (word2vec)
• Random Cut Forest
New!
New!
https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html
New!

29. Image Classification
ResNet
• ResNet CNN
• ILSVRC 2015 1
• ImageNet
• use_pretrained_model
1 0
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb
https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf
https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/image-classification.html
https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html

30. SageMaker
S3

31. estimator.fit()
SageMaker
S3

32. estimator.deploy()
SageMaker
S3

33. https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html

34. Chainer / PyTorch / Tensorflow /
MXNet

35. Chainer
• __main__
• model_fn:
•
https://docs.aws.amazon.com/sagemaker/latest/dg/chainer.html
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/chainer_mnist/chainer_mnist_single_machine.py

36. PyTorch
• __main__
• model_fn:
•
https://docs.aws.amazon.com/sagemaker/latest/dg/pytorch.html
https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/pytorch/README.rst

37. Tensorflow
• model_fn
• estimator_fn tensorflow.estimator
• keras_model_fn tf.keras
• train_input_fn
• eval_input_fn
• serving_input_fn
• input_fn
• output_fn
https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html

38. MXNet
• train
• save
• model_fn:
• transform_fn:
•
https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet-training-inference-code-template.html

39. *
S3
SageMaker
S3

40. estimator.fit() .
AWS
SageMaker
S3

41. estimator.deploy()
AWS
SageMaker
S3

44. ECR
push
SageMaker
S3

45. S3ECR
SageMaker
estimator.fit()

46. S3ECR
SageMaker
estimator.deploy()

49. SageMaker 3
1:
• SageMaker
2: GPU
• AWS

50. AWS SDK SageMaker SDK
• SageMaker SDK
Jupyter Notebook
• AWS SDK

51. AWS SDK SageMaker SDK
• SageMaker SDK
Jupyter Notebook
• AWS SDK
SageMaker SDK AWS SDK

52. ML
• SageMaker
1
ML
•
• 0.14 USD/GB/
•
• 0.016 USD/GB
https://aws.amazon.com/jp/sagemaker/pricing/

53. 3
SageMaker Example Notebooks
• https://github.com/awslabs/amazon-sagemaker-examples
SageMaker SDK
• https://github.com/aws/sagemaker-python-sdk
(Doc : https://readthedocs.org/projects/sagemaker/)
SageMaker
• https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/whatis.html

54. https://github.com/awslabs/amazon-sagemaker-examples/tree/master/introduction_to_amazon_algorithms
https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/algos.html
•
•
•
•
•
•
New!
•
•
•
•
•
•

55. Tensorflow
• Tensorflow
• https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/tensorflow_abalone_age_predictor_using_keras/tensorflow_abalone_age_predictor_using_keras.ipynb
• Tensorflow Tensorboard
• https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-
sdk/tensorflow_resnet_cifar10_with_tensorboard
• Tensorflow
• https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/tensorflow_distributed_mnist/tensorflow_distributed_mnist.ipynb
• Keras + Tensorflow
• https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-
sdk/tensorflow_abalone_age_predictor_using_keras
• MXNet
• https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk

56. • scikit-learn SageMaker
SageMaker
• https://github.com /aw slabs/am azon-sagem aker-
exam ples/blob/m aster/advanced_functionality/scikit_bring_your_ow n/scikit_bring_your_ow n.ipynb
• SageMaker R
• https://github.com /aw slabs/am azon-sagem aker-
exam ples/blob/m aster/advanced_functionality/install_r_kernel/exam ple_r_notebook.ipynb
• SageMaker
• https://github.com /aw slabs/am azon-sagem aker-
exam ples/blob/m aster/advanced_functionality/handling_km s_encrypted_data/handling_km s_encrypted_data.ipynb
• XGBoost SageMaker
• https://github.com /aw slabs/am azon-sagem aker-
exam ples/blob/m aster/advanced_functionality/xgboost_bring_your_ow n_m odel/xgboost_bring_your_ow n_m odel.ipynb
• Tensorflow Sagemaker
• https://github.com /aw slabs/am azon-sagem aker-
exam ples/blob/m aster/advanced_functionality/tensorflow _iris_byom /tensorflow _BYO M _iris.ipynb
• Redshift MXNet R
• https://github.com /aw slabs/am azon-sagem aker-exam ples/tree/m aster/advanced_functionality

57. Amazon Web Services Japan, K. K.
Amazon SageMaker

58. SageMaker

59. •
•
ml.t2.medium
• IAM

60. IAM
S3
S3

61. • VPC,
•

62. Jupyter Notebook
• InService
Jupyter Notebook
• New → Terminal

63. Terminal
cd SageMaker/
wget http://bit.ly/sagemaker-notebooks -O sagemaker-notebooks.zip
unzip sagemaker-notebooks.zip
O( )

64. Jupyter Notebook
sagemaker-notebooks

65. • MNIST
•
• Amazon SageMaker 1
XGBoost
• Chainer MLP ( )
Amazon SageMaker