This document discusses building machine learning models for mobile apps using TensorFlow. It describes the process of gathering training data, training a model using Cloud ML Engine, optimizing the model for mobile, and integrating it into an Android app. Key steps involve converting video training data to images, retraining an InceptionV3 model, optimizing the model size with graph transformations, and loading the model into an Android app. TensorFlow allows developing machine learning models that can run efficiently on mobile devices.

1. @YufengG
On-device machine learning:
TensorFlow on Android
Yufeng Guo
Developer Advocate
@YufengG
yufengg.com

2. @YufengG
We now live in an
AI-first world

3. @YufengG
Over half of the companies
on the Fortune 500 have
disappeared since 2000
Source: World economic forum

4. @YufengG
Train on the server
Predict on mobile

5. @YufengG@YufengG

6. @YufengG@YufengG

7. @YufengG
How can I make that?
ML is hard...
@YufengG

8. @YufengG
Demo: custom image
recognition
Let's make an Android app
@YufengG

9. @YufengG

10. @YufengG
From training to app
Gather training data
Load into Android app
? ? ?
??

11. @YufengG
Video
Capture video of what you want to classify

12. @YufengG
From training to app
Gather training data
Load into Android app
?
??
Convert to images
Folders of images

13. @YufengG
Split the video up
into pictures for
training
Pictures

14. @YufengG
From training to app
Gather training data
Load into Android app
?
Convert to images
Folders of images
Storage
Cloud Storage
Object storage with global
edge-caching
Training
Cloud ML Engine
Fast, scalable, and
easy-to-use ML services

15. @YufengG
Sidebar: Convolutional neural networks

16. @YufengG@YufengG

17. @YufengG

18. @YufengG

19. @YufengG

20. @YufengG

21. @YufengG

22. @YufengG

23. @YufengG
Training the InceptionV3 model
Use the pictures to retrain the InceptionV3 model

24. @YufengG
From training to app
Gather training data
Load into Android app
Convert to images
Folders of images
Storage
Cloud Storage
Object storage with global
edge-caching
Training
Cloud ML Engine
Fast, scalable, and
easy-to-use ML services
Optimize for mobile

25. @YufengG
Downsizing with the graph transform tool

26. @YufengG
One more thing...
App
Model
App
Model

27. @YufengG
From training to app
Gather training data Convert to images
Training
Cloud ML Engine
Fast, scalable, and
easy-to-use ML services
Storage
Cloud Storage
Object storage with global
edge-caching
Folders of images
Optimize for mobile
Load into Android app

28. @YufengG
Load into Android app
Add the model into your app and ship it!

29. @YufengG
What makes all of this possible?

30. @YufengG
Supported platforms
Raspberry
Pi
Android iOS
TPU
GPUCPU

31. @YufengG
Community
● 475+ non-Google contributors to TensorFlow 1.0
● 14,000+ commits in 14 months
● Many community created tutorials, models,
translations, and projects
○ ~5,500 GitHub repositories with ‘TensorFlow’
in the title

32. @YufengG
Machine learning on mobile — awesomeness

33. @YufengG
From training to app
Gather training data Convert to images
Training
Cloud ML Engine
Fast, scalable, and
easy-to-use ML services
Storage
Cloud Storage
Object storage with global
edge-caching
Folders of images
Optimize for mobile
Load into Android app

34. @YufengG
Codelab to retrain your own InceptionV3 network
bit.ly/tf-retrain
Mobile Tensorflow
tensorflow.org/mobile
Cloud Machine Learning Engine
cloud.google.com/ml
Thank you!
Yufeng Guo
Developer Advocate
yufengg.com

35. @YufengG
Mobile machine learning, now with extra fast
Just-In-Time Compilation
via XLA, "Accelerated Linear Algebra" compiler
0x00000000 movq (%rdx), %rax
0x00000003 vmovaps (%rax), %xmm0
0x00000007 vmulps %xmm0, %xmm0, %xmm0
0x0000000b vmovaps %xmm0, (%rdi)
...
TF graphs go in
Optimized & specialized
assembly comes out.