For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2021/01/using-learning-at-the-edge-to-deliver-business-value-a-presentation-from-lg-electronics/ Vera Serdiukova, Product Manager in the Advanced AI team at LG Electronics, presents the “Using Learning at the Edge to Deliver Business Value” tutorial at the September 2020 Embedded Vision Summit. Learning at the Edge is the next frontier of Edge AI. Unlike inference, so far it has stayed on the fringe of industry focus, usually thought of as infeasible for practical applications. In her talk, Serdiukova sheds light on the benefits and challenges of learning at the edge, as well as practical insights on unlocking its potential and using it to create business value.

1. © 2020 Vera Serdiukova
Using Learning at the Edge to
Deliver Business Value
Vera Serdiukova
AI Product Manager
September 2020

2. © 2020 Vera Serdiukova
01GOALS
02 FIT
03 EXECUTION
AGENDA:

3. © 2020 Vera Serdiukova
DEFINITIONS:
EDGE DEVICE - a device whose compute, memory, and
energy resources are constrained and cannot be easily
changed.
LEARNING AT THE EDGE - training of a machine learning
model on the edge device.

4. © 2020 Vera Serdiukova 4

5. © 2020 Vera Serdiukova 5
- What is your objective?
01GOALS

6. © 2020 Vera Serdiukova 6
CAN LEARNING AT
THE EDGE HELP?
02 FIT
01GOALS

7. © 2020 Vera Serdiukova
AUTONOMY ACCURACY
SPEED
PRIVACY
EVOLUTION
EDGE AI
SECURITY
02 FIT
01GOALS
- What are the benefits?

8. © 2020 Vera Serdiukova
AUTONOMY ACCURACY
PRIVACY
EVOLUTION
LEARNING
AT THE EDGE
SECURITY
02 FIT
01GOALS
- What are the benefits?

9. © 2020 Vera Serdiukova 9
02 FIT
01GOALS
- What are the benefits?

10. © 2020 Vera Serdiukova 10
02 FIT
01GOALS
- What are the benefits?
Source: https://www.android.com/versions/pie-9-0/
Android 9 Pie
EXAMPLE

11. © 2020 Vera Serdiukova 11
FEASIBLE?
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?

12. © 2020 Vera Serdiukova 12
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?

13. © 2020 Vera Serdiukova 13
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
TRAINING DATA
TRAIN
ITERATE
IMPROVE
COMPUTE MEMORY POWER
RESOURCES

14. © 2020 Vera Serdiukova 14
“Model is only as good
as the training data
- Anybody who has ever built a ML model
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?

15. © 2020 Vera Serdiukova 15
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
UNAVAILABLE

16. © 2020 Vera Serdiukova 16
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
LIMITED

17. © 2020 Vera Serdiukova 17
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
MINIMAL/ NO
DATA
PREPROCESSING

18. © 2020 Vera Serdiukova 18
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
LIMITED/ NO LABELED
DATA
????
????
????
????

19. © 2020 Vera Serdiukova 19
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
TRAIN
ITERATE
IMPROVE
COMPUTE MEMORY POWER

20. © 2020 Vera Serdiukova 20
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
Scenario
#1
Build a new
product/ device
from scratch

21. © 2020 Vera Serdiukova 21
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
VS.
Scenario
#1
Build a new
product/ device
from scratch
Scenario
#2
Improve an
existing product/
device

22. © 2020 Vera Serdiukova 22
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
V S .
YOU ARE SOMEWHERE HERE

23. © 2020 Vera Serdiukova 23
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
- What are the costs?
V S .

24. © 2020 Vera Serdiukova 24
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
- What are the costs?
V S .

25. © 2020 Vera Serdiukova 25
BOTTOM LINE
02 FIT
01GOALS
- What are the benefits?
- What are the
prerequisites?
- What are the costs?
- Does it make sense?

26. © 2020 Vera Serdiukova 26
01GOALS
03 EXECUTION
02 FIT

27. © 2020 Vera Serdiukova 27
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
LIMITED & MINIMALLY
PREPROSSED DATA

28. © 2020 Vera Serdiukova 28
OBTAIN QUALITY DATA
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.

29. © 2020 Vera Serdiukova 29
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
EXAMPLE

30. © 2020 Vera Serdiukova 30
TRAIN
ITERATE
IMPROVE
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?

31. © 2020 Vera Serdiukova 31
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
LEARNING AT THE EDGE:
APPROACHES
GENERIC MODEL
TRAINED ON
CLOUD/ PREM
+
MODEL REFINED
ON-DEVICE

32. © 2020 Vera Serdiukova 32
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
Source: https://developer.apple.com/machine-learning/core-ml/
EXAMPLE

33. © 2020 Vera Serdiukova 33
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
Old
model
New
model
KEEPING MODELS UP-TO-DATE

34. © 2020 Vera Serdiukova 34
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
LEARNING AT THE EDGE:
APPROACHES
GENERIC MODEL
TRAINED ON
CLOUD/ PREM
+
MODEL REFINED
ON-DEVICE
COLLABORATIVE
LEARNING
(E.G. FEDERATED
LEARNING)

35. © 2020 Vera Serdiukova 35
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
Gboard
Source: https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
EXAMPLE

36. © 2020 Vera Serdiukova 36
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
LEARNING AT THE EDGE:
APPROACHES
GENERIC MODEL
TRAINED ON
CLOUD/ PREM
+
MODEL REFINED
ON-DEVICE
COLLABORATIVE
LEARNING
(E.G. FEDERATED
LEARNING)
MODEL TRAINED
FROM SCRATCH
+
POTENTIALLY
RETRAINED
ON-DEVICE ONLY

37. © 2020 Vera Serdiukova 37
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
KNOWLEDGE SILOS

38. © 2020 Vera Serdiukova 38
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
- What else shall I take into
account?
A FEW ADDITIONAL
CONSIDERATIONS

39. © 2020 Vera Serdiukova 39
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
- What else shall I take into
account?
MODEL LOSS

40. © 2020 Vera Serdiukova 40
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
- What else shall I take into
account?
EXPERIENCE CONSISTENCY

41. © 2020 Vera Serdiukova 41
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
- What else shall I take into
account?
MODEL EVOLUTION TRACKABILITY

42. © 2020 Vera Serdiukova 42
01GOALS
03 EXECUTION
02 FIT
- Where do I start? Data.
- How does learning
happen?
- What else shall I take into
account?
FUN? YES!
TRIVIAL? NO!

43. © 2020 Vera Serdiukova 43

44. © 2020 Vera Serdiukova 44
RESOURCES:
Publications
On-Device Machine Learning: An
Algorithms and Learning Theory
Perspective
https://arxiv.org/abs/1911.00623
On-device Learning: Examples
Android 9 Pie, Adaptive Battery and
Brightness
https://www.android.com/versions/pie-9-
0/
Core ML 3, On-device Training
https://developer.apple.com/machine-
learning/core-ml/
Federated Learning: Collaborative
Machine Learning without Centralized
Training Data
https://ai.googleblog.com/2017/04/federa
ted-learning-collaborative.html

45. © 2020 Vera Serdiukova 45
THANK YOU
Vera Serdiukova
in/veraserdiukova/