SlideShare a Scribd company logo
1 of 50
Download to read offline
Text
Deep Learning and Business Models
Tran Quoc Hoan
The University of Tokyo
VNITC@2015-09-13
2
Today agendas
Deep Learning boom
Essentials of Deep Learning
Deep Learning - the newest
Deep Learning and Business Models
3
What is Deep Learning?
Deep Learning is a new trend of machine learning that enables
machines to unravel high level abstraction in large amount of data
Image
recognition
Speech
recognition
Customer Centric
Management
Natural Language
Processing
Drug Discovery
& Toxicology
Deep Learning Boom
–Albert Einstein
“If we knew what it was we were doing, it would not
be called research, would it?”
Big News - ILSVRC 2012
ILSVRC 2012 (ImageNet
Large Scale Visual
Recognition)
SuperVision
ISI
OXFORD_VGG
XRCE/INRIA
UnivofAmsterdam
29.576%
27.058%26.979%26.172%
15.315%
Error rate
Image source: http://cs.stanford.edu/people/karpathy/cnnembed/
5
6
Big News - Google brain
Self-taught learning with
unlabelled youtube
videos and 16,000
computers
AI grandmother cell (2012)
(Cat)(Human)
7
Data and Machine Learning
Most learning
algorithms
New AI method
(Deep Learning)
Amount of data
Performance
Image source: http://cs229.stanford.edu/materials/CS229-DeepLearning.pdf
8
New boom for the Giants & Startups
Google brain
project (2012)
Facebook AI Research
Lab (2013 Dec.)
Project Adam (2014)
Enlitic
Ersatz Labs
MetaMind
Nervana Systems
Skymind
Deepmind
One of 10 breakthrough technologies 2013
(MIT Technology Review)
PFN (Japan)
9
Pioneers
Geoffrey Hinton

(Toronto, Google)
Yoshua Bengio

(Montreal)
Yann LeCun

(NewYork, Facebook)
Andrew Ng

(Stanford, Baidu)
Jeffrey Dean

(Google)
Le Viet Quoc

(Stanford, Google)
Theory +
algorithms
Implementation
10
Sub-summary
・More than 2000 papers in 2014 ~ 2015 with 47 Google services.
Essentials of Deep Learning
–Albert Einstein
“If we knew what it was we were doing, it would not
be called research, would it?”
12
The basic calculation
x1
x2
x3
+1
z
f
Active function
z = f(x1w1 + x2w2 + x3w3 + w4)
w1
w2
w3
w4
Input
Forward propagation
wi: weight parameters
13
Neural network (1/2)
x1
x2
x3
+1
Input layer
+1 +1
y
Hidden layers
Output layer
f
f
f
f
h1
h2
z1
z2
Put deeper
The deeper layer displays for comprehensive
combination of small parts
Prob(cat) = 99%
14
Neural network (2/2)
x1
x2
x3
+1
Input layer
+1 +1
y1
Hidden layers
• Loss function E measures
how output labels are similar
to true labels.

• For N input data
{w1ij}
{w2ij}
{w3ij}
Supervisor learning
y2
yk
Output
(Ex. probability of each in k class
in classification task)
Find parameters {w} to minimize E → How?
15
Backward propagation
x1
x2
x3
+1
Input layer
r s
y
Output layer
y*
E
w
∂E
∂y
∂E
∂y
∂y
∂s
∂E
∂y
∂y
∂s
∂s
∂w
∂E
∂w
=
Backward from loss to update parameters
(true label)
How E change
when move w
How E change when move s
How E change when
move y
16
Vanishing gradient problem
x1
x2
x3
+1
+1 +1
y
y*
E
…
Error vanish with back-propagation at low layer
・Neural network architecture reached the limit before
deep learning boom
17
Features representation
・Deep learning reduced the human’s time-consuming of
features selection process in classification task
Features selection
Parameter learning
Answer
Features selection
+
Parameter learning
Answer
Previous methods Deep Learning
Convolutional Neural Network
ConvNet diagram from Torch Tutorial
Sub sampling
Emphasis special
features
The most popular kind of deep learning model
18
How to train parameters
Trainable parameters
Values in each kernel of convolutional layer
Weigh values and bias in fully connection layer
Stochastic Gradient Descent (SGD) (others: AdaGrad, Adam,…)
Parameters update
W ← W - ß
∂E
∂W
Backward propagation
Learning rate
SGD - Mini batch
Batch learning: update all samples in each update (over-fitting)
Mini batch: update parameters with some samples
20
Training trick
Data normalization
Reduce learning rate after some iterations
Momentum SGD
21
22
Auto-encoder (1/2)
y
…
How to initialize these parameters
23
Auto-encoder (2/2)
The ability of
reconstruct input
makes a good
initialization for
parameters
x x’
h(x)
Reconstruct error
E = ||x - x’||
Encode Decode
24
Robust algorithms
Data augmentation: more
noisy data → more robust
model
Nodes in network don’t
need to active all (mimic
human brain) → 

Dropout concept
…
Deep Learning - The Newest
–Albert Einstein
“If we knew what it was we were doing, it would not
be called research, would it?”
26
In Services
Google
Translate App
Deep learning inside (work offline)
July 29, 2015 version
27
In Services
My implemented service http://yelpio.hongo.wide.ad.jp/
28
In Medical Imaging
Segmentation
Offline due to image’s copyright
In Automatic Devices
http://blogs.nvidia.com/blog/2015/02/24/deep-learning-drive/
Self-driving car
29
In Speech Interfaces
99% is much different with 95%
(Andrew Ng. - Baidu)
https://medium.com/s-c-a-l-e/how-baidu-mastered-mandarin-with-deep-
learning-and-lots-of-data-1d94032564a5 30
In Drug Discovery
Active
Active
Inactive
1
0
1
0
0
1
10010000110011
Chemical
compound
Assay data Finger print
+ Activity
Deep Neural Net
PubChem
Database
Prediction of Drug Activity
multiple targets
http://on-demand.gputechconf.com/gtc/2015/presentation/S5813-Nobuyuki-Ota.pdf 31
Applications in IoT(Internet of Things)
More difficult tasks in AI, robotics,
information processing
Huge amount of time series data
and states of sensor & devices data
RNN (recurrent neural network)
Difficult to get supervisor data
VAE (variational auto encoder)
Take action in conditions,
environments
Deep Reinforcement Learning
Image source:

http://on-demand.gputechconf.com/gtc/2015/presentation/S5813-
Nobuyuki-Ota.pdf
32
RNN (1/3)
Recurrent Neural Network: loop inside neural network
Represent for time series data, sequence of inputs
(speech model, natural language model,…)
.
.
.
.
.
.
.
.
.
x(t) y(t)
z(t) = f( z(t-1), x(t) )
33
RNN (2/3)
Time series extension of RNN
…
…
t = 1 t = 2 t = 3 t = T
Back propagation through time (BPTT)
+ Long Short-Term Memory
How to learn
parameters?
34
…
…
…
…
…
…
…
…
…
…
…
RNN (3/3)
Neural Turing Machine [A. Graves, 2014]
Neural network which has the capability of coupling the external memories
Applications: COPY, PRIORITY SORT
35
NN with
parameters to
coupling to
external
memories
36
VAE (variational auto encoder)
Learn a mapping from some latent variable z to a
complicated distribution on x
p(x) = ∫p(x,z)dz where p(x, z) = p(x|z)p(z)
p(z) = something simple and p(x|z) = f(z) = neural network
Encode
VAE
Example: generate face image from 29 hidden
variables
http://vdumoulin.github.io/morphing_faces/online_demo.html
37
VAE
Example: learning from number
and writing style
Experiment (PFN): given a
number x, produce outputs as
remained numbers with writing
style of x
VAE: useful in half-supervised
learning process (especially when
training data are not enough)
Image from ディープラーニングが活かすIoT, Preferred Networks, Inc. 2015/06/09 Interop 2015 seminar
39
Reinforcement Learning (RL)
Environment
(street, factory,…)
Agent

(car, robot,…)
Action
a(t)
Reward r(t)
State s(t)
Total rewards in the future
R = r(t) + µr(t+1) + µ2r(t+2) + …
(µ < 1)
Design next action
40
Deep Reinforcement Learning (RL)
Q - learning: need to know future expectation reward Q(s, a)
of action a at state s (Bellman update equation)

Q(s, a) <— Q(s, a) + ß ( r + µ maxa’ Q(s’, a’) - Q(s, a) )
Deep Q-Learning Network [V. Mnih, 2015]
Get Q(s, a) by a deep neural network Q(s, a, w)
Maximize: L(w) = E[ (r + µ max Q(s’, a’, w) - Q(s, a, w))2 ]
Useful when there are many states
41
PFN - demo https://www.youtube.com/watch?v=RH2TmreYkdA
Deep Learning and Business Models
–Albert Einstein
“If we knew what it was we were doing, it would not
be called research, would it?”
The future of the field
Better hardware and bigger
machine cluster
Better implementations and
optimizations
Understand video, text and
signals
Develop in business models
Deep learning is just
the tip of the iceberg
43
Model 1 - Framework development
Bridge the gap between
algorithms and
implementation
Provide common
interfaces and
understandable API for
users
Pylearn2
44
Model 2 - Building a deep-able hardware
New hardware (chipset)
architecture for Deep
Learning
Synopsys
http://www.bdti.com/InsideDSP/2015/04/21/Synopsys
Processors’ convolutional network capabilities
Qualcomm Zeroth Platform
Cognitive computing and custom CPU
drive Snapdragon 820 processors
https://www.qualcomm.com/news/snapdragon/2015/03/02/cognitive-
computing-and-custom-cpu-drive-next-gen-snapdragon-processors
NVIDIA DIGITS
DevBox
https://developer.nvidia.com/devbox
45
Model 3 - Deep Intelligences for IoT
Sensing
Action
Feedback
Analysis
Union
Image from ディープラーニングが活かすIoT, Preferred Networks, Inc. 2015/06/09 Interop 2015 seminar
Model 4 - Host API and deep learning service
Data
in
Model
out
https://www.metamind.io/vision/train
https://www.metamind.io/language/train
Usability + Tuning + Scale out
47
Model 5 - Personal Deep Learning
Data
in
Label
out
DeepFace
(Facebook)
http://www.adweek.com/socialtimes/deepface/433401
Facial verification & tagging
Applied previous solutions
Almost free 48
Summary
Deep Learning has the capability to find patterns
among data by enabling wide range of
abstraction.
Deep Learning shown significant results in voice
and image recognition compared with
conventional machine learning methods.
Deep Learning has potential applications and
business models in some important key sectors.
The challenges are your ideas !
49
“Thank you for listening.”
• Some good materials for learning
• Papers, code, blog, seminars, online courses
• For beginner: 深層学習 (機械学習プロフェッショナルシリーズ)
• Deep Learning - an MIT Press book in preparation
• http://www.iro.umontreal.ca/~bengioy/dlbook/
50

More Related Content

What's hot

Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningMustafa Aldemir
 
Yann le cun
Yann le cunYann le cun
Yann le cunYandex
 
Deep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsDeep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsBuhwan Jeong
 
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...Greg Makowski
 
Machine Learning and Deep Learning with R
Machine Learning and Deep Learning with RMachine Learning and Deep Learning with R
Machine Learning and Deep Learning with RPoo Kuan Hoong
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual IntroductionLukas Masuch
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep LearningDavid Rostcheck
 
Intro To Convolutional Neural Networks
Intro To Convolutional Neural NetworksIntro To Convolutional Neural Networks
Intro To Convolutional Neural NetworksMark Scully
 
Deep learning intro
Deep learning introDeep learning intro
Deep learning introbeamandrew
 
Deep Learning - A Literature survey
Deep Learning - A Literature surveyDeep Learning - A Literature survey
Deep Learning - A Literature surveyAkshay Hegde
 
From Conventional Machine Learning to Deep Learning and Beyond.pptx
From Conventional Machine Learning to Deep Learning and Beyond.pptxFrom Conventional Machine Learning to Deep Learning and Beyond.pptx
From Conventional Machine Learning to Deep Learning and Beyond.pptxChun-Hao Chang
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorRoelof Pieters
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningOleg Mygryn
 
Deep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image ProcessingDeep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image ProcessingGrigory Sapunov
 
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...Andrew Gardner
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work IIMohamed Loey
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep LearningPoo Kuan Hoong
 
Deep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with styleDeep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with styleRoelof Pieters
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...GeeksLab Odessa
 

What's hot (20)

Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Yann le cun
Yann le cunYann le cun
Yann le cun
 
Deep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsDeep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applications
 
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
Using Deep Learning to do Real-Time Scoring in Practical Applications - 2015-...
 
Machine Learning and Deep Learning with R
Machine Learning and Deep Learning with RMachine Learning and Deep Learning with R
Machine Learning and Deep Learning with R
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
 
An introduction to Deep Learning
An introduction to Deep LearningAn introduction to Deep Learning
An introduction to Deep Learning
 
Intro To Convolutional Neural Networks
Intro To Convolutional Neural NetworksIntro To Convolutional Neural Networks
Intro To Convolutional Neural Networks
 
Deep learning intro
Deep learning introDeep learning intro
Deep learning intro
 
Deep Learning - A Literature survey
Deep Learning - A Literature surveyDeep Learning - A Literature survey
Deep Learning - A Literature survey
 
From Conventional Machine Learning to Deep Learning and Beyond.pptx
From Conventional Machine Learning to Deep Learning and Beyond.pptxFrom Conventional Machine Learning to Deep Learning and Beyond.pptx
From Conventional Machine Learning to Deep Learning and Beyond.pptx
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Deep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image ProcessingDeep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image Processing
 
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
Deep Learning for Data Scientists - Data Science ATL Meetup Presentation, 201...
 
Deep Learning - Overview of my work II
Deep Learning - Overview of my work IIDeep Learning - Overview of my work II
Deep Learning - Overview of my work II
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
 
Deep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with styleDeep Neural Networks 
that talk (Back)… with style
Deep Neural Networks 
that talk (Back)… with style
 
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...AI&BigData Lab. Артем Чернодуб  "Распознавание изображений методом Lazy Deep ...
AI&BigData Lab. Артем Чернодуб "Распознавание изображений методом Lazy Deep ...
 
Andrew Ng, Chief Scientist at Baidu
Andrew Ng, Chief Scientist at BaiduAndrew Ng, Chief Scientist at Baidu
Andrew Ng, Chief Scientist at Baidu
 

Similar to Deep Learning And Business Models (VNITC 2015-09-13)

Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learningAmr Rashed
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Julien SIMON
 
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakLearn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakPyData
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeSiby Jose Plathottam
 
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Wanjin Yu
 
2017 07 03_meetup_d
2017 07 03_meetup_d2017 07 03_meetup_d
2017 07 03_meetup_dDana Brophy
 
2017 07 03_meetup_d
2017 07 03_meetup_d2017 07 03_meetup_d
2017 07 03_meetup_dDana Brophy
 
1st review android malware.pptx
1st review  android malware.pptx1st review  android malware.pptx
1st review android malware.pptxNambiraju
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning ApplicationsNVIDIA Taiwan
 
Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk
Deep Learning Enabled Question Answering System to Automate Corporate HelpdeskDeep Learning Enabled Question Answering System to Automate Corporate Helpdesk
Deep Learning Enabled Question Answering System to Automate Corporate HelpdeskSaurabh Saxena
 
Edge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningEdge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningZiqiang Feng
 
The deep learning tour - Q1 2017
The deep learning tour - Q1 2017 The deep learning tour - Q1 2017
The deep learning tour - Q1 2017 Eran Shlomo
 
Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)Julien SIMON
 
Machine learning the next revolution or just another hype
Machine learning   the next revolution or just another hypeMachine learning   the next revolution or just another hype
Machine learning the next revolution or just another hypeJorge Ferrer
 
Open source ai_technical_trend
Open source ai_technical_trendOpen source ai_technical_trend
Open source ai_technical_trendMario Cho
 
Camp IT: Making the World More Efficient Using AI & Machine Learning
Camp IT: Making the World More Efficient Using AI & Machine LearningCamp IT: Making the World More Efficient Using AI & Machine Learning
Camp IT: Making the World More Efficient Using AI & Machine LearningKrzysztof Kowalczyk
 
Using Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsUsing Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsGreg Makowski
 
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...APJ ABDUL KALAM TECHNICAL UNIVERSITY
 

Similar to Deep Learning And Business Models (VNITC 2015-09-13) (20)

AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)
 
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr TeterwakLearn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
Learn to Build an App to Find Similar Images using Deep Learning- Piotr Teterwak
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
 
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
 
2017 07 03_meetup_d
2017 07 03_meetup_d2017 07 03_meetup_d
2017 07 03_meetup_d
 
2017 07 03_meetup_d
2017 07 03_meetup_d2017 07 03_meetup_d
2017 07 03_meetup_d
 
1st review android malware.pptx
1st review  android malware.pptx1st review  android malware.pptx
1st review android malware.pptx
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
 
Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk
Deep Learning Enabled Question Answering System to Automate Corporate HelpdeskDeep Learning Enabled Question Answering System to Automate Corporate Helpdesk
Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk
 
Null
NullNull
Null
 
Edge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningEdge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine Learning
 
The deep learning tour - Q1 2017
The deep learning tour - Q1 2017 The deep learning tour - Q1 2017
The deep learning tour - Q1 2017
 
Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)Deep Learning: concepts and use cases (October 2018)
Deep Learning: concepts and use cases (October 2018)
 
Machine learning the next revolution or just another hype
Machine learning   the next revolution or just another hypeMachine learning   the next revolution or just another hype
Machine learning the next revolution or just another hype
 
Open source ai_technical_trend
Open source ai_technical_trendOpen source ai_technical_trend
Open source ai_technical_trend
 
Camp IT: Making the World More Efficient Using AI & Machine Learning
Camp IT: Making the World More Efficient Using AI & Machine LearningCamp IT: Making the World More Efficient Using AI & Machine Learning
Camp IT: Making the World More Efficient Using AI & Machine Learning
 
Using Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsUsing Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical Applications
 
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...
Industrial training (Artificial Intelligence, Machine Learning & Deep Learnin...
 

More from Ha Phuong

QTML2021 UAP Quantum Feature Map
QTML2021 UAP Quantum Feature MapQTML2021 UAP Quantum Feature Map
QTML2021 UAP Quantum Feature MapHa Phuong
 
CCS2019-opological time-series analysis with delay-variant embedding
CCS2019-opological time-series analysis with delay-variant embeddingCCS2019-opological time-series analysis with delay-variant embedding
CCS2019-opological time-series analysis with delay-variant embeddingHa Phuong
 
SIAM-AG21-Topological Persistence Machine of Phase Transition
SIAM-AG21-Topological Persistence Machine of Phase TransitionSIAM-AG21-Topological Persistence Machine of Phase Transition
SIAM-AG21-Topological Persistence Machine of Phase TransitionHa Phuong
 
018 20160902 Machine Learning Framework for Analysis of Transport through Com...
018 20160902 Machine Learning Framework for Analysis of Transport through Com...018 20160902 Machine Learning Framework for Analysis of Transport through Com...
018 20160902 Machine Learning Framework for Analysis of Transport through Com...Ha Phuong
 
017_20160826 Thermodynamics Of Stochastic Turing Machines
017_20160826 Thermodynamics Of Stochastic Turing Machines017_20160826 Thermodynamics Of Stochastic Turing Machines
017_20160826 Thermodynamics Of Stochastic Turing MachinesHa Phuong
 
016_20160722 Molecular Circuits For Dynamic Noise Filtering
016_20160722 Molecular Circuits For Dynamic Noise Filtering016_20160722 Molecular Circuits For Dynamic Noise Filtering
016_20160722 Molecular Circuits For Dynamic Noise FilteringHa Phuong
 
015_20160422 Controlling Synchronous Patterns In Complex Networks
015_20160422 Controlling Synchronous Patterns In Complex Networks015_20160422 Controlling Synchronous Patterns In Complex Networks
015_20160422 Controlling Synchronous Patterns In Complex NetworksHa Phuong
 
Tutorial of topological data analysis part 3(Mapper algorithm)
Tutorial of topological data analysis part 3(Mapper algorithm)Tutorial of topological data analysis part 3(Mapper algorithm)
Tutorial of topological data analysis part 3(Mapper algorithm)Ha Phuong
 
Tutorial of topological_data_analysis_part_1(basic)
Tutorial of topological_data_analysis_part_1(basic)Tutorial of topological_data_analysis_part_1(basic)
Tutorial of topological_data_analysis_part_1(basic)Ha Phuong
 
013_20160328_Topological_Measurement_Of_Protein_Compressibility
013_20160328_Topological_Measurement_Of_Protein_Compressibility013_20160328_Topological_Measurement_Of_Protein_Compressibility
013_20160328_Topological_Measurement_Of_Protein_CompressibilityHa Phuong
 
011_20160321_Topological_data_analysis_of_contagion_map
011_20160321_Topological_data_analysis_of_contagion_map011_20160321_Topological_data_analysis_of_contagion_map
011_20160321_Topological_data_analysis_of_contagion_mapHa Phuong
 
010_20160216_Variational Gaussian Process
010_20160216_Variational Gaussian Process010_20160216_Variational Gaussian Process
010_20160216_Variational Gaussian ProcessHa Phuong
 
009_20150201_Structural Inference for Uncertain Networks
009_20150201_Structural Inference for Uncertain Networks009_20150201_Structural Inference for Uncertain Networks
009_20150201_Structural Inference for Uncertain NetworksHa Phuong
 
PRML Reading Chapter 11 - Sampling Method
PRML Reading Chapter 11 - Sampling MethodPRML Reading Chapter 11 - Sampling Method
PRML Reading Chapter 11 - Sampling MethodHa Phuong
 
Approximate Inference (Chapter 10, PRML Reading)
Approximate Inference (Chapter 10, PRML Reading)Approximate Inference (Chapter 10, PRML Reading)
Approximate Inference (Chapter 10, PRML Reading)Ha Phuong
 
008 20151221 Return of Frustrating Easy Domain Adaptation
008 20151221 Return of Frustrating Easy Domain Adaptation008 20151221 Return of Frustrating Easy Domain Adaptation
008 20151221 Return of Frustrating Easy Domain AdaptationHa Phuong
 
007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics
007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics
007 20151214 Deep Unsupervised Learning using Nonequlibrium ThermodynamicsHa Phuong
 
006 20151207 draws - Deep Recurrent Attentive Writer
006 20151207 draws - Deep Recurrent Attentive Writer006 20151207 draws - Deep Recurrent Attentive Writer
006 20151207 draws - Deep Recurrent Attentive WriterHa Phuong
 
005 20151130 adversary_networks
005 20151130 adversary_networks005 20151130 adversary_networks
005 20151130 adversary_networksHa Phuong
 
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamicsHa Phuong
 

More from Ha Phuong (20)

QTML2021 UAP Quantum Feature Map
QTML2021 UAP Quantum Feature MapQTML2021 UAP Quantum Feature Map
QTML2021 UAP Quantum Feature Map
 
CCS2019-opological time-series analysis with delay-variant embedding
CCS2019-opological time-series analysis with delay-variant embeddingCCS2019-opological time-series analysis with delay-variant embedding
CCS2019-opological time-series analysis with delay-variant embedding
 
SIAM-AG21-Topological Persistence Machine of Phase Transition
SIAM-AG21-Topological Persistence Machine of Phase TransitionSIAM-AG21-Topological Persistence Machine of Phase Transition
SIAM-AG21-Topological Persistence Machine of Phase Transition
 
018 20160902 Machine Learning Framework for Analysis of Transport through Com...
018 20160902 Machine Learning Framework for Analysis of Transport through Com...018 20160902 Machine Learning Framework for Analysis of Transport through Com...
018 20160902 Machine Learning Framework for Analysis of Transport through Com...
 
017_20160826 Thermodynamics Of Stochastic Turing Machines
017_20160826 Thermodynamics Of Stochastic Turing Machines017_20160826 Thermodynamics Of Stochastic Turing Machines
017_20160826 Thermodynamics Of Stochastic Turing Machines
 
016_20160722 Molecular Circuits For Dynamic Noise Filtering
016_20160722 Molecular Circuits For Dynamic Noise Filtering016_20160722 Molecular Circuits For Dynamic Noise Filtering
016_20160722 Molecular Circuits For Dynamic Noise Filtering
 
015_20160422 Controlling Synchronous Patterns In Complex Networks
015_20160422 Controlling Synchronous Patterns In Complex Networks015_20160422 Controlling Synchronous Patterns In Complex Networks
015_20160422 Controlling Synchronous Patterns In Complex Networks
 
Tutorial of topological data analysis part 3(Mapper algorithm)
Tutorial of topological data analysis part 3(Mapper algorithm)Tutorial of topological data analysis part 3(Mapper algorithm)
Tutorial of topological data analysis part 3(Mapper algorithm)
 
Tutorial of topological_data_analysis_part_1(basic)
Tutorial of topological_data_analysis_part_1(basic)Tutorial of topological_data_analysis_part_1(basic)
Tutorial of topological_data_analysis_part_1(basic)
 
013_20160328_Topological_Measurement_Of_Protein_Compressibility
013_20160328_Topological_Measurement_Of_Protein_Compressibility013_20160328_Topological_Measurement_Of_Protein_Compressibility
013_20160328_Topological_Measurement_Of_Protein_Compressibility
 
011_20160321_Topological_data_analysis_of_contagion_map
011_20160321_Topological_data_analysis_of_contagion_map011_20160321_Topological_data_analysis_of_contagion_map
011_20160321_Topological_data_analysis_of_contagion_map
 
010_20160216_Variational Gaussian Process
010_20160216_Variational Gaussian Process010_20160216_Variational Gaussian Process
010_20160216_Variational Gaussian Process
 
009_20150201_Structural Inference for Uncertain Networks
009_20150201_Structural Inference for Uncertain Networks009_20150201_Structural Inference for Uncertain Networks
009_20150201_Structural Inference for Uncertain Networks
 
PRML Reading Chapter 11 - Sampling Method
PRML Reading Chapter 11 - Sampling MethodPRML Reading Chapter 11 - Sampling Method
PRML Reading Chapter 11 - Sampling Method
 
Approximate Inference (Chapter 10, PRML Reading)
Approximate Inference (Chapter 10, PRML Reading)Approximate Inference (Chapter 10, PRML Reading)
Approximate Inference (Chapter 10, PRML Reading)
 
008 20151221 Return of Frustrating Easy Domain Adaptation
008 20151221 Return of Frustrating Easy Domain Adaptation008 20151221 Return of Frustrating Easy Domain Adaptation
008 20151221 Return of Frustrating Easy Domain Adaptation
 
007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics
007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics
007 20151214 Deep Unsupervised Learning using Nonequlibrium Thermodynamics
 
006 20151207 draws - Deep Recurrent Attentive Writer
006 20151207 draws - Deep Recurrent Attentive Writer006 20151207 draws - Deep Recurrent Attentive Writer
006 20151207 draws - Deep Recurrent Attentive Writer
 
005 20151130 adversary_networks
005 20151130 adversary_networks005 20151130 adversary_networks
005 20151130 adversary_networks
 
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics
004 20151116 deep_unsupervisedlearningusingnonequlibriumthermodynamics
 

Recently uploaded

2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17Celine George
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICESayali Powar
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptxmary850239
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational PhilosophyShuvankar Madhu
 

Recently uploaded (20)

2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17How to Add a many2many Relational Field in Odoo 17
How to Add a many2many Relational Field in Odoo 17
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICE
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptx
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational Philosophy
 

Deep Learning And Business Models (VNITC 2015-09-13)

  • 1. Text Deep Learning and Business Models Tran Quoc Hoan The University of Tokyo VNITC@2015-09-13
  • 2. 2 Today agendas Deep Learning boom Essentials of Deep Learning Deep Learning - the newest Deep Learning and Business Models
  • 3. 3 What is Deep Learning? Deep Learning is a new trend of machine learning that enables machines to unravel high level abstraction in large amount of data Image recognition Speech recognition Customer Centric Management Natural Language Processing Drug Discovery & Toxicology
  • 4. Deep Learning Boom –Albert Einstein “If we knew what it was we were doing, it would not be called research, would it?”
  • 5. Big News - ILSVRC 2012 ILSVRC 2012 (ImageNet Large Scale Visual Recognition) SuperVision ISI OXFORD_VGG XRCE/INRIA UnivofAmsterdam 29.576% 27.058%26.979%26.172% 15.315% Error rate Image source: http://cs.stanford.edu/people/karpathy/cnnembed/ 5
  • 6. 6 Big News - Google brain Self-taught learning with unlabelled youtube videos and 16,000 computers AI grandmother cell (2012) (Cat)(Human)
  • 7. 7 Data and Machine Learning Most learning algorithms New AI method (Deep Learning) Amount of data Performance Image source: http://cs229.stanford.edu/materials/CS229-DeepLearning.pdf
  • 8. 8 New boom for the Giants & Startups Google brain project (2012) Facebook AI Research Lab (2013 Dec.) Project Adam (2014) Enlitic Ersatz Labs MetaMind Nervana Systems Skymind Deepmind One of 10 breakthrough technologies 2013 (MIT Technology Review) PFN (Japan)
  • 9. 9 Pioneers Geoffrey Hinton
 (Toronto, Google) Yoshua Bengio
 (Montreal) Yann LeCun
 (NewYork, Facebook) Andrew Ng
 (Stanford, Baidu) Jeffrey Dean
 (Google) Le Viet Quoc
 (Stanford, Google) Theory + algorithms Implementation
  • 10. 10 Sub-summary ・More than 2000 papers in 2014 ~ 2015 with 47 Google services.
  • 11. Essentials of Deep Learning –Albert Einstein “If we knew what it was we were doing, it would not be called research, would it?”
  • 12. 12 The basic calculation x1 x2 x3 +1 z f Active function z = f(x1w1 + x2w2 + x3w3 + w4) w1 w2 w3 w4 Input Forward propagation wi: weight parameters
  • 13. 13 Neural network (1/2) x1 x2 x3 +1 Input layer +1 +1 y Hidden layers Output layer f f f f h1 h2 z1 z2 Put deeper The deeper layer displays for comprehensive combination of small parts Prob(cat) = 99%
  • 14. 14 Neural network (2/2) x1 x2 x3 +1 Input layer +1 +1 y1 Hidden layers • Loss function E measures how output labels are similar to true labels.
 • For N input data {w1ij} {w2ij} {w3ij} Supervisor learning y2 yk Output (Ex. probability of each in k class in classification task) Find parameters {w} to minimize E → How?
  • 15. 15 Backward propagation x1 x2 x3 +1 Input layer r s y Output layer y* E w ∂E ∂y ∂E ∂y ∂y ∂s ∂E ∂y ∂y ∂s ∂s ∂w ∂E ∂w = Backward from loss to update parameters (true label) How E change when move w How E change when move s How E change when move y
  • 16. 16 Vanishing gradient problem x1 x2 x3 +1 +1 +1 y y* E … Error vanish with back-propagation at low layer ・Neural network architecture reached the limit before deep learning boom
  • 17. 17 Features representation ・Deep learning reduced the human’s time-consuming of features selection process in classification task Features selection Parameter learning Answer Features selection + Parameter learning Answer Previous methods Deep Learning
  • 18. Convolutional Neural Network ConvNet diagram from Torch Tutorial Sub sampling Emphasis special features The most popular kind of deep learning model 18
  • 19. How to train parameters Trainable parameters Values in each kernel of convolutional layer Weigh values and bias in fully connection layer Stochastic Gradient Descent (SGD) (others: AdaGrad, Adam,…) Parameters update W ← W - ß ∂E ∂W Backward propagation Learning rate
  • 20. SGD - Mini batch Batch learning: update all samples in each update (over-fitting) Mini batch: update parameters with some samples 20
  • 21. Training trick Data normalization Reduce learning rate after some iterations Momentum SGD 21
  • 22. 22 Auto-encoder (1/2) y … How to initialize these parameters
  • 23. 23 Auto-encoder (2/2) The ability of reconstruct input makes a good initialization for parameters x x’ h(x) Reconstruct error E = ||x - x’|| Encode Decode
  • 24. 24 Robust algorithms Data augmentation: more noisy data → more robust model Nodes in network don’t need to active all (mimic human brain) → 
 Dropout concept …
  • 25. Deep Learning - The Newest –Albert Einstein “If we knew what it was we were doing, it would not be called research, would it?”
  • 26. 26 In Services Google Translate App Deep learning inside (work offline) July 29, 2015 version
  • 27. 27 In Services My implemented service http://yelpio.hongo.wide.ad.jp/
  • 28. 28 In Medical Imaging Segmentation Offline due to image’s copyright
  • 30. In Speech Interfaces 99% is much different with 95% (Andrew Ng. - Baidu) https://medium.com/s-c-a-l-e/how-baidu-mastered-mandarin-with-deep- learning-and-lots-of-data-1d94032564a5 30
  • 31. In Drug Discovery Active Active Inactive 1 0 1 0 0 1 10010000110011 Chemical compound Assay data Finger print + Activity Deep Neural Net PubChem Database Prediction of Drug Activity multiple targets http://on-demand.gputechconf.com/gtc/2015/presentation/S5813-Nobuyuki-Ota.pdf 31
  • 32. Applications in IoT(Internet of Things) More difficult tasks in AI, robotics, information processing Huge amount of time series data and states of sensor & devices data RNN (recurrent neural network) Difficult to get supervisor data VAE (variational auto encoder) Take action in conditions, environments Deep Reinforcement Learning Image source:
 http://on-demand.gputechconf.com/gtc/2015/presentation/S5813- Nobuyuki-Ota.pdf 32
  • 33. RNN (1/3) Recurrent Neural Network: loop inside neural network Represent for time series data, sequence of inputs (speech model, natural language model,…) . . . . . . . . . x(t) y(t) z(t) = f( z(t-1), x(t) ) 33
  • 34. RNN (2/3) Time series extension of RNN … … t = 1 t = 2 t = 3 t = T Back propagation through time (BPTT) + Long Short-Term Memory How to learn parameters? 34 … … … … … … … … … … …
  • 35. RNN (3/3) Neural Turing Machine [A. Graves, 2014] Neural network which has the capability of coupling the external memories Applications: COPY, PRIORITY SORT 35 NN with parameters to coupling to external memories
  • 36. 36 VAE (variational auto encoder) Learn a mapping from some latent variable z to a complicated distribution on x p(x) = ∫p(x,z)dz where p(x, z) = p(x|z)p(z) p(z) = something simple and p(x|z) = f(z) = neural network Encode
  • 37. VAE Example: generate face image from 29 hidden variables http://vdumoulin.github.io/morphing_faces/online_demo.html 37
  • 38. VAE Example: learning from number and writing style Experiment (PFN): given a number x, produce outputs as remained numbers with writing style of x VAE: useful in half-supervised learning process (especially when training data are not enough) Image from ディープラーニングが活かすIoT, Preferred Networks, Inc. 2015/06/09 Interop 2015 seminar
  • 39. 39 Reinforcement Learning (RL) Environment (street, factory,…) Agent
 (car, robot,…) Action a(t) Reward r(t) State s(t) Total rewards in the future R = r(t) + µr(t+1) + µ2r(t+2) + … (µ < 1) Design next action
  • 40. 40 Deep Reinforcement Learning (RL) Q - learning: need to know future expectation reward Q(s, a) of action a at state s (Bellman update equation)
 Q(s, a) <— Q(s, a) + ß ( r + µ maxa’ Q(s’, a’) - Q(s, a) ) Deep Q-Learning Network [V. Mnih, 2015] Get Q(s, a) by a deep neural network Q(s, a, w) Maximize: L(w) = E[ (r + µ max Q(s’, a’, w) - Q(s, a, w))2 ] Useful when there are many states
  • 41. 41 PFN - demo https://www.youtube.com/watch?v=RH2TmreYkdA
  • 42. Deep Learning and Business Models –Albert Einstein “If we knew what it was we were doing, it would not be called research, would it?”
  • 43. The future of the field Better hardware and bigger machine cluster Better implementations and optimizations Understand video, text and signals Develop in business models Deep learning is just the tip of the iceberg 43
  • 44. Model 1 - Framework development Bridge the gap between algorithms and implementation Provide common interfaces and understandable API for users Pylearn2 44
  • 45. Model 2 - Building a deep-able hardware New hardware (chipset) architecture for Deep Learning Synopsys http://www.bdti.com/InsideDSP/2015/04/21/Synopsys Processors’ convolutional network capabilities Qualcomm Zeroth Platform Cognitive computing and custom CPU drive Snapdragon 820 processors https://www.qualcomm.com/news/snapdragon/2015/03/02/cognitive- computing-and-custom-cpu-drive-next-gen-snapdragon-processors NVIDIA DIGITS DevBox https://developer.nvidia.com/devbox 45
  • 46. Model 3 - Deep Intelligences for IoT Sensing Action Feedback Analysis Union Image from ディープラーニングが活かすIoT, Preferred Networks, Inc. 2015/06/09 Interop 2015 seminar
  • 47. Model 4 - Host API and deep learning service Data in Model out https://www.metamind.io/vision/train https://www.metamind.io/language/train Usability + Tuning + Scale out 47
  • 48. Model 5 - Personal Deep Learning Data in Label out DeepFace (Facebook) http://www.adweek.com/socialtimes/deepface/433401 Facial verification & tagging Applied previous solutions Almost free 48
  • 49. Summary Deep Learning has the capability to find patterns among data by enabling wide range of abstraction. Deep Learning shown significant results in voice and image recognition compared with conventional machine learning methods. Deep Learning has potential applications and business models in some important key sectors. The challenges are your ideas ! 49
  • 50. “Thank you for listening.” • Some good materials for learning • Papers, code, blog, seminars, online courses • For beginner: 深層学習 (機械学習プロフェッショナルシリーズ) • Deep Learning - an MIT Press book in preparation • http://www.iro.umontreal.ca/~bengioy/dlbook/ 50