SlideShare a Scribd company logo
1 of 42
Convolutional Deep Belief Networks for
Scalable Unsupervised Learning of
Hierarchical Representations
Jaehyun Ahn
(jaehyunahn@sogang.ac.kr)
서강대학교 데이터마이닝 연구실
Computer Science Department
Sogang University
Honglak Lee (ICML 2009, 744 quotes)
Key Reference
•  Convolutional Deep Belief Networks for Scalable
Unsupervised Learning of Hierarchical Representation
(Honglak Lee et al., ICML 2009)
•  Lecture at ICML 2009 (http://videolectures.net/icml09_lee_cdb/)
•  Learning Multiple Layers of Features from Tiny Images
(Alex Krizhevsky, 2009)
•  To be Bernoulli or to be Gaussian, for a Restricted
Boltzmann Machine (Takayoshi Yamashita et al., ICPR 2014)
7/21/15 2
Ideas
7/21/15 3
What this paper wants to say..
•  Taking advantages of Deep Belief Network
through Convolutional networks
–  Translation invariance
•  Max-pooling
–  Scalable to realistic image sizes
•  Max-pooling
–  Hierarchical probabilistic inference by combining
bottom-up and top-down information
•  Probabilistic max-pooling
7/21/15 4
Convolutional Neural Networks
7/21/15 5
Basic Design of Convolutional Networks
7/21/15 6What	
  is	
  this?	
  h)p://www.slideshare.net/sogo1127/101-­‐convolu>onal-­‐neural-­‐networks	
  
Alternate between “Detection” and “Pooling” layers
Advantages of Convolutional Networks
1. Translation Invariance
•  In LeNet, LeCun et al., 1998, Max-pooling provides a
form of translation invariance. If max-pooling is done
over a 2x2 region, 4 possible configurations will produce
exactly the same output at the convolutional layer.
7/21/15 7
Image:	
  IEEE	
  2013,	
  h)p://www.computer.org/csdl/trans/tp/2013/08/)p2013081930-­‐abs.html	
  
Advantages of Convolutional Networks
2. Scalable to realistic image sizes
•  This paper starts from this basic question “How can we
scale to realistic image sizes (e.g. 200x200 pixels)?”
•  Max-pooling shrinks the representation in higher layers
7/21/15 8
Image:	
  IEEE	
  2013,	
  h)p://www.computer.org/csdl/trans/tp/2013/08/)p2013081930-­‐abs.html	
  
Key Idea with advantages of Convolutional Networks
3. Hierarchical probabilistic inference Max-pooling
•  Max-pooling is deterministic and feed-forward only.
However, this paper gave a max-pooling to a
probabilistic semantics that enables to combine bottom-
up and top-down information.
7/21/15 9
Deep Belief Networks
7/21/15 10
General Deep Belief Network
7/21/15 11
Z: state probability
If the visible units are binary-valued,
If the visible layers are real-valued,
The visible units are Gaussian with diagonal covariance.
General Deep Belief Network
7/21/15 12
The visible units are Gaussian with diagonal covariance.
[Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
General Deep Belief Network
7/21/15 13
The visible units are Gaussian with diagonal covariance.
[Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
General Deep Belief Network
7/21/15 14
The visible units are Gaussian with diagonal covariance.
[Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
V-dimensional Gaussian Distribution
With Diagonal Covariance given by
And mean in dimension i given by
General Deep Belief Network
7/21/15 15
Therefore, we can perform efficient block Gibbs sampling by alternately
sampling each layer’s unit.
General Deep Belief Network
7/21/15 16[Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
General Deep Belief Network
7/21/15 17
! = − exp!(−! !, ℎ )
!,!
!
To	
  find	
  maximizing	
  model	
  weights	
  W,	
  we	
  have	
  to	
  calculate	
  
Z: state probability
with	
  gradient	
  ascent	
  
Carreira-Perpinan and Hinton (2005) showed that the derivate of the log-likelihood of data using chain
-rule. Since computing the average over the true model distribution is intractable, Hinton et al. (2006)
use approximation of that derivative called contrastive divergence: one replaces the average infinite
to small k.
Quotes from: [Representational Power of RBM and DBN, Nicolas Le Roux et al. p.3]
Convolutional DBN
7/21/15 18
Basic structure of Convolutional RBM
(CRBM)
7/21/15 19
Convolutional RBM
7/21/15 20
Bα := {(i, j):hij belongs to the block α }
Convolutional RBM’s Energy Function
7/21/15 21
K contains an information of Hidden layer’s units “group”
E(v,h) = − hij
k
r,s=1
NW
∑
i, j=1
NH
∑
k=1
K
∑ Wrs
k
vi+r−1, j+s−1 − bk
k=1
K
∑ hij
k
−c vij
i, j=1
NV
∑
i, j=1
NH
∑
NV NW
NH
NP
Bα := {(i, j):hij belongs to the block α }
α := {(C ×C): pooling block }
Object function for Gibbs sampling
7/21/15 22
Gibbs sampling
P(hij
k
=1| v) =σ ((W
~
*v)ij + bk )
P(vij =1| h) =σ (( W k
k
∑ *hk
)ij +c)
NV NW
NH
NP
Bα := {(i, j):hij belongs to the block α }
α := {(C ×C): pooling block }
P(v,h)model
Gibbs sampling
Convolutional RBM
7/21/15 23
Bα := {(i, j):hij belongs to the block α }
Probabilistic Max pooling
7/21/15 24
Probabilistic Max pooling
7/21/15 25
P(Y = 0)?
I(hij
k
):= bk +(W k
~
*v)ij
Convolu>onal	
  summa>on	
  (signal)	
  
Meaning of probabilistic max-pooling
7/21/15 26
Max-pooling was intended only for feed-forward
architectures. In contrast, we are interested in a
generative model of images which supports both
top-down and bottom-up interface. Therefore, we
designed our generative model so that inference inv
-loves max-pooling like behavior.
3.3 Probabilistic max-pooling
Hierarchical Probabilistic Inference
7/21/15 27
Object: Image restoration via probabilistic way
7/21/15 28
Object: Image restoration via probabilistic way
7/21/15 29
Difference Between DBN and DBM
7/21/15 30
Difference Between DBN and DBM
7/21/15 31
Model	
  construc>on	
  	
  
with	
  sta%s%cal	
  dependency	
  
Training Deep Boltzmann Machine
7/21/15 32Deep	
  learning,	
  Russ	
  Salakhutdinov,	
  University	
  of	
  Toronto	
  (h)p://bit.ly/1Mg9mAi)	
  
Training Deep Boltzmann Machine
7/21/15 33Deep	
  learning,	
  Russ	
  Salakhutdinov,	
  University	
  of	
  Toronto	
  (h)p://bit.ly/1Mg9mAi)	
  
Let’s reconstruct our energy function
7/21/15 34Deep	
  learning,	
  Russ	
  Salakhutdinov,	
  University	
  of	
  Toronto	
  (h)p://bit.ly/1Mg9mAi)	
  
h'
Γ
p
h
v
ω
E(v,h, p,h') = − v
k
∑ •(wk
*hk
)− bk
k
∑ ij
k
hij
∑ − pk
k,l
∑ •(Γkl
*h'kl
)− b'l
l
∑ ij
l
h'ij
∑
Let’s reconstruct our energy function
7/21/15 35Deep	
  learning,	
  Russ	
  Salakhutdinov,	
  University	
  of	
  Toronto	
  (h)p://bit.ly/1Mg9mAi)	
  
h'
Γ
p
h
v
ω
E(v,h, p,h') = − v
k
∑ •(wk
*hk
)− bk
k
∑ ij
k
hij
∑ − pk
k,l
∑ •(Γkl
*h'kl
)− b'l
l
∑ ij
l
h'ij
∑
Compare with DBM learning method
7/21/15 36Deep	
  learning,	
  Russ	
  Salakhutdinov,	
  University	
  of	
  Toronto	
  (h)p://bit.ly/1Mg9mAi)	
  
E(v,h, p,h') = − v
k
∑ •(wk
*hk
)− bk
k
∑ ij
k
hij
∑ − pk
k,l
∑ •(Γkl
*h'kl
)− b'l
l
∑ ij
l
h'ij
∑
visible	
   hidden	
  
Conditional Probability will be given by
7/21/15 37
P(hi, j
k
=1| v,h') =
exp(I(hi, j
k
)+ I(pα
k
))
1+ exp(I(hi, j
k
)+ I(pα
k
))
(i', j')∈Bα
∑
P(pα
k
=1| v,h') =
exp(I(hi, j
k
)+ I(pα
k
))
(i', j')∈Bα
∑
1+ exp(I(hi, j
k
)+ I(pα
k
))
(i', j')∈Bα
∑
Top-­‐down	
  
Top-­‐down	
  
Bo)om-­‐up	
  
Bo)om-­‐up	
  
Experimental Results
7/21/15 38
Supervised Learning from MNIST
7/21/15 39
Unsupervised Learning from Natural Images
7/21/15 40
Unsupervised learning of object-parts
7/21/15 41
Hierarchical Probabilistic Inference
7/21/15 42

More Related Content

What's hot

Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...IOSR Journals
 
Deeplabv1, v2, v3, v3+
Deeplabv1, v2, v3, v3+Deeplabv1, v2, v3, v3+
Deeplabv1, v2, v3, v3+Sungchul Kim
 
Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)SungminYou
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetSungminYou
 
Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Dongmin Choi
 
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_Report
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_ReportSaptashwa_Mitra_Sitakanta_Mishra_Final_Project_Report
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_ReportSitakanta Mishra
 
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...Ashray Bhandare
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketakiKetaki Patwari
 
[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun YooJaeJun Yoo
 
Unsupervised Object Detection
Unsupervised Object DetectionUnsupervised Object Detection
Unsupervised Object DetectionMahan Fathi
 
Image Compression Using Neural Network
 Image Compression Using Neural Network Image Compression Using Neural Network
Image Compression Using Neural NetworkOmkar Lokhande
 
Deep Learning - CNN and RNN
Deep Learning - CNN and RNNDeep Learning - CNN and RNN
Deep Learning - CNN and RNNAshray Bhandare
 
ImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural NetworksImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural NetworksWilly Marroquin (WillyDevNET)
 
Neural network based image compression with lifting scheme and rlc
Neural network based image compression with lifting scheme and rlcNeural network based image compression with lifting scheme and rlc
Neural network based image compression with lifting scheme and rlceSAT Publishing House
 
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...A Framework for Scene Recognition Using Convolutional Neural Network as Featu...
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...Tahmid Abtahi
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkItachi SK
 
PR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
PR-108: MobileNetV2: Inverted Residuals and Linear BottlenecksPR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
PR-108: MobileNetV2: Inverted Residuals and Linear BottlenecksJinwon Lee
 
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...Sunghoon Joo
 

What's hot (20)

Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...
Using Multi-layered Feed-forward Neural Network (MLFNN) Architecture as Bidir...
 
GPU Kernels for Block-Sparse Weights
GPU Kernels for Block-Sparse WeightsGPU Kernels for Block-Sparse Weights
GPU Kernels for Block-Sparse Weights
 
Deeplabv1, v2, v3, v3+
Deeplabv1, v2, v3, v3+Deeplabv1, v2, v3, v3+
Deeplabv1, v2, v3, v3+
 
Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)Deep learning lecture - part 1 (basics, CNN)
Deep learning lecture - part 1 (basics, CNN)
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNet
 
Network Deconvolution review [cdm]
Network Deconvolution review [cdm]Network Deconvolution review [cdm]
Network Deconvolution review [cdm]
 
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_Report
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_ReportSaptashwa_Mitra_Sitakanta_Mishra_Final_Project_Report
Saptashwa_Mitra_Sitakanta_Mishra_Final_Project_Report
 
deep CNN vs conventional ML
deep CNN vs conventional MLdeep CNN vs conventional ML
deep CNN vs conventional ML
 
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...
Bio-inspired Algorithms for Evolving the Architecture of Convolutional Neural...
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketaki
 
[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo
 
Unsupervised Object Detection
Unsupervised Object DetectionUnsupervised Object Detection
Unsupervised Object Detection
 
Image Compression Using Neural Network
 Image Compression Using Neural Network Image Compression Using Neural Network
Image Compression Using Neural Network
 
Deep Learning - CNN and RNN
Deep Learning - CNN and RNNDeep Learning - CNN and RNN
Deep Learning - CNN and RNN
 
ImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural NetworksImageNet Classification with Deep Convolutional Neural Networks
ImageNet Classification with Deep Convolutional Neural Networks
 
Neural network based image compression with lifting scheme and rlc
Neural network based image compression with lifting scheme and rlcNeural network based image compression with lifting scheme and rlc
Neural network based image compression with lifting scheme and rlc
 
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...A Framework for Scene Recognition Using Convolutional Neural Network as Featu...
A Framework for Scene Recognition Using Convolutional Neural Network as Featu...
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural network
 
PR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
PR-108: MobileNetV2: Inverted Residuals and Linear BottlenecksPR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
PR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
 
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...
[PR-325] Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Tran...
 

Viewers also liked

« Convolutional Deep Belief Networks »
« Convolutional Deep Belief Networks »« Convolutional Deep Belief Networks »
« Convolutional Deep Belief Networks » Loghin Dumitru
 
Modélisation de la dynamique d’occupation du sol par réseaux de neurones
Modélisation de la dynamique d’occupation du sol par réseaux de neuronesModélisation de la dynamique d’occupation du sol par réseaux de neurones
Modélisation de la dynamique d’occupation du sol par réseaux de neuronestuxette
 
Exposé réseaux des neurones (NN) - (RN)
Exposé réseaux des neurones (NN) - (RN)Exposé réseaux des neurones (NN) - (RN)
Exposé réseaux des neurones (NN) - (RN)Soumia Elyakote HERMA
 
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...Andino Maseleno
 
Proposed smart traffic signal control in brunei darussalam
Proposed smart traffic signal control in brunei darussalamProposed smart traffic signal control in brunei darussalam
Proposed smart traffic signal control in brunei darussalamAndino Maseleno
 
réseaux de neurones artificiels
réseaux de neurones artificiels réseaux de neurones artificiels
réseaux de neurones artificiels Oussama Werfelli
 
Machine learning, deep learning et search : à quand ces innovations dans nos ...
Machine learning, deep learning et search : à quand ces innovations dans nos ...Machine learning, deep learning et search : à quand ces innovations dans nos ...
Machine learning, deep learning et search : à quand ces innovations dans nos ...Antidot
 
Deep Belief Networks
Deep Belief NetworksDeep Belief Networks
Deep Belief NetworksHasan H Topcu
 
Neuroscience enseignement - Apprendre par le jeu
Neuroscience enseignement - Apprendre par le jeuNeuroscience enseignement - Apprendre par le jeu
Neuroscience enseignement - Apprendre par le jeuAntoine Taly
 
IoT Toulouse : introduction à mqtt
IoT Toulouse : introduction à mqttIoT Toulouse : introduction à mqtt
IoT Toulouse : introduction à mqttJulien Vermillard
 
Brief Introduction to Boltzmann Machine
Brief Introduction to Boltzmann MachineBrief Introduction to Boltzmann Machine
Brief Introduction to Boltzmann MachineArunabha Saha
 
Deep learning Tutorial - Part II
Deep learning Tutorial - Part IIDeep learning Tutorial - Part II
Deep learning Tutorial - Part IIQuantUniversity
 
Réseaux des neurones
Réseaux des neuronesRéseaux des neurones
Réseaux des neuronesMed Zaibi
 
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree Search
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree SearchAlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree Search
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree SearchKarel Ha
 
Expériences intégrées avec l'IoT - NUI Day 2015
Expériences intégrées avec l'IoT - NUI Day 2015Expériences intégrées avec l'IoT - NUI Day 2015
Expériences intégrées avec l'IoT - NUI Day 2015NUI Day
 
IoT (M2M) - Big Data - Analyses : Simulation et Démonstration
IoT (M2M) - Big Data - Analyses : Simulation et DémonstrationIoT (M2M) - Big Data - Analyses : Simulation et Démonstration
IoT (M2M) - Big Data - Analyses : Simulation et DémonstrationCHAKER ALLAOUI
 

Viewers also liked (20)

« Convolutional Deep Belief Networks »
« Convolutional Deep Belief Networks »« Convolutional Deep Belief Networks »
« Convolutional Deep Belief Networks »
 
Modélisation de la dynamique d’occupation du sol par réseaux de neurones
Modélisation de la dynamique d’occupation du sol par réseaux de neuronesModélisation de la dynamique d’occupation du sol par réseaux de neurones
Modélisation de la dynamique d’occupation du sol par réseaux de neurones
 
Exposé réseaux des neurones (NN) - (RN)
Exposé réseaux des neurones (NN) - (RN)Exposé réseaux des neurones (NN) - (RN)
Exposé réseaux des neurones (NN) - (RN)
 
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...
Viva Presentation - Fuzzy Logic and Dempster-Shafer Theory to Detect The Risk...
 
Proposed smart traffic signal control in brunei darussalam
Proposed smart traffic signal control in brunei darussalamProposed smart traffic signal control in brunei darussalam
Proposed smart traffic signal control in brunei darussalam
 
réseaux de neurones artificiels
réseaux de neurones artificiels réseaux de neurones artificiels
réseaux de neurones artificiels
 
Machine learning, deep learning et search : à quand ces innovations dans nos ...
Machine learning, deep learning et search : à quand ces innovations dans nos ...Machine learning, deep learning et search : à quand ces innovations dans nos ...
Machine learning, deep learning et search : à quand ces innovations dans nos ...
 
Deep Belief Networks
Deep Belief NetworksDeep Belief Networks
Deep Belief Networks
 
Neuroscience enseignement - Apprendre par le jeu
Neuroscience enseignement - Apprendre par le jeuNeuroscience enseignement - Apprendre par le jeu
Neuroscience enseignement - Apprendre par le jeu
 
IoT Toulouse : introduction à mqtt
IoT Toulouse : introduction à mqttIoT Toulouse : introduction à mqtt
IoT Toulouse : introduction à mqtt
 
Réseaux neurons
Réseaux neuronsRéseaux neurons
Réseaux neurons
 
Brief Introduction to Boltzmann Machine
Brief Introduction to Boltzmann MachineBrief Introduction to Boltzmann Machine
Brief Introduction to Boltzmann Machine
 
Deep learning Tutorial - Part II
Deep learning Tutorial - Part IIDeep learning Tutorial - Part II
Deep learning Tutorial - Part II
 
Réseaux des neurones
Réseaux des neuronesRéseaux des neurones
Réseaux des neurones
 
Les systèmes intelligents
Les systèmes intelligentsLes systèmes intelligents
Les systèmes intelligents
 
Deep learning - Part I
Deep learning - Part IDeep learning - Part I
Deep learning - Part I
 
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree Search
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree SearchAlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree Search
AlphaGo: Mastering the Game of Go with Deep Neural Networks and Tree Search
 
Expériences intégrées avec l'IoT - NUI Day 2015
Expériences intégrées avec l'IoT - NUI Day 2015Expériences intégrées avec l'IoT - NUI Day 2015
Expériences intégrées avec l'IoT - NUI Day 2015
 
XebiConFr15 - Les algorithmes du machine learning
XebiConFr15 - Les algorithmes du machine learningXebiConFr15 - Les algorithmes du machine learning
XebiConFr15 - Les algorithmes du machine learning
 
IoT (M2M) - Big Data - Analyses : Simulation et Démonstration
IoT (M2M) - Big Data - Analyses : Simulation et DémonstrationIoT (M2M) - Big Data - Analyses : Simulation et Démonstration
IoT (M2M) - Big Data - Analyses : Simulation et Démonstration
 

Similar to Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations

RunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkRunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkPutra Wanda
 
UNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxUNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxNoorUlHaq47
 
NIPS2007: deep belief nets
NIPS2007: deep belief netsNIPS2007: deep belief nets
NIPS2007: deep belief netszukun
 
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...AMD Developer Central
 
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...IJERA Editor
 
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...sipij
 
k means clustering-based data compression
k means clustering-based data compressionk means clustering-based data compression
k means clustering-based data compressionmohammed alrekabe
 
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)Universitat Politècnica de Catalunya
 
Regulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann MachinesRegulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann MachinesPatrick Michl
 
Robust foreground modelling to segment and detect multiple moving objects in ...
Robust foreground modelling to segment and detect multiple moving objects in ...Robust foreground modelling to segment and detect multiple moving objects in ...
Robust foreground modelling to segment and detect multiple moving objects in ...IJECEIAES
 
Remote Sensing IEEE 2015 Projects
Remote Sensing IEEE 2015 ProjectsRemote Sensing IEEE 2015 Projects
Remote Sensing IEEE 2015 ProjectsVijay Karan
 
Image Restoration Using Joint Statistical Modeling in a Space-Transform Domain
Image Restoration Using Joint Statistical Modeling in a Space-Transform DomainImage Restoration Using Joint Statistical Modeling in a Space-Transform Domain
Image Restoration Using Joint Statistical Modeling in a Space-Transform Domainjohn236zaq
 
Spine net learning scale permuted backbone for recognition and localization
Spine net learning scale permuted backbone for recognition and localizationSpine net learning scale permuted backbone for recognition and localization
Spine net learning scale permuted backbone for recognition and localizationDevansh16
 
CNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent AdvancesCNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent AdvancesDmytro Mishkin
 
02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄Jeong-gyu Kim
 

Similar to Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations (20)

Ok3425102516
Ok3425102516Ok3425102516
Ok3425102516
 
rbm_hls
rbm_hlsrbm_hls
rbm_hls
 
PointNet
PointNetPointNet
PointNet
 
RunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural NetworkRunPool: A Dynamic Pooling Layer for Convolution Neural Network
RunPool: A Dynamic Pooling Layer for Convolution Neural Network
 
1809.05680.pdf
1809.05680.pdf1809.05680.pdf
1809.05680.pdf
 
UNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptxUNetEliyaLaialy (2).pptx
UNetEliyaLaialy (2).pptx
 
NIPS2007: deep belief nets
NIPS2007: deep belief netsNIPS2007: deep belief nets
NIPS2007: deep belief nets
 
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...
HC-4012, Complex Network Clustering Using GPU-based Parallel Non-negative Mat...
 
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...
Kernel Estimation of Videodeblurringalgorithm and Motion Compensation of Resi...
 
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...
ADVANCED SINGLE IMAGE RESOLUTION UPSURGING USING A GENERATIVE ADVERSARIAL NET...
 
k means clustering-based data compression
k means clustering-based data compressionk means clustering-based data compression
k means clustering-based data compression
 
U4408108113
U4408108113U4408108113
U4408108113
 
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)
Unsupervised Learning (D2L6 2017 UPC Deep Learning for Computer Vision)
 
Regulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann MachinesRegulation Analysis using Restricted Boltzmann Machines
Regulation Analysis using Restricted Boltzmann Machines
 
Robust foreground modelling to segment and detect multiple moving objects in ...
Robust foreground modelling to segment and detect multiple moving objects in ...Robust foreground modelling to segment and detect multiple moving objects in ...
Robust foreground modelling to segment and detect multiple moving objects in ...
 
Remote Sensing IEEE 2015 Projects
Remote Sensing IEEE 2015 ProjectsRemote Sensing IEEE 2015 Projects
Remote Sensing IEEE 2015 Projects
 
Image Restoration Using Joint Statistical Modeling in a Space-Transform Domain
Image Restoration Using Joint Statistical Modeling in a Space-Transform DomainImage Restoration Using Joint Statistical Modeling in a Space-Transform Domain
Image Restoration Using Joint Statistical Modeling in a Space-Transform Domain
 
Spine net learning scale permuted backbone for recognition and localization
Spine net learning scale permuted backbone for recognition and localizationSpine net learning scale permuted backbone for recognition and localization
Spine net learning scale permuted backbone for recognition and localization
 
CNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent AdvancesCNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent Advances
 
02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄02.cnn - CNN 파헤치기 3탄
02.cnn - CNN 파헤치기 3탄
 

More from Mad Scientists

LSTM 네트워크 이해하기
LSTM 네트워크 이해하기LSTM 네트워크 이해하기
LSTM 네트워크 이해하기Mad Scientists
 
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회Mad Scientists
 
트라우마틱 메모리(Traumatic Memory) 수정사항 발표
트라우마틱 메모리(Traumatic Memory) 수정사항 발표트라우마틱 메모리(Traumatic Memory) 수정사항 발표
트라우마틱 메모리(Traumatic Memory) 수정사항 발표Mad Scientists
 
트라우마틱 메모리(Traumatic Memory) 제작 기획
트라우마틱 메모리(Traumatic Memory) 제작 기획트라우마틱 메모리(Traumatic Memory) 제작 기획
트라우마틱 메모리(Traumatic Memory) 제작 기획Mad Scientists
 
Learning RBM(Restricted Boltzmann Machine in Practice)
Learning RBM(Restricted Boltzmann Machine in Practice)Learning RBM(Restricted Boltzmann Machine in Practice)
Learning RBM(Restricted Boltzmann Machine in Practice)Mad Scientists
 
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정Sampling-Importance-Sampling을 이용한 선수 경기능력 측정
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정Mad Scientists
 
핀테크 기업조사- TransferWise, CurrencyCloud, TOSS
핀테크 기업조사- TransferWise, CurrencyCloud, TOSS핀테크 기업조사- TransferWise, CurrencyCloud, TOSS
핀테크 기업조사- TransferWise, CurrencyCloud, TOSSMad Scientists
 
나를 자극하는 Real world_algorithm (1/2)
나를 자극하는 Real world_algorithm (1/2)나를 자극하는 Real world_algorithm (1/2)
나를 자극하는 Real world_algorithm (1/2)Mad Scientists
 
101: Convolutional Neural Networks
101: Convolutional Neural Networks 101: Convolutional Neural Networks
101: Convolutional Neural Networks Mad Scientists
 
how to visualize: interstellar from example
how to visualize: interstellar from examplehow to visualize: interstellar from example
how to visualize: interstellar from exampleMad Scientists
 
ideas worth spreading: 2015 Club Release Seminar
ideas worth spreading: 2015 Club Release Seminarideas worth spreading: 2015 Club Release Seminar
ideas worth spreading: 2015 Club Release SeminarMad Scientists
 
Relational Mate Value: Consensus and Uniqueness in Romantic Eavaluations
Relational Mate Value: Consensus and Uniqueness in Romantic EavaluationsRelational Mate Value: Consensus and Uniqueness in Romantic Eavaluations
Relational Mate Value: Consensus and Uniqueness in Romantic EavaluationsMad Scientists
 
Personal Matching Recommendation system in TinderBox
Personal Matching Recommendation system in TinderBoxPersonal Matching Recommendation system in TinderBox
Personal Matching Recommendation system in TinderBoxMad Scientists
 
핀란드, 내가 가보고 느낀 스타트업 생태계
핀란드, 내가 가보고 느낀 스타트업 생태계핀란드, 내가 가보고 느낀 스타트업 생태계
핀란드, 내가 가보고 느낀 스타트업 생태계Mad Scientists
 
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...Mad Scientists
 
Superhero discourse in Americanization
Superhero discourse in AmericanizationSuperhero discourse in Americanization
Superhero discourse in AmericanizationMad Scientists
 
[SW Maestro] Team Loclas 1-2 Final Presentation
[SW Maestro] Team Loclas 1-2 Final Presentation[SW Maestro] Team Loclas 1-2 Final Presentation
[SW Maestro] Team Loclas 1-2 Final PresentationMad Scientists
 
Basic Understanding of the Deep
Basic Understanding of the DeepBasic Understanding of the Deep
Basic Understanding of the DeepMad Scientists
 
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론Mad Scientists
 
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계Mad Scientists
 

More from Mad Scientists (20)

LSTM 네트워크 이해하기
LSTM 네트워크 이해하기LSTM 네트워크 이해하기
LSTM 네트워크 이해하기
 
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회
트라우마틱 메모리(Traumatic Memory) : 리스크리스트 및 작품방향 선회
 
트라우마틱 메모리(Traumatic Memory) 수정사항 발표
트라우마틱 메모리(Traumatic Memory) 수정사항 발표트라우마틱 메모리(Traumatic Memory) 수정사항 발표
트라우마틱 메모리(Traumatic Memory) 수정사항 발표
 
트라우마틱 메모리(Traumatic Memory) 제작 기획
트라우마틱 메모리(Traumatic Memory) 제작 기획트라우마틱 메모리(Traumatic Memory) 제작 기획
트라우마틱 메모리(Traumatic Memory) 제작 기획
 
Learning RBM(Restricted Boltzmann Machine in Practice)
Learning RBM(Restricted Boltzmann Machine in Practice)Learning RBM(Restricted Boltzmann Machine in Practice)
Learning RBM(Restricted Boltzmann Machine in Practice)
 
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정Sampling-Importance-Sampling을 이용한 선수 경기능력 측정
Sampling-Importance-Sampling을 이용한 선수 경기능력 측정
 
핀테크 기업조사- TransferWise, CurrencyCloud, TOSS
핀테크 기업조사- TransferWise, CurrencyCloud, TOSS핀테크 기업조사- TransferWise, CurrencyCloud, TOSS
핀테크 기업조사- TransferWise, CurrencyCloud, TOSS
 
나를 자극하는 Real world_algorithm (1/2)
나를 자극하는 Real world_algorithm (1/2)나를 자극하는 Real world_algorithm (1/2)
나를 자극하는 Real world_algorithm (1/2)
 
101: Convolutional Neural Networks
101: Convolutional Neural Networks 101: Convolutional Neural Networks
101: Convolutional Neural Networks
 
how to visualize: interstellar from example
how to visualize: interstellar from examplehow to visualize: interstellar from example
how to visualize: interstellar from example
 
ideas worth spreading: 2015 Club Release Seminar
ideas worth spreading: 2015 Club Release Seminarideas worth spreading: 2015 Club Release Seminar
ideas worth spreading: 2015 Club Release Seminar
 
Relational Mate Value: Consensus and Uniqueness in Romantic Eavaluations
Relational Mate Value: Consensus and Uniqueness in Romantic EavaluationsRelational Mate Value: Consensus and Uniqueness in Romantic Eavaluations
Relational Mate Value: Consensus and Uniqueness in Romantic Eavaluations
 
Personal Matching Recommendation system in TinderBox
Personal Matching Recommendation system in TinderBoxPersonal Matching Recommendation system in TinderBox
Personal Matching Recommendation system in TinderBox
 
핀란드, 내가 가보고 느낀 스타트업 생태계
핀란드, 내가 가보고 느낀 스타트업 생태계핀란드, 내가 가보고 느낀 스타트업 생태계
핀란드, 내가 가보고 느낀 스타트업 생태계
 
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...
Face Feature Recognition System with Deep Belief Networks, for Korean/KIISE T...
 
Superhero discourse in Americanization
Superhero discourse in AmericanizationSuperhero discourse in Americanization
Superhero discourse in Americanization
 
[SW Maestro] Team Loclas 1-2 Final Presentation
[SW Maestro] Team Loclas 1-2 Final Presentation[SW Maestro] Team Loclas 1-2 Final Presentation
[SW Maestro] Team Loclas 1-2 Final Presentation
 
Basic Understanding of the Deep
Basic Understanding of the DeepBasic Understanding of the Deep
Basic Understanding of the Deep
 
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론
문화연구입문 발표자료, <현대캐피탈>광고 분석 및 담론
 
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계
러셀과 데카르트 탈 구조주의와 컴퓨터공학의 관계
 

Recently uploaded

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 

Recently uploaded (20)

Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 

Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations

  • 1. Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations Jaehyun Ahn (jaehyunahn@sogang.ac.kr) 서강대학교 데이터마이닝 연구실 Computer Science Department Sogang University Honglak Lee (ICML 2009, 744 quotes)
  • 2. Key Reference •  Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representation (Honglak Lee et al., ICML 2009) •  Lecture at ICML 2009 (http://videolectures.net/icml09_lee_cdb/) •  Learning Multiple Layers of Features from Tiny Images (Alex Krizhevsky, 2009) •  To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine (Takayoshi Yamashita et al., ICPR 2014) 7/21/15 2
  • 4. What this paper wants to say.. •  Taking advantages of Deep Belief Network through Convolutional networks –  Translation invariance •  Max-pooling –  Scalable to realistic image sizes •  Max-pooling –  Hierarchical probabilistic inference by combining bottom-up and top-down information •  Probabilistic max-pooling 7/21/15 4
  • 6. Basic Design of Convolutional Networks 7/21/15 6What  is  this?  h)p://www.slideshare.net/sogo1127/101-­‐convolu>onal-­‐neural-­‐networks   Alternate between “Detection” and “Pooling” layers
  • 7. Advantages of Convolutional Networks 1. Translation Invariance •  In LeNet, LeCun et al., 1998, Max-pooling provides a form of translation invariance. If max-pooling is done over a 2x2 region, 4 possible configurations will produce exactly the same output at the convolutional layer. 7/21/15 7 Image:  IEEE  2013,  h)p://www.computer.org/csdl/trans/tp/2013/08/)p2013081930-­‐abs.html  
  • 8. Advantages of Convolutional Networks 2. Scalable to realistic image sizes •  This paper starts from this basic question “How can we scale to realistic image sizes (e.g. 200x200 pixels)?” •  Max-pooling shrinks the representation in higher layers 7/21/15 8 Image:  IEEE  2013,  h)p://www.computer.org/csdl/trans/tp/2013/08/)p2013081930-­‐abs.html  
  • 9. Key Idea with advantages of Convolutional Networks 3. Hierarchical probabilistic inference Max-pooling •  Max-pooling is deterministic and feed-forward only. However, this paper gave a max-pooling to a probabilistic semantics that enables to combine bottom- up and top-down information. 7/21/15 9
  • 11. General Deep Belief Network 7/21/15 11 Z: state probability If the visible units are binary-valued, If the visible layers are real-valued, The visible units are Gaussian with diagonal covariance.
  • 12. General Deep Belief Network 7/21/15 12 The visible units are Gaussian with diagonal covariance. [Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
  • 13. General Deep Belief Network 7/21/15 13 The visible units are Gaussian with diagonal covariance. [Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
  • 14. General Deep Belief Network 7/21/15 14 The visible units are Gaussian with diagonal covariance. [Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs] V-dimensional Gaussian Distribution With Diagonal Covariance given by And mean in dimension i given by
  • 15. General Deep Belief Network 7/21/15 15 Therefore, we can perform efficient block Gibbs sampling by alternately sampling each layer’s unit.
  • 16. General Deep Belief Network 7/21/15 16[Learning features from Tiny Images, 2009 p.13, 1.4.3 Gaussian-Bernoulli RBMs]
  • 17. General Deep Belief Network 7/21/15 17 ! = − exp!(−! !, ℎ ) !,! ! To  find  maximizing  model  weights  W,  we  have  to  calculate   Z: state probability with  gradient  ascent   Carreira-Perpinan and Hinton (2005) showed that the derivate of the log-likelihood of data using chain -rule. Since computing the average over the true model distribution is intractable, Hinton et al. (2006) use approximation of that derivative called contrastive divergence: one replaces the average infinite to small k. Quotes from: [Representational Power of RBM and DBN, Nicolas Le Roux et al. p.3]
  • 19. Basic structure of Convolutional RBM (CRBM) 7/21/15 19
  • 20. Convolutional RBM 7/21/15 20 Bα := {(i, j):hij belongs to the block α }
  • 21. Convolutional RBM’s Energy Function 7/21/15 21 K contains an information of Hidden layer’s units “group” E(v,h) = − hij k r,s=1 NW ∑ i, j=1 NH ∑ k=1 K ∑ Wrs k vi+r−1, j+s−1 − bk k=1 K ∑ hij k −c vij i, j=1 NV ∑ i, j=1 NH ∑ NV NW NH NP Bα := {(i, j):hij belongs to the block α } α := {(C ×C): pooling block }
  • 22. Object function for Gibbs sampling 7/21/15 22 Gibbs sampling P(hij k =1| v) =σ ((W ~ *v)ij + bk ) P(vij =1| h) =σ (( W k k ∑ *hk )ij +c) NV NW NH NP Bα := {(i, j):hij belongs to the block α } α := {(C ×C): pooling block } P(v,h)model Gibbs sampling
  • 23. Convolutional RBM 7/21/15 23 Bα := {(i, j):hij belongs to the block α }
  • 25. Probabilistic Max pooling 7/21/15 25 P(Y = 0)? I(hij k ):= bk +(W k ~ *v)ij Convolu>onal  summa>on  (signal)  
  • 26. Meaning of probabilistic max-pooling 7/21/15 26 Max-pooling was intended only for feed-forward architectures. In contrast, we are interested in a generative model of images which supports both top-down and bottom-up interface. Therefore, we designed our generative model so that inference inv -loves max-pooling like behavior. 3.3 Probabilistic max-pooling
  • 28. Object: Image restoration via probabilistic way 7/21/15 28
  • 29. Object: Image restoration via probabilistic way 7/21/15 29
  • 30. Difference Between DBN and DBM 7/21/15 30
  • 31. Difference Between DBN and DBM 7/21/15 31 Model  construc>on     with  sta%s%cal  dependency  
  • 32. Training Deep Boltzmann Machine 7/21/15 32Deep  learning,  Russ  Salakhutdinov,  University  of  Toronto  (h)p://bit.ly/1Mg9mAi)  
  • 33. Training Deep Boltzmann Machine 7/21/15 33Deep  learning,  Russ  Salakhutdinov,  University  of  Toronto  (h)p://bit.ly/1Mg9mAi)  
  • 34. Let’s reconstruct our energy function 7/21/15 34Deep  learning,  Russ  Salakhutdinov,  University  of  Toronto  (h)p://bit.ly/1Mg9mAi)   h' Γ p h v ω E(v,h, p,h') = − v k ∑ •(wk *hk )− bk k ∑ ij k hij ∑ − pk k,l ∑ •(Γkl *h'kl )− b'l l ∑ ij l h'ij ∑
  • 35. Let’s reconstruct our energy function 7/21/15 35Deep  learning,  Russ  Salakhutdinov,  University  of  Toronto  (h)p://bit.ly/1Mg9mAi)   h' Γ p h v ω E(v,h, p,h') = − v k ∑ •(wk *hk )− bk k ∑ ij k hij ∑ − pk k,l ∑ •(Γkl *h'kl )− b'l l ∑ ij l h'ij ∑
  • 36. Compare with DBM learning method 7/21/15 36Deep  learning,  Russ  Salakhutdinov,  University  of  Toronto  (h)p://bit.ly/1Mg9mAi)   E(v,h, p,h') = − v k ∑ •(wk *hk )− bk k ∑ ij k hij ∑ − pk k,l ∑ •(Γkl *h'kl )− b'l l ∑ ij l h'ij ∑ visible   hidden  
  • 37. Conditional Probability will be given by 7/21/15 37 P(hi, j k =1| v,h') = exp(I(hi, j k )+ I(pα k )) 1+ exp(I(hi, j k )+ I(pα k )) (i', j')∈Bα ∑ P(pα k =1| v,h') = exp(I(hi, j k )+ I(pα k )) (i', j')∈Bα ∑ 1+ exp(I(hi, j k )+ I(pα k )) (i', j')∈Bα ∑ Top-­‐down   Top-­‐down   Bo)om-­‐up   Bo)om-­‐up  
  • 39. Supervised Learning from MNIST 7/21/15 39
  • 40. Unsupervised Learning from Natural Images 7/21/15 40
  • 41. Unsupervised learning of object-parts 7/21/15 41