Deep Learning: AI Breakthrough
•
3 likes•1,010 views
A short talk on principles of video processing and deep learning
Recommended
Recommended
More Related Content
Similar to Deep Learning: AI Breakthrough
Similar to Deep Learning: AI Breakthrough (20)
Recently uploaded
Recently uploaded (20)
Deep Learning: AI Breakthrough
- 1. Deep Learning: AI Breakthrough Mohsen Fayyaz Sensifai Tehran University – 15 Dey 1395 (4 Jan 2017)
- 2. Video Processing and Deep Learning
- 3. What is Video? • Batches of Frames • Can we process video as batches of frames? Motion cannot be inferred from single frame
- 4. Why do we need video processing? • Self-Driving Cars: Video Semantic Segmentation Feature Space Optimization for Semantic Video Segmentation, Kundu et. al., 2016
- 5. Why do we need video processing? • Robots: Action Recognition Simonyan et. al., 2014
- 6. Why do we need video processing? • Google, YouTube, Aparat : Video Tagging Densecap, Johnson et. al., 2016 (Image captioning)
- 7. Why do we need video processing? • Network Video Broadcasting: Frame Prediction Patraucean et. al., 2016
- 8. From Images to Video 3 Image CNN Extracted Features Frames ? Extracted Features Image Video
- 9. From Images to Video CNN Extracted Spatio-Temporal Features Frames LSTM Donahe et. al., 2015
- 10. From Images to Video CNN Extracted Spatio-Temporal Features Frames LSTM Donahe et. al., 2015 What if we want regional features?
- 11. From Images to Video - STFCN CNN Extracted Regional Spatio-Temporal FeaturesFrames Convolutional LSTM Fayyaz et. al., 2016
- 12. From Images to Video – C3D 3D CNN Extracted Regional Spatio-Temporal FeaturesFrames Tran et. al., 2015
- 13. Now that we have the appropriate tool Let’s see some real world applications
- 14. Video Semantic Segmentation - STFCN Fayyaz et. al., 2016
- 15. Video Semantic Segmentation – C3D Tran et. al., 2015
- 16. Action Recognition & Video Classification Simonyan et. al., 2014
- 17. Does video have visual data only?
- 18. Action Recognition & Video Classification Wu et al., 2015 Audio + Vision
- 19. Let’s briefly take a look at some state-of-the- art Image based Networks
- 20. Extremely Deep Networks Residual Networks • Problem: Gradients Vanish in Back-propagation • Solution: Let’s make a shortcut for them! • Y = 𝐻(𝑋, 𝑊𝐻) -> Y = 𝐻 𝑋, 𝑊𝐻 + 𝑋
- 21. Extremely Deep Networks Highway Networks • Similar to ResNets • The shortcuts are controlled using a learnable parameter to have a better trade-off between being • Y = 𝐻 𝑋, 𝑊𝐻 . 𝑇 𝑋, 𝑊𝑇 + 𝑋. (1 − 𝑇 𝑋, 𝑊𝑇 )
- 22. Extremely Deep Networks DenseNets • If ResNet works with just connecting previous layers, why not connecting all?! • 𝑌 = 𝐹(𝑋 𝑛, 𝑋 𝑛−1, …, 𝑋0) • Improvements in both Forward & • Backward
- 23. Now what if we use the idea of propagating data and gradients between shallow and deep layers in video based networks?
- 24. Up to here everything was Supervised But there are bunch of data across the Internet with weak labels … Let’s go through Weakly-Supervised methods
- 25. Weakly Supervised Learning Weakly Supervised Learning with CNNs • Multiple Labeling • Weakly Localization • Data can be crawled over Internet • Can be adopted to Video Oquab et. al., 2015
- 26. How about some Unsupervised methods …
- 27. Unsupervised Learning Anticipating Visual Representations From Unlabeled Video • Training on Big Huge amount of unlabeled Video across the net • Training Classifiers on the final output Vondrick et. al., 2016
- 29. What Hardware do I use? • NVIDIA GPU + SSD + HDD • More info on: http://www.DeepLearning.ir
- 30. What framework do I use? Caffe Torch Tensorflow Theano Keras Microsoft CNTK Deeplearning4j …
- 31. What framework do I use? Tensorflow Torch Theano From Karpathy’s slides
- 32. Distributed Training: Will be presented at my next presentation at Sharif University of Technology on 22 Dey 1395 (11 Jan 2017) From Karpathy’s slides