Caffe - A deep learning framework (Ramin Fahimi)

1. Caffe:
Deep learning
Framework
Ramin Fahimi
PyCon 2016 , IUST
Many contents has been taken from Caffe CVPR’15 tutorial and CS231n lectures, Stanford.

2. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
Took million years from nature to
form effective visual recognition
system.
It didn’t happened in one night!
Evolution.

3. Computer Vision
1. Controlling processes: an industrial robot
2. Navigation: an autonomous vehicle
3. Detecting events: visual surveillance
4. Organizing information: indexing databases of
images and image sequences
5. Modeling objects or environments: medical
image analysis or topographical modeling
6. Interaction: input to a device for computer-
human interaction
7. Automatic inspection in manufacturing
applications.
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

4. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
ILSVRC – Visual recognition challenges

5. What is neural network?
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

6. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

7. What is deep learning? (DL)
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
Input image
Weights
Loss
The number of neurons in each layer is given by 253440,
186624, 64896, 64896, 43264, 4096, 4096, 1000

8. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

9. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

10. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

11. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

12. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

13. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

14. Why?
What changed?
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

15. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

16. Why?
What changed?
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
1. Improvements in hardware
2. Data Size
3. Initialization
4. Successfully applying back propagation
5. Many other things

17. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
Training Deep networks

18. Back propagation
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

19. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
Back propagation – cont.

20. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

21. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

22. Frameworks
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

23. Use Cases
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
• Extract AlexNet or VGG features? Use Caffe
• Fine-tune AlexNet for new classes? Use Caffe
• Image Captioning with fine-tuning?
o Need pre-trained models (Caffe, Torch, Lasagne)
o Need RNNs (Torch or Lasagne)
o Use Torch or Lasagna
• Segmentation? (Classify every pixel)
o Use Caffe If loss function exists in Caffe else Use Torch
• Object Detection?
o Need pre-trained model (Torch, Caffe, Lasagne)
o Need lots of custom imperative code (NOT Lasagne), Use Caffe or Torch

24. Use Cases – Cont.
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
• Feature extraction / fine-tuning existing models: Use Caffe
• Complex uses of pre-trained models: Use Lasagne or Torch
• Write your own layers: Use Torch
• Crazy RNNs: Use Theano or Tensorflow
• Huge model, need model parallelism: Use TensorFlow

25. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

26. Why Caffe?
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
§ Expression: models + optimizations are plaintext schemas, not code.
§ Speed: for state-of-the-art models and massive data.
§ Modularity: to extend to new tasks and architectures.
§ Openness: common code and reference models for reproducibility.
§ Community: joint discussion, development, and modeling
● Frameworks are more alike than different
o All express deep models
o All are nicely open-source
o All include scripting for hacking and prototyping
● No strict winners – experiment and choose the framework that best fits your
work

27. Open model collections
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
• The Caffe model zoo contains open collection of deep models
o VGG ILSVRC14 + Devil models in the zoo
o Network-in-Network / CCCP model in the zoo
o MIT Places scene recognition model in the zoo
• help disseminate and reproduce research
• bundled tools for loading and publishing models
• Share Your Models! with your citation + license of course

28. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

29. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

30. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

31. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

32. Blobs
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

33. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

34. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

35. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

36. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

37. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
Models and solvers are schema, not code

38. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

39. Prototxt : Define Layer
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

40. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

41. Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016

42. References.
Caffe: Deep learning Framework Ramin Fahimi, #irpycon 2016
[1]. http://caffe.berkeleyvision.org/
[1]. http://cs231n.stanford.edu/
[2]. http://www.panderson.me/images/Caffe.pdf
Thanks for your attention.
Questions?