Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

In image analysis, #convolutional neural networks (#CNNs or #ConvNets for short) are time and memory efficient than fully connected (#FC) networks. But why? What are the advantages of ConvNets over FC networks in image analysis? How is #ConvNet derived from FC networks? Where the term #convolution in CNNs came from? These questions are to be answered in this #presentation.
Image analysis has a number of challenges such as classification, object detection, recognition, description, etc. If an image classifier, for example, is to be created, it should be able to work with a high accuracy even with variations such as occlusion, illumination changes, viewing angles, and others. The traditional pipeline of image classification with its main step of feature engineering is not suitable for working in rich environments. Even experts in the field won’t be able to give a single or a group of features that are able to reach high accuracy under different variations. Motivated by this problem, the idea of feature learning came out. The suitable features to work with images are learned automatically. This is the reason why artificial neural networks (ANNs) are one of the robust ways of image analysis. Based on a learning algorithm such as gradient descent (GD), ANN learns the image features automatically. The raw image is applied to the ANN and ANN is responsible for generating the features describing it.

  • Login to see the comments

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

  1. 1. Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ‫المنوفية‬ ‫جامعة‬ ‫الحاسبات‬ ‫كلية‬‫والمعلومات‬ ‫المنوفية‬ ‫جامعة‬ Ahmed F. Gad 18-May-2018
  2. 2. 15 8 9 10 17 22 20 3015 Input Image 3x3 Ahmed F. Gad
  3. 3. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  4. 4. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  5. 5. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  6. 6. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  7. 7. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  8. 8. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  9. 9. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  10. 10. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  11. 11. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  12. 12. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  13. 13. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  14. 14. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer Ahmed F. Gad
  15. 15. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  16. 16. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  17. 17. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  18. 18. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 Fully Connected Network Ahmed F. Gad
  19. 19. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 Fully Connected Network Ahmed F. Gad
  20. 20. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 Fully Connected Network Ahmed F. Gad
  21. 21. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  22. 22. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Weight 16 𝒘 𝟎 𝟎 𝒘 𝟏𝟓 𝟎 Total Number of Weights For First Pixel Ahmed F. Gad
  23. 23. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  24. 24. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Ahmed F. Gad
  25. 25. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Weight 16 𝒘 𝟎 𝟏 𝒘 𝟏𝟓 𝟏 Total Number of Weights For Second Pixel Ahmed F. Gad
  26. 26. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 𝒘 𝟒 𝟐 𝒘 𝟓 𝟐 𝒘 𝟔 𝟐 𝒘 𝟕 𝟐 𝒘 𝟖 𝟐 𝒘 𝟗 𝟐 𝒘 𝟏𝟎 𝟐 𝒘 𝟏𝟏 𝟐 𝒘 𝟏𝟐 𝟐 𝒘 𝟏𝟑 𝟐 𝒘 𝟏𝟒 𝟐 𝒘 𝟏𝟓 𝟐 Ahmed F. Gad
  27. 27. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 𝒘 𝟒 𝟑 𝒘 𝟓 𝟑 𝒘 𝟔 𝟑 𝒘 𝟕 𝟑 𝒘 𝟖 𝟑 𝒘 𝟗 𝟑 𝒘 𝟏𝟎 𝟑 𝒘 𝟏𝟏 𝟑 𝒘 𝟏𝟐 𝟑 𝒘 𝟏𝟑 𝟑 𝒘 𝟏𝟒 𝟑 𝒘 𝟏𝟓 𝟑 Ahmed F. Gad
  28. 28. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 𝒘 𝟒 𝟒 𝒘 𝟓 𝟒 𝒘 𝟔 𝟒 𝒘 𝟕 𝟒 𝒘 𝟖 𝟒 𝒘 𝟗 𝟒 𝒘 𝟏𝟎 𝟒 𝒘 𝟏𝟏 𝟒 𝒘 𝟏𝟐 𝟒 𝒘 𝟏𝟑 𝟒 𝒘 𝟏𝟒 𝟒 𝒘 𝟏𝟓 𝟒 Ahmed F. Gad
  29. 29. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 𝒘 𝟒 𝟓 𝒘 𝟓 𝟓 𝒘 𝟔 𝟓 𝒘 𝟕 𝟓 𝒘 𝟖 𝟓 𝒘 𝟗 𝟓 𝒘 𝟏𝟎 𝟓 𝒘 𝟏𝟏 𝟓 𝒘 𝟏𝟐 𝟓 𝒘 𝟏𝟑 𝟓 𝒘 𝟏𝟒 𝟓 𝒘 𝟏𝟓 𝟓 Ahmed F. Gad
  30. 30. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 𝒘 𝟒 𝟔 𝒘 𝟓 𝟔 𝒘 𝟔 𝟔 𝒘 𝟕 𝟔 𝒘 𝟖 𝟔 𝒘 𝟗 𝟔 𝒘 𝟏𝟎 𝟔 𝒘 𝟏𝟏 𝟔 𝒘 𝟏𝟐 𝟔 𝒘 𝟏𝟑 𝟔 𝒘 𝟏𝟒 𝟔 𝒘 𝟏𝟓 𝟔 Ahmed F. Gad
  31. 31. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 𝒘 𝟒 𝟕 𝒘 𝟓 𝟕 𝒘 𝟔 𝟕 𝒘 𝟕 𝟕 𝒘 𝟖 𝟕 𝒘 𝟗 𝟕 𝒘 𝟏𝟎 𝟕 𝒘 𝟏𝟏 𝟕 𝒘 𝟏𝟐 𝟕 𝒘 𝟏𝟑 𝟕 𝒘 𝟏𝟒 𝟕 𝒘 𝟏𝟓 𝟕 Ahmed F. Gad
  32. 32. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 𝒘 𝟒 𝟖 𝒘 𝟓 𝟖 𝒘 𝟔 𝟖 𝒘 𝟕 𝟖 𝒘 𝟖 𝟖 𝒘 𝟗 𝟖 𝒘 𝟏𝟎 𝟖 𝒘 𝟏𝟏 𝟖 𝒘 𝟏𝟐 𝟖 𝒘 𝟏𝟑 𝟖 𝒘 𝟏𝟒 𝟖 𝒘 𝟏𝟓 𝟖 Ahmed F. Gad
  33. 33. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  34. 34. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Ahmed F. Gad
  35. 35. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  36. 36. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Bias is Neglected. Ahmed F. Gad
  37. 37. Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  38. 38. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  39. 39. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500 Too Many Parameters Ahmed F. Gad
  40. 40. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500+ 810+4,500=5,310= Too Many Parameters Ahmed F. Gad
  41. 41. Input Image 32x32 Too Many Parameters Ahmed F. Gad
  42. 42. Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron Input Image 32x32 Too Many Parameters Ahmed F. Gad
  43. 43. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 Too Many Parameters Ahmed F. Gad
  44. 44. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 CNN can create a large network but with less number of parameters than FC networks Too Many Parameters Ahmed F. Gad
  45. 45. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  46. 46. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  47. 47. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  48. 48. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  49. 49. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  50. 50. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  51. 51. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  52. 52. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  53. 53. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  54. 54. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Number of Parameters for First Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  55. 55. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Ahmed F. Gad
  56. 56. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Number of Parameters for Second Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  57. 57. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 Continue Ahmed F. Gad
  58. 58. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 Continue Ahmed F. Gad
  59. 59. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 Continue Ahmed F. Gad
  60. 60. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 Continue Ahmed F. Gad
  61. 61. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 Continue Ahmed F. Gad
  62. 62. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 Continue Ahmed F. Gad
  63. 63. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 Continue Ahmed F. Gad
  64. 64. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 All Connections Ahmed F. Gad
  65. 65. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Ahmed F. Gad
  66. 66. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Hidden Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  67. 67. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Total Parameters After Grouping Neurons 9 Pixels 4 Weight Total Parameters = 9*4=36Ahmed F. Gad
  68. 68. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Ahmed F. Gad
  69. 69. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% Ahmed F. Gad
  70. 70. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% More Reduction Ahmed F. Gad
  71. 71. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  72. 72. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  73. 73. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  74. 74. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  75. 75. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  76. 76. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  77. 77. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  78. 78. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  79. 79. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad
  80. 80. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons
  81. 81. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons Each neuron will process an image region of a specific size. In this example, region size is 2x2.
  82. 82. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  83. 83. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  84. 84. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  85. 85. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  86. 86. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  87. 87. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  88. 88. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  89. 89. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad
  90. 90. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  91. 91. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  92. 92. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  93. 93. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  94. 94. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  95. 95. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  96. 96. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  97. 97. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  98. 98. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  99. 99. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  100. 100. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  101. 101. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  102. 102. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  103. 103. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  104. 104. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  105. 105. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  106. 106. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  107. 107. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  108. 108. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  109. 109. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  110. 110. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels 4 Weights Parameters for First Hidden Neurons Group Ahmed F. Gad
  111. 111. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels Parameters for First Hidden Neurons Group 4*4=16 Weights Parameters for Entire Network 4 Weights Ahmed F. Gad
  112. 112. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Ahmed F. Gad
  113. 113. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Saved Parameters 144-16=128 Reduction % (128/144)*100 =88.89% Ahmed F. Gad
  114. 114. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 Ahmed F. Gad
  115. 115. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  116. 116. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘 𝟑 `𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  117. 117. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 8*𝒘` 𝟎 𝟎 +9*𝒘` 𝟏 𝟎 +17*𝒘` 𝟐 𝟎 +22𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  118. 118. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 10*𝒘` 𝟎 𝟎 +17*𝒘` 𝟏 𝟎 +20*𝒘` 𝟐 𝟎 +15𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  119. 119. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 17*𝒘 𝟎 𝟎 +22*𝒘 𝟏 𝟎 +15*𝒘 𝟐 𝟎 +30𝒘 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  120. 120. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  121. 121. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  122. 122. References • Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic- Sign Detection and Classification. Springer, 2017. • Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step • https://www.linkedin.com/pulse/derivation-convolutional-neural-network- from-fully-connected-gad • https://www.slideshare.net/AhmedGadFCIT/derivation-of-convolutional- neural-network-convnet-from-fully-connected-network • https://www.kdnuggets.com/2018/04/derivation-convolutional-neural- network-fully-connected-step-by-step.html Ahmed F. Gad

×