SlideShare a Scribd company logo

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

•
•
Ahmed Gad
Ahmed Gad

In image analysis, #convolutional neural networks (#CNNs or #ConvNets for short) are time and memory efficient than fully connected (#FC) networks. But why? What are the advantages of ConvNets over FC networks in image analysis? How is #ConvNet derived from FC networks? Where the term #convolution in CNNs came from? These questions are to be answered in this #presentation. Image analysis has a number of challenges such as classification, object detection, recognition, description, etc. If an image classifier, for example, is to be created, it should be able to work with a high accuracy even with variations such as occlusion, illumination changes, viewing angles, and others. The traditional pipeline of image classification with its main step of feature engineering is not suitable for working in rich environments. Even experts in the field won’t be able to give a single or a group of features that are able to reach high accuracy under different variations. Motivated by this problem, the idea of feature learning came out. The suitable features to work with images are learned automatically. This is the reason why artificial neural networks (ANNs) are one of the robust ways of image analysis. Based on a learning algorithm such as gradient descent (GD), ANN learns the image features automatically. The raw image is applied to the ANN and ANN is responsible for generating the features describing it.

Data & Analytics
1 of 122
Download to read offline
Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ‫المنوفية‬ ‫جامعة‬ ‫الحاسبات‬ ‫كلية‬‫والمعلومات‬ ‫المنوفية‬ ‫جامعة‬ Ahmed F. Gad 18-May-2018
15 8 9 10 17 22 20 3015 Input Image 3x3 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Weight 16 𝒘 𝟎 𝟎 𝒘 𝟏𝟓 𝟎 Total Number of Weights For First Pixel Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Weight 16 𝒘 𝟎 𝟏 𝒘 𝟏𝟓 𝟏 Total Number of Weights For Second Pixel Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 𝒘 𝟒 𝟐 𝒘 𝟓 𝟐 𝒘 𝟔 𝟐 𝒘 𝟕 𝟐 𝒘 𝟖 𝟐 𝒘 𝟗 𝟐 𝒘 𝟏𝟎 𝟐 𝒘 𝟏𝟏 𝟐 𝒘 𝟏𝟐 𝟐 𝒘 𝟏𝟑 𝟐 𝒘 𝟏𝟒 𝟐 𝒘 𝟏𝟓 𝟐 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 𝒘 𝟒 𝟑 𝒘 𝟓 𝟑 𝒘 𝟔 𝟑 𝒘 𝟕 𝟑 𝒘 𝟖 𝟑 𝒘 𝟗 𝟑 𝒘 𝟏𝟎 𝟑 𝒘 𝟏𝟏 𝟑 𝒘 𝟏𝟐 𝟑 𝒘 𝟏𝟑 𝟑 𝒘 𝟏𝟒 𝟑 𝒘 𝟏𝟓 𝟑 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 𝒘 𝟒 𝟒 𝒘 𝟓 𝟒 𝒘 𝟔 𝟒 𝒘 𝟕 𝟒 𝒘 𝟖 𝟒 𝒘 𝟗 𝟒 𝒘 𝟏𝟎 𝟒 𝒘 𝟏𝟏 𝟒 𝒘 𝟏𝟐 𝟒 𝒘 𝟏𝟑 𝟒 𝒘 𝟏𝟒 𝟒 𝒘 𝟏𝟓 𝟒 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 𝒘 𝟒 𝟓 𝒘 𝟓 𝟓 𝒘 𝟔 𝟓 𝒘 𝟕 𝟓 𝒘 𝟖 𝟓 𝒘 𝟗 𝟓 𝒘 𝟏𝟎 𝟓 𝒘 𝟏𝟏 𝟓 𝒘 𝟏𝟐 𝟓 𝒘 𝟏𝟑 𝟓 𝒘 𝟏𝟒 𝟓 𝒘 𝟏𝟓 𝟓 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 𝒘 𝟒 𝟔 𝒘 𝟓 𝟔 𝒘 𝟔 𝟔 𝒘 𝟕 𝟔 𝒘 𝟖 𝟔 𝒘 𝟗 𝟔 𝒘 𝟏𝟎 𝟔 𝒘 𝟏𝟏 𝟔 𝒘 𝟏𝟐 𝟔 𝒘 𝟏𝟑 𝟔 𝒘 𝟏𝟒 𝟔 𝒘 𝟏𝟓 𝟔 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 𝒘 𝟒 𝟕 𝒘 𝟓 𝟕 𝒘 𝟔 𝟕 𝒘 𝟕 𝟕 𝒘 𝟖 𝟕 𝒘 𝟗 𝟕 𝒘 𝟏𝟎 𝟕 𝒘 𝟏𝟏 𝟕 𝒘 𝟏𝟐 𝟕 𝒘 𝟏𝟑 𝟕 𝒘 𝟏𝟒 𝟕 𝒘 𝟏𝟓 𝟕 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 𝒘 𝟒 𝟖 𝒘 𝟓 𝟖 𝒘 𝟔 𝟖 𝒘 𝟕 𝟖 𝒘 𝟖 𝟖 𝒘 𝟗 𝟖 𝒘 𝟏𝟎 𝟖 𝒘 𝟏𝟏 𝟖 𝒘 𝟏𝟐 𝟖 𝒘 𝟏𝟑 𝟖 𝒘 𝟏𝟒 𝟖 𝒘 𝟏𝟓 𝟖 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Bias is Neglected. Ahmed F. Gad
Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500 Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500+ 810+4,500=5,310= Too Many Parameters Ahmed F. Gad
Input Image 32x32 Too Many Parameters Ahmed F. Gad
Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron Input Image 32x32 Too Many Parameters Ahmed F. Gad
1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 Too Many Parameters Ahmed F. Gad
1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 CNN can create a large network but with less number of parameters than FC networks Too Many Parameters Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Number of Parameters for First Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Number of Parameters for Second Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 All Connections Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Hidden Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Total Parameters After Grouping Neurons 9 Pixels 4 Weight Total Parameters = 9*4=36Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% More Reduction Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons Each neuron will process an image region of a specific size. In this example, region size is 2x2.
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels 4 Weights Parameters for First Hidden Neurons Group Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels Parameters for First Hidden Neurons Group 4*4=16 Weights Parameters for Entire Network 4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Saved Parameters 144-16=128 Reduction % (128/144)*100 =88.89% Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘 𝟑 `𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 8*𝒘` 𝟎 𝟎 +9*𝒘` 𝟏 𝟎 +17*𝒘` 𝟐 𝟎 +22𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 10*𝒘` 𝟎 𝟎 +17*𝒘` 𝟏 𝟎 +20*𝒘` 𝟐 𝟎 +15𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 17*𝒘 𝟎 𝟎 +22*𝒘 𝟏 𝟎 +15*𝒘 𝟐 𝟎 +30𝒘 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
References • Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic- Sign Detection and Classification. Springer, 2017. • Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step • https://www.linkedin.com/pulse/derivation-convolutional-neural-network- from-fully-connected-gad • https://www.slideshare.net/AhmedGadFCIT/derivation-of-convolutional- neural-network-convnet-from-fully-connected-network • https://www.kdnuggets.com/2018/04/derivation-convolutional-neural- network-fully-connected-step-by-step.html Ahmed F. Gad

Recommended

Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...
Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...
Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...Ahmed Gad
 
The neural tangent link between CNN denoisers and non-local filters
The neural tangent link between CNN denoisers and non-local filtersThe neural tangent link between CNN denoisers and non-local filters
The neural tangent link between CNN denoisers and non-local filtersJulián Tachella
 
Introduction to OpenCV
Introduction to OpenCVIntroduction to OpenCV
Introduction to OpenCVLuigi De Russis
 
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
[PR12] You Only Look Once (YOLO): Unified Real-Time Object DetectionTaegyun Jeon
 
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...Edureka!
 
"Deep Learning" Chap.6 Convolutional Neural Net
"Deep Learning" Chap.6 Convolutional Neural Net"Deep Learning" Chap.6 Convolutional Neural Net
"Deep Learning" Chap.6 Convolutional Neural NetKen'ichi Matsui
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial NetworksDong Heon Cho
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksBennoG1
 

Recommended

Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...
Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...
Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & T...Ahmed Gad
 
The neural tangent link between CNN denoisers and non-local filters
The neural tangent link between CNN denoisers and non-local filtersThe neural tangent link between CNN denoisers and non-local filters
The neural tangent link between CNN denoisers and non-local filtersJulián Tachella
 
Introduction to OpenCV
Introduction to OpenCVIntroduction to OpenCV
Introduction to OpenCVLuigi De Russis
 
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection[PR12] You Only Look Once (YOLO): Unified Real-Time Object Detection
[PR12] You Only Look Once (YOLO): Unified Real-Time Object DetectionTaegyun Jeon
 
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...
Naive Bayes Classifier Tutorial | Naive Bayes Classifier Example | Naive Baye...Edureka!
 
"Deep Learning" Chap.6 Convolutional Neural Net
"Deep Learning" Chap.6 Convolutional Neural Net"Deep Learning" Chap.6 Convolutional Neural Net
"Deep Learning" Chap.6 Convolutional Neural NetKen'ichi Matsui
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial NetworksDong Heon Cho
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksBennoG1
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks남주 김
 
Optimization problems and algorithms
Optimization problems and algorithmsOptimization problems and algorithms
Optimization problems and algorithmsAboul Ella Hassanien
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsMd. Main Uddin Rony
 
Autoencoders in Deep Learning
Autoencoders in Deep LearningAutoencoders in Deep Learning
Autoencoders in Deep Learningmilad abbasi
 
Variational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math BehindVariational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math BehindVarun Reddy
 
Explicit Density Models
Explicit Density ModelsExplicit Density Models
Explicit Density ModelsSangwoo Mo
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsBrief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsAhmed Gad
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNNChanuk Lim
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machinesnextlib
 
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Simplilearn
 
[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yoo[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yooJaeJun Yoo
 
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs) A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)Thomas da Silva Paula
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine LearningRaveen Perera
 
What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?Kazuki Yoshida
 
Image classification using cnn
Image classification using cnnImage classification using cnn
Image classification using cnnSumeraHangi
 
GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and ApplicationsHoang Nguyen
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial NetworksMark Chang
 
Neural Networks
Neural NetworksNeural Networks
Neural NetworksAdri Jovin
 
Machine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree LearningMachine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree Learningbutest
 
Tensorizing Neural Network
Tensorizing Neural NetworkTensorizing Neural Network
Tensorizing Neural NetworkRuochun Tzeng
 
ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmAhmed Gad
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...Ahmed Gad
 

More Related Content

What's hot

Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks남주 김
 
Optimization problems and algorithms
Optimization problems and algorithmsOptimization problems and algorithms
Optimization problems and algorithmsAboul Ella Hassanien
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsMd. Main Uddin Rony
 
Autoencoders in Deep Learning
Autoencoders in Deep LearningAutoencoders in Deep Learning
Autoencoders in Deep Learningmilad abbasi
 
Variational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math BehindVariational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math BehindVarun Reddy
 
Explicit Density Models
Explicit Density ModelsExplicit Density Models
Explicit Density ModelsSangwoo Mo
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsBrief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsAhmed Gad
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machinesnextlib
 
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Simplilearn
 
[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yoo[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yooJaeJun Yoo
 
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs) A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)Thomas da Silva Paula
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine LearningRaveen Perera
 
What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?Kazuki Yoshida
 
Image classification using cnn
Image classification using cnnImage classification using cnn
Image classification using cnnSumeraHangi
 
GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and ApplicationsHoang Nguyen
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial NetworksMark Chang
 
Neural Networks
Neural NetworksNeural Networks
Neural NetworksAdri Jovin
 
Machine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree LearningMachine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree Learningbutest
 
Tensorizing Neural Network
Tensorizing Neural NetworkTensorizing Neural Network
Tensorizing Neural NetworkRuochun Tzeng
 

What's hot (20)

Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Optimization problems and algorithms
Optimization problems and algorithmsOptimization problems and algorithms
Optimization problems and algorithms
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning Algorithms
 
Autoencoders in Deep Learning
Autoencoders in Deep LearningAutoencoders in Deep Learning
Autoencoders in Deep Learning
 
Variational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math BehindVariational Auto Encoder and the Math Behind
Variational Auto Encoder and the Math Behind
 
Explicit Density Models
Explicit Density ModelsExplicit Density Models
Explicit Density Models
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsBrief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNs
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
 
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
Naive Bayes Classifier | Naive Bayes Algorithm | Naive Bayes Classifier With ...
 
[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yoo[PR12] intro. to gans jaejun yoo
[PR12] intro. to gans jaejun yoo
 
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs) A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?What is the Expectation Maximization (EM) Algorithm?
What is the Expectation Maximization (EM) Algorithm?
 
Image classification using cnn
Image classification using cnnImage classification using cnn
Image classification using cnn
 
GANs and Applications
GANs and ApplicationsGANs and Applications
GANs and Applications
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
 
Neural Networks
Neural NetworksNeural Networks
Neural Networks
 
Machine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree LearningMachine Learning 3 - Decision Tree Learning
Machine Learning 3 - Decision Tree Learning
 
Tensorizing Neural Network
Tensorizing Neural NetworkTensorizing Neural Network
Tensorizing Neural Network
 

More from Ahmed Gad

ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmAhmed Gad
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...Ahmed Gad
 
Python for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionPython for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionAhmed Gad
 
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Ahmed Gad
 
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesM.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesAhmed Gad
 
Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Ahmed Gad
 
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Ahmed Gad
 
Avoid Overfitting with Regularization
Avoid Overfitting with RegularizationAvoid Overfitting with Regularization
Avoid Overfitting with RegularizationAhmed Gad
 
Genetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleGenetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleAhmed Gad
 
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisAhmed Gad
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepBackpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepAhmed Gad
 
Computer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientComputer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientAhmed Gad
 
Python for Computer Vision - Revision
Python for Computer Vision - RevisionPython for Computer Vision - Revision
Python for Computer Vision - RevisionAhmed Gad
 
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAnime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAhmed Gad
 
Operations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleOperations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleAhmed Gad
 
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingMATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingAhmed Gad
 
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...Ahmed Gad
 
Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Ahmed Gad
 
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Ahmed Gad
 
Introduction to Digital Signal Processing (DSP) - Course Notes
Introduction to Digital Signal Processing (DSP) - Course NotesIntroduction to Digital Signal Processing (DSP) - Course Notes
Introduction to Digital Signal Processing (DSP) - Course NotesAhmed Gad
 

More from Ahmed Gad (20)

ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic Algorithm
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
 
Python for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionPython for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd Edition
 
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
 
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesM.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
 
Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)
 
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
 
Avoid Overfitting with Regularization
Avoid Overfitting with RegularizationAvoid Overfitting with Regularization
Avoid Overfitting with Regularization
 
Genetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleGenetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step Example
 
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepBackpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
 
Computer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientComputer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and Gradient
 
Python for Computer Vision - Revision
Python for Computer Vision - RevisionPython for Computer Vision - Revision
Python for Computer Vision - Revision
 
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAnime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
 
Operations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleOperations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by Example
 
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingMATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
 
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
 
Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...
 
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
 
Introduction to Digital Signal Processing (DSP) - Course Notes
Introduction to Digital Signal Processing (DSP) - Course NotesIntroduction to Digital Signal Processing (DSP) - Course Notes
Introduction to Digital Signal Processing (DSP) - Course Notes
 

Recently uploaded

Defining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryDefining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryJeremy Anderson
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...dajasot375
 
While-For-loop in python used in college
While-For-loop in python used in collegeWhile-For-loop in python used in college
While-For-loop in python used in collegessuser7a7cd61
 
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort servicejennyeacort
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdfHuman37
 
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改yuu sss
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Colleen Farrelly
 
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...ssuserf63bd7
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFAAndrei Kaleshka
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queensdataanalyticsqueen03
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...limedy534
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]📊 Markus Baersch
 
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一F sss
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfJohn Sterrett
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Cantervoginip
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsVICTOR MAESTRE RAMIREZ
 
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样vhwb25kk
 
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Jack DiGiovanna
 

Recently uploaded (20)

Defining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data StoryDefining Constituents, Data Vizzes and Telling a Data Story
Defining Constituents, Data Vizzes and Telling a Data Story
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
 
While-For-loop in python used in college
While-For-loop in python used in collegeWhile-For-loop in python used in college
While-For-loop in python used in college
 
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
 
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
 
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024
 
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFA
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queens
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]
 
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdf
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Canter
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business Professionals
 
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
 
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
 

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

  • 1. Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ‫المنوفية‬ ‫جامعة‬ ‫الحاسبات‬ ‫كلية‬‫والمعلومات‬ ‫المنوفية‬ ‫جامعة‬ Ahmed F. Gad 18-May-2018
  • 2. 15 8 9 10 17 22 20 3015 Input Image 3x3 Ahmed F. Gad
  • 3. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 4. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 5. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 6. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 7. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 8. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 9. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 10. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 11. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 12. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 13. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 14. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer Ahmed F. Gad
  • 15. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  • 16. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  • 17. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  • 18. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 Fully Connected Network Ahmed F. Gad
  • 19. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 Fully Connected Network Ahmed F. Gad
  • 20. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 Fully Connected Network Ahmed F. Gad
  • 21. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  • 22. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Weight 16 𝒘 𝟎 𝟎 𝒘 𝟏𝟓 𝟎 Total Number of Weights For First Pixel Ahmed F. Gad
  • 23. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 24. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Ahmed F. Gad
  • 25. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Weight 16 𝒘 𝟎 𝟏 𝒘 𝟏𝟓 𝟏 Total Number of Weights For Second Pixel Ahmed F. Gad
  • 26. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 𝒘 𝟒 𝟐 𝒘 𝟓 𝟐 𝒘 𝟔 𝟐 𝒘 𝟕 𝟐 𝒘 𝟖 𝟐 𝒘 𝟗 𝟐 𝒘 𝟏𝟎 𝟐 𝒘 𝟏𝟏 𝟐 𝒘 𝟏𝟐 𝟐 𝒘 𝟏𝟑 𝟐 𝒘 𝟏𝟒 𝟐 𝒘 𝟏𝟓 𝟐 Ahmed F. Gad
  • 27. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 𝒘 𝟒 𝟑 𝒘 𝟓 𝟑 𝒘 𝟔 𝟑 𝒘 𝟕 𝟑 𝒘 𝟖 𝟑 𝒘 𝟗 𝟑 𝒘 𝟏𝟎 𝟑 𝒘 𝟏𝟏 𝟑 𝒘 𝟏𝟐 𝟑 𝒘 𝟏𝟑 𝟑 𝒘 𝟏𝟒 𝟑 𝒘 𝟏𝟓 𝟑 Ahmed F. Gad
  • 28. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 𝒘 𝟒 𝟒 𝒘 𝟓 𝟒 𝒘 𝟔 𝟒 𝒘 𝟕 𝟒 𝒘 𝟖 𝟒 𝒘 𝟗 𝟒 𝒘 𝟏𝟎 𝟒 𝒘 𝟏𝟏 𝟒 𝒘 𝟏𝟐 𝟒 𝒘 𝟏𝟑 𝟒 𝒘 𝟏𝟒 𝟒 𝒘 𝟏𝟓 𝟒 Ahmed F. Gad
  • 29. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 𝒘 𝟒 𝟓 𝒘 𝟓 𝟓 𝒘 𝟔 𝟓 𝒘 𝟕 𝟓 𝒘 𝟖 𝟓 𝒘 𝟗 𝟓 𝒘 𝟏𝟎 𝟓 𝒘 𝟏𝟏 𝟓 𝒘 𝟏𝟐 𝟓 𝒘 𝟏𝟑 𝟓 𝒘 𝟏𝟒 𝟓 𝒘 𝟏𝟓 𝟓 Ahmed F. Gad
  • 30. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 𝒘 𝟒 𝟔 𝒘 𝟓 𝟔 𝒘 𝟔 𝟔 𝒘 𝟕 𝟔 𝒘 𝟖 𝟔 𝒘 𝟗 𝟔 𝒘 𝟏𝟎 𝟔 𝒘 𝟏𝟏 𝟔 𝒘 𝟏𝟐 𝟔 𝒘 𝟏𝟑 𝟔 𝒘 𝟏𝟒 𝟔 𝒘 𝟏𝟓 𝟔 Ahmed F. Gad
  • 31. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 𝒘 𝟒 𝟕 𝒘 𝟓 𝟕 𝒘 𝟔 𝟕 𝒘 𝟕 𝟕 𝒘 𝟖 𝟕 𝒘 𝟗 𝟕 𝒘 𝟏𝟎 𝟕 𝒘 𝟏𝟏 𝟕 𝒘 𝟏𝟐 𝟕 𝒘 𝟏𝟑 𝟕 𝒘 𝟏𝟒 𝟕 𝒘 𝟏𝟓 𝟕 Ahmed F. Gad
  • 32. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 𝒘 𝟒 𝟖 𝒘 𝟓 𝟖 𝒘 𝟔 𝟖 𝒘 𝟕 𝟖 𝒘 𝟖 𝟖 𝒘 𝟗 𝟖 𝒘 𝟏𝟎 𝟖 𝒘 𝟏𝟏 𝟖 𝒘 𝟏𝟐 𝟖 𝒘 𝟏𝟑 𝟖 𝒘 𝟏𝟒 𝟖 𝒘 𝟏𝟓 𝟖 Ahmed F. Gad
  • 33. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 34. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Ahmed F. Gad
  • 35. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  • 36. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Bias is Neglected. Ahmed F. Gad
  • 37. Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  • 38. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  • 39. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500 Too Many Parameters Ahmed F. Gad
  • 40. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500+ 810+4,500=5,310= Too Many Parameters Ahmed F. Gad
  • 41. Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 42. Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 43. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 44. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 CNN can create a large network but with less number of parameters than FC networks Too Many Parameters Ahmed F. Gad
  • 45. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 46. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 47. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 48. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 49. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 50. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 51. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 52. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 53. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 54. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Number of Parameters for First Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  • 55. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Ahmed F. Gad
  • 56. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Number of Parameters for Second Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  • 57. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 Continue Ahmed F. Gad
  • 58. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 Continue Ahmed F. Gad
  • 59. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 Continue Ahmed F. Gad
  • 60. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 Continue Ahmed F. Gad
  • 61. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 Continue Ahmed F. Gad
  • 62. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 Continue Ahmed F. Gad
  • 63. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 Continue Ahmed F. Gad
  • 64. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 All Connections Ahmed F. Gad
  • 65. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Ahmed F. Gad
  • 66. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Hidden Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  • 67. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Total Parameters After Grouping Neurons 9 Pixels 4 Weight Total Parameters = 9*4=36Ahmed F. Gad
  • 68. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Ahmed F. Gad
  • 69. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% Ahmed F. Gad
  • 70. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% More Reduction Ahmed F. Gad
  • 71. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 72. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 73. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 74. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  • 75. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  • 79. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad
  • 80. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons
  • 81. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons Each neuron will process an image region of a specific size. In this example, region size is 2x2.
  • 82. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 83. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 84. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 85. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 86. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 87. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  • 88. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  • 89. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad
  • 90. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 91. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 92. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 93. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 94. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 95. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 96. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 97. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 98. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 99. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 100. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 101. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 102. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 103. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 104. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 105. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 106. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 107. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 108. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 109. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 110. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels 4 Weights Parameters for First Hidden Neurons Group Ahmed F. Gad
  • 111. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels Parameters for First Hidden Neurons Group 4*4=16 Weights Parameters for Entire Network 4 Weights Ahmed F. Gad
  • 112. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Ahmed F. Gad
  • 113. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Saved Parameters 144-16=128 Reduction % (128/144)*100 =88.89% Ahmed F. Gad
  • 114. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 115. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 116. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘 𝟑 `𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 117. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 8*𝒘` 𝟎 𝟎 +9*𝒘` 𝟏 𝟎 +17*𝒘` 𝟐 𝟎 +22𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 118. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 10*𝒘` 𝟎 𝟎 +17*𝒘` 𝟏 𝟎 +20*𝒘` 𝟐 𝟎 +15𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 119. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 17*𝒘 𝟎 𝟎 +22*𝒘 𝟏 𝟎 +15*𝒘 𝟐 𝟎 +30𝒘 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 120. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  • 121. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  • 122. References • Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic- Sign Detection and Classification. Springer, 2017. • Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step • https://www.linkedin.com/pulse/derivation-convolutional-neural-network- from-fully-connected-gad • https://www.slideshare.net/AhmedGadFCIT/derivation-of-convolutional- neural-network-convnet-from-fully-connected-network • https://www.kdnuggets.com/2018/04/derivation-convolutional-neural- network-fully-connected-step-by-step.html Ahmed F. Gad