Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Artificial Neural Networks (ANNs)
Step-By-Step Training & Testing
Example 2
MENOUFIA UNIVERSITY
FACULTY OF COMPUTERS AND I...
Classification Example
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
Neural Networks
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
Input Hidden Output
Neural Networks
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
Input Layer
Input Output
𝑭 𝟏
𝑭 𝟐
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
Output Layer
Input Output
C1/C2
𝒀𝒋
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
𝑭 𝟏
𝑭 𝟐
Weights
Input Output
Weights=𝑾𝒊
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C1/C2
𝒀𝒋
𝑭 𝟏
...
Activation Function
Input Output
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C1/C2
𝒀𝒋
𝑭 𝟏...
Activation Function
Input Output
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C1/C2
𝒀𝒋
𝑭 𝟏...
Activation Function
Input Output
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C1/C2
𝒀𝒋
𝑭 𝟏...
Activation Function
Components
Input Output
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C...
Activation Function
Inputs
Input Output
s
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
C1/...
Activation Function
Inputs
Input Output
ss=SOP(𝑿𝒊, 𝑾𝒊)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5...
Activation Function
Inputs
Input Output
ss=SOP(𝑿𝒊, 𝑾𝒊)
𝑿𝒊=Inputs 𝑾𝒊=Weights
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
5...
Activation Function
Inputs
Input Output
ss=SOP(𝑿𝒊, 𝑾𝒊)
𝑿𝒊=Inputs 𝑾𝒊=Weights
𝑿 𝟏
𝑿 𝟐
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8....
Activation Function
Inputs
Input Output
ss=SOP(𝑿𝒊, 𝑾𝒊)
𝑿𝒊=Inputs 𝑾𝒊=Weights
S= 𝟏
𝒎
𝑿𝒊 𝑾𝒊
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
...
Activation Function
Inputs
Input Output
ss=SOP(𝑿𝒊, 𝑾𝒊)
𝑿𝒊=Inputs 𝑾𝒊=Weights s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
...
Activation Function
Outputs
Input Output
F(s)s Class Label
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250...
Activation Functions
Piecewise
Linear Sigmoid Signum
Activation Functions
Which activation function to use?
Outputs
Class
Labels
Activation
Function
TWO Class
Labels
TWO
Outpu...
Activation Functions
Piecewise
Linear Sigmoid SignumSignum
Activation Function
Input Output
F(s)s sgn
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
0
50
100
150
200
250
0 5 10 15 20
𝑿 ...
Bias
Input Output
F(s)s sgn
=+1𝑿 𝟎
W0
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
𝑿 𝟏
𝑿 𝟐
C1/C2
𝒀𝒋
𝑭 𝟏
...
Bias
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1195
=+1...
Bias
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑿 𝟎 = +𝟏
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8...
Bias
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(+𝟏 ∗ 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1...
Bias
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2
8.1...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4210
C2...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=ax+b
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9....
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
𝑭 𝟐𝑭 𝟏
16.8121
C1
15.2114
9.4...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
𝑭 𝟐𝑭 𝟏
16.8121
C1...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=0
𝑭 𝟐𝑭 𝟏
16.812...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=0
𝑭 𝟐𝑭 𝟏
16.812...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=+v
𝑭 𝟐𝑭 𝟏
16.81...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=+v
𝑭 𝟐𝑭 𝟏
16.81...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=-v
𝑭 𝟐𝑭 𝟏
16.81...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
X
Y y=x+b
Y-Intercept
b=-v
𝑭 𝟐𝑭 𝟏
16.81...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
Same Concept Applies to Bias
S= 𝟏
𝒎
𝑿𝒊 ...
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
S= 𝟏
𝒎
𝑿𝒊 𝑾𝒊+BIAS
𝑭 𝟐𝑭 𝟏
16.8121
C1
15....
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
S= 𝟏
𝒎
𝑿𝒊 𝑾𝒊+BIAS
𝑭 𝟐𝑭 𝟏
16.8121
C1
15....
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
S= 𝟏
𝒎
𝑿𝒊 𝑾𝒊+BIAS
𝑭 𝟐𝑭 𝟏
16.8121
C1
15....
Bias Importance
Input Output
F(s)s sgn
s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
S= 𝟏
𝒎
𝑿𝒊 𝑾𝒊+BIAS
𝑭 𝟐𝑭 𝟏
16.8121
C1
15....
Learning Rate
F(s)s sgn
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏=+1𝑿 𝟎
W0
𝑿 𝟏
𝑿 𝟐
𝑭 𝟏
𝑭 𝟐
𝑾 𝟏
𝑾 𝟐
C1/C2
𝒀𝒋
Summary of Parameters
Inputs 𝑿 𝒎
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
F(s)s sgn
=+1𝑿 𝟎
W0
𝑿 𝟏
𝑿 𝟐
𝑭 𝟏...
Summary of Parameters
Weights 𝑾 𝒎
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐)
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
F(s)s sgn
...
Summary of Parameters
Bias 𝒃
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐)
F(s)s sgn
=+1𝑿 ...
Summary of Parameters
Sum Of Products (SOP) 𝒔
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐...
Summary of Parameters
Activation Function 𝒔𝒈𝒏
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐...
Summary of Parameters
Outputs 𝒀𝒋
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐)
F(s)s sgn
=...
Summary of Parameters
Learning Rate η
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐)
F(s)s ...
Other Parameters
Step n
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏𝒏 = 𝟎, 𝟏, 𝟐, …
𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐)
W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐)
F(s)s ...
Other Parameters
Desired Output 𝒅𝒋
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
𝟎 ≤ η ≤ 𝟏𝒏 = 𝟎, 𝟏, 𝟐, …
𝒅 𝒏 =
+𝟏, 𝒙 𝒏 𝒃𝒆𝒍𝒐𝒏𝒈𝒔 𝒕𝒐 𝑪𝟏
−𝟏, 𝒙 𝒏...
Neural Networks Training Steps
Weights Initialization
Inputs Application
Sum of Inputs-Weights Products
Activation Functio...
Regarding 5th Step: Weights Adaptation
• If the predicted output Y is not the same as the desired output d,
then weights a...
Neural Networks
Training Example
Step n=0
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=0
F(s)s sgn
=+1𝑿 𝟎
-1230
𝑿 𝟏
𝑿 𝟐
121
𝟏𝟔. 𝟖
-30
300
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
...
Neural Networks
Training Example
Step n=0 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1230+121*-30+16.8*300
=180
𝑭 𝟐𝑭 𝟏
16.8121...
Neural Networks
Training Example
Step n=0 - Output
𝒀 𝒏 = 𝒀 𝟎
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 𝟏𝟖𝟎
= +𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐𝑭 𝟏
16....
Neural Networks
Training Example
Step n=0 - Output
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎
-...
Neural Networks
Training Example
Step n=0
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟎 = +𝟏
𝐝 𝒏 = 𝒅 𝟎 = +𝟏
∵ 𝒀 𝒏 = 𝒅 𝒏
∴ Weights are Co...
Neural Networks
Training Example
Step n=1
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=1
F(s)s sgn
=+1𝑿 𝟎
-1230
𝑿 𝟏
𝑿 𝟐
114
𝟏𝟓. 𝟐
-30
300
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
...
Neural Networks
Training Example
Step n=1 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1230+114*-30+15.2*300
=-90
𝑭 𝟐𝑭 𝟏
16.8121...
Neural Networks
Training Example
Step n=1 - Output
𝒀 𝒏 = 𝒀 𝟏
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 −𝟗𝟎
= −𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐𝑭 𝟏
16....
Neural Networks
Training Example
Step n=1 - Output
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎
-...
Neural Networks
Training Example
Step n=1
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟏 = −𝟏
𝐝 𝒏 = 𝒅 𝟏 = +𝟏
∵ 𝒀 𝒏 ≠ 𝒅 𝒏
∴ Weights are In...
Weights Adaptation
• According to
𝑾 𝒏 + 𝟏 = 𝑾 𝒏 + η 𝒅 𝒏 − 𝒀 𝒏 𝑿(𝒏)
• Where n = 1
𝑾 𝟏 + 𝟏 = 𝑾 𝟏 + η 𝒅 𝟏 − 𝒀 𝟏 𝑿(𝟏)
𝑾 𝟐 = −𝟏...
Neural Networks
Training Example
Step n=2
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=2
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08...
Neural Networks
Training Example
Step n=2 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1229.08+210*-
27.72+9.4*300.304
=-4227.42...
Neural Networks
Training Example
Step n=2 - Output
𝒀 𝒏 = 𝒀 𝟐
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 −4227.4224
= +𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐...
Neural Networks
Training Example
Step n=2 - Output
𝒀 𝒏 = 𝒀 𝟐
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 −4227.4224
= −𝟏
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
...
Neural Networks
Training Example
Step n=2
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟐 = −𝟏
𝐝 𝒏 = 𝒅 𝟐 = −𝟏
∵ 𝒀 𝒏 = 𝒅 𝒏
∴ Weights are Co...
Neural Networks
Training Example
Step n=3
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=3
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08...
Neural Networks
Training Example
Step n=3 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1229.08+195*-
27.72+8.1*300.304
=-4202.01...
Neural Networks
Training Example
Step n=3 - Output
𝒀 𝒏 = 𝒀 𝟑
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 −4202.0176
= −𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐...
Neural Networks
Training Example
Step n=3 - Output
𝒀 𝒏 = 𝒀 𝟑
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 −4202.0176
= −𝟏
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
...
Neural Networks
Training Example
Step n=3
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟑 = −𝟏
𝐝 𝒏 = 𝒅 𝟑 = −𝟏
∵ 𝒀 𝒏 = 𝒅 𝒏
∴ Weights are Co...
Neural Networks
Training Example
Step n=4
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=4
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08...
Neural Networks
Training Example
Step n=4 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1229.08+121*-
27.72+16.8*300.304
=461.91
...
Neural Networks
Training Example
Step n=4 - Output
𝒀 𝒏 = 𝒀 𝟒
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 461.91
= +𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐𝑭 𝟏
...
Neural Networks
Training Example
Step n=4 - Output
𝒀 𝒏 = 𝒀 𝟒
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 461.91
= +𝟏
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.42...
Neural Networks
Training Example
Step n=4
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟒 = +𝟏
𝐝 𝒏 = 𝒅 𝟒 = +𝟏
∵ 𝒀 𝒏 = 𝒅 𝒏
∴ Weights are Co...
Neural Networks
Training Example
Step n=5
• In each step in the solution, the parameters of the neural network
must be kno...
Neural Networks
Training Example
Step n=5
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4210
C2 = -1
8.1195
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08...
Neural Networks
Training Example
Step n=5 - SOP
s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐)
=+1*-1229.08+114*-
27.72+15.2*300.304
= 1𝟕𝟓. 𝟒...
Neural Networks
Training Example
Step n=5 - Output
𝒀 𝒏 = 𝒀 𝟓
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 1𝟕𝟓. 𝟒𝟔
= +𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
𝑭 𝟐𝑭 𝟏...
Neural Networks
Training Example
Step n=5 - Output
𝒀 𝒏 = 𝒀 𝟓
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 1𝟕𝟓. 𝟒𝟔
= +𝟏
𝑭 𝟐𝑭 𝟏
16.8121
C1 = +1
15.2114
9.4...
Neural Networks
Training Example
Step n=5
Predicted Vs. Desired
𝒀 𝒏 = 𝒀 𝟒 = +𝟏
𝐝 𝒏 = 𝒅 𝟒 = +𝟏
∵ 𝒀 𝒏 = 𝒅 𝒏
∴ Weights are Co...
Correct Weights
• After testing the weights across all samples and results were correct
then we can conclude that current ...
Testing Trained Neural Network
(F1, F2) = (140, 17.9)
Trained Neural Networks Parameters
𝑊 = −1229.08, −27.72, 300.304
Testing Trained Neural Network
(F1, F2) = (140, 17.9)
SOP
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08
𝑿 𝟏
𝑿 𝟐
140
𝟏𝟕. 𝟗
-27.72
300.3
04
C1...
Testing Trained Neural Network
(F1, F2) = (140, 17.9)
Output
𝒀
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 𝟐𝟔𝟓
= +𝟏
𝒔𝒈𝒏 𝒔 =
+𝟏, 𝒔 ≥ 𝟎
−𝟏, 𝒔 < 𝟎
F(s)s sg...
Testing Trained Neural Network
(F1, F2) = (140, 17.9)
Output
𝒀
= 𝑺𝑮𝑵 𝒔
= 𝑺𝑮𝑵 𝟐𝟔𝟓
= +𝟏
F(s)s sgn
=+1𝑿 𝟎 -
1229.
08
𝑿 𝟏
𝑿 𝟐
...
Upcoming SlideShare
Loading in …5
×

Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & Testing Example 2

This is an introduction to artificial neural networks (ANNs) including the idea of classification and how ANNs can classify data into number of distinct classes based on some features.
A basic neural network example is given that uses a single layer perceptron with three inputs and one output to classify data linearly using the Signum activation function.
The presented example is about classifying data about colors into two categories (Red and Blue).
Artificial neural networks (ANNs) or connectionist systems are a computational model used in machine learning, computer science and other research disciplines, which is based on a large collection of connected simple units called artificial neurons, loosely analogous to axons in a biological brain. Connections between neurons carry an activation signal of varying strength. If the combined incoming signals are strong enough, the neuron becomes activated and the signal travels to other neurons connected to it. Such systems can be trained from examples, rather than explicitly programmed, and excel in areas where the solution or feature detection is difficult to express in a traditional computer program. Like other machine learning methods, neural networks have been used to solve a wide variety of tasks, like computer vision and speech recognition, that are difficult to solve using ordinary rule-based programming.

Find me on:
AFCIT
http://www.afcit.xyz

YouTube
https://www.youtube.com/channel/UCuewOYbBXH5gwhfOrQOZOdw

Google Plus
https://plus.google.com/u/0/+AhmedGadIT

SlideShare
https://www.slideshare.net/AhmedGadFCIT

LinkedIn
https://www.linkedin.com/in/ahmedfgad/

ResearchGate
https://www.researchgate.net/profile/Ahmed_Gad13

Academia
https://www.academia.edu/

Google Scholar
https://scholar.google.com.eg/citations?user=r07tjocAAAAJ&hl=en

Mendelay
https://www.mendeley.com/profiles/ahmed-gad12/

ORCID
https://orcid.org/0000-0003-1978-8574

StackOverFlow
http://stackoverflow.com/users/5426539/ahmed-gad

Twitter
https://twitter.com/ahmedfgad

Facebook
https://www.facebook.com/ahmed.f.gadd

Pinterest
https://www.pinterest.com/ahmedfgad/

  • Be the first to comment

Introduction to Artificial Neural Networks (ANNs) - Step-by-Step Training & Testing Example 2

  1. 1. Artificial Neural Networks (ANNs) Step-By-Step Training & Testing Example 2 MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ALL DEPARTMENTS ARTIFICIAL INTELLIGENCE ‫المنوفية‬ ‫جامعة‬ ‫والمعلومات‬ ‫الحاسبات‬ ‫كلية‬ ‫األقسام‬ ‫جميع‬ ‫الذكاء‬‫اإلصطناعي‬ ‫المنوفية‬ ‫جامعة‬ Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg
  2. 2. Classification Example 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195
  3. 3. Neural Networks 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 Input Hidden Output
  4. 4. Neural Networks 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20
  5. 5. Input Layer Input Output 𝑭 𝟏 𝑭 𝟐 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20
  6. 6. Output Layer Input Output C1/C2 𝒀𝒋 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 𝑭 𝟏 𝑭 𝟐
  7. 7. Weights Input Output Weights=𝑾𝒊 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  8. 8. Activation Function Input Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  9. 9. Activation Function Input Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  10. 10. Activation Function Input Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  11. 11. Activation Function Components Input Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  12. 12. Activation Function Inputs Input Output s 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  13. 13. Activation Function Inputs Input Output ss=SOP(𝑿𝒊, 𝑾𝒊) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  14. 14. Activation Function Inputs Input Output ss=SOP(𝑿𝒊, 𝑾𝒊) 𝑿𝒊=Inputs 𝑾𝒊=Weights 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  15. 15. Activation Function Inputs Input Output ss=SOP(𝑿𝒊, 𝑾𝒊) 𝑿𝒊=Inputs 𝑾𝒊=Weights 𝑿 𝟏 𝑿 𝟐 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  16. 16. Activation Function Inputs Input Output ss=SOP(𝑿𝒊, 𝑾𝒊) 𝑿𝒊=Inputs 𝑾𝒊=Weights S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  17. 17. Activation Function Inputs Input Output ss=SOP(𝑿𝒊, 𝑾𝒊) 𝑿𝒊=Inputs 𝑾𝒊=Weights s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  18. 18. Activation Function Outputs Input Output F(s)s Class Label 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  19. 19. Activation Functions Piecewise Linear Sigmoid Signum
  20. 20. Activation Functions Which activation function to use? Outputs Class Labels Activation Function TWO Class Labels TWO Outputs One that gives two outputs. Which activation function to use? 𝑪𝒋𝒀𝒋 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195
  21. 21. Activation Functions Piecewise Linear Sigmoid SignumSignum
  22. 22. Activation Function Input Output F(s)s sgn 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 0 50 100 150 200 250 0 5 10 15 20 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  23. 23. Bias Input Output F(s)s sgn =+1𝑿 𝟎 W0 s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  24. 24. Bias Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  25. 25. Bias Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑿 𝟎 = +𝟏 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  26. 26. Bias Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(+𝟏 ∗ 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  27. 27. Bias Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  28. 28. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  29. 29. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  30. 30. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=ax+b 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  31. 31. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  32. 32. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  33. 33. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=0 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  34. 34. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=0 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  35. 35. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=+v 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  36. 36. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=+v 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  37. 37. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=-v 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  38. 38. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) X Y y=x+b Y-Intercept b=-v 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  39. 39. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) Same Concept Applies to Bias S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  40. 40. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  41. 41. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  42. 42. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  43. 43. Bias Importance Input Output F(s)s sgn s=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) s=(𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 C1/C2 𝒀𝒋 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐
  44. 44. Learning Rate F(s)s sgn s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏=+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  45. 45. Summary of Parameters Inputs 𝑿 𝒎 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  46. 46. Summary of Parameters Weights 𝑾 𝒎 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  47. 47. Summary of Parameters Bias 𝒃 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  48. 48. Summary of Parameters Sum Of Products (SOP) 𝒔 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  49. 49. Summary of Parameters Activation Function 𝒔𝒈𝒏 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  50. 50. Summary of Parameters Outputs 𝒀𝒋 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  51. 51. Summary of Parameters Learning Rate η s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  52. 52. Other Parameters Step n s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏𝒏 = 𝟎, 𝟏, 𝟐, … 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  53. 53. Other Parameters Desired Output 𝒅𝒋 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) 𝟎 ≤ η ≤ 𝟏𝒏 = 𝟎, 𝟏, 𝟐, … 𝒅 𝒏 = +𝟏, 𝒙 𝒏 𝒃𝒆𝒍𝒐𝒏𝒈𝒔 𝒕𝒐 𝑪𝟏 −𝟏, 𝒙 𝒏 𝒃𝒆𝒍𝒐𝒏𝒈𝒔 𝒕𝒐 𝑪𝟐 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐) 𝑭 𝟐𝑭 𝟏 16.8121 C1 15.2114 9.4210 C2 8.1195 F(s)s sgn =+1𝑿 𝟎 W0 𝑿 𝟏 𝑿 𝟐 𝑭 𝟏 𝑭 𝟐 𝑾 𝟏 𝑾 𝟐 C1/C2 𝒀𝒋
  54. 54. Neural Networks Training Steps Weights Initialization Inputs Application Sum of Inputs-Weights Products Activation Function Response Calculation Weights Adaptation Back to Step 2 1 2 3 4 5 6
  55. 55. Regarding 5th Step: Weights Adaptation • If the predicted output Y is not the same as the desired output d, then weights are to be adapted according to the following equation: 𝑾 𝒏 + 𝟏 = 𝑾 𝒏 + η 𝒅 𝒏 − 𝒀 𝒏 𝑿(𝒏) Where 𝑾 𝒏 = [𝒃 𝒏 , 𝑾 𝟏(𝒏), 𝑾 𝟐(𝒏), 𝑾 𝟑(𝒏), … , 𝑾 𝒎(𝒏)]
  56. 56. Neural Networks Training Example Step n=0 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=0: η = .01 𝑋 𝑛 = 𝑋 0 = +1, 121, 16.8 𝑊 𝑛 = 𝑊 0 = −1230, −30, 300 𝑑 𝑛 = 𝑑 0 = +1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  57. 57. Neural Networks Training Example Step n=0 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -30 300 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 C1/C2 𝒀(𝒏)
  58. 58. Neural Networks Training Example Step n=0 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1230+121*-30+16.8*300 =180 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -30 300 C1/C2 𝒀(𝒏)
  59. 59. Neural Networks Training Example Step n=0 - Output 𝒀 𝒏 = 𝒀 𝟎 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 𝟏𝟖𝟎 = +𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -30 300 C1/C2 𝒀(𝒏)
  60. 60. Neural Networks Training Example Step n=0 - Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -30 300 𝒀 𝒏 = 𝒀 𝟎 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 𝟏𝟖𝟎 = +𝟏 C1 +1
  61. 61. Neural Networks Training Example Step n=0 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟎 = +𝟏 𝐝 𝒏 = 𝒅 𝟎 = +𝟏 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -30 300 C1 +1
  62. 62. Neural Networks Training Example Step n=1 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=1: η = .01 𝑋 𝑛 = 𝑋 1 = +1, 121, 16.8 𝑊 𝑛 = 𝑊 1 = 𝑊 0 = −1230, −30, 300 𝑑 𝑛 = 𝑑 1 = +1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  63. 63. Neural Networks Training Example Step n=1 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -30 300 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 C1/C2 𝒀(𝒏)
  64. 64. Neural Networks Training Example Step n=1 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1230+114*-30+15.2*300 =-90 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -30 300 C1/C2 𝒀(𝒏)
  65. 65. Neural Networks Training Example Step n=1 - Output 𝒀 𝒏 = 𝒀 𝟏 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −𝟗𝟎 = −𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -30 300 C1/C2 𝒀(𝒏)
  66. 66. Neural Networks Training Example Step n=1 - Output 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -30 300 𝒀 𝒏 = 𝒀 𝟏 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −𝟗𝟎 = −𝟏 C2 -1
  67. 67. Neural Networks Training Example Step n=1 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟏 = −𝟏 𝐝 𝒏 = 𝒅 𝟏 = +𝟏 ∵ 𝒀 𝒏 ≠ 𝒅 𝒏 ∴ Weights are Incorrect. Adaptation Required. 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 -1230 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -30 300 C2 -1
  68. 68. Weights Adaptation • According to 𝑾 𝒏 + 𝟏 = 𝑾 𝒏 + η 𝒅 𝒏 − 𝒀 𝒏 𝑿(𝒏) • Where n = 1 𝑾 𝟏 + 𝟏 = 𝑾 𝟏 + η 𝒅 𝟏 − 𝒀 𝟏 𝑿(𝟏) 𝑾 𝟐 = −𝟏𝟐𝟑𝟎, 𝟑𝟎, 𝟑𝟎𝟎 + .01 +𝟏 − (−𝟏) +𝟏, 𝟏𝟏𝟒, 𝟏𝟓. 𝟐 𝑾 𝟐 = −𝟏𝟐𝟑𝟎, 𝟑𝟎, 𝟑𝟎𝟎 + .01 +𝟐 +𝟏, 𝟏𝟏𝟒, 𝟏𝟓. 𝟐 𝑾 𝟐 = −𝟏𝟐𝟑𝟎, 𝟑𝟎, 𝟑𝟎𝟎 + .0 𝟐 +𝟏, 𝟏𝟏𝟒, 𝟏𝟓. 𝟐 𝑾 𝟐 = −𝟏𝟐𝟑𝟎, 𝟑𝟎, 𝟑𝟎𝟎 + +. 𝟎𝟐, 𝟐. 𝟐𝟖, . 𝟑𝟎𝟒 𝑾 𝟐 = −𝟏𝟐𝟐𝟗. 𝟎𝟖, −𝟐𝟕. 𝟕𝟐, 𝟑𝟎𝟎. 𝟑𝟎𝟒
  69. 69. Neural Networks Training Example Step n=2 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=2: η = .01 𝑋 𝑛 = 𝑋 2 = +1, 210, 9.4 𝑊 𝑛 = 𝑊 2 = −1229.08, −27.72, 300.304 𝑑 𝑛 = 𝑑 2 = −1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  70. 70. Neural Networks Training Example Step n=2 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 210 𝟗. 𝟒 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  71. 71. Neural Networks Training Example Step n=2 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1229.08+210*- 27.72+9.4*300.304 =-4227.4224 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 210 𝟗. 𝟒 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  72. 72. Neural Networks Training Example Step n=2 - Output 𝒀 𝒏 = 𝒀 𝟐 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −4227.4224 = +𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 210 𝟗. 𝟒 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  73. 73. Neural Networks Training Example Step n=2 - Output 𝒀 𝒏 = 𝒀 𝟐 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −4227.4224 = −𝟏 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 210 𝟗. 𝟒 -27.72 300.3 04 C2 −𝟏
  74. 74. Neural Networks Training Example Step n=2 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟐 = −𝟏 𝐝 𝒏 = 𝒅 𝟐 = −𝟏 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 210 𝟗. 𝟒 -27.72 300.3 04 C2 −𝟏
  75. 75. Neural Networks Training Example Step n=3 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=3: η = .01 𝑋 𝑛 = 𝑋 3 = +1, 210, 9.4 𝑊 𝑛 = 𝑊 3 = 𝑊 2 = −1229.08, −27.72, 300.304 𝑑 𝑛 = 𝑑 3 = −1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  76. 76. Neural Networks Training Example Step n=3 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 195 𝟖. 𝟏 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  77. 77. Neural Networks Training Example Step n=3 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1229.08+195*- 27.72+8.1*300.304 =-4202.0176 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 195 𝟖. 𝟏 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  78. 78. Neural Networks Training Example Step n=3 - Output 𝒀 𝒏 = 𝒀 𝟑 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −4202.0176 = −𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 195 𝟖. 𝟏 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  79. 79. Neural Networks Training Example Step n=3 - Output 𝒀 𝒏 = 𝒀 𝟑 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 −4202.0176 = −𝟏 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 195 𝟖. 𝟏 -27.72 300.3 04 C2 −𝟏
  80. 80. Neural Networks Training Example Step n=3 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟑 = −𝟏 𝐝 𝒏 = 𝒅 𝟑 = −𝟏 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 195 𝟖. 𝟏 -27.72 300.3 04 C1 −𝟏
  81. 81. Neural Networks Training Example Step n=4 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=4: η = .01 𝑋 𝑛 = 𝑋 4 = +1, 121, 16.8 𝑊 𝑛 = 𝑊 4 = 𝑊 3 = −1229.08, −27.72, 300.304 𝑑 𝑛 = 𝑑 4 = +1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  82. 82. Neural Networks Training Example Step n=4 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  83. 83. Neural Networks Training Example Step n=4 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1229.08+121*- 27.72+16.8*300.304 =461.91 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  84. 84. Neural Networks Training Example Step n=4 - Output 𝒀 𝒏 = 𝒀 𝟒 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 461.91 = +𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  85. 85. Neural Networks Training Example Step n=4 - Output 𝒀 𝒏 = 𝒀 𝟒 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 461.91 = +𝟏 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -27.72 300.3 04 C1 +𝟏
  86. 86. Neural Networks Training Example Step n=4 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟒 = +𝟏 𝐝 𝒏 = 𝒅 𝟒 = +𝟏 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 121 𝟏𝟔. 𝟖 -27.72 300.3 04 C1 +𝟏
  87. 87. Neural Networks Training Example Step n=5 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=5: η = .01 𝑋 𝑛 = 𝑋 5 = +1, 114, 15.2 𝑊 𝑛 = 𝑊 5 = 𝑊 4 = −1229.08, −27.72, 300.304 𝑑 𝑛 = 𝑑 5 = +1 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195
  88. 88. Neural Networks Training Example Step n=5 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  89. 89. Neural Networks Training Example Step n=5 - SOP s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1229.08+114*- 27.72+15.2*300.304 = 1𝟕𝟓. 𝟒𝟔 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  90. 90. Neural Networks Training Example Step n=5 - Output 𝒀 𝒏 = 𝒀 𝟓 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 1𝟕𝟓. 𝟒𝟔 = +𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  91. 91. Neural Networks Training Example Step n=5 - Output 𝒀 𝒏 = 𝒀 𝟓 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 1𝟕𝟓. 𝟒𝟔 = +𝟏 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -27.72 300.3 04 C1 +𝟏
  92. 92. Neural Networks Training Example Step n=5 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟒 = +𝟏 𝐝 𝒏 = 𝒅 𝟒 = +𝟏 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation 𝑭 𝟐𝑭 𝟏 16.8121 C1 = +1 15.2114 9.4210 C2 = -1 8.1195 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 114 𝟏𝟓. 𝟐 -27.72 300.3 04 C1 +𝟏
  93. 93. Correct Weights • After testing the weights across all samples and results were correct then we can conclude that current weights are correct ones for training the neural network. • After training phase we come to testing the neural network. • What is the class of the unknown color of values of F1=140 and F2=17.9?
  94. 94. Testing Trained Neural Network (F1, F2) = (140, 17.9) Trained Neural Networks Parameters 𝑊 = −1229.08, −27.72, 300.304
  95. 95. Testing Trained Neural Network (F1, F2) = (140, 17.9) SOP F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 140 𝟏𝟕. 𝟗 -27.72 300.3 04 C1/C2 𝒀(𝒏) s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐) =+1*-1229.08+140*- 27.72+17.9*300.304 = 𝟐𝟔𝟓
  96. 96. Testing Trained Neural Network (F1, F2) = (140, 17.9) Output 𝒀 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 𝟐𝟔𝟓 = +𝟏 𝒔𝒈𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 140 𝟏𝟕. 𝟗 -27.72 300.3 04 C1/C2 𝒀(𝒏)
  97. 97. Testing Trained Neural Network (F1, F2) = (140, 17.9) Output 𝒀 = 𝑺𝑮𝑵 𝒔 = 𝑺𝑮𝑵 𝟐𝟔𝟓 = +𝟏 F(s)s sgn =+1𝑿 𝟎 - 1229. 08 𝑿 𝟏 𝑿 𝟐 140 𝟏𝟕. 𝟗 -27.72 300.3 04 C1 +1

×