6. Fully Connected Neural Network
PR-163: CNN_Attention_Networks
Input
Feature
Hidden State
Representation
𝑤1
𝑤2
…
𝑤 𝑛
Fully Connected NN
Represent Blue
using Weighted Sum of Inputs
without Constraint
7. Convolutional Neural Network
PR-163: CNN_Attention_Networks
Input
Feature
Hidden State
Representation
(V)
(V)
(V)
(V)
(V)
(V)
𝑤1
𝑤2
𝑤3
Convolutional NN
Represent Blue
using Weighted Sum of Inputs
from Current Position
Q
58. RAN / SE / BAM / CBAM Comparison
PR-163: CNN_Attention_Networks
Network Module Position Attention
RAN (CVPR17) Modified ChannelxSpatial 3D
SE (CVPR18) In the ResBlock Channel
BAM (BMVC18)
Before the Stride=2
ResBlock
Channel, Spatial Parallel
CBAM (ECCV18) In the ResBlock Channel, Spatial Sequential