Neural Networkを可視化してみる（数学カフェ忘年会 LT資料） Christopher Olahさんのブログ "Neural Networks, Manifolds, and Topology" http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ にインスピレーションを受け、Pythonで実装、検証してみたものです。

1. Neural Network
2016/12/11(Sun)
LT
@kenmatsu4

2. Twitter

3. Neural Network
0
0
1
0
x2
x783
x784
x1
x0
z10u10
z1u1
z2u1
z9u9
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z(l+1)
= f(u(l+1)
)
softmax
y = softmax(z(L)
)
…
0.01
…
0.02
0.92
0.03

4. u(l+1)
= W(l+1)
z(l)
+ b(l+1)
z(1)
= x
(l = 1, 2, · · · , L 1, L)
x2
x783
x784
x1
x0
z10u10
z1u1
z2u1
z9u9
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z(l+1)
= f(u(l+1)
)
Neural NetworkNeural Network
(1)
= x
W, b
M
1
N
1
N
M
1
N
z(l+1)
= f(u(l+1)
)
y = softmax(z(L)
)
softmaxk(z(L)
) =
exp(u
(L)
k )
PK
j=1 exp(u
(L)
j )

5. f(u) =
1
1 + exp( u)
f(u) = tanh(u)
tanh
ReLu
f(u) = max(0, u)

6. u(l+1)
= W(l+1)
z(l)
+ b(l+1)
z(l+1)
= f(u(l+1)
)
z(1)
= x
x2
x783
x784
x1
x0
z10u10
z1u1
z2u1
z9u9
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
z100
0
u1000
z999u999
u1 z1
u2 z2
z0
Neural Network
(1)
= x
z(l+1)
= f(u(l+1)
)
bW
M
1
N
1
N
M
1
N
W, b
y = softmax(z(L)
)
(l = 1, 2, · · · , L 1, L)
softmaxk(z(L)
) =
exp(u
(L)
k )
PK
j=1 exp(u
(L)
j )
L(w) =
NY
n=1
KY
k=1
(yk(x, w))

8. x =
✓
x1
x2
◆
W(1)
=
w
(1)
11 w
(1)
12
w
(1)
21 w
(1)
22
!
W(2)
=
w
(2)
11 w
(2)
12
w
(2)
21 w
(2)
22
!
b(1)
=
b
(1)
1
b
(1)
2
!
b(2)
=
b
(2)
1
b
(2)
2
!
https://github.com/matsuken92/Qiita_Contents/blob/master/General/mathcafe_20161212.ipynb

9. x =
✓
x1
x2
◆
W(1)
=
w
(1)
11 w
(1)
12
w
(1)
21 w
(1)
22
!
W(2)
=
w
(2)
11 w
(2)
12
w
(2)
21 w
(2)
22
!
b(1)
=
b
(1)
1
b
(1)
2
!
b(2)
=
b
(2)
1
b
(2)
2
!
✔
https://github.com/matsuken92/Qiita_Contents/blob/master/General/mathcafe_20161212.ipynb

10. http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
https://github.com/matsuken92/Qiita_Contents/blob/master/
General/mathcafe_20161212.ipynb
Chainer MNIST
https://github.com/matsuken92/Qiita_Contents/blob/master/
chainer-MNIST/chainer-MNIST_forPubs.ipynb