3. アジェンダ
半教師あり深層学習の最先端の手法を紹介します
Deep Generative Models
– Semi-supervised learning with deep generative models (Kingma et al., 2014)
– Improving Semi-Supervised Learning with Auxiliary Deep Generative Models
(Maaloe et al., 2015)
Virtual Adversarial Training
– Distributional smoothing with virtual adversarial training (Miyato et al., 2015)
Ladder Networks (これをメインに)
– Semi-supervised learning with Ladder network (Rasmus et al., 2015)
– Deconstructing the ladder network architecture (Mohammad et al., 2016)
12. 半教師あり学習の手法
Permutation-invariant MNISTのstate-of-the-art
12
100 labels 60000 (all)
Feed-forward NN 25.8% 1.18%
Deep generative model (M1+M2)
(Kingma et al., 2014)
3.33% 0.96%
Virtual adversarial training
(Miyato et al., 2015)
2.12% 0.64%
Ladder network (Original)
(Rasmus et al., 2015)
1.06% 0.61%
Ladder network (AMLP)
(Mohammad et al., 2016)
1.00% 0.57%
Auxiliary deep generative model
(Maaloe et al., 2015)
0.96% -
①
①
②
③
③
今日話す順番 (半教師手法は全教師でもつよい)
13. 半教師あり学習の手法
Permutation-invariant MNISTのstate-of-the-art
13
100 labels 60000 (all)
Feed-forward NN 25.8% 1.18%
Deep generative model (M1+M2)
(Kingma et al., 2014)
3.33% 0.96%
Virtual adversarial training
(Miyato et al., 2015)
2.12% 0.64%
Ladder network (Original)
(Rasmus et al., 2015)
1.06% 0.61%
Ladder network (AMLP)
(Mohammad et al., 2016)
1.00% 0.57%
Auxiliary deep generative model
(Maaloe et al., 2015)
0.96% -
☆
☆
14. 半教師あり学習の手法 – Deep Generative Models
Deep Generative Modelのアイデア (VAE, AAEなどなど)
データの分布
本当はもっと高次元で複雑
Inference
Generation
狙った形の分布に押し込める
(画像は二次元正規分布)
まだラベルデータは使ってない
http://www.informatik.uni-
bremen.de/~afabisch/files/tsne/tsne_mnist_all.png arXiv:1511.05644
Deep NN
15. 半教師あり学習の手法 – Deep Generative Models
Semi-supervised Deep Generative Modelのアイデア
データの分布
本当はもっと高次元で複雑
Inference
Generation
こんな感じの分布を狙っても良い。
ラベルがあるデータは、
どの羽根に行くかもlossに入れる
http://www.informatik.uni-
bremen.de/~afabisch/files/tsne/tsne_mnist_all.png arXiv:1511.05644
Deep NN
0 1
2
3
16. 半教師あり学習の手法 – Deep Generative Models
Deep Generative Model (M1+M2) (Kingma et al., 2014)
16
Gen.
Inf.
Gen. Inf.
http://approximateinference.org/accepted/MaaloeEtAl2015.pdf
矢印は全部
Deep NN
17. 半教師あり学習の手法 – Deep Generative Models
Auxiliary Deep Generative Model (ADGM) (Maaloe et al., 2015)
17
NIPS2015のワークショップ論文
100 label MNISTで0.97%のerror (現在最高記録)を主張している…
(まだあまり検証されていない)
http://approximateinference.org/accepted/MaaloeEtAl2015.pdf
18. 半教師あり学習の手法
Permutation-invariant MNISTのstate-of-the-art
18
100 labels 60000 (all)
Feed-forward NN 25.8% 1.18%
Deep generative model (M1+M2)
(Kingma et al., 2014)
3.33% 0.96%
Virtual adversarial training
(Miyato et al., 2015)
2.12% 0.64%
Ladder network (Original)
(Rasmus et al., 2015)
1.06% 0.61%
Ladder network (AMLP)
(Mohammad et al., 2016)
1.00% 0.57%
Auxiliary deep generative model
(Maaloe et al., 2015)
0.96% -
☆