More Related Content Similar to mxnetで頑張る深層学習 (20) More from Takashi Kitano (15) mxnetで頑張る深層学習7. 多層パーセプトロン
• 学習⽤と検証⽤にデータを分割
> data(iris)
> # 奇数⾏を学習⽤データ、偶数⾏を検証⽤データとする
> train.ind <- seq(1, nrow(iris), 2)
>
> train.x <- data.matrix(iris[train.ind, 1:4])
> train.y <- as.numeric(iris[train.ind, 5]) - 1
> test.x <- data.matrix(iris[-train.ind, 1:4])
> test.y <- as.numeric(iris[-train.ind, 5]) - 1
> table(train.y)
train.y
0 1 2
25 25 25
9. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
10. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
中間層のユニット数
11. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
出⼒層のユニット数
今回は3class分類なので
3を指定
12. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
活性化関数
今回はReLUを指定
13. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
誤差関数
14. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
学習回数
15. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
1回の学習で
使⽤する
学習データ数
16. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
学習率
17. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
モーメンタム
18. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
評価⽅法
回帰の場合はRMSEなど
19. 多層パーセプトロン
• モデル作成
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = 6,
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
学習毎に実⾏する関数
20. 多層パーセプトロン
• モデル作成結果
Start training with 1 devices
[1] Train-accuracy=0.38
[1] Validation-accuracy=0.333333333333333
[2] Train-accuracy=0.333333333333333
[2] Validation-accuracy=0.333333333333333
[3] Train-accuracy=0.333333333333333
[3] Validation-accuracy=0.333333333333333
(中略)
[199] Train-accuracy=0.96
[199] Validation-accuracy=0.96
[200] Train-accuracy=0.986666666666667
[200] Validation-accuracy=0.96
21. 多層パーセプトロン
• 学習毎の精度をプロット
> library(dplyr)
> library(plotly)
> data.frame(epoc = seq(1, 200, 1),
+ train = logger$train, test = logger$eval) %>%
+ plot_ly(
+ x = epoc, y = train,
+ type = "scatter", mode = "lines", name = "train"
+ ) %>%
+ add_trace(
+ x = epoc, y = test,
+ type = "scatter", mode = "lines", name = "test"
+ ) %>%
+ layout(yaxis = list(title = "accuracy"))
23. 多層パーセプトロン
• モデルを検証⽤データに当てはめ
> # 検証⽤データに当てはめ
> preds <- predict(model, test.x)
> head(t(preds))
[,1] [,2] [,3]
[1,] 0.9999290 7.107238e-05 1.109092e-12
[2,] 0.9997237 2.762006e-04 1.440176e-11
[3,] 0.9999521 4.790557e-05 6.202604e-13
[4,] 0.9999722 2.779509e-05 1.976960e-13
[5,] 0.9999521 4.791964e-05 5.282145e-13
[6,] 0.9998796 1.204296e-04 3.145415e-12
24. 多層パーセプトロン
• 混合⾏列/精度の確認
> # 最も確率が⾼い列を予測した品種とする
> pred.label <- max.col(t(preds)) - 1
> # 混合⾏列をプロット
> library(caret)
> confusionMatrix(pred.label, test.y)
Confusion Matrix and Statistics
Reference
Prediction 0 1 2
0 25 0 0
1 0 24 2
2 0 1 23
26. 多層パーセプトロン
• 2層にしてみる
> model <- mx.mlp(
+ train.x, train.y,
+ eval.data = list(data = test.x, label = test.y),
+ hidden_node = c(6, 6),
+ out_node = 3,
+ activation = "relu",
+ out_activation = "softmax",
+ num.round = 200,
+ array.batch.size = 25,
+ learning.rate = 0.07,
+ momentum = 0.9,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(5, logger)
30. 多層パーセプトロン
• オッカムの剃⼑
“Pluralitas non est ponenda sine
neccesitate. Frustra fit per plura
quod potest fieri per pauciora.”
必要が無いなら多くのものを定⽴
してはならない。少数の論理でよい
場合は多数の論理を定⽴してはなら
ない。
40. 畳み込みニューラルネットワーク
• 次元の変換とプロット
> # mx.symbol.Convolutionで処理できる4次元配列に変換
> train.x <- t(train$x)
> dim(train.x) <- c(28, 28, 1, 60000)
>
> test.x <- t(test$x)
> dim(test.x) <- c(28, 28, 1, 10000)
>
> # 1⽂字⽬をプロット
> library(plotly)
> plot_ly(z=train.x[, , 1, 1], colorscale = "Greys", type = "heatmap")
42. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
43. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
フィルタのサイズ
44. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
フィルタの数
45. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
活性化関数
46. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
プーリングのタイプ
47. 畳み込みニューラルネットワーク
• ⼊⼒層とC1, S2の定義
> library(mxnet)
> input <- mx.symbol.Variable('data')
>
> # C1
> c1 <- mx.symbol.Convolution(
+ data = input, kernel = c(5, 5), num_filter = 6)
> a1 <- mx.symbol.Activation(data = c1, act_type = "tanh")
>
> # S2
> s2 <- mx.symbol.Pooling(
+ data = a1, pool_type = "max", kernel = c(2, 2))
フィルタのサイズ
48. 畳み込みニューラルネットワーク
• C3, S4, C5の定義
> # C3
> c3 <- mx.symbol.Convolution(
+ data = s2, kernel = c(5, 5), num_filter = 16)
> a2 <- mx.symbol.Activation(data = c3, act_type = "tanh")
>
> # S4
> s4 <- mx.symbol.Pooling(
+ data = a2, pool_type = "max", kernel = c(2, 2))
>
> # C5
> c5 <- mx.symbol.Flatten(data = s4)
49. 畳み込みニューラルネットワーク
• F6, OUTPUT, 損失関数の定義
> # F6
> f6 <- mx.symbol.FullyConnected(data = c5, num_hidden = 84)
> a3 <- mx.symbol.Activation(data = f6, act_type = "tanh")
>
> # Output
> output <- mx.symbol.FullyConnected(data = a3, num_hidden = 10)
>
> # loss
> lenet <- mx.symbol.SoftmaxOutput(data = output)
50. 畳み込みニューラルネットワーク
• ネットワークモデルの学習
> model <- mx.model.FeedForward.create(
+ lenet,
+ X = train.x,
+ y = train$y,
+ ctx = mx.gpu(),
+ num.round = 20,
+ array.batch.size = 1000,
+ learning.rate = 0.05,
+ momentum = 0.9,
+ wd = 0.00001,
+ eval.metric = mx.metric.accuracy,
+ epoch.end.callback = mx.callback.log.train.metric(100)
+ )
51. 畳み込みニューラルネットワーク
• 学習毎の精度をプロット
> data.frame(epoc = 1:20,
+ train = logger$train,
+ test = logger$eval) %>%
+ plot_ly(
+ x = epoc, y = train,
+ type = "scatter", mode = "lines", name = "train") %>%
+ add_trace(
+ x = epoc, y = test,
+ type = "scatter", mode = "lines", name = "test") %>%
+ layout(yaxis = list(title = "accuracy"))
53. 畳み込みニューラルネットワーク
• 混合⾏列/精度の確認
> confusionMatrix(pred.label, test$y)
Confusion Matrix and Statistics
Reference
Prediction 0 1 2 3 4 5 6 7 8 9
0 970 0 1 0 0 2 5 0 5 3
1 0 1126 2 0 0 0 2 1 0 2
2 3 1 1020 1 1 0 1 4 3 0
3 0 1 0 997 0 7 1 2 3 4
4 1 0 3 0 971 0 2 0 2 13
5 0 0 0 5 0 876 2 0 2 2
6 2 2 1 0 2 4 943 0 1 0
7 1 2 4 5 3 1 0 1019 6 10
58. •MXNet R Package - mxnet 0.7.0
documentation
•Deep Learningライブラリ{mxnet}のR版で
Convolutional Neural Networkをサクッと
試してみた(追記3件あり)
•Deep Learningライブラリ「MXNet」のR版
をKaggle Otto Challengeで実践してみた
•Mxnetで回帰 #TokyoR 53th