Topic discovery through data dependent and random projections
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
1. Data Noising as Smoothing
in Neural Network Language Models
担当:Quasi_quant2010
ICLR2017読み会 @Dena1
2. Authers Insistence and Reviewer’s Summary
Authers
Data Nosing is data augmentation as another
regularization methods
Reviwers
{Rating_6:2人, Rating_8:1人}
Strengths
data noising as a regularization technique for LM
Weeknesses
Main concern is that the proposed method is too simple
and sounds ad hoc
there are no comparisons between the noising/smoothing
idea and more conventional regularizers
ICLR2017読み会 @Dena2
引用 : goo.gl/ZKRyjr
17. 参考文献
[Ziang Xie, etc] Data Noising as Smoothing in Neural Network
Language Models ICLR17
[M.U.Gutmann, etc] Noise-Contrastive Estimation of Unnormalized
Statistical Models, with Applications to Natural Image Statistics,
JMLR, 13, 307-361, 2012
[M.U.Gutmann, etc] Likelihood-Free Inference via Classification,
Statistics and Computing17
[Mnih, etc] Learning word embeddings efficiently with noise-
contrastive estimation, Advances in Neural Information Processing
Systems 26, NIPS, 2013
[Tianxing He, etc] On Training Bi-directional Neural Network
Language Model with Noise Contrastive Estimation arxiv16
[松林優一郎, etc]ニューラルネットワークによる日本語述語項構造解析の素性の汎
化, 言語処理学会17
[Quasi_quant2010] Noise Contrastive Estimationについて,
goo.gl/c9tw7r
ICLR2017読み会 @Dena17