The document summarizes a paper presented at NIPS 2016 titled "Supervised Word Mover's Distance" by Huang et al. that extends the Word Mover's Distance (WMD) to incorporate supervision. WMD measures document distance based on word embeddings, and the supervised version improves its ability to capture semantic relationships through supervision in training word embeddings. The document also references related work on word2vec, Earth Mover's Distance, Neighborhood Component Analysis, and Word Mover's Distance.

1. NIPS 2016
Supervised Word Mover's Distance

2. http://qiita.com/tmshn/items/3ccc5d84daa23a98d4be

3. Supervied Word Mover's Distance
G. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha, In NIPS 2016.
‣
https://papers.nips.cc/paper/6139-supervised-word-movers-distance
‣
https://channel9.msdn.com/Events/Neural-Information-Processing-Systems-
Conference/Neural-Information-Processing-Systems-Conference-NIPS-2016/
Supervised-Word-Movers-Distance
‣
https://github.com/gaohuang/S-WMD
‣

4. Supervied Word Mover's Distance
G. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha, In NIPS 2016.
‣
https://papers.nips.cc/paper/6139-supervised-word-movers-distance
‣
https://channel9.msdn.com/Events/Neural-Information-Processing-Systems-
Conference/Neural-Information-Processing-Systems-Conference-NIPS-2016/
Supervised-Word-Movers-Distance
‣
https://github.com/gaohuang/S-WMD
‣

44. [Mikolov et al., '13]
Paris - France + Italy = Rome.
Paris
France
Rome
Italy

48. [Rubner et al., '98]

49. Icon designed by Freepik from Flaticon
http://www.flaticon.com/free-icon/truck-cleaner-snow_96129

58.
T
i
=
db

59.
T da
j

60.
T da
j
i
=
db

61. [Kusner et al., '15]
X ∈ Rd×n
da, db

62. [Kusner et al., '15]
X ∈ Rd×n
da, db

63. [Kusner et al., '15]
X ∈ Rd×n
da, db
L2

64. [Kusner et al., '15]
X ∈ Rd×n
da, db
L2

65. [Kusner et al., '15]
X ∈ Rd×n
da, db
L2

67. O(p3 log p)
O(p)
p da, db
O(dp) d
L2

71. A
dA(x, y) = d(Ax, Ay)
A
A

72. k
[Goldberger et al., '05]

73. k
P(i j)
A1 A2
A
pij
A1 A2
A
A2
A1
i
j
k

77. [Huang et al., '16]
A w
k

78. [Huang et al., '16]
A w
k

79. ‣ w0 = 1

80. ‣ w0 = 1
‣ A0 = WCD NCA

81. ‣ w0 = 1
‣ A0 = WCD NCA
A0

82. ‣ w0 = 1
‣ A0 = WCD NCA
A0
A0, w0

83. k
L2

84. k
L2

85. k
L2
P(a's NN is b) b

86. O(p3 log p) O(p2)

87. O(p3 log p) O(p2)

88. k

89. L2

90. [Huang et al., '16] Supervised Word Mover's Distance
G. Huang, C. Guo, M.J. Kusner, Y. Sun, K.Q. Weinberger, F. Sha.
Supervised Word Mover's Distance. In NIPS 2016.
https://papers.nips.cc/paper/6139-supervised-word-movers-distance
[Mikolov et al., '13] word2vec
T. Mikolov, K. Chen, G. Corrado, J. Dean. Efficient Estimation of Word
Representations in Vector Space. In ICLR 2013 Workshop.
https://arxiv.org/abs/1301.3781
[Rubner et al., '98] Eearth Mover's Distance
Y. Rubner, C. Tomasi, L.J. Guibas. A Metric for Distributions with
Applications to Image Databases. In ICCV 1998.
http://ieeexplore.ieee.org/abstract/document/710701/
http://ai.stanford.edu/~rubner/papers/rubnerIccv98.pdf

91. [Kusner et al., '15] Word Mover's Distance
M.J. Kusner, Y. Sun, N.I. Kolkin, K.Q. Weinberger. From Word
Embedding To Documents Distances. In ICML 2015.
http://www.jmlr.org/proceedings/papers/v37/kusnerb15.html
[Goldberger et al., '05] Neighborhood Component Analysis
J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov. Neighborhood
Component Analysis. In NIPS 2005.
https://papers.nips.cc/paper/2566-neighbourhood-components-analysis

92. Word Mover's Distance
http://yubessy.hatenablog.com/entry/2017/01/10/122737
http://www.slideshare.net/kentonozawa75/from-word-embeddings-
to-document-distances
Earth Mover's Distance
http://aidiary.hatenablog.com/entry/20120804/1344058475