SlideShare a Scribd company logo

Semi-Supervised Classification with Graph Convolutional Networks @ICLR2017読み会

•
•
Eiji Sekiya
Eiji Sekiya

ICLR2017@DeNA（https://connpass.com/event/57631/）の論文紹介資料

Data & Analytics
1 of 27
Download to read offline
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. June 17, 2017 ICLR @DeNA Eiji Sekiya AI Research and Development Gr. AI System Dept. 
 DeNA Co., Ltd. 1 h"ps://openreview.net/pdf?id=SJU4ayYgl
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • • (@eratostennis) • • Master • DeNA (2014) • : (~2016) • Hadoop, Vertica • : (2016~) • • GameAI 2
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Abstract • Semi-supervised learning on graph-structured data • Localized first-order approximation of spectral graph • : • • 3
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Agenda • Introduction • Related Work • Fast Approximate Convolutions on Graphs • Semi-Supervised Node Classification • Experiments • Results • Conclusion 4
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Relation to Weisfeiler-Lehman Algorithm • WL-1 • Hash Neural Network 5 hash
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Node Embeddings with Random Weights 6 output 2
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Semi-Supervised Node Embeddings • 4 1 7 : h"p://tkipf.github.io/graph-convoluDonal-networks/
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work • Graph-Based Semi-Supervised Learning • Graph Laplacian regularization • Label Propagation (Zhu et al., 2003) • Manifold Regularization (Belkin et al., 2006) • Graph embedding-based • DeepWalk (Perozzi et al., 2014) • Planetoid (Yang et al., 2016) 8
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work (DeepWalk: Perozzi et al., 2014) • • SkipGram embedding • Random Walk 9
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work • Graph-Based Semi-Supervised Learning • Graph Laplacian regularization • Label Propagation (Zhu et al., 2003) • Manifold Regularization (Belkin et al., 2006) • Graph embedding-based • DeepWalk (Perozzi et al., 2014) • Planetoid (Yang et al., 2016) • Neural Networks on Graphs ( ) • Spectral graph convolutional neural network (Bruna et al., 2014) • Fast localized convolution (Defferrard et al., 2016) 10
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Graph-Based Semi-Supervised Learning • (citation network ) (documents) • • • • encode • 11 Neural Networks on Graph
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Multi-layer Graph Convolutional Network (GCN) • • • • 12 Contribution
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Spectral Graph Convolutions • Convolution with filter • : • • …(Hammond at el., 2011) • • ( ) 13
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Spectral Graph Convolutions (Defferrad at el., 2016) • Graph Convolution ( , ) • K K • : • 14
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Layer-Wise Linear Model (Contribution) • K=1 … • k k • • Renormalization Trick ( ) • Input C, F … • 15
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Semi-Supervised Node Classification • Forward Model ( ) • Cross-Entropy: 16
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Dataset) • (Citeseer, Cora, Pubmed) • Sparse bag-of-words feature vectors for each document • a list of citation links between documents • (NELL) • A set of entities connected with directed, labeled edges • Random Graph • A dataset with N nodes we create a random graph assigning 2N edges uniformly at random 17
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Setup) • • 2 GCN • • Prediction Accuracy • 1000 • • 500 • Dropout rate, L2 regularization factor, Number of hidden units • Cora Citeseer Pubmed 18
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Baseline) • Baseline • Label Propagation (LP) • Semi-supervised embedding (SemiEmb) • Manifold regularization (ManiReg) • Skip-gram based graph embeddings (DeepWalk) • Iterative classification algorithm (ICA) • Planetoid 19
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Semi-Supervised Node Classification) • Results for all other baseline methods are taken from the Planetoid paper • Environment • The same hardware as planetoid 20 Training Dme in seconds unDl convergence
• Compare different variants of our proposed per-layer propagation model on the citation network datasets Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Evaluation Propagation Model) 21
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Training Time per Epoch) • The mean training time per epoch for 100 epochs on simulated random graphs 22
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments on Model Depth • Residual connections • 2, 3 • 7 … • residual connections • 23
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Discussion • Keypoints • Overcome both limitations • Graph-Laplacian regularization • Edges encode mere similarity of nodes • Skip-gram based methods • Limited by the fact that they are based on a multi-step pipeline which difficult to optimize • Findings • Propagation from neighboring nodes improves classification performance • Proposed renormalized propagation model offers both improved efficiency 24
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Future Work • Memory Requirement • Full-batch gradient descent • Grows linearly in the size of the dataset • Mini-batch stochastic gradient descent can alleviate this issue • Directed edges and edge features • Limited to undirected graph • NELL show it is possible to handle • By representing the original directed graph as an undirected bipartite graph • Limiting assumptions • Locality • dependence on the Kth-order neighborhood for a GCN with K layers • Equal importance of self-connections vs. edges to neighboring nodes 25
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Reference • Graph Semi-Supervised Learning • Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In International Conference on Machine Learning (ICML), volume 3, pp. 912–919, 2003. • Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric frame- work for learning from labeled and unlabeled examples. Journal of machine learning research (JMLR), 7(Nov):2399–2434, 2006. • JasonWeston,Fre ́de ́ricRatle,HosseinMobahi,andRonanCollobert. Deeplearningviasemi- supervised embedding. In Neural Networks: Tricks of the Trade, pp. 639–655. Springer, 2012. • Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Advances in neural information processing systems (NIPS), pp. 3111–3119, 2013. • Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social repre- sentations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–710. ACM, 2014. • Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning (ICML), 2016. 26
Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Reference • Dataset & Pre-Processing • Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008. • Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr, and Tom M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, volume 5, pp. 3, 2010. • Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning (ICML), 2016. • Website • http://tkipf.github.io/graph-convolutional-networks/ • Implementation • https://github.com/tkipf/gcn 27

Recommended

[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks
[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks
[DL Hacks]Semi-Supervised Classification with Graph Convolutional NetworksDeep Learning JP
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Preferred Networks
 
Graph Convolutional Neural Networks
Graph Convolutional Neural Networks Graph Convolutional Neural Networks
Graph Convolutional Neural Networks 신동 강
 
High Dimensional Data Visualization using t-SNE
High Dimensional Data Visualization using t-SNEHigh Dimensional Data Visualization using t-SNE
High Dimensional Data Visualization using t-SNEKai-Wen Zhao
 
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Convolutional Neural Networks on Graphs with Fast Localized Spectral FilteringConvolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Convolutional Neural Networks on Graphs with Fast Localized Spectral FilteringSOYEON KIM
 
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習台灣資料科學年會
 
Visualizing Data Using t-SNE
Visualizing Data Using t-SNEVisualizing Data Using t-SNE
Visualizing Data Using t-SNEDavid Khosid
 
Graph Convolutional Network 概説
Graph Convolutional Network 概説Graph Convolutional Network 概説
Graph Convolutional Network 概説KCS Keio Computer Society
 

Recommended

[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks
[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks[DL Hacks]Semi-Supervised Classification with Graph Convolutional Networks
[DL Hacks]Semi-Supervised Classification with Graph Convolutional NetworksDeep Learning JP
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...Preferred Networks
 
Graph Convolutional Neural Networks
Graph Convolutional Neural Networks Graph Convolutional Neural Networks
Graph Convolutional Neural Networks 신동 강
 
High Dimensional Data Visualization using t-SNE
High Dimensional Data Visualization using t-SNEHigh Dimensional Data Visualization using t-SNE
High Dimensional Data Visualization using t-SNEKai-Wen Zhao
 
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Convolutional Neural Networks on Graphs with Fast Localized Spectral FilteringConvolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Convolutional Neural Networks on Graphs with Fast Localized Spectral FilteringSOYEON KIM
 
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動：李宏毅 / 一天搞懂深度學習台灣資料科學年會
 
Visualizing Data Using t-SNE
Visualizing Data Using t-SNEVisualizing Data Using t-SNE
Visualizing Data Using t-SNEDavid Khosid
 
Graph Convolutional Network 概説
Graph Convolutional Network 概説Graph Convolutional Network 概説
Graph Convolutional Network 概説KCS Keio Computer Society
 
Introduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetupIntroduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetupLiad Magen
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNNChanuk Lim
 
Convolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in TheanoConvolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in TheanoSeongwon Hwang
 
Graph neural networks overview
Graph neural networks overviewGraph neural networks overview
Graph neural networks overviewRodion Kiryukhin
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsShunta Saito
 
MIRU2014 tutorial deeplearning
MIRU2014 tutorial deeplearningMIRU2014 tutorial deeplearning
MIRU2014 tutorial deeplearningTakayoshi Yamashita
 
距離とクラスタリング
距離とクラスタリング距離とクラスタリング
距離とクラスタリング大貴 末廣
 
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−Deep Learning JP
 
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative ModelDeep Learning JP
 
Python基礎その２
Python基礎その２Python基礎その２
Python基礎その２大貴 末廣
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Graph Neural Network - Introduction
Graph Neural Network - IntroductionGraph Neural Network - Introduction
Graph Neural Network - IntroductionJungwon Kim
 
画像処理応用
画像処理応用画像処理応用
画像処理応用大貴 末廣
 
TabNetの論文紹介
TabNetの論文紹介TabNetの論文紹介
TabNetの論文紹介西岡 賢一郎
 
Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)yukihiro domae
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 
深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎Takumi Ohkuma
 
Machine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowMachine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowDataWorks Summit/Hadoop Summit
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN TutorialSungjoon Choi
 
SIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLSIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLMark Kilgard
 
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-Takahiro Kubo
 
医療データ解析界隈から見たICLR2017
医療データ解析界隈から見たICLR2017医療データ解析界隈から見たICLR2017
医療データ解析界隈から見たICLR2017RIKEN, Medical Sciences Innovation Hub Program (MIH)
 

More Related Content

What's hot

Introduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetupIntroduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetupLiad Magen
 
Convolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in TheanoConvolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in TheanoSeongwon Hwang
 
Graph neural networks overview
Graph neural networks overviewGraph neural networks overview
Graph neural networks overviewRodion Kiryukhin
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsShunta Saito
 
距離とクラスタリング
距離とクラスタリング距離とクラスタリング
距離とクラスタリング大貴 末廣
 
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−Deep Learning JP
 
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative ModelDeep Learning JP
 
Python基礎その２
Python基礎その２Python基礎その２
Python基礎その２大貴 末廣
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Graph Neural Network - Introduction
Graph Neural Network - IntroductionGraph Neural Network - Introduction
Graph Neural Network - IntroductionJungwon Kim
 
Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)yukihiro domae
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 
深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎Takumi Ohkuma
 
SIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLSIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLMark Kilgard
 

What's hot (20)

Introduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetupIntroduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetup
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
Convolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in TheanoConvolutional Neural Network (CNN) presentation from theory to code in Theano
Convolutional Neural Network (CNN) presentation from theory to code in Theano
 
Graph neural networks overview
Graph neural networks overviewGraph neural networks overview
Graph neural networks overview
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methods
 
MIRU2014 tutorial deeplearning
MIRU2014 tutorial deeplearningMIRU2014 tutorial deeplearning
MIRU2014 tutorial deeplearning
 
距離とクラスタリング
距離とクラスタリング距離とクラスタリング
距離とクラスタリング
 
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
[DL輪読会]The Neural Process Family−Neural Processes関連の実装を読んで動かしてみる−
 
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
【DL輪読会】NeRF-VAE: A Geometry Aware 3D Scene Generative Model
 
Python基礎その２
Python基礎その２Python基礎その２
Python基礎その２
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
Graph Neural Network - Introduction
Graph Neural Network - IntroductionGraph Neural Network - Introduction
Graph Neural Network - Introduction
 
画像処理応用
画像処理応用画像処理応用
画像処理応用
 
TabNetの論文紹介
TabNetの論文紹介TabNetの論文紹介
TabNetの論文紹介
 
Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)Graph convolution (スペクトルアプローチ)
Graph convolution (スペクトルアプローチ)
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
 
深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎深層学習によるHuman Pose Estimationの基礎
深層学習によるHuman Pose Estimationの基礎
 
Machine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlowMachine Intelligence at Google Scale: TensorFlow
Machine Intelligence at Google Scale: TensorFlow
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN Tutorial
 
SIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLSIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGL
 

Viewers also liked

言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-Takahiro Kubo
 
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @DenaICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @DenaTakanori Nakai
 
ICLR読み会 奥村純 20170617
ICLR読み会 奥村純 20170617ICLR読み会 奥村純 20170617
ICLR読み会 奥村純 20170617Jun Okumura
 
[ICLR2017読み会 @ DeNA] ICLR2017紹介
[ICLR2017読み会 @ DeNA] ICLR2017紹介[ICLR2017読み会 @ DeNA] ICLR2017紹介
[ICLR2017読み会 @ DeNA] ICLR2017紹介Takeru Miyato
 
エフェクト用 Shader 機能紹介
エフェクト用 Shader 機能紹介エフェクト用 Shader 機能紹介
エフェクト用 Shader 機能紹介Hajime Sanno
 
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10エンジニアがデザインやってみた @ Aimning MeetUp 2017/10
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10Hidenori Doi
 
エフェクトにしっかり色を付ける方法
エフェクトにしっかり色を付ける方法エフェクトにしっかり色を付ける方法
エフェクトにしっかり色を付ける方法kmasaki
 
当たり前を当たり前に：Agile2017レポート
当たり前を当たり前に：Agile2017レポート当たり前を当たり前に：Agile2017レポート
当たり前を当たり前に：Agile2017レポートLINE Corporation
 
共変戻り値型って知ってますか?
共変戻り値型って知ってますか?共変戻り値型って知ってますか?
共変戻り値型って知ってますか?Ryota Murohoshi
 
Node and Micro-Services at IBM
Node and Micro-Services at IBMNode and Micro-Services at IBM
Node and Micro-Services at IBMDejan Glozic
 
Effective web performance tuning for smartphone
Effective web performance tuning for smartphoneEffective web performance tuning for smartphone
Effective web performance tuning for smartphonedena_study
 
RのffでGLMしてみたけど...
RのffでGLMしてみたけど...RのffでGLMしてみたけど...
RのffでGLMしてみたけど...Kazuya Wada
 
Googleのインフラ技術から考える理想のDevOps
Googleのインフラ技術から考える理想のDevOpsGoogleのインフラ技術から考える理想のDevOps
Googleのインフラ技術から考える理想のDevOpsEtsuji Nakai
 
ディープボルツマンマシン入門
ディープボルツマンマシン入門ディープボルツマンマシン入門
ディープボルツマンマシン入門Saya Katafuchi
 
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2Hiroyuki Kato
 
FINAL FANTASY Record Keeper の作り方
FINAL FANTASY Record Keeper の作り方FINAL FANTASY Record Keeper の作り方
FINAL FANTASY Record Keeper の作り方dena_study
 

Viewers also liked (20)

言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
言葉のもつ広がりを、モデルの学習に活かそう -one-hot to distribution in language modeling-
 
医療データ解析界隈から見たICLR2017
医療データ解析界隈から見たICLR2017医療データ解析界隈から見たICLR2017
医療データ解析界隈から見たICLR2017
 
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @DenaICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
ICLR2017読み会 Data Noising as Smoothing in Neural Network Language Models @Dena
 
170614 iclr reading-public
170614 iclr reading-public170614 iclr reading-public
170614 iclr reading-public
 
Q prop
Q propQ prop
Q prop
 
ICLR読み会 奥村純 20170617
ICLR読み会 奥村純 20170617ICLR読み会 奥村純 20170617
ICLR読み会 奥村純 20170617
 
[ICLR2017読み会 @ DeNA] ICLR2017紹介
[ICLR2017読み会 @ DeNA] ICLR2017紹介[ICLR2017読み会 @ DeNA] ICLR2017紹介
[ICLR2017読み会 @ DeNA] ICLR2017紹介
 
エフェクト用 Shader 機能紹介
エフェクト用 Shader 機能紹介エフェクト用 Shader 機能紹介
エフェクト用 Shader 機能紹介
 
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10エンジニアがデザインやってみた @ Aimning MeetUp 2017/10
エンジニアがデザインやってみた @ Aimning MeetUp 2017/10
 
エフェクトにしっかり色を付ける方法
エフェクトにしっかり色を付ける方法エフェクトにしっかり色を付ける方法
エフェクトにしっかり色を付ける方法
 
当たり前を当たり前に：Agile2017レポート
当たり前を当たり前に：Agile2017レポート当たり前を当たり前に：Agile2017レポート
当たり前を当たり前に：Agile2017レポート
 
共変戻り値型って知ってますか?
共変戻り値型って知ってますか?共変戻り値型って知ってますか?
共変戻り値型って知ってますか?
 
Node and Micro-Services at IBM
Node and Micro-Services at IBMNode and Micro-Services at IBM
Node and Micro-Services at IBM
 
Effective web performance tuning for smartphone
Effective web performance tuning for smartphoneEffective web performance tuning for smartphone
Effective web performance tuning for smartphone
 
RのffでGLMしてみたけど...
RのffでGLMしてみたけど...RのffでGLMしてみたけど...
RのffでGLMしてみたけど...
 
Googleのインフラ技術から考える理想のDevOps
Googleのインフラ技術から考える理想のDevOpsGoogleのインフラ技術から考える理想のDevOps
Googleのインフラ技術から考える理想のDevOps
 
ディープボルツマンマシン入門
ディープボルツマンマシン入門ディープボルツマンマシン入門
ディープボルツマンマシン入門
 
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2
PRML上巻勉強会 at 東京大学 資料 第4章4.3.1 〜 4.5.2
 
FINAL FANTASY Record Keeper の作り方
FINAL FANTASY Record Keeper の作り方FINAL FANTASY Record Keeper の作り方
FINAL FANTASY Record Keeper の作り方
 
Prml 4.3
Prml 4.3Prml 4.3
Prml 4.3
 

Similar to Semi-Supervised Classification with Graph Convolutional Networks @ICLR2017読み会

20191107 deeplearningapproachesfornetworks
20191107 deeplearningapproachesfornetworks20191107 deeplearningapproachesfornetworks
20191107 deeplearningapproachesfornetworkstm1966
 
Rack Cluster Deployment for SDSC Supercomputer
Rack Cluster Deployment for SDSC SupercomputerRack Cluster Deployment for SDSC Supercomputer
Rack Cluster Deployment for SDSC SupercomputerRebekah Rodriguez
 
High-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and ModelingHigh-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and ModelingNesreen K. Ahmed
 
PEARC17:A real-time machine learning and visualization framework for scientif...
PEARC17:A real-time machine learning and visualization framework for scientif...PEARC17:A real-time machine learning and visualization framework for scientif...
PEARC17:A real-time machine learning and visualization framework for scientif...Feng Li
 
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習 Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習 Herman Wu
 
Ling liu part 02：big graph processing
Ling liu part 02：big graph processingLing liu part 02：big graph processing
Ling liu part 02：big graph processingjins0618
 
Grid is Dead ? Nimrod on the Cloud
Grid is Dead ? Nimrod on the CloudGrid is Dead ? Nimrod on the Cloud
Grid is Dead ? Nimrod on the CloudAdianto Wibisono
 
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facilityinside-BigData.com
 
Deep Learning and Automatic Differentiation from Theano to PyTorch
Deep Learning and Automatic Differentiation from Theano to PyTorchDeep Learning and Automatic Differentiation from Theano to PyTorch
Deep Learning and Automatic Differentiation from Theano to PyTorchinside-BigData.com
 
Detection of Related Semantic Datasets Based on Frequent Subgraph Mining
Detection of Related Semantic Datasets Based on Frequent Subgraph MiningDetection of Related Semantic Datasets Based on Frequent Subgraph Mining
Detection of Related Semantic Datasets Based on Frequent Subgraph MiningMikel Emaldi Manrique
 
The Analytics Frontier of the Hadoop Eco-System
The Analytics Frontier of the Hadoop Eco-SystemThe Analytics Frontier of the Hadoop Eco-System
The Analytics Frontier of the Hadoop Eco-Systeminside-BigData.com
 
Deep Learning and What's Next?
Deep Learning and What's Next?Deep Learning and What's Next?
Deep Learning and What's Next?Tao Wang
 
Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009Ian Foster
 
Xiaoxin_Resume_combined_NCL
Xiaoxin_Resume_combined_NCLXiaoxin_Resume_combined_NCL
Xiaoxin_Resume_combined_NCLXiaoxin Ren
 
g-Social - Enhancing e-Science Tools with Social Networking Functionality
g-Social - Enhancing e-Science Tools with Social Networking Functionalityg-Social - Enhancing e-Science Tools with Social Networking Functionality
g-Social - Enhancing e-Science Tools with Social Networking FunctionalityNicholas Loulloudes
 
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...MLconf
 

Similar to Semi-Supervised Classification with Graph Convolutional Networks @ICLR2017読み会 (20)

20191107 deeplearningapproachesfornetworks
20191107 deeplearningapproachesfornetworks20191107 deeplearningapproachesfornetworks
20191107 deeplearningapproachesfornetworks
 
Rack Cluster Deployment for SDSC Supercomputer
Rack Cluster Deployment for SDSC SupercomputerRack Cluster Deployment for SDSC Supercomputer
Rack Cluster Deployment for SDSC Supercomputer
 
High-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and ModelingHigh-Performance Graph Analysis and Modeling
High-Performance Graph Analysis and Modeling
 
PEARC17:A real-time machine learning and visualization framework for scientif...
PEARC17:A real-time machine learning and visualization framework for scientif...PEARC17:A real-time machine learning and visualization framework for scientif...
PEARC17:A real-time machine learning and visualization framework for scientif...
 
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習 Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
 
Big data analytics_7_giants_public_24_sep_2013
Big data analytics_7_giants_public_24_sep_2013Big data analytics_7_giants_public_24_sep_2013
Big data analytics_7_giants_public_24_sep_2013
 
Spark
SparkSpark
Spark
 
Ling liu part 02：big graph processing
Ling liu part 02：big graph processingLing liu part 02：big graph processing
Ling liu part 02：big graph processing
 
Grid is Dead ? Nimrod on the Cloud
Grid is Dead ? Nimrod on the CloudGrid is Dead ? Nimrod on the Cloud
Grid is Dead ? Nimrod on the Cloud
 
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
 
Deep Learning and Automatic Differentiation from Theano to PyTorch
Deep Learning and Automatic Differentiation from Theano to PyTorchDeep Learning and Automatic Differentiation from Theano to PyTorch
Deep Learning and Automatic Differentiation from Theano to PyTorch
 
Detection of Related Semantic Datasets Based on Frequent Subgraph Mining
Detection of Related Semantic Datasets Based on Frequent Subgraph MiningDetection of Related Semantic Datasets Based on Frequent Subgraph Mining
Detection of Related Semantic Datasets Based on Frequent Subgraph Mining
 
Chug dl presentation
Chug dl presentationChug dl presentation
Chug dl presentation
 
The Analytics Frontier of the Hadoop Eco-System
The Analytics Frontier of the Hadoop Eco-SystemThe Analytics Frontier of the Hadoop Eco-System
The Analytics Frontier of the Hadoop Eco-System
 
Deeplearning in finance
Deeplearning in financeDeeplearning in finance
Deeplearning in finance
 
Deep Learning and What's Next?
Deep Learning and What's Next?Deep Learning and What's Next?
Deep Learning and What's Next?
 
Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009
 
Xiaoxin_Resume_combined_NCL
Xiaoxin_Resume_combined_NCLXiaoxin_Resume_combined_NCL
Xiaoxin_Resume_combined_NCL
 
g-Social - Enhancing e-Science Tools with Social Networking Functionality
g-Social - Enhancing e-Science Tools with Social Networking Functionalityg-Social - Enhancing e-Science Tools with Social Networking Functionality
g-Social - Enhancing e-Science Tools with Social Networking Functionality
 
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
 

Recently uploaded

Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queensdataanalyticsqueen03
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanMYRABACSAFRA2
 
RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.natarajan8993
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdfHuman37
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxMike Bennett
 
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一fhwihughh
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...limedy534
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFAAndrei Kaleshka
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceSapana Sha
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 217djon017
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Seán Kennedy
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsVICTOR MAESTRE RAMIREZ
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectBoston Institute of Analytics
 
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfRadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfgstagge
 
Multiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfMultiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfchwongval
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfJohn Sterrett
 
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...ssuserf63bd7
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...Boston Institute of Analytics
 

Recently uploaded (20)

Top 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In QueensTop 5 Best Data Analytics Courses In Queens
Top 5 Best Data Analytics Courses In Queens
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population Mean
 
RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptx
 
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFA
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts Service
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business Professionals
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis Project
 
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdfRadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdf
 
Multiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdfMultiple time frame trading analysis -brianshannon.pdf
Multiple time frame trading analysis -brianshannon.pdf
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdf
 
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
Statistics, Data Analysis, and Decision Modeling, 5th edition by James R. Eva...
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
 

Semi-Supervised Classification with Graph Convolutional Networks @ICLR2017読み会

  • 1. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. June 17, 2017 ICLR @DeNA Eiji Sekiya AI Research and Development Gr. AI System Dept. 
 DeNA Co., Ltd. 1 h"ps://openreview.net/pdf?id=SJU4ayYgl
  • 2. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • • (@eratostennis) • • Master • DeNA (2014) • : (~2016) • Hadoop, Vertica • : (2016~) • • GameAI 2
  • 3. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Abstract • Semi-supervised learning on graph-structured data • Localized first-order approximation of spectral graph • : • • 3
  • 4. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Agenda • Introduction • Related Work • Fast Approximate Convolutions on Graphs • Semi-Supervised Node Classification • Experiments • Results • Conclusion 4
  • 5. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Relation to Weisfeiler-Lehman Algorithm • WL-1 • Hash Neural Network 5 hash
  • 6. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Node Embeddings with Random Weights 6 output 2
  • 7. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Semi-Supervised Node Embeddings • 4 1 7 : h"p://tkipf.github.io/graph-convoluDonal-networks/
  • 8. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work • Graph-Based Semi-Supervised Learning • Graph Laplacian regularization • Label Propagation (Zhu et al., 2003) • Manifold Regularization (Belkin et al., 2006) • Graph embedding-based • DeepWalk (Perozzi et al., 2014) • Planetoid (Yang et al., 2016) 8
  • 9. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work (DeepWalk: Perozzi et al., 2014) • • SkipGram embedding • Random Walk 9
  • 10. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Related Work • Graph-Based Semi-Supervised Learning • Graph Laplacian regularization • Label Propagation (Zhu et al., 2003) • Manifold Regularization (Belkin et al., 2006) • Graph embedding-based • DeepWalk (Perozzi et al., 2014) • Planetoid (Yang et al., 2016) • Neural Networks on Graphs ( ) • Spectral graph convolutional neural network (Bruna et al., 2014) • Fast localized convolution (Defferrard et al., 2016) 10
  • 11. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Graph-Based Semi-Supervised Learning • (citation network ) (documents) • • • • encode • 11 Neural Networks on Graph
  • 12. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Multi-layer Graph Convolutional Network (GCN) • • • • 12 Contribution
  • 13. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Spectral Graph Convolutions • Convolution with filter • : • • …(Hammond at el., 2011) • • ( ) 13
  • 14. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Spectral Graph Convolutions (Defferrad at el., 2016) • Graph Convolution ( , ) • K K • : • 14
  • 15. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Layer-Wise Linear Model (Contribution) • K=1 … • k k • • Renormalization Trick ( ) • Input C, F … • 15
  • 16. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Semi-Supervised Node Classification • Forward Model ( ) • Cross-Entropy: 16
  • 17. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Dataset) • (Citeseer, Cora, Pubmed) • Sparse bag-of-words feature vectors for each document • a list of citation links between documents • (NELL) • A set of entities connected with directed, labeled edges • Random Graph • A dataset with N nodes we create a random graph assigning 2N edges uniformly at random 17
  • 18. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Setup) • • 2 GCN • • Prediction Accuracy • 1000 • • 500 • Dropout rate, L2 regularization factor, Number of hidden units • Cora Citeseer Pubmed 18
  • 19. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments (Baseline) • Baseline • Label Propagation (LP) • Semi-supervised embedding (SemiEmb) • Manifold regularization (ManiReg) • Skip-gram based graph embeddings (DeepWalk) • Iterative classification algorithm (ICA) • Planetoid 19
  • 20. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Semi-Supervised Node Classification) • Results for all other baseline methods are taken from the Planetoid paper • Environment • The same hardware as planetoid 20 Training Dme in seconds unDl convergence
  • 21. • Compare different variants of our proposed per-layer propagation model on the citation network datasets Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Evaluation Propagation Model) 21
  • 22. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Results (Training Time per Epoch) • The mean training time per epoch for 100 epochs on simulated random graphs 22
  • 23. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Experiments on Model Depth • Residual connections • 2, 3 • 7 … • residual connections • 23
  • 24. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Discussion • Keypoints • Overcome both limitations • Graph-Laplacian regularization • Edges encode mere similarity of nodes • Skip-gram based methods • Limited by the fact that they are based on a multi-step pipeline which difficult to optimize • Findings • Propagation from neighboring nodes improves classification performance • Proposed renormalized propagation model offers both improved efficiency 24
  • 25. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Future Work • Memory Requirement • Full-batch gradient descent • Grows linearly in the size of the dataset • Mini-batch stochastic gradient descent can alleviate this issue • Directed edges and edge features • Limited to undirected graph • NELL show it is possible to handle • By representing the original directed graph as an undirected bipartite graph • Limiting assumptions • Locality • dependence on the Kth-order neighborhood for a GCN with K layers • Equal importance of self-connections vs. edges to neighboring nodes 25
  • 26. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Reference • Graph Semi-Supervised Learning • Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In International Conference on Machine Learning (ICML), volume 3, pp. 912–919, 2003. • Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric frame- work for learning from labeled and unlabeled examples. Journal of machine learning research (JMLR), 7(Nov):2399–2434, 2006. • JasonWeston,Fre ́de ́ricRatle,HosseinMobahi,andRonanCollobert. Deeplearningviasemi- supervised embedding. In Neural Networks: Tricks of the Trade, pp. 639–655. Springer, 2012. • Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Advances in neural information processing systems (NIPS), pp. 3111–3119, 2013. • Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social repre- sentations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–710. ACM, 2014. • Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning (ICML), 2016. 26
  • 27. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Reference • Dataset & Pre-Processing • Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008. • Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R. Hruschka Jr, and Tom M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, volume 5, pp. 3, 2010. • Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning (ICML), 2016. • Website • http://tkipf.github.io/graph-convolutional-networks/ • Implementation • https://github.com/tkipf/gcn 27