SlideShare a Scribd company logo
1 of 30
Graph Neural Network
2019.03.24
Jungwon Kim
시작하기 전...
● GCN(Graph Convolutional Network)에 대한 이야기가 아닙니다
○ 추후에 볼 예정…
● GNN의 기본 컨셉에 대해서만 다룹니다
Graph Neural Network란?
● Graph 데이터 구조를 위한 Neural Network
○ 입력: Graph
○ 출력: Label
■ 숫자(0.95, 0.81, 0.4, …)
■ 라벨링 (Protein, Carbon-dioxide, etc.)
■ 분류 (Drug/Not Drug, etc.)
왜 Graph Neural Network?
● 다양한 관계에 대한 이해를 위한 구조
○ RNN의 경우 Cell간에 Recursive한 관계를 가짐
● 네트워크를 표현하기 위함
○ 자기 자신에 대한 Connection
○ 어떤것들은 연결이 아예 존재하지 않는 경우도 있음
Euclidean Space (유클리드 공간)
원점에서 좌표 (x, y, z)까지의 거리
= 원점과 좌표 사이의 가장 짧은 직선의 길이
Non-Euclidean Space (비 유클리드 공간)
파랑부분의 처음 지점 ~ 붉은부분의 끝지점
● Euclidean 공간에서는 거리가 계속 변함
○ 거리가 계속 좁혀지는 형태
● Non-Euclidean 공간에서는 거리가 변하지
않음
https://deepmind.com/blog/alphafold/
Euclidean Space vs Non-Euclidean Space
When to use GNN?
● 화학
○ 마약인가/아닌가? (확률을 통해 산출)
● Program 이해
○ Expression Statement
○ 프로그래밍을 통한 해결
○ RNN 모델보다 성능이 더 좋음
● Etc.
For example...
For example...
RNN (Recurrent Neural Network)
● RNN에서는 각 Cell이 이전 Cell에서부터 전달되는 형식으로 정보가 누적된다
○ 이러한 연속된 형태의 예시로는...
■ Time Series
■ 단어 순서
■ 사건의 시퀀스
■ Etc.
RNN (Recurrent Neural Network)
GNN (Graph Neural Network)
GNN (Graph Neural Network)
GNN (Graph Neural Network)
Adjacency = 1
GNN (Graph Neural Network)
Adjacency Matrix
● 연결 강도로 시각화
Adjacency Matrix
DeepMind: AlphaFold (https://deepmind.com/blog/alphafold/#image-33420)
GCN (Graph Convolutional Network)
● (l): Layer
● H: Input from Layer (l)
● W: Weights (Trainable)
● A: Adjacency Matrix
● D: Degree Matrix
○ Number of connections
attached to each node
● Hat notations
○ Normalized with
standard Identity matrix
● Activation Function: Sigmoid
GCN (Graph Convolutional Network)
GNN (Graph Neural Network)
● RNN
○ 현재 Cell에 이전 Cell의 정보를 전달하는 방식으로 propagate
● GNN
○ ‘Adjacency’에 대해 고려
■ Adjacency = 1에 대한 정보
■ Adjacency = 2에 대한 정보
■ Adjacency = 3에 대한 정보
■ …
■ Adjacency = n(최대 갯수)에 대한 정보 까지
GNN Variations
● Graph Neural Network (2005)
● Spectral Networks (2014)
● Neural Message Passing (2017)
○ Gated Graph Neural Network (2016)
○ ChebyNets (2016)
○ Graph Convolution Network (2017)
● Async. Neural Message Passing (2018)
● AMPNet (2018)
● Programs As Graphs (2018)
GNN Variations
● Graph AutoEncoder (2016)
● GraphGAN (2017)
● GraphSAGE (2017)
● Graph Attention Networks (ICRL 2018)
● GraphRNN (2018)
● Splitter (2019)
Types of GNN
Datasets for GNN
https://github.com/shiruipan/graph_datasets
** Not listed in link, but ‘Zachary’s karate
club’ is a commonly used social network. **
(https://towardsdatascience.com/how-to-do-
deep-learning-on-graphs-with-graph-
convolutional-networks-7d2250723780)
Graph Nets Library (DeepMind)
https://github.com/deepmind/graph_nets
● DeepMind에서 제작한 프레임워크
● Doc이 상세하게 기술되어 있음
● Tensorflow-based
DGL (Deep Graph Library)
https://www.dgl.ai/
● 최근 개발됨 (2018)
● NYU, NYU-상하이, Amazon
에서 개발
PyTorch Geometric
https://github.com/rusty1s/pytorch_geometric
● PyTorch Extension
○ 기본 설치 X
● DGL보다 최대 15배 가량 빠
르다는 정보가 있음
Geometric Deep Learning
http://geometricdeeplearning.com/
● Not a Framework
● Many references for GNN
(Not a lot of descriptions)
References
Microsoft - Graph Neural Networks: Variations and Applications
(https://www.youtube.com/watch?v=cWIeTMklzNg)
Graph Theory - Adjacency Matrices
(https://www.ebi.ac.uk/training/online/course/network-analysis-protein-interaction-data-
introduction/introduction-graph-theory/graph-0)
DeepMind - AlphaFold
(https://deepmind.com/blog/alphafold/)

More Related Content

What's hot

Webinar on Graph Neural Networks
Webinar on Graph Neural NetworksWebinar on Graph Neural Networks
Webinar on Graph Neural NetworksLucaCrociani1
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기NAVER Engineering
 
Graph Neural Network in practice
Graph Neural Network in practiceGraph Neural Network in practice
Graph Neural Network in practicetuxette
 
Deep Learning for Graphs
Deep Learning for GraphsDeep Learning for Graphs
Deep Learning for GraphsDeepLearningBlr
 
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Christopher Morris
 
Graph Neural Network 1부
Graph Neural Network 1부Graph Neural Network 1부
Graph Neural Network 1부seungwoo kim
 
グラフニューラルネットワーク入門
グラフニューラルネットワーク入門グラフニューラルネットワーク入門
グラフニューラルネットワーク入門ryosuke-kojima
 
第52回SWO研究会チュートリアル資料
第52回SWO研究会チュートリアル資料第52回SWO研究会チュートリアル資料
第52回SWO研究会チュートリアル資料Takanori Ugai
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Muhammad Haroon
 
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.Yongho Ha
 
오토인코더의 모든 것
오토인코더의 모든 것오토인코더의 모든 것
오토인코더의 모든 것NAVER Engineering
 
Graph Neural Networks
Graph Neural NetworksGraph Neural Networks
Graph Neural Networkstm1966
 
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜SSII
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature EngineeringSri Ambati
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks남주 김
 
Graph Convolutional Neural Networks
Graph Convolutional Neural Networks Graph Convolutional Neural Networks
Graph Convolutional Neural Networks 신동 강
 
IIBMP2016 深層生成モデルによる表現学習
IIBMP2016 深層生成モデルによる表現学習IIBMP2016 深層生成モデルによる表現学習
IIBMP2016 深層生成モデルによる表現学習Preferred Networks
 
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB ImagesDeep Learning JP
 

What's hot (20)

Webinar on Graph Neural Networks
Webinar on Graph Neural NetworksWebinar on Graph Neural Networks
Webinar on Graph Neural Networks
 
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
1시간만에 GAN(Generative Adversarial Network) 완전 정복하기
 
Graph Neural Network in practice
Graph Neural Network in practiceGraph Neural Network in practice
Graph Neural Network in practice
 
Deep Learning for Graphs
Deep Learning for GraphsDeep Learning for Graphs
Deep Learning for Graphs
 
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
 
Graph Neural Network 1부
Graph Neural Network 1부Graph Neural Network 1부
Graph Neural Network 1부
 
Graph Convolutional Network 概説
Graph Convolutional Network 概説Graph Convolutional Network 概説
Graph Convolutional Network 概説
 
グラフニューラルネットワーク入門
グラフニューラルネットワーク入門グラフニューラルネットワーク入門
グラフニューラルネットワーク入門
 
第52回SWO研究会チュートリアル資料
第52回SWO研究会チュートリアル資料第52回SWO研究会チュートリアル資料
第52回SWO研究会チュートリアル資料
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
 
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.
자습해도 모르겠던 딥러닝, 머리속에 인스톨 시켜드립니다.
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
오토인코더의 모든 것
오토인코더의 모든 것오토인코더의 모든 것
오토인코더의 모든 것
 
Graph Neural Networks
Graph Neural NetworksGraph Neural Networks
Graph Neural Networks
 
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜
SSII2020SS: グラフデータでも深層学習 〜 Graph Neural Networks 入門 〜
 
Feature Engineering
Feature EngineeringFeature Engineering
Feature Engineering
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Graph Convolutional Neural Networks
Graph Convolutional Neural Networks Graph Convolutional Neural Networks
Graph Convolutional Neural Networks
 
IIBMP2016 深層生成モデルによる表現学習
IIBMP2016 深層生成モデルによる表現学習IIBMP2016 深層生成モデルによる表現学習
IIBMP2016 深層生成モデルによる表現学習
 
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
[DL輪読会]Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
 

Graph Neural Network (한국어)

  • 2. 시작하기 전... ● GCN(Graph Convolutional Network)에 대한 이야기가 아닙니다 ○ 추후에 볼 예정… ● GNN의 기본 컨셉에 대해서만 다룹니다
  • 3. Graph Neural Network란? ● Graph 데이터 구조를 위한 Neural Network ○ 입력: Graph ○ 출력: Label ■ 숫자(0.95, 0.81, 0.4, …) ■ 라벨링 (Protein, Carbon-dioxide, etc.) ■ 분류 (Drug/Not Drug, etc.)
  • 4. 왜 Graph Neural Network? ● 다양한 관계에 대한 이해를 위한 구조 ○ RNN의 경우 Cell간에 Recursive한 관계를 가짐 ● 네트워크를 표현하기 위함 ○ 자기 자신에 대한 Connection ○ 어떤것들은 연결이 아예 존재하지 않는 경우도 있음
  • 5. Euclidean Space (유클리드 공간) 원점에서 좌표 (x, y, z)까지의 거리 = 원점과 좌표 사이의 가장 짧은 직선의 길이
  • 6. Non-Euclidean Space (비 유클리드 공간) 파랑부분의 처음 지점 ~ 붉은부분의 끝지점 ● Euclidean 공간에서는 거리가 계속 변함 ○ 거리가 계속 좁혀지는 형태 ● Non-Euclidean 공간에서는 거리가 변하지 않음 https://deepmind.com/blog/alphafold/
  • 7. Euclidean Space vs Non-Euclidean Space
  • 8. When to use GNN? ● 화학 ○ 마약인가/아닌가? (확률을 통해 산출) ● Program 이해 ○ Expression Statement ○ 프로그래밍을 통한 해결 ○ RNN 모델보다 성능이 더 좋음 ● Etc.
  • 11. RNN (Recurrent Neural Network) ● RNN에서는 각 Cell이 이전 Cell에서부터 전달되는 형식으로 정보가 누적된다 ○ 이러한 연속된 형태의 예시로는... ■ Time Series ■ 단어 순서 ■ 사건의 시퀀스 ■ Etc.
  • 13. GNN (Graph Neural Network)
  • 14. GNN (Graph Neural Network)
  • 15. GNN (Graph Neural Network) Adjacency = 1
  • 16. GNN (Graph Neural Network)
  • 17. Adjacency Matrix ● 연결 강도로 시각화
  • 18. Adjacency Matrix DeepMind: AlphaFold (https://deepmind.com/blog/alphafold/#image-33420)
  • 19. GCN (Graph Convolutional Network) ● (l): Layer ● H: Input from Layer (l) ● W: Weights (Trainable) ● A: Adjacency Matrix ● D: Degree Matrix ○ Number of connections attached to each node ● Hat notations ○ Normalized with standard Identity matrix ● Activation Function: Sigmoid
  • 21. GNN (Graph Neural Network) ● RNN ○ 현재 Cell에 이전 Cell의 정보를 전달하는 방식으로 propagate ● GNN ○ ‘Adjacency’에 대해 고려 ■ Adjacency = 1에 대한 정보 ■ Adjacency = 2에 대한 정보 ■ Adjacency = 3에 대한 정보 ■ … ■ Adjacency = n(최대 갯수)에 대한 정보 까지
  • 22. GNN Variations ● Graph Neural Network (2005) ● Spectral Networks (2014) ● Neural Message Passing (2017) ○ Gated Graph Neural Network (2016) ○ ChebyNets (2016) ○ Graph Convolution Network (2017) ● Async. Neural Message Passing (2018) ● AMPNet (2018) ● Programs As Graphs (2018)
  • 23. GNN Variations ● Graph AutoEncoder (2016) ● GraphGAN (2017) ● GraphSAGE (2017) ● Graph Attention Networks (ICRL 2018) ● GraphRNN (2018) ● Splitter (2019)
  • 25. Datasets for GNN https://github.com/shiruipan/graph_datasets ** Not listed in link, but ‘Zachary’s karate club’ is a commonly used social network. ** (https://towardsdatascience.com/how-to-do- deep-learning-on-graphs-with-graph- convolutional-networks-7d2250723780)
  • 26. Graph Nets Library (DeepMind) https://github.com/deepmind/graph_nets ● DeepMind에서 제작한 프레임워크 ● Doc이 상세하게 기술되어 있음 ● Tensorflow-based
  • 27. DGL (Deep Graph Library) https://www.dgl.ai/ ● 최근 개발됨 (2018) ● NYU, NYU-상하이, Amazon 에서 개발
  • 28. PyTorch Geometric https://github.com/rusty1s/pytorch_geometric ● PyTorch Extension ○ 기본 설치 X ● DGL보다 최대 15배 가량 빠 르다는 정보가 있음
  • 29. Geometric Deep Learning http://geometricdeeplearning.com/ ● Not a Framework ● Many references for GNN (Not a lot of descriptions)
  • 30. References Microsoft - Graph Neural Networks: Variations and Applications (https://www.youtube.com/watch?v=cWIeTMklzNg) Graph Theory - Adjacency Matrices (https://www.ebi.ac.uk/training/online/course/network-analysis-protein-interaction-data- introduction/introduction-graph-theory/graph-0) DeepMind - AlphaFold (https://deepmind.com/blog/alphafold/)