Recommended
More Related Content
What's hot
What's hot (20)
Similar to Graph Neural Networks Explained
Similar to Graph Neural Networks Explained (20)
Recently uploaded
Recently uploaded (20)
Graph Neural Networks Explained
- 1. Graph Neural Networks Liad Magen March October 2020
- 3. Prof. Dr. Stefan Thurner
- 5. Seven Bridges of Königsberg → →
- 6. Location & Street representation
- 9. Paper citations as a graph
- 10. Mesh & Point cloud as a graph
- 14. Process other (non-Euclidean) formats of data
- 15. Google PageRank
- 16. Inspired usages
- 19. Graph Neural Networks ICLR Submissions Source: Twitter - @prlz77
- 20. Back to basics
- 21. Graphs, Mathematically G = (Vetrices, Edges) V = {v1 … vn) E = {(vi, vj), … }
- 22. Graph - algebraic representation G = (Vetrices, Edges) Adjacency matrix 1 - if {vi, vj} ∈ E && i != j Aij 0 - Otherwise
- 23. Graph - algebraic representation G = (Vetrices, Edges) Laplacian matrix * L = D-A Degree Adjacency Laplacian * For undirected graphs
- 24. GCN - Algorithm
- 25. GCN GCN Layer: Y = ReLU(AXW) ...but with message passing
- 26. Message Passing For each node : 1. Aggregate neighbor nodes into an intermediate representation 2. Transform the aggregated representation with a linear projection followed by a non-linearity (ReLU) Mathematically: H →Network Layer W→Network Weights A →Adjacency Matrix D →Degree Matrix
- 27. Model types ● Graph classification ○ Chemical properties of a molecule ○ Comparing user preferences / activities ● Node classification - node label prediction ○ Malicious users in a social network ○ Visually inferred Named Entity Recognition (NER) ○ Node clustering ● Edge prediction ○ Recommendation system ○ Protein-protein interaction ○ “Friend” suggestion
- 28. Frameworks
- 29. NetworkX ● Store and mutate Graphs ● Graph algorithms (Shortest path - Dijkstra, TreeWidth, clustering, centrality) ● Network analysis ● Node / edge data ● Visualization tools
- 30. DGL ● Building blocks ● Great tutorials ● Generative graphs ● Great for research and complicated tasks
- 31. ● An extension library for pyTorch ● Officially part of the pyTorch ecosystem ● Easily extensible ● Papers are implemented directly in it ● Looooooooooooooong list of ready-to-use methods and algorithms: ○ TransformerConf (2020) ○ GCN2Conv (2020) ○ DeeperGCN (2020) ○ Top-K Pooling ○ PairNorm PyTorch Geometric
- 32. Model types ● Graph classification ○ Chemical properties of a molecule ○ Comparing user preferences / activities ● Node classification - node label prediction ○ Malicious users in a social network ○ Visually inferred Named Entity Recognition (NER) ○ Node clustering ● Edge prediction ○ Recommendation system ○ Protein-protein interaction ○ “Friend” suggestion
- 34. PyTorch Geometric Example ● DataLoader ● GCN Layers
- 35. Potential Pitfalls when going ‘deep’ ● Vanishing Gradient ● Overfitting ● Over-smoothing Node-vectors become too similar ● Bottleneck (Over-Squashing) A single node vector contains data of too many nodes https://arxiv.org/abs/2006.07107 - Effective Training Strategies for Deep Graph Neural Networks
- 36. Scaling ● GraphSAGE https://arxiv.org/abs/1706.02216 ● ClusterGCN https://arxiv.org/abs/1905.07953 ● Graph-BERT https://arxiv.org/abs/2001.05140 https://github.com/jwzhanggy/Graph-Bert Additional reading: https://towardsdatascience.com/simple-scalable-graph-neural-networks-7eb04f366d07
- 37. Model Evaluation https://ogb.stanford.edu/ ● Datasets + Data Loaders (DGL / pyTorch Geometric) ● Node pred ● Edge pred ● Graph pred
- 38. Please feel free to ask questions ;)