Graphs are brilliant. They're really well studied and have some amazing properties, especially for predictive analytics. But then there's AI: it's super-cool and also has some amazing abilities to guess the future given the past.
So, which are we meant to choose? I'd argue we should use both.
In this talk we'll see how graphs give us a framework for contextualizing the world around us. We'll explore how simple rules from graph theory can evolve our model to show how it might be in the future.
But that's not all, we'll also see how we can take our graphs and feed them into our ML pipelines for better scores than simple row-wise data using graph neural nets. And we'll see how to learn on graphs directly with graph convolutional networks.
Finally, we'll close the loop by asking our machine learning to tell us about other queries we should be running against the input graph to find patterns in data we don’t even know are valuable today.
1. 10/07/2019
1
Graphs for AI and ML
Dr. Jim Webber
Chief Scientist, Neo4j
@jimwebber
● Some no-BS definitions
● Graphs and an accidental Skynet
● Graph theory
● Contemporary graph ML
● The future of graph AI
Overview
2. 10/07/2019
2
● ML - Machine Learning
○ Finding functions from historical data to guide future
interactions within a given domain
● AI - Artificial Intelligence
● The property of a system that it appears intelligent to its
users
● Often, but not always, using ML techniques
● Or ML implementations that can be cheaply retrained to
address neighbouring domains
A Bluffer’s Guide to AI-cronyms
● Predictive analytics
● Use past data to predict the future
● General purpose AI
● ML with transfer learning such that learned experiences in
one domain can be applied elsewhere
● Human-like AI
Often conflated with
4. 10/07/2019
4
Extract all the features!
• What do we do? Turn it to
vectors and pump it through a
classification or regression
model
• That’s actually not a bad
thing
• But we can do so much before
we even get to ML…
• … if we have graph data
Credit: Graph Algorithms, Holder and Needham, O’Reilly 2019
6. 10/07/2019
6
stole
from
loves
loves
enemy
enemy
A Good
Man Goes
to War
appeared
in
appeared
in
appeared
in
appeared
in
Victory of
the Daleks
appeared
in
appeared
in
companion
companion
enemy
stole
from
loves
loves
enemy
enemy
A Good
Man Goes
to War
appeared
in
appeared
in
appeared
in
appeared
in
Victory of
the Daleks
appeared
in
appeared
in
companion
companion
enemy
planet
prop
species
species
species
character
character
character
episode
episode
8. 10/07/2019
8
Fearless querying
MATCH path = (:author {name:’Jim Webber’}
-[*]->(:character {name:’The Doctor’})
RETURN path
OR
MATCH (me:author {name:’Jim Webber’},
(doc:character {name:’The Doctor’}),
path = shortestPath((me)-[*]->(doc))
RETURN path
regeneration
character
actor
character
actor
landlord
author
tenant
flat
book
author
wrote
wrote
owns
sublet
played
seriesin
in
played
played
character
played
21. 10/07/2019
21
Search structure
Graph Theory
• Rich knowledge of how graphs
operate in many domains
• Off the shelf algorithms to
process those graphs for
information, insight,
predictions
• Low barrier to entry
• Amazingly powerful
31. 10/07/2019
31
It if a node has strong relationships to two neighbours, then these
neighbours must have at least a weak relationship between them.
[Wikipedia]
Strong Triadic Closure
Triadic Closure
(weak relationship)
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND
32. 10/07/2019
32
Triadic Closure
(weak relationship)
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND
name: Kenny
name: Stan name: Cartman
FRIENDFRIEND
FRIEND 50%
• Relationships can have “strength” as well as intent
• Think: weighting on a relationship in a property graph
• Weak links play another super-important structural role in graph
theory
• They bridge neighbourhoods
Weak relationships
33. 10/07/2019
33
Local Bridges
FRIEND
name: Kenny
name: Stanname: Kyle
FRIENDFRIEND
FRIEND
name: Sally
name: Bebename: Wendy
FRIENDFRIEND
FRIEND 50%
name:
Cartman
FRIEND
FRIEND
50%ENEMY
“If a node A in a network satisfies the Strong Triadic Closure Property
and is involved in at least two strong relationships, then any local
bridge it is involved in must be a weak relationship.”
[Easley and Kleinberg]
Local Bridge Property
34. 10/07/2019
34
University Karate Club
• (NP) Hard problem
• Repeatedly remove the spanning links between dense regions
• Or recursively merge nodes into ever larger “subgraph” nodes
• Choose your algorithm carefully – some are better than others
for a given domain
• Can use to (almost exactly) predict the
break up of the karate club!
Graph Partitioning
38. 10/07/2019
38
Find and stop spammers
Extract graph structure over time
Not message content!
(Fakhraei et al, KDD 2015)
Learning to stop bad guys
Result: find and classify 70% spammers with 90% accuracy
Much of modern graph ML is still about turning graphs to vectors
Graph2Vec and friends
Highly complementary techniques
Mixing structural data and features gives better results
Better data into the model, better results out
But we don’t have to always vectorize graphs...
Graph ML
39. 10/07/2019
39
Knowledge Graphs
• Semantic domain knowledge for
inference and understanding
• E.g. eBay Google Assistant
• What’s the next best question to
ask when a potential customer
says they want a bag?
• Price? Function? Colour?
• Depends on context! Demographic,
history, user journey.
• Richly connected data makes the
system seem intelligent
• But it’s “just” data and algorithms in
reality
Graph Convolutional
Neural Networks
A general architecture for
predicting node and
relationship attributes in
graphs.
(Kipf and Welling, ICLR 2017)
Credit: Andrew Docherty (CSIRO), YowData 2017
https://www.youtube.com/watch?v=Gmxz41L70Fg
40. 10/07/2019
40
Graph Networks for
Structured Causal Models
• Position paper from Google,
MIT, Edinburgh
• Structured representations
and computations (graphs)
are key
• Goal: generalize beyond direct
experience
• Like human infants can
https://arxiv.org/pdf/1806.01261.pdf
credit: @markhneedham