2. about me
who am i ...
➡ working as an it lead / software architect @ janssen pharmaceutica
• dealing with big scientific data sets
• hands-on expertise in big data and NoSQL technologies
➡ founder of datablend
• provide big data and NoSQL consultancy
Davy Suvee • share practical knowledge and big data use cases via blog
@DSUVEE
5. graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ...
★ neo-versioning by david montag 1
2
★ representing time dependent graphs in neo4j by the isi foundation
★ modeling a multilevel index in neo4j by peter neubauer 3
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
6. graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ...
★ neo-versioning by david montag 1
2
★ representing time dependent graphs in neo4j by the isi foundation
★ modeling a multilevel index in neo4j by peter neubauer 3
copy and relink semantics
๏ graph size
๏ object identity
๏ mixing data-model and time-model
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
8. FluxGraph ...
➡ towards a time-aware graph ...
➡ implement a blueprints-compatible graph on top of Datomic
9. FluxGraph ...
➡ towards a time-aware graph ...
➡ implement a blueprints-compatible graph on top of Datomic
➡ make FluxGraph fully time-aware
★ travel your graph through time
★ time-scoped iteration of vertices and edges
★ temporal graph comparison
11. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
12. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
13. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
Vertex michael = ...
Michael
14. travel through time
FluxGraph fg = new FluxGraph();
Davy
kn
ow
Vertex davy = fg.addVertex();
s
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
Vertex michael = ...
Edge e1 = Michael
fg.addEdge(davy, peter,“knows”);
15. travel through time
Davy
Date checkpoint = new Date();
kn
ow
s
Peter
Michael
16. travel through time
David
Date checkpoint = new Date();
kn
ow
s
davy.setProperty(“name”,”David”); Peter
Michael
17. travel through time
David
Date checkpoint = new Date();
kn
ow
s
davy.setProperty(“name”,”David”); Peter
kn
Edge e2 =
ow
fg.addEdge(davy, michael,“knows”);
s
Michael
28. time-scoped iteration
t1 t2 t3 tcurrrent
change change change
Davy Davy’ Davy’’ Davy’’’
➡ how to find the version of the vertex you are interested in?
36. time-scoped iteration
➡ When does an element change?
➡ vertex:
★ setting or removing a property
★ add or remove it from an edge
★ being removed
37. time-scoped iteration
➡ When does an element change?
➡ vertex: ➡ edge:
★ setting or removing a property ★ setting or removing a property
★ add or remove it from an edge ★ being removed
★ being removed
38. time-scoped iteration
➡ When does an element change?
➡ vertex: ➡ edge:
★ setting or removing a property ★ setting or removing a property
★ add or remove it from an edge ★ being removed
★ being removed
➡ ... and each element is time-scoped!
44. use case: longitudinal patient data
t1 t2 t3 t4 t5
smoking smoking death
patient patient patient patient patient
cancer cancer
45. use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
46. use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
➡ example analysis:
★ if a male patient is no longer smoking in 2005
★ what are the chances of getting lung cancer in 2010, comparing
patients that smoked before 2005
patients that never smoked
47. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
48. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males =
fg.getVertices("gender", "male").iterator()
49. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males =
fg.getVertices("gender", "male").iterator()
while (males.hasNext()) {
Vertex p2005 = males.next();
boolean smoking2005 =
p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();
}
50. use case: longitudinal patient data
➡ which patients were smoking before 2005?
boolean smokingBefore2005 =
((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() {
public TimeAwareElement filter(TimeAwareVertex element) {
return element.getEdges(OUT, "smokingStatus").iterator().hasNext()
? element : null;
}
}).iterator().hasNext();
51. use case: longitudinal patient data
➡ which patients have cancer in 2010
working set of smokers
Graph g =
fg.difference(smokerws,
time2010.toDate(),
time2005.toDate());
52. use case: longitudinal patient data
➡ which patients have cancer in 2010
working set of smokers
Graph g =
fg.difference(smokerws,
time2010.toDate(),
time2005.toDate());
➡ extract the patients that have an edge to the cancer node
56. gephi plugin for blueprints!
1
➡ available on github
http://github.com/datablend/gephi-blueprints-plugin
➡ Support for neo4j, orientdb, dex, rexter, ...
1. Kudos to Timmy Storms (@timmystorms)
