Kevin Van Gundy presents on transforming your RDBMS data into a graph at Neo4j GraphDay NYC

1. RDBMS TO GRAPH
Graph Day NYC

4. ACCOUNT
HOLDER 2
ACCOUNT
HOLDER 1
ACCOUNT
HOLDER 3
CREDIT
CARD
BANK
ACCOUNT
BANK
ACCOUNT
BANK
ACCOUNT
ADDRESS
PHONE NUMBER
PHONE NUMBER
SSN 2
UNSECURE LOAN
SSN 2
UNSECURE LOAN
CREDIT
CARD

5. I'm Kevin
Deployment Strategy at Neo4j
/in/kevinvangundy
@kevinvangundy
kevin@neo4j.com

6. Does the underlying
data-structure matter?

7. Name Country Dept University
John UK Prime Brokerage Princeton
Mary USA Sales and Trade Yale
Li China Investment Banking Princeton
Kate UK Sales and Trade Princeton
Michal CA Investment Banking Brown
Employees

8. ID Country
17 UK
12 USA
19 China
17 UK
112 CA
Countries

9. ID Country Leader
17 UK Cameron
12 USA Obama
19 China Xi Jinping
17 UK Cameron
112 CA Trudeau
Countries

10. Name Country Dept University
John 17 Prime Brokerage Princeton
Mary 12 Sales and Trade Yale
Li 19 Investment Banking Princeton
Kate 17 Sales and Trade Princeton
Michal 112 Investment Banking Brown
Employees

11. ID Name President State
92 Princeton Eisgrubt NJ
34 Yale Salovey CT
1 Brown Paxson RI
University

12. Name Country Dept University
John 17 Prime Brokerage 92
Mary 12 Sales and Trade 34
Li 19 Investment Banking 92
Kate 17 Sales and Trade 92
Michal 112 Investment Banking 1
Employees

13. Name Country Dept University
John 17 Prime
Brokerage
92
Mary 12 Sales and
Trade
34
Li 19 Investment
Banking
92
Kate 17 Sales and
Trade
92
Michal 112 Investment
Banking
1
ID Country
17 UK
12 USA
19 China
17 UK
112 CA
ID Name Presiden
t
State
92 Princeton Eisgrubt NJ
34 Yale Salovey CT
1 Brown Paxson RI

14. SELECT
p.name,
c.country, c.leader, p.hair,
u.name, u.pres, u.state
FROM
people p
LEFT JOIN country c ON c.ID=p.country
LEFT JOIN uni u ON p.uni=u.id
WHERE
u.state=‘CT’

15. Name Country Dept University
John 17
Prime
Brokerage
92
Mary 12 Sales and Trade 34
Li 19
Investment
Banking
92
Kate 17 Sales and Trade 92
Michal 112 Investment
Banking
1
ID Country
17 UK
12 USA
19 China
17 UK
112 CA
ID Name President State
92 Princeton Eisgrubt NJ
34 Yale Salovey CT
1 Brown Paxson RI
Name Country Dept University
John 17
Prime
Brokerage
92
Mary 12 Sales and Trade 34
Li 19
Investment
Banking
92
Kate 17 Sales and Trade 92
Michal 112 Investment
Banking
1
ID Country
17 UK
12 USA
19 China
17 UK
112 CA
ID Name President State
92 Princeton Eisgrubt NJ
34 Yale Salovey CT
1 Brown Paxson RI
Name Country Dept University
John 17
Prime
Brokerage
92
Mary 12 Sales and Trade 34
Li 19
Investment
Banking
92
Kate 17 Sales and Trade 92
Michal 112 Investment
Banking
1
ID Country
17 UK
12 USA
19 China
17 UK
112 CA
19 China
17 UK
112 CA
Name Country Dept University
John 17
Prime
Brokerage
92
Mary 12 Sales and Trade 34
Li 19
Investment
Banking
92
Kate 17 Sales and Trade 92
Michal 112 Investment
Banking
1
ID Country
17 UK
12 USA
19 China
17 UK
112 CA
Name Country Dept University
John 17
Prime
Brokerage
92
Mary 12 Sales and Trade 34
Li 19
Investment
Banking
92
Kate 17 Sales and Trade 92
Michal 112 Investment
Banking
1
ID Name President State
92 Princeton Eisgrubt NJ
34 Yale Salovey CT
1 Brown Paxson RI

16. (SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.pid AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT manager.pid AS directReportees, count(manager.directly_manages)
AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(reportee.directly_manages) AS
count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(L2Reportees.directly_manages)
AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT reportee.pid AS directReportees, count(reportee.directly_manages)
AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
SELECT depth1Reportees.pid AS directReportees,
count(depth2Reportees.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM(
SELECT reportee.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT L2Reportees.pid AS directReportees, count(L2Reportees.directly_manages) AS
count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT L2Reportees.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
)

17. JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN

18. JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN

19. JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN
JOIN

21. Have you seen
Ted's UUID?

23. • Complex to model and store relationships
• Performance degrades with increases in data
• Queries get long and complex
• Maintenance is painful
SQL Trouble

24. • Easy to model and store relationships
• Performance of relationship traversal remains constant with growth
in data size
• Queries are shortened and more readable
• Adding additional properties and relationships can be done on the
fly - no migrations
Graph Motivations

25. Graph Gains

26. (SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.pid AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT manager.pid AS directReportees, count(manager.directly_manages)
AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(reportee.directly_manages) AS
count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT manager.pid AS directReportees, count(L2Reportees.directly_manages)
AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM (
SELECT manager.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
UNION
SELECT reportee.pid AS directReportees, count(reportee.directly_manages)
AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
SELECT depth1Reportees.pid AS directReportees,
count(depth2Reportees.directly_manages) AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT T.directReportees AS directReportees, sum(T.count) AS count
FROM(
SELECT reportee.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee reportee
ON manager.directly_manages = reportee.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
UNION
SELECT L2Reportees.pid AS directReportees, count(L2Reportees.directly_manages) AS
count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
GROUP BY directReportees
) AS T
GROUP BY directReportees)
UNION
(SELECT L2Reportees.directly_manages AS directReportees, 0 AS count
FROM person_reportee manager
JOIN person_reportee L1Reportees
ON manager.directly_manages = L1Reportees.pid
JOIN person_reportee L2Reportees
ON L1Reportees.directly_manages = L2Reportees.pid
WHERE manager.pid = (SELECT id FROM person WHERE name = "fName lName")
)

27. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate, count(report) AS Total

28. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total

29. How Fast is Fast?
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

30. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

31. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
RDBMs 1,000 2,000 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

32. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
RDBMs 1,000 2,000 ms
Neo4j 1,000 2 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

33. How Fast is Fast?
DATABASE # OF PERSONS QUERY TIME
RDBMs 1,000 2,000 ms
Neo4j 1,000 2 ms
Neo4j 10,000,000 2 ms
• Sample Social Graph with roughly 1,000 persons
• On average each person has 50 friends
• pathExists(a,b) limited to depth 4
• Caches warmed up to eliminate disk I/O

34. What is the
"Graph Impact?"

35. David Meza of NASA said: "Neo helped NASA save millions of
dollars and up to two years by locating existing research they
could use in his work on the Orion, the spacecraft
NASA hopes eventually will take humans to Mars."

36. we're getting mars 2
years sooner
because of Neo4j…

37. “We needed to understand consumer behavior across
devices in order to capture a complete picture. Conceptually
we could have done this in a relational database, but the
multiple JOINS would have made it much too complicated.”
- Qualia CTO, Niels Meersschaert

38. We're smashing a billion
queries a day that'd be
impossible in relational…

39. "I found graph databases, which perform well with
queries on connected data. With more than 10 years of
experience of using relational database, I know that
complicated joins are the performance killer. But graph
databases kick ass of other databases."
- LinkedIn China Development Lead, Dong Bin

40. Neo4j is awesome…

41. How do you use Neo4j?
KEY QUESTIONS

42. How do you use Neo4j?
KEY QUESTIONS
MODEL

43. How do you use Neo4j?
KEY QUESTIONS
MODEL+
LOAD DATA

44. How do you use Neo4j?
KEY QUESTIONS
MODEL
QUERY DATA
+
LOAD DATA

45. How do you use Neo4j?

47. How do you use Neo4j?

48. Language Drivers

49. Native Procedures and Functions

50. From RDBMs To Graphs

52. Northwind

53. Northwind - a Canonical RDBMS Example

54. ( )-[:TO]->(Graph)

56. ( )-[:IS_BETTER_AS]->(Graph)

57. Starting with the ER Diagram

58. Locate the Foreign Keys

59. Drop the Foreign Keys

60. Find the JOIN Tables

61. (Simple) JOIN Tables Become Relationships

62. Attributed JOIN Tables -> Relationships with Properties

63. Querying a Subset Today

64. As a Graph

65. QUERYING THE GRAPH

66. Property Graph Model
CREATE (:Employee{ firstName:“Steven”} ) -[:REPORTS_TO]-> (:Employee{ firstName:“Andrew”} )
REPORTS_TO
Steven Andrew
LABEL PROPERTY
NODE NODE
LABEL PROPERTY

67. Who do people report to?
MATCH
(e:Employee)<-[:REPORTS_TO]-(sub:Employee)
RETURN
*

68. Who do people report to?

69. Who do people report to?
MATCH
(e:Employee)<-[:REPORTS_TO]-(sub:Employee)
RETURN
e.employeeID AS managerID,
e.firstName AS managerName,
sub.employeeID AS employeeID,
sub.firstName AS employeeName;

70. Who do people report to?

71. Who does Robert report to?
MATCH
p=(e:Employee)<-[:REPORTS_TO]-(sub:Employee)
WHERE
sub.firstName = ‘Robert’
RETURN
p

72. Who does Robert report to?

73. What is Robert’s reporting chain?
MATCH
p=(e:Employee)<-[:REPORTS_TO*]-(sub:Employee)
WHERE
sub.firstName = ‘Robert’
RETURN
p

74. What is Robert’s reporting chain?

75. Who’s the Big Boss?
MATCH
(e:Employee)
WHERE
NOT (e)-[:REPORTS_TO]->()
RETURN
e.firstName as bigBoss

76. Who’s the Big Boss?

77. Product Cross-Selling
MATCH
(choc:Product {productName: 'Chocolade'})
<-[:INCLUDES]-(:Order)<-[:SOLD]-(employee),
(employee)-[:SOLD]->(o2)-[:INCLUDES]->(other:Product)
RETURN
employee.firstName,
other.productName,
COUNT(DISTINCT o2) as count
ORDER BY
count DESC
LIMIT 5;

78. Product Cross-Selling

79. POWERING AN APP

80. NYC Meetup Recommendation App