The document discusses why graph databases and the graph model are becoming more popular. It provides three key reasons: 1) Big Data is increasingly large in volume and variety, requiring new database approaches; 2) NoSQL databases like graph databases provide an alternative to relational databases for big, complex datasets; 3) Many real-world datasets have an inherent graph structure that is difficult to represent and query in relational databases.
1. #neo4j
The Neo4j Election
Data @GDG SF
Andreas Kollegger Peter Neubauer Michael Hunger
@akollegger @peterneubauer @mesirii
1
Saturday, September 29, 12
3. #neo4j
Follow the Data
FEC Campaign Data
Andreas Kollegger Peter Neubauer Michael Hunger
@akollegger @peterneubauer @mesirii
2
Saturday, September 29, 12
8. Follow the Plan
1.Graph Database Primer
4
Saturday, September 29, 12
9. Follow the Plan
1.Graph Database Primer
1.Why graphs?
4
Saturday, September 29, 12
10. Follow the Plan
1.Graph Database Primer
1.Why graphs?
2.What's a graph database?
4
Saturday, September 29, 12
11. Follow the Plan
1.Graph Database Primer
1.Why graphs?
2.What's a graph database?
2.FEC Campaign Data
4
Saturday, September 29, 12
12. Follow the Plan
1.Graph Database Primer
1.Why graphs?
2.What's a graph database?
2.FEC Campaign Data
1.Data Model
4
Saturday, September 29, 12
13. Follow the Plan
1.Graph Database Primer
1.Why graphs?
2.What's a graph database?
2.FEC Campaign Data
1.Data Model
2.Import Strategy
4
Saturday, September 29, 12
14. Follow the Plan
1.Graph Database Primer
1.Why graphs?
2.What's a graph database?
2.FEC Campaign Data
1.Data Model
2.Import Strategy
3.Queries
4
Saturday, September 29, 12
17. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
5
Saturday, September 29, 12
18. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
5
Saturday, September 29, 12
19. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
5
Saturday, September 29, 12
20. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
4. Add a Neo4j addon instance to it (heroku addons:add neo4j)
5
Saturday, September 29, 12
21. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
4. Add a Neo4j addon instance to it (heroku addons:add neo4j)
5. Upload existing data to the graph
5
Saturday, September 29, 12
22. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
4. Add a Neo4j addon instance to it (heroku addons:add neo4j)
5. Upload existing data to the graph
6. Create a custom Ruby proxy app on Heroku
5
Saturday, September 29, 12
23. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
4. Add a Neo4j addon instance to it (heroku addons:add neo4j)
5. Upload existing data to the graph
6. Create a custom Ruby proxy app on Heroku
7. Connect to the app using a Google Spreadsheet
5
Saturday, September 29, 12
24. Follow the Plan - Part 2
1. Intro to Google Apps Script by Alex
2. Register at Heroku and install the heroku gem
3. Create and install a Heroku app (heroku apps:create)
4. Add a Neo4j addon instance to it (heroku addons:add neo4j)
5. Upload existing data to the graph
6. Create a custom Ruby proxy app on Heroku
7. Connect to the app using a Google Spreadsheet
8. Build a small bar chart from a Cypher query
5
Saturday, September 29, 12
28. Why graphs, why now?
D
$
;
* , ☕⛵
) ⛽. 7 4B N % 8 1
=
+O
'
@
5% < J
- ☕ 9:
(
⚾ !
?
E F$ P
2
✈ 3 * - ), L C
(
⚽ IQ H 6 7
>G KM
Saturday, September 29, 12
29. Why graphs, why now?
1.Big Data is the trend
8 /! ⛽C &
:D A ⛵0 , #
G
7 <7 J$ O " ⚾
! T⚽ %
+
3
X. E
C M S D@ A; B
*
9N ✈
P 10 4 #
Q K D 5 Y EI @ ⛽
, 4 J4 Z &R "☕> G
% 5 3+9 =% 8 @F ?
$ +7
; [W . $ B
L⚾.
⛵
* ,< ⛵ U
)☕ ⛽ XO F> 5 $=1 H
=
BN' ☕✈
! 1⚽ P
- ☕ 8 2⚾: LE H % < J
/9 V
( ?
F?
( ]
✈ 3 * - ), ; : C O *6
⚽ IQ H 7
> G KM
Saturday, September 29, 12
30. Why graphs, why now?
1.Big Data is the trend
2.NOSQL is the answer
P ' L 97 N W
Y
T 5 K 0 S @! ;
Q 8 /B . ⛽C&
& ⛽ : D !2$% A ⛵0 , #
? G
# O- 3⛵ ! T⚽ C " ⚾
X 7$ 7JR+ O
J %< !
3
8 [ * X. E
⚾] " N , ⚽9 C > S D@ A; B
D 1 P9 O > E M 10 4 #
M UJ I + D
7 ✈
#: Q K & WA ⛽
✈ R34 S 5 Y EI @ '
, 5 $ 4 < R T ⛵☕> G
Z
9 K %. " @F ?
X $ 4& J
⛽[) % 0 7 XOL⚾ - =☕
;" WI 5A 3B N = . 8 1 H
H V * ⛵ ☕. U L + $ B5
+
⛽ N
⛵ U 9 = , > B$1 ✈
G☕ , < M P
Z ☕ - ☕ ⚾8 2⚾:[FE H % < J P
'<: !
F / ]8 V; % FQ ⚽
?
✈ 4 H Z GV ), 2L E? *6
⚽
1
F ✈ 3 (
( - ] ; : CO
?
* @
⚽ C D Y M Q I H 7
+ > G K
Saturday, September 29, 12
31. Why graphs, why now?
1.Big Data is the trend
2.NOSQL is the answer
3.Large in volume, and in
P ' L 97 N W
Y
P ' L 97 N W
Y T5K 0 S @ . ;
T 5 K 0 S @! ; ⛽
. & Q & B
& ⛽ Q 8 /B O-G⛵ ⛽ C # !2R C
: D !2$% A "
# ⛵X
3 0, ?
# O- 3⛵ ! T⚽ ? 8 " J 3%$[ ! >
X 7$ 7JR+ 1 ] X⚾ A;⚽9 E
J % < M DJO C⚾ ,
8 9 [ C >✈S
* 7✈ O >
.
⚾] " N , ⚽9 ! U M IR D@ + E B WA
D 1 P + O >E # : 0 4 S
M UJ I 1 3 45 T'
7 S D X I@ WA " # &
✈ Q K
, 4 J &
Y EG T ⛽
$ +7 < Z= ⛵R UG
# :R34 5 $ 54HZV ☕"' I >0 L < $K% .
⛽ ⛵☕ FN? [ : -
X ;" WI 5A 3B 9 K %.8 1⚾P
[ % 0 L + N ⚾.
H V * ⛵ )☕. U
⛽
G☕ , < M PU&
N XO ☕
= L B- ☕
1@ A ]8 ;
MH
F ,
< BQ
⛽
⛵ [F> 4 Q J
';✈ 5 $= GV
1 F / 98 V, ⚽F ? < ✈
( : F
Z ☕ - ☕ ⚾8 2⚾: $ E H % 1⚽ P @
< %B Z
!
?H 2? E
✈ 4 H Z GV ]] 2L + *6 D Y
F ✈ 3( - ), ; : C O
⚽ ⚽ * @Y IQ E ? H C
+ >C D K M
G 7
Saturday, September 29, 12
32. P L0 7 N W
T 5 K Q S @ . ; L9 Y W
& ⛽ BP ' 07 N
9 Y O- 3⛵ !2 ? K
T5 S @B ;.
7 N W P ' L 97 N "
# Y X
8 WJ %$ & R 9 Y Q
⛽C7
' L >⛵ N W !2 ?
01 [ !
S @ B ; T 5 KDJ ⚾] @ . , ⚽# O- 3 X
. P> 0
9
Q
!2 ? ⛽
M UQ IS Why graphs, why now?
7 + O 58 Q WA @ $[ !R C
BS T K
; E J
S %. 9
" B⚽ >
&
R C - 3⛵
O XJ # :R34 5& ⛽ J ⚾] T ' , ; E >
✈ !I ?M U <I K 7 ! O ?
2RU LD 1
%$[ ! # 8 H"V ☕"
⚽9 X G %$⛽ 0O
C
AN ✈$ % . S
⛵ [ ! #& - 3⛵:R34 + 2R C
- WA
,
+ O M UJ W ⚾] , F >X #"< I [U L < K T '
> E >1 A Z ☕ ⚽9 M P8]8 [J %$5 & X: BQ
,
1E H⚾V ; , ⚽9 ! >
⚾ .
✈ 4 H > GV 1 AG☕" E 0 E N
S D I
5U LL < # :.
' ⚽ 1.Big Data is the trend
F Z DJ W] 2 ⛵
R3 7 S 9
9 Y ✈T ' 4 + O M U @IR ☕7 + 1⚾P
D Y
⛽ M A
Z' ? S O > ]8 [A % B Q
F W<
;$
,: -
P& 0[ K -
0N 7: "
A H ,G ⛵ &'
N
P A 0[ < YNT ✈ 4 5 '
✈
X $ N W I 50 LL XC # :.34 H Z GV9 Y 2. E
+ K - ⚽F
W I P& L < K T '
M P K8 ; % ⛽ Q M U
FT V @ .
⚾ 5 ] Q Z ☕B ;⚾P
< 7: "
H $ %⛽
S ☕ B 1 F 5 K8 ; G☕⛵ Q+M A
V
, @B . U @Y
0 LC0[ :%W
N D7 N - ?
T <
G⛽ @
V E
2 ✈ !2 Z GV ] Q Z F
F
? ⚽4 H ? ⛽ 1F 5 ] Q $
S ☕ B ;⚾P K8 ; @B Q
T ,<
S .
8
3 Y' 9 Y &
O-D⛵
C PJ L + R C C 3⛵
X %$7 N #
" W 2.NOSQL is the answer
O-D X 9
⚾] T , ⚽9 ! > ⚾ NJ %$[ ! #
E
@Y 2 ✈ !2 Z G⛽ @
? ⚽4 H ? V
&
R C C D⛵
O- 3 Y 2 E ;
B
? !2 ?
1 0 [>
5 K Q S E B1; 8
@ J WA
Y L ' 8
+
. W " 7 , ⚽9 P > X %$
J RC
J I 7+ O M U . ] 7+ O5T
D I@ 0 > E 1 A " , ⚽9 ! >
⚾] [
# :R34 ⛵
✈& ⛽ S !# :.34 S K M UJ W ' 7 + O > E
; ✈T ' S Q
?
BR D I WA
G# 8
⛽
⛵
O-I 5 &
3 XNJ
2R C
<? -
0 L H $!2 ⛽
V ☕" ⚾ M" A %$[V % "
U X : 3.Large in volume, and in
K ! I 5 & ⛽ & ✈T 34 S
[ < ☕⛵ Q
F , ⚽9R! > $%M J
P ,
CG B
8 ;E [
< K -
U L X # :.
R
0 ⛵3 -[ # % " I 50 L < K T
XA HOV
N &
N
1⚾
☕4 1 ] 7 + O Z ☕ E⚽ 1⚾P
M UJ I GV
✈D HZ
F ] > F " ]8 8; $ G☕⛵ Q YU
<
⚾ ZP
: ⛽
, ' 9M P
B A : -
[< %
$
,
✈ YS W 9
> E 4 WAGV
⚽ # :R34 @9 2 ✈ > H Z+ 7 ]9 2D ☕F L0 7 N W8 ;
? ⚽F O , ' ' I JU ME 1⚾
1 Y✈ 4 F
W NI 5 & ' P K TS
Y9
AL @Y ✈ ? ⚽ K Z GV @ . 2
L0 7 T 5 H
W ] E
X + . " C' 0N Y < + - P43R: # N
7
D UL .CD S @Y ;?
H V ☕⛵ P S' 0 K 5N W % 5 I K
⛽
G B@ MQ.T K < :&
A [< N BQ @B Q
L0 7 T $ L U T 5 "Q & ⛽ +. ; C D B ?
SX
;☕ ,.A
Z
F T 5⚾P :8 @! ;F⛽ ⛽☕#Z -!⛵ J
F
1 K- ] ⛽ ; 0 M
$[ & &⚾
? ⚽4 H Z GV% < /? P 1 ⛵0 ☕C
⛵⛽G& H
V 3
O !2
✈2! ⛵Y, 8
S-O
8
Q B Q 3D2B # O- 3⛵ F ✈ 8
; E 2R C %$[ !R C
X
"?
& ⛽ ED ? 8 ] $%8 Z " ⚽ %$ ! , ⚽9
C R + % CX :2 !# ? G A HX, # ]
J@
O- 3⛵ " ⚾ Y⚽ V ⚾G M UJ ⚾9 7 + O > E > A
[$ J 7 ! T@ CO " 4 3 1[
! 9⚽ 2R JD ⚽
#
> E >8 , " X ]$ JD D J I X⚾ A;IR3 >
1 7J MC S
% <9UC + 1 ] + , O > E SWA
I[ M U * . EB 4
✈
@ +# : W
O S 9N R ⚽
⚾] 7 , ✈ >
+ !D 7
D 1 5 43 : # E # A H #
M UJ I P + O
7 S D X I@
>✈ 10DX S
M :R34 ✈ I 50 L < K T
45 G☕" < KU .
⛽
⛵ &
T' [ : -
N .
K < # N R34 K 5 V Y E W"' I Z & ?
"☕ X
L 0 I ⛽ G HZV T ⛽ >0 L
✈U Q
& & "☕ V A G
A M, 5+ 4 < RG☕⛵ ⛵ M F F [ ⚾ % ]8 ; % B
N
U☕ MP
F
A $
,
,X : ⛵J
: [ 8 ;"⚾ I 5N 3 = Z= .
$ $7
% 0☕ + N ⚾.
H
$ 4 & Z 9 K %⛽ 8 @P $ V B Q 2< E
1: -
; V * P W . U L XOL B -
< F ⛽U
⛽ 1M 4
G VG< P
, Z⚾ A ☕4 $1☕ ✈ ] H ;
H ] ☕⛵ )☕⛵ F ✈ B $✈ 5 =1 H⚽48 Z< @
[ [F % B < J1⚾
'; > F Q GV F ,
:⚽ F Z ✈ + 2 C D Y
, E H% ⚽ P
!
G
E ?
? Z @F ☕ H8 ⚽98 V L
2 F 2⚾:
☕4 ✈1 / ]] < E
- ?
Y D C Z GV ), 2 : + ? *6 D Y
✈ ( H @ ?
( +
⚽ H ⚽3 * - Y ; ? CO C
@ I H
+ >C D K M Q
G 7
Saturday, September 29, 12
38. A graph database...
๏ no: not for charts & diagrams, or vector artwork
9
Saturday, September 29, 12
39. A graph database...
๏ no: not for charts & diagrams, or vector artwork
๏ yes: for storing data that is structured as a graph
9
Saturday, September 29, 12
40. A graph database...
๏ no: not for charts & diagrams, or vector artwork
๏ yes: for storing data that is structured as a graph
•remember linked lists, trees?
9
Saturday, September 29, 12
41. A graph database...
๏ no: not for charts & diagrams, or vector artwork
๏ yes: for storing data that is structured as a graph
•remember linked lists, trees?
•graphs are the general-purpose data structure
9
Saturday, September 29, 12
42. A graph database...
๏ no: not for charts & diagrams, or vector artwork
๏ yes: for storing data that is structured as a graph
•remember linked lists, trees?
•graphs are the general-purpose data structure
๏ “A relational database may tell you
the average age of everyone in the USA,
but a graph database will tell you
who is most likely to buy you a beer.”
9
Saturday, September 29, 12
63. We're talking about a
Property Graph
Nodes
Relationships
12
Saturday, September 29, 12
64. We're talking about a
Property Graph
Em Joh
il a n
knows knows
Alli Tob Lar
Nodes
son ias knows s
knows
And And knows
knows rea rés
s
knows knows knows
Pet Miic
Mc knows Ian
er knows a
a
knows knows
De Mic
lia h ael
Relationships
Properties (each a key+value)
+ Indexes (for easy look-ups)
12
Saturday, September 29, 12
69. Cypher - a graph query language
๏ a pattern-matching query language
๏ declarative grammar with clauses (like SQL)
๏ aggregation, ordering, limits
๏ create, read, update, delete
14
Saturday, September 29, 12
70. Cypher - a graph query language
๏ a pattern-matching query language
๏ declarative grammar with clauses (like SQL)
๏ aggregation, ordering, limits
๏ create, read, update, delete
// get node 1, traverse 2 steps away
start a=node(1) match (a)--()--(c) return c
// create a node with a 'name' property
CREATE (me {name: 'Andreas'}) return me
๏ more on this later...
14
Saturday, September 29, 12
99. Cypher - common clauses
// get node 1, traverse 2 steps away
START a=node(1) MATCH (a)--()--(c) RETURN c
// get node from an index, return it
START a=node:people(name='Andreas')
RETURN a
// get node from an index, match, filter
// with where, then return results
START a=node:people(name='Andreas')
MATCH (a)-[r]-(b) WHERE b.last='Sparrow'
RETURN r,b
22
Saturday, September 29, 12
101. FEC Campaign Data
yeah, this is the good stuff..
23
Saturday, September 29, 12
102. and now, it's time for
FEC Campaign Data
yeah, this is the good stuff..
23
Saturday, September 29, 12
103. FEC Campaign Data
๏In 1975, Congress created the Federal Election
Commission (FEC) to administer and enforce the
Federal Election Campaign Act (FECA) – The statute
that governs the financing of federal elections.
๏The duties of the FEC, which is an independent
regulatory agency, are to disclose campaign finance
information
24
Saturday, September 29, 12
104. FEC Campaign Data
๏Detailed files about...
• Candidates Committee Candidate
• Committees
• Individual Contributions Individual Contributions
๏10 years of data
๏Updated every Sunday
25
Saturday, September 29, 12
105. FEC Campaign Data - Committees
๏Committees
• one record for each committee registered with the
Federal Election Commission.
Committee - cm12.txt
CMTE_ID: String
CMTE_NM: String
TRES_NM: String
CMTE_ST1: String
CMTE_ST2: String
CMTE_CITY: String
CMTE_ST: String
CMTE_ZIP: String
CMTE_DSGN: String
CMTE_TP: String
CMTE_PTY_AFFILIATION: String
CMTE_FILING_FREQ: String
ORG_TP: String
CONNECTED_ORG_NM: String
CAND_ID: String
26
Saturday, September 29, 12
106. FEC Campaign Data
๏Candidates
• one record for each candidateappeared on a ballot
registered with the FEC or
who has either
list prepared by a state elections office.
Candidate - cn12.txt
CAND_ID: String
CAND_NAME: String
CAND_PTY_AFFILIATION: String
CAND_ELECTION_YR: String
CAND_OFFICE_ST: String
CAND_OFFICE: String
CAND_OFFICE_DISTRICT: String
CAND_ICI: String
CAND_STATUS: String
CAND_PCC: String
CAND_ST1: String
CAND_ST2: String
CAND_CITY: String
CAND_ST: String
CAND_ZIP: String
27
Saturday, September 29, 12
107. FEC Campaign Data
๏Individual Contributions
• each contribution from an individual to least $200.
committee if the contribution was at
a federal
Individual Contrib - itcont.txt
CMTE_ID: String
AMNDT_IND: String
RPT_TP: String
TRANSACTION_PGI: String
IMAGE_NUM: String
TRANSACTION_TP: String
ENTITY_TP: String
NAME: String
CITY: String
STATE: String
ZIP_CODE: String
EMPLOYER: String
OCCUPATION: String
TRANSACTION_DT: String
TRANSACTION_AMT: Double
OTHER_ID: String
TRAN_ID: String
FILE_NUM: Integer
MEMO_CD: String
MEMO_TEXT: String
SUB_ID: Integer
28
Saturday, September 29, 12
108. FEC Campaign Data - Extra Records
๏Candidate to Committee Linkage
• registered candidate to committee linkage
๏Transactions between Committees
• inter-committee contribution or independent
expenditure during the two-year election cycle
๏Contribution to Candidate
• contribution or independent expenditure from
committee to candidate during the two-year
election cycle 29
Saturday, September 29, 12
110. Raw Data Import
Committee Candidate
Candidate to Committee
Inter Committee Contributions
Candidate Contributions
Individual Contributions
31
Saturday, September 29, 12
111. Raw Data Import
Committee Candidate
CMTE_ID CAND_ID
Candidate to Committee
CMTE_ID CAND_ID
Inter Committee Contributions
CMTE_ID
Candidate Contributions
CAND_ID
Individual Contributions
CMTE_ID
31
Saturday, September 29, 12
119. Advanced Import - Dave Fauth
๏ includes SuperPAC data
๏ custom transform, then import
๏ model then looks like this...
Expenditures
Committee SUPPORTS Candidate
FUNDS
superPac
Contributions Contribution
GIVES
Individual
35
Saturday, September 29, 12
120. Advanced Import - Dave Fauth
๏ Extract and Transform
• Stored files on S3
• Used MortarData to run Hadoop jobs to prepare data
(@MortarData)
๏ Load
• Used Neo4J BatchInserter to load
• Thanks to Michael Hunger (@mesirii)
• Loaded 2M+ nodes in <5 minutes
36
Saturday, September 29, 12
121. Advanced Import - Dave Fauth
Java BatchInsert
Download Use S3
data Storage
Process with
Hadoop/Pig
Created Neo4J
DB
37
Saturday, September 29, 12
122. Wanna learn more?
๏Come hear Dave Fauth
present at...
38
Saturday, September 29, 12
123. Next...
Your Turn
39
Saturday, September 29, 12
124. From scratch
๏ git clone https://github.com/akollegger/FEC_GRAPH.git
๏ cd FEC_GRAPH
๏ ant initialize
• (need Apache ant? install from http://ant.apache.org)
๏ ant
• ant will build the importers and create a script
๏ ./bin/fec2graph --force --importer=RELATED
๏ ant neo4j-start
• will download and unpack neo4j, then start it
40
Saturday, September 29, 12
125. Investigate with Neo4j's Web UI
๏open http://localhost:7474
๏Dashboard - overview of data records
๏Data browser - examine data records, with
visualization options
๏Console - query the database using Cypher
41
Saturday, September 29, 12
126. Querying FEC with Cypher
๏ For Cypher documentation
• http://docs.neo4j.org/
๏ FEC Data Definitions
• http://www.fec.gov/finance/disclosure/ftpdet.shtml
๏ Ready for a challenge?
42
Saturday, September 29, 12
128. Cypher Challenges
http://1.usa.gov/uIGzZ 43
Saturday, September 29, 12
129. Cypher Challenges
// All presidential candidates for 2012
// Top 10 Presidential candidates according to number of
campaign committees
// find President Barack Obama
// lookup Obama by his candidate ID
// find Presidential Candidate Mitt Romney
// lookup Romney by his candidate ID
// find the shortest path of funding between Obama and Romney
// 10 top individual contributions to Obama
// 10 top individual contributions to Romney
http://1.usa.gov/uIGzZ 43
Saturday, September 29, 12
132. Cypher Challenges
// All presidential candidates for 2012
start candidate=node:candidates('CAND_ID:*')
where candidate.CAND_OFFICE='P' AND
candidate.CAND_ELECTION_YR='2012'
return candidate.CAND_NAME;
// Top 10 Presidential candidates according to
// number of campaign committees
start candidate=node:candidates('CAND_ID:*')
match candidate<-[r:SUPPORTS]-(campaign)
where candidate.CAND_OFFICE='P' AND
candidate.CAND_ELECTION_YR='2012'
return candidate.CAND_NAME, COUNT(campaign) as count
ORDER BY count desc LIMIT 10;
// find President Barack Obama
start obama=node:candidates('CAND_ID:*')
WHERE obama.CAND_NAME =~ '.*OBAMA.*'
return obama.CAND_NAME, obama.CAND_ID;
44
Saturday, September 29, 12
135. Cypher Challenges
// lookup Obama by his candidate ID
start obama=node:candidates(CAND_ID='P80003338') return obama;
// find Presidential Candidate Mitt Romney
start romney=node:candidates('CAND_ID:*')
WHERE romney.CAND_NAME =~ '.*ROMNEY.*'
return romney.CAND_NAME, romney.CAND_ID;
// lookup Romney by his candidate ID
start romney=node:candidates(CAND_ID='P80003353')
return romney;
// find the shortest path of funding between Obama and Romney
start romney=node:candidates(CAND_ID='P80003353'),
obama=node:candidates(CAND_ID='P80003338') MATCH
p=shortestPath(romney-[*..10]-obama) return p;
45
Saturday, September 29, 12
138. Cypher Challenges
// 10 top individual contributions to Obama
start obama=node:candidates(CAND_ID='P80003338') match obama<-
[:SUPPORTS]-(campaign)<-[:INDIVIDUAL_CONTRIBUTION]-
(contribution) return contribution.NAME,
contribution.TRANSACTION_AMT order by
contribution.TRANSACTION_AMT desc limit 10;
// 10 top individual contributions to Romney
start romney=node:candidates(CAND_ID='P80003353') match
romney<-[:SUPPORTS]-(campaign)<-[:INDIVIDUAL_CONTRIBUTION]-
(contribution) return contribution.NAME,
contribution.TRANSACTION_AMT order by
contribution.TRANSACTION_AMT desc limit 10;
46
Saturday, September 29, 12
139. Customize the Data Importer
๏Java-savvy and feeling brave?
๏make a copy of
• CODE/fecGraph/src/importer/fec/RelatedFecImporter.java
๏add your class to
• CODE/fecGraph/src/importer/Tool.java
๏read docs about batch insertion
• http://docs.neo4j.org/chunked/milestone/batchinsert.html
๏Ideas:
• extract States and Zip Codes into "location index"
• extract individual contributors from contribution list 47
Saturday, September 29, 12
146. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
50
Saturday, September 29, 12
147. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
50
Saturday, September 29, 12
148. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
3. Add a Neo4j addon (http://addons.heroku.com/neo4j)
instance to it (heroku addons:add neo4j)
50
Saturday, September 29, 12
149. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
3. Add a Neo4j addon (http://addons.heroku.com/neo4j)
instance to it (heroku addons:add neo4j)
4. Create a custom Ruby app (code below, GitHub) https://
github.com/neo4j-examples/heroku-neo4j-proxy
50
Saturday, September 29, 12
150. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
3. Add a Neo4j addon (http://addons.heroku.com/neo4j)
instance to it (heroku addons:add neo4j)
4. Create a custom Ruby app (code below, GitHub) https://
github.com/neo4j-examples/heroku-neo4j-proxy
5. Upload the data from example-data.neo4j.org
50
Saturday, September 29, 12
151. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
3. Add a Neo4j addon (http://addons.heroku.com/neo4j)
instance to it (heroku addons:add neo4j)
4. Create a custom Ruby app (code below, GitHub) https://
github.com/neo4j-examples/heroku-neo4j-proxy
5. Upload the data from example-data.neo4j.org
6. Connect to the app using a Google Spreadsheet , http://
bit.ly/GDG-GCALC
50
Saturday, September 29, 12
152. Follow the Plan - Part 2
1. Register at Heroku and install the heroku gem
2. Create and install a Heroku app (heroku apps:create)
3. Add a Neo4j addon (http://addons.heroku.com/neo4j)
instance to it (heroku addons:add neo4j)
4. Create a custom Ruby app (code below, GitHub) https://
github.com/neo4j-examples/heroku-neo4j-proxy
5. Upload the data from example-data.neo4j.org
6. Connect to the app using a Google Spreadsheet , http://
bit.ly/GDG-GCALC
7. Build a small bar chart from a Cypher query
50
Saturday, September 29, 12
154. Heroku Challenges
http://1.usa.gov/uIGzZ 51
Saturday, September 29, 12
155. Heroku Challenges
//Point the Database Instance to FEC http://bit.ly/SmkwUx/db/
data
// Build a Google Data table endpoint
// https://developers.google.com/chart/interactive/docs/
php_example
http://1.usa.gov/uIGzZ 51
Saturday, September 29, 12
158. The Google Spreadsheet Cypher driver
https://docs.google.com/spreadsheet/ccc?
key=0AsSBFHSo5OaPdGhzT1RTbDVaR0R3NW5iNUFpejVuSHc#gid=0 53
Saturday, September 29, 12
159. The Google Spreadsheet Cypher driver
function cypherUrlREST(payload, url, user, pwd) {
var auth = Utilities.base64Encode(user+":"+pwd);
var response = UrlFetchApp.fetch(
url,
{"method":"POST",
"payload": payload,
"contentType": "application/json",
"headers":{
"Authorization":"Basic "+auth,
"accept":"application/json",
}
});
return response.getContentText();
}
https://docs.google.com/spreadsheet/ccc?
key=0AsSBFHSo5OaPdGhzT1RTbDVaR0R3NW5iNUFpejVuSHc#gid=0 53
Saturday, September 29, 12
161. Google Challenges
http://1.usa.gov/uIGzZ 54
Saturday, September 29, 12
162. Google Challenges
// Build a cypher parser in GoogleAppsScript
// Build a Cypher query Google Widget
// Visualize Cypher Results with Google Data Table
// Geographic data viz
http://1.usa.gov/uIGzZ 54
Saturday, September 29, 12
163. The heatmap from Cypher to Google
55
Saturday, September 29, 12