Graph Search and Discovery for your Dark Data
To achieve new insights within your company you don't necessarily have to accumulate and crunch a lot of new tracking and tracing data - much key information already exists in disconnected data silos, stored in different databases and systems. By bringing the relevant pieces together in a single place and connecting the pieces of the puzzle at the right points you can gain new insights into your existing data and business. Graph databases offer the unique opportunity to easily cross-reference and connect disparate, variably structured datasets from many sources in a single place. It is then available for ad-hoc querying and exploration as well as strategic decision making. As a graph is a flexible and malleable data structure, it can evolve as you add new datasets or draw new connections from the insights or assumptions gained. This source of decision making will not be your old-school MDM data cemetery, but a flexible instrument that you can use, reuse, discard and recreate for different goals and purposes. Getting the data into the graph is only one interesting side of the coin, the other similarly challenging one is how to make the freshly spun web of knowledge available for different types of users. Technical users have it easy - they can yield Cypher, a powerful graph query language for arbitrary graph pattern matching, filtering, projection and aggregation of relevant data. For non-technical users a variety of tools and toolkits are available that use the inherent structures and meta information of the graph (labels, types, relationships and properties) to provide visual or natural language interaction to drill down, look at interesting facets, connect and correlate interesting information. Under the hood those tools use the same graph query language to do their job. This talk will cover both aspects of this exciting opportunity. We'll look at how to aggregate, import and connect information from disparate data-sources into a dynamic, flexible graph model and the means, tools and techniques for graph search and discovery on top of that fabric. The tools we use are the graph database Neo4j, its query language Cypher which also provides comprehensive import and data transformation abilities, as well as several tools and libraries from the Neo4j ecosystem to provide visual and textual graph search and exploration. Storyline Draft: http://gist.asciidoctor.org/?dropbox-14493611%2Ftalk_javazone.txt
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Graph Search and Discovery for your Dark Data
Similar to Graph Search and Discovery for your Dark Data (15)
More from Neo4j
More from Neo4j (20)
Recently uploaded
Recently uploaded (20)
Graph Search and Discovery for your Dark Data
- 11. The+world+is+a+graph+–+everything+is+connected+ • people,&places,&events& • companies,&markets& • countries,&history,&poli4cs& • sciences,&art,&teaching& • technology,&networks,&machines,&& applica4ons,&users& • so7ware,&code,&dependencies,&& architecture,&deployments&
- 15. Meet+Jan+
- 16. Kontext+is+King+
- 24. NODE& key:&“value”& proper4es& Property+Graph+Model+ Nodes& • The&en44es&in&the&graph& • Can&have&name?value&proper%es' • Can&be&labeled& RelaEonships& • Relate&nodes&by&type&and&direc4on& • Can&have&name?value&proper%es' RELATIONSHIP& NODE& NODE& key:&“value”& proper4es& key:&“value”& proper4es& key:&“value”& proper4es&
- 25. CAR& name:&“Jan”& last:&Hansson& born:&1978& name:&“Anne”& last:&Olsen& since:&& Jan&10,&2011& brand:&“Volvo”& model:&“V70”& Property+Graph+Model+&+Example+ Nodes& • The&en44es&in&the&graph& • Can&have&name?value&proper%es' • Can&be&labeled& RelaEonships& • Relate&nodes&by&type&and&direc4on& • Can&have&name?value&proper%es' LOVES& LOVES& LIVES&WITH& PERSON& PERSON&
- 28. Your+friend+Neo4j+ An&openLsource$graph$database& • Manager+and+store&your&connected+ data+as&a&graph+ • Query+relaEonships&& easily&and&quickly& • Evolve+model+and+applicaEons++ to&support&new&requirements&and& insights& • Built&to&solve&relaEonal+pains++ &
- 29. Your+friend+Neo4j+ An&openLsource$graph$database$ & • Built+for+RelaEonships+ • Open+Source+ • Java+&+Scala+ • High+Performance+ • ACIDSDatabase+ &
- 32. Graph+Query+Language:+Cypher+ &&&&&&&&&&&&&&(jan:Person&{&name:“Jan”}&)&?[:LOVES]?>&(anne:Person&{&name:“Anne”}&)&& LOVES& Jan+ Anne+ NODE& NODE& LABEL& PROPERTY&LABEL& PROPERTY& MATCH+
- 35. Cypher:+Clauses+ • WITH&expr&AS&alias,&...& • UNWIND&coll&AS&item& • LOAD&CSV&FROM&„url“&AS&row&
- 37. GeYng+Data+into+Neo4j+ Load+JSON+with+Cypher+ • Send&JSON&as¶meter& • Deconstruct&the&document& • Into&a&non?duplicated&graph&model& UNWIND%{json}.items%as%item% MERGE%(:Person%{name:item.name,%% %%%%%%%%%%%%%%%%%age:toInt(item.age)});%
- 38. GeYng+Data+into+Neo4j+ CSV+Bulk+Loader++++neo4j/import' • For&ini4al&database&popula4on& • For&loads&with&10B+&records& • Up&to&1M&records&per&second& bin/neo4jOimport%–Ointo%people.db%% %OOnodes:Person%people.csv% %OOrelationship:FRIEND_OF%friendship.csv%
- 40. Let‘s+LOAD+
- 43. Twi]er+ • Run&a&twiber&search,&exclude&Neo4j&sources& • „neo4j&OR&#neo4j&OR&@neo4j&& &?from:@neo4j&?from:@neotechnology“& • Pass&resul4ng&JSON&directly&to&Cypher& (:Person)O[:TWEETED]O>(:Tweet:Content)O[:TAGGED]O>(:Tag)% (:Tweet)O[:MENTIONS]O>(:Person)% (:Tweet)O[:RETWEET]O>(:Tweet)% (:Tweet)O[:REPLY]O>(:Tweet)% (:Tweet)O[:CONTAINS]O>(:Link)%
- 45. Data+imported&
- 46. Connect!+& • twiber&handle& • email& • website& • name& • tags& • url&
- 53. So_ware+AnalyEcs+ So7ware&is&connected&informa4on& • Source&?>&AST& • Inheritance,&Composi4on,&Delega4on& • Call&Trees& • Run4me&Memory& • Dependencies& • Modules,&Libraries& • Tests& • ...& & hbps://jqassistant.org&
- 54. jQAssistant+ • GeekCruise:&My&first&Neo4j&project& • So_ware+deteriorates+ • Develop&rules&and&enforce&them& • Commercial&Tools&too&inflexible& • Open&Source&So7ware&...&& • Scanner&?>&Enhancer&?>&Analyzer& • Enrichment,&Concepts&and&Rulez&in&Cypher& • Scanner&Plugins& • Integrate&in&Build&Process& • Fail,&Generate&Reports,&...& hbp://jqassistant.org&
- 58. Background& Business&problem& Solu4on&&&Benefits& ©&All&Rights&Reserved&2014&|&Neo&Technology,&Inc.& Industry:'''Communica%ons' Use'case:''Network'Management' Paris,&France& • Large&French&communica4ons&company& • Tons&of&various&networking&infrasctructure& • Has&to&shut&down&equipment&for&maintenance& • Who+is+impacted?+How+to+compensate?+ • Data&lives&in&30+&systems& • Took&a&week&to&print,&reseach,&inform&...& • Un4l&...& 5 8
- 60. Background& Business&problem& Solu4on&&&Benefits& ©&All&Rights&Reserved&2014&|&Neo&Technology,&Inc.& Industry:'''Communica%ons' Use'case:''Network'Management' Paris,&France& • Second&largest&communica4ons&company& in&France& • Part&of&Vivendi&Group,&& partnering&with&Vodafone& 6 0