TITANBIG GRAPH DATA WITH CASSANDRA#TITANDB
#GRAPHDB #CASSANDRA12Matthias Broecheler, CTO AURELIUSAugust VIII, MMXII THINKAURELIUS.COM

AbstractTitan is an open source
distributed graph database build on top ofCassandra that can power real-time applications with thousands ofconcurrent users over graphs with billions of edges. Graphs are a versatiledata model for capturing and analyzing rich relational structures. Graphsare an increasingly popular way to represent data in a wide range ofdomains such as social networking, recommendation engines,advertisement optimization, knowledge representation, health care,education, and security.This presentation discusses Titans data model, query language, and noveltechniques in edge compression, data layout, and vertex-centric indiceswhich facilitate the representation and processing of Big Graph Dataacross a Cassandra cluster. We demonstrate Titans performance on alarge scale benchmark evaluation using Twitter data.

Titan Graph Database  supports
real time local traversals (OLTP)  is highly scalable  in the number of concurrent users  in the size of the graph  is open source under the Apache2 license  builds on top of Apache Cassandra for distribution and replication

IThe Graph Data Model AURELIUS
THINKAURELIUS.COM

Hercules: demigodAlcmene: humanJupiter: godSaturn: titanPluto:
godNeptune: godCerberus: monster Entities

Name TypeHercules demigodAlcmene humanJupiter godSaturn
titanPluto godNeptune godCerberus monster Table

Name: Name: Name: Name:Hercules Alcmene
Jupiter SaturnType: Type: Type: Type:demigod human god titanName: Name: Name:Pluto Neptune CerberusType: Type: Type:god god monster Documents

Hercules type:demigodAlcmene type:humanJupiter type:godSaturn type:titanPluto
type:godNeptune type:godCerberus type:monster Key->Value

name: Neptune name: Alcmene type:
god type: godVertex Property name: Saturn name: Jupiter name: Hercules type: titan type: god type: demigod name: Pluto name: Cerberus type: god type: monster Graph

name: Neptune name: Alcmene type:
god type: godEdge brother mother name: Saturn name: Jupiter name: Hercules type: titan type: god type: demigod father father Edge battled brother time:12 Property name: Pluto name: Cerberus type: god type: monster Edge Type pet Graph

IGraph = Agile Data Model

IIGraph Use Cases AURELIUS THINKAURELIUS.COM

Recommendations

Recommendation?name: Hercules

name: “Muscle building for beginners”name:
Hercules type: book bought

name: “Muscle building for beginners”name:
Hercules type: book bought boughtname: Newton

name: “Muscle building for beginners”name:
Hercules type: book bought bought name: “How to deal with Father issues”name: Newton type: book bought

name: “Muscle building for beginners”
name: Hercules type: book boughtrecommend bought name: “How to deal with Father issues” name: Newton type: book bought Traversal

name: “Dancing with the Stars”
type: DVD in-Cart name: “Muscle building for beginners”name: Hercules type: book bought bought name: “How to deal with Father issues”name: Newton type: book bought viewed name: “Friends forever bracelet” type: Accessory

name: “Dancing with the Stars”
type: DVD in-Cart name: “Muscle building for beginners” name: Hercules type: book boughtfriends bought name: “How to deal with Father issues” name: Newton type: book bought viewed name: “Friends forever bracelet” type: Accessory

name: “Dancing with the Stars”
type: DVD in-Cart name: “Muscle building for beginners” name: Hercules type: book bought time:24friends bought time:22 name: “How to deal with Father issues” name: Newton type: book bought time:20 viewed name: “Friends forever bracelet” type: Accessory

Recommendations Path Finding

name: Neptune name: Alcmene type:
god type: god X brother mother name: Saturn name: Jupiter name: Hercules type: titan type: god type: demigodX father father battled brother time:12 name: Pluto name: Cerberus type: god type: monster pet Path Finding

name: Neptune name: Alcmene type:
god type: god X brother mother name: Saturn name: Jupiter name: Hercules type: titan type: god type: demigodX father father battled brother time:12 name: Pluto name: Cerberus type: god type: monster pet Path Finding

yahoo.com geocities.com cnn.com /johnlittlesite<html> …
<html> …</html>! <html> … </html>! </html>! Credibility?

url: yahoo.com url: cnn.comhtml: <html>…!
html: <html>…! url: geocities.com/johnlittlesite Link html: <html>…! Graph

url: yahoo.com url: cnn.comhtml: <html>…!
html: <html>…! elections funny cat foreign policy url: geocities.com/johnlittlesite Link html: <html>…! Graph

IIGraph = Milk Your Connections

IIIThe Titan Graph Database AURELIUS
THINKAURELIUS.COM

Titan Features numerous concurrent users
real-time traversals (OLTP) high availability dynamic scalability built on Apache Cassandra

Titan Ecosystem Native Blueprints Graph
Server Implementation Graph Algorithms Gremlin Query Object-Graph Mapper Language Traversal Language Rexster Server Dataflow Processing  any Titan graph can be exposed Generic Graph API as a REST endpoint

Titan InternalsI. Data ManagementII. Edge
CompressionIII. Vertex-Centric Indices

IVRebuilding Twitter with Titan AURELIUS
THINKAURELIUS.COM

text: string name: string! time:
long! follows User Tweettime: long! tweets time: long!

time: long! stream text: string
name: string! time: long! follows User Tweettime: long! tweets time: long!

Titan Storage Model Adjacency list
in one 5 column family Row key = vertex id Each property and edge 5 in one column  Denormalized, i.e. stored twice Direction and label/key as column prefix  Use slice predicate for quick retrieval

Connecting Titantitan$ bin/gremlin.sh! ,,,/! (o
o)!-----oOOo-(_)-oOOo-----!gremlin> conf = new BaseConfiguration();!==>org.apache.commons.configuration.BaseConfiguration@763861e6!gremlin> conf.setProperty("storage.backend","cassandra");!gremlin> conf.setProperty("storage.hostname","77.77.77.77");!gremlin> g = TitanFactory.open(conf);==>titangraph[cassandra:77.77.77.77]!gremlin>!

Defining Property Keysgremlin> g.makeType().name(“time”).! !
! dataType(Long.class).! ! ! functional().! ! ! makePropertyKey();!gremlin> g.makeType().name(“text”).dataType(String.class).! ! ! functional().makePropertyKey();!gremlin> g.makeType().name(“name”).dataType(String.class).! ! ! indexed().! ! ! unique().! ! ! functional().makePropertyKey();!

Defining Property Keys Each type
has a unique namegremlin> g.makeType().name(“time”).! The allowed data type ! ! dataType(Long.class).! ! ! functional().! If a key is functional, each vertex can ! ! makePropertyKey();! have at most one property for this keygremlin> g.makeType().name(“text”).dataType(String.class).! ! ! functional().makePropertyKey();!gremlin> g.makeType().name(“name”).dataType(String.class).! ! ! indexed().! ! ! unique().! ! ! functional().makePropertyKey();!

Defining Property Keysgremlin> g.makeType().name(“time”).! !
! dataType(Long.class).! ! ! functional().! ! ! makePropertyKey();!gremlin> g.makeType().name(“text”).dataType(String.class).! ! ! functional().makePropertyKey();!gremlin> g.makeType().name(“name”).dataType(String.class).! ! ! indexed().! Creates and maintains an index over property values ! ! unique().! Ensures that each property value is uniquely ! ! functional().makePropertyKey();! associated with only one vertex by acquiring a lock.

Titan Indexing Vertices can be
retrieved by property key + value name : Hercules 5 Titan maintains index in a name : Jupiter 9 separate column family as graph is updated Only need to define a property key as .index()

Titan Locking Locking ensures consistency
when it is needed name : Hercules 5 Titan uses time stamped quorum reads and writes on 9 separate CFs for locking Uses name : name : Jupiter Hercules  Property uniqueness: .unique() father  Functional edges: .functional()  Global ID management x name : father Pluto

Defining Edge Labelsgremlin> g.makeType().name(“follows”).! !
! primaryKey(time).! ! ! makeEdgeLabel();!gremlin> g.makeType().name(“tweets”).! ! ! primaryKey(time).makeEdgeLabel();!gremlin> g.makeType().name(“stream).! ! ! primaryKey(time).! ! ! unidirected().! ! ! makeEdgeLabel();!

Defining Edge Labelsgremlin> g.makeType().name(“follows”).! !
! primaryKey(time).! Sort/index key for edges of this label ! ! makeEdgeLabel();!gremlin> g.makeType().name(“tweets”).! ! ! primaryKey(time).makeEdgeLabel();!gremlin> g.makeType().name(“stream).! ! ! primaryKey(time).! ! ! unidirected().! ! ! makeEdgeLabel();!

Defining Edge Labelsgremlin> g.makeType().name(“follows”).! !
! primaryKey(time).! ! ! makeEdgeLabel();!gremlin> g.makeType().name(“tweets”).! ! ! primaryKey(time).makeEdgeLabel();!gremlin> g.makeType().name(“stream).! ! ! primaryKey(time).! ! ! unidirected().! Store edges of this label only in outgoing direction ! ! makeEdgeLabel();!

Vertex-Centric Indices Sort and index
edges per vertex by primary key  Primary key can be composite Enables efficient focused traversals  Only retrieve edges that matter Uses slice predicate for quick, index-driven retrieval

tweets tweets tweetstime: 123 time:
334 time: 624 v.query()! tweets v time: 1112 follows follows follows follows

tweets tweets tweetstime: 123 time:
334 time: 624 v.query()! tweets .direction(OUT)! v time: 1112 follows follows

tweets tweets tweetstime: 123 time:
334 time: 624 v.query()! tweets .direction(OUT)! v time: 1112 .labels(“tweets”)!

v.query()! tweets .direction(OUT)!v time: 1112
.labels(“tweets”)! .has(“time”,T.gt,1000)!

name: Hercules name: PlutoCreate Accounts
gremlin> hercules = g.addVertex([name:Hercules]);! gremlin> pluto = g.addVertex([name:Pluto]);!

name: Hercules name: PlutoAdd Followship
follows time:2 gremlin> hercules = g.addVertex([name:Hercules]);! gremlin> pluto = g.addVertex([name:Pluto]);! gremlin> g.addEdge(hercules,pluto,"follows",[time:2]);!

name: Hercules name: PlutoPublish Tweet
follows time:2 tweets time:4 text: A tweet! time: 4! gremlin> hercules = g.addVertex([name:Hercules]);! gremlin> pluto = g.addVertex([name:Pluto]);! gremlin> g.addEdge(hercules,pluto,"follows",[time:2]);! gremlin> tweet = g.addVertex([text:A tweet!,time:4])! gremlin> g.addEdge(pluto,tweet,"tweets",[time:4]) !

name: Hercules name: PlutoUpdate Streams
follows time:2 stream tweets time:4 time:4 text: A tweet! time: 4! gremlin> hercules = g.addVertex([name:Hercules]);! gremlin> pluto = g.addVertex([name:Pluto]);! gremlin> g.addEdge(hercules,pluto,"follows",[time:2]);! gremlin> tweet = g.addVertex([text:A tweet!,time:4])! gremlin> g.addEdge(pluto,tweet,"tweets",[time:4]) ! gremlin> pluto.in("follows").each{g.addEdge(it,tweet,"stream",[time:4])} !

name: Hercules name: PlutoRead Stream
follows time:2 stream tweets time:4 time:4 text: A tweet! time: 4! gremlin> hercules = g.addVertex([name:Hercules]);! gremlin> pluto = g.addVertex([name:Pluto]);! gremlin> g.addEdge(hercules,pluto,"follows",[time:2]);! gremlin> tweet = g.addVertex([text:A tweet!,time:4])! gremlin> g.addEdge(pluto,tweet,"tweets",[time:4]) ! gremlin> pluto.in("follows").each{g.addEdge(it,tweet,"stream",[time:4])} ! gremlin> hercules.outE(stream)[0..9].inV.map! Sorted by time because its ‘stream’s primary key

Followship name: Hercules name: PlutoRecommendation
follows time:2 follows follows time:9 name: Neptune follows = g.V(name,’Hercules’).out(follows).toList()! follows20 = follows[(0..19).collect{random.nextInt(follows.size)}]! m = [:]! follows20.each ! { it.outE(follows’[0..29].inV.except(follows).groupCount(m).iterate() }! m.sort{a,b -> b.value <=> a.value}[0..4]!

IVTitan Performance Evaluation onTwitter-like Benchmark
AURELIUS THINKAURELIUS.COM

Twitter Benchmark 1.47 billion followship
edges and 41.7 million users  Loaded into Titan using BatchGraph  Twitter in 2009, crawled by Kwak et. al 4 Transaction Types  Create Account (1%)  Publish tweet (15%)  Read stream (76%)  Recommendation (8%) Kwak, H., Lee, C., Park, H., Moon, S., “What is  Follow recommended user (30%) Twitter, a Social Network or a News Media?,” World Wide Web Conference, 2010.

Benchmark Setup 6 cc1.4xl Cassandra
nodes  in one placement group  Cassandra 1.10 40 m1.small worker machines  repeatedly running transactions  simulating servers handling user requests EC2 cost: $11/hour

Benchmark ResultsTransaction Type Number of
tx Mean tx time Std of tx timeCreate account 379,019 115.15 ms 5.88 msPublish tweet 7,580,995 18.45 ms 6.34 msRead stream 37,936,184 6.29 ms 1.62 msRecommendation 3,793,863 67.65 ms 13.89 ms Total 49,690,061 Runtime 2.3 hours 5,900 tx/sec

Peak Load ResultsTransaction Type Number
of tx Mean tx time Std of tx timeCreate account 374,860 172.74 ms 10.52 msPublish tweet 7,517,667 70.07 ms 19.43 msRead stream 37,618,648 24.40 ms 3.18 msRecommendation 3,758,266 229.83 ms 29.08 ms Total 49,269,441 Runtime 1.3 hours 10,200 tx/sec

Benchmark ConclusionTitan can handle 10s
of thousands of concurrent users with short response 5mes even for complex traversals on a simulated social networking applica5on based on real-­‐world network data with billions of edges and millions of users in a standard EC2 deployment. For more informa5on on the benchmark: hDp://thinkaurelius.com/2012/08/06/5tan-­‐provides-­‐real-­‐5me-­‐big-­‐graph-­‐data/

Future Titan Titan+Cassandra embedding 
sending Gremlin queries into the cluster Graph partitioning together with ByteOrderedPartitioner  data locality = better performance Let us know what you need!

Titan goes OLAP Map/Reduce Load
& Compress Analysis results back into Titan Stores a massive-scale Batch processing of large Runs global graph algorithmsproperty graph allowing real- graphs with Hadoop on large, compressed, time traversals and updates in-memory graphs

IIIGraph = Scalable + Practical

TITANTHINKAURELIUS.GITHUB.COM/TITAN

AURELIUSTHINKAURELIUS.COM