Intro to NoSQL, Cassandra and Hector I gave at Globant Laminar in Buenos Aires Argentina Dec 13th 2012.

1. Introduction to NoSQL and
Apache Cassandra
Patricio Echagüe
patricioe@gmail.com
@patricioe

2. About me
Present:
Relateiq (Data Processing and Scalability)
Hector committer
Past:
DataStax (The Cassandra Company)
Cassandra/Hadoop distribution (former Brisk)
Cassandra FS
CQL connection pool
Cassandra contributions

3. Trends: “NoSQL”

4. 2011

5. 2012

6. What is “NoSQL” ?
systems able to store and retrieve great
quantities of data with none or little
information about the relationships
between them.
Generally they don't have a SQL like
language for data manipulation and
their schema is more relaxed than
traditional RDBM systems.
Full ACID is not often guaranteed.

7. Brewer's CAP theorem
Consistency: all replicas agree on the
same value
Availability: always get an answer from
a replica
Partition Tolerance: the system works
even if replicas can't talk
You can have 2 of these

8. Brewer's CAP theorem

9. CAP Classification
Consistency
Availability Partitioning

10. Types
- Relationals
- Key-Value stores
- Columnar (column-oriented)
- Graph databases
- Document

11. What's eventual consistency?
It is a promise that eventually, in the
absence of new writes, all replicas that
are responsible for a data item will
agree on the same version

12. How eventual is eventual?
Write to 1 replica and Read from 1 replica of a total
of 3

13. How eventual is eventual?
Write to 2 replicas and Read from 2 replicas of a total
of 3

14. Why is it good?
because, by contacting fewer
replicas, read and write operations
complete more quickly, lowering
latency.

15. Cassandra is a distributed
, fault
tolerant, scalable, column
oriented and tunable
consistency data store.

16. Cassandra has
CAP
But C is tunable

17. What is Apache Cassandra?

18. Key Concepts
Multi-Master, Multi-DC
Linearly scalable
Integrated Caching
Performs well with Larger-than-memory Datasets
Tunable consistency
Idempotent (client clock)
Schema Optional
No ACID transactions, No Locking

19. Generally complements another system(s)
(Not intended to be one-size-fits-all)
You should always use the right tool for the right job

20. Speaking Cassandra

21. Data Model
“4-Dimensional Hash Table”
A Keyspace contains a collection of Column Families
(Controls replication)
A Column Family contains Rows
A Row have a key, and each row has columns
(No need to define the columns before hand)
Each column has a name and a value and a
timestamp
(TTL is optional)

22. Data Model – (RDBMS)
Keyspace (Schema)
Column Family(CF) (table)
Row (row)
Column (column*) → may not be present in all
rows

23. Data Model – Column Family
Static Column Family
- Model my object data
Dynamic Column Family
- Precalculated / Prematerialized query results
Nothing stopping you from mixing them!

24. Data Model – Static Column Family

25. Data Model – Dynamic CF
stats for a specific date

26. Data Model – Dynamic CF
Timeline of tweets by a user
Timeline of tweets by all of the people a user is
following
List of comments sorted by score
List of friends grouped by state
Metrics for a time bucket

27. ...
Let's store “foo”

28. ...
Let's store “foo”
Foo

29. …
But if that node is down?
Foo

30. ...
Let's store “foo” in 3 nodes.
This is the Replication Factor(N)
Foo
Foo
Foo

31. ...
Now we need to know what nodes the key was written
to so we can read it later

32. ...
The Initial Token specifies the upper value of the key
range each node is responsible for
#1
#5 <= 'd'
<= 'z' 'e f g h I j k '
#2
<= 'k'
#4
<= 'u'
#3
<= 'p'
a b c d e f g h I j k l m n …. z

33. ...
Gossip is the protocol Cassandra uses to interchange
information with nodes in the cluster (a.k.a. Ring)

34. …
Gossip is the protocol Cassandra uses to interchange
information with nodes in the cluster (a.k.a. Ring)
For example, what nodes owns the key “foo”

35. ...
Gossip is the protocol Cassandra uses to interchange
information with nodes in the cluster (a.k.a. Ring)
For example, what nodes owns the key “foo”
#1
Read 'foo'
#5 <= 'd'
Client 'e f g h I j k '
<= 'z'
#2
'foo'
<= 'k'
#4
<= 'u'
#3
<= 'p'

36. ...
A Partitioner is used to transform the key.
“foo1” and “foo2” may end up in different nodes

37. ...
A Partitioner is used to transform the key.
“foo1” and “foo2” may end up in different nodes
The most commonly used is Random Partitioner
“foo1” md5(“foo1”) “A99A0B....”

38. ...
A Partitioner is used to transform the key.
“foo1” and “foo2” may end up in different nodes
The most commonly used is Random Partitioner
#1 'foo1'
#5
#2
'foo2'
#4
#3

39. ...
A Replica Placement Strategy determines which
nodes contain replicas

40. ...
A Replica Placement Strategy determines which
nodes contain replicas
Simple Strategy place them clockwise
'foo1'
#1
#5
'foo1'
#2
#4
#3 'foo1'

41. ...
A Replica Placement Strategy determines which
nodes contain replicas
Network Topology Strategy place them in different
DCs
DC1:3 DC2:1
'foo1'
#1 #1 'foo1'
#5 #5
'foo1'
#2 #2
#4 #4
#3 #3
'foo1'

42. ...
Consistency Level determines how many replicas to
contact to

43. ...
Consistency Level determines how many replicas to
contact to
CL = 1
#1 'foo1'
Client #
5
'foo1'
#2
#
4
#3 'foo1'

44. ...
Consistency Level determines how many replicas to
contact to
CL = QUORUM
#1 'foo1'
Client #
5
'foo1'
#2
#
4
#3 'foo1'

45. Consistency For Writes
ANY
ONE
TWO
THREE
QUORUM
LOCAL_QUORUM
EACH_QUORUM
ALL

46. Consistency For Reads
ONE
TWO
THREE
QUORUM
LOCAL_QUORUM
EACH_QUORUM
ALL

47. Consistency In Math Term
Cassandra guarantees strong consistency if
(nodes_written + nodes_read) >
replication_factor
R+W>N

48. Back to the example..
Consistency Level determines how many replicas to
contact to
CL = QUORUM
#1 'foo1'
Client #
5
'foo1'
#2
#
4
#3 'foo1'

49. ...
But what if node #3 is down?

50. ...
But what if node #3 is down?
hint
#1 'foo1'
Client #
5
'foo1'
#2
#
4
#3

51. ...
But what if node #3 is down?
The coordinator nodes will store a hint and will replay
that mutation when the down node comes back up.
This is known as Hinted Handoff

52. ...
Node #5 will replay the hint to node #3 when it comes
back online
hint
'foo1'
#1
Client #5
'foo1'
#2
#4
#3 'foo1'

53. ...
And if node #5 dies before sending the hints to node
#3?
hint
#1 'foo1'
Client #5
'foo1'
#2
#4
#3

54. ...
If using Quorum, node #4 will request for 'foo' to all
the replicas
hint
#1 'foo1'
Client #5
'foo1'
#2
#4
#3 ''

55. ...
If the result received do not match, a Read Repair
process is performed in the background
hint
#1 'foo1'
Client #5
'foo1'
#2
#4
#3 ''

56. ...
And the missing or not up-to-date value is pushed to
the out of date node. #3 in this case
hint
#1 'foo1'
Client #5
'foo1'
#2
#4
'foo' != '' #3 'foo'

57. ...
The last feature to achieve consistency is the Anti
Entropy Service (AES)
Should run periodically as part of the cluster
maintenance or when a node was down

58. Recap Consistency Features
Read Repair
Anti Entropy Service (AES)
Hinted Handoff

59. scaling
“e”
“z”
“j”
“t”
“o”

60. scaling
“e”
“?”
“z”
“j”
“t”
“o”

61. scaling
“e”
“z”
“g”
“j”
“t”
“o”
Nodetool move ?

62. Want 2x performance ?!
Add 2x nodes
'No downtime' included!

63. Want 2x performance ?!
“e”
“z”
“j”
“t”
“o”

64. Want 2x performance ?!
“b”
“e”
“z”
“g”
“v”
“j”
“t”
“l”
“q” “o”

65. With RF= 3 we could lose
“b”
“e”
“z”
X “g”
X
“v”
“j”
“t”
X
“l”
“q” “o”

66. With RF= 3 we could lose
?
“b”
“e”
X
“z”
X “g”
X“v”
“j”
“t”
X
“l”
“q” “o”

67. Vs others
b e
z
g
v
j
t l
q o

68. Recap
Replication Factor
Tokens
Gossip
Partitioner
Replica Placement
Consistency
Hinted Handoff
Read Repair
AES
Clustering

70. Performance
Reads on par with writes

71. Scalability

72. Internals

73. Read and Write path

74. Storage - SSTable
- SSTables are sorted
- Immutable (“Merge on read”)
- Newest timestamp wins

75. Storage – Compaction

76. Storage – Compaction
Merges SSTables together into a larger SSTables
Removes Tombstones
Rebuild primary and secondary indexes

77. Storage – Compaction
Two types:
- Size-tiered compaction
- Leveled compaction

78. Storage – Compaction
Size-tiered compaction
Performance no guaranteed
Row may be across many SSTables
Waste of space
Good for write heavy ops
Rows are written once
100% more space than SSTables

79. Storage – Compaction
Leveled compaction
Grouped into levels
No overlapping within a level
Each level is ten times as large
90% of reads satisfied with 1 SSTable
Twice as much I/O

80. Recap
SSTable
Memtable
Row Cache
Compaction

81. SSDs and caching
Before - 48 Cassandra on m2.4xlarge. 36 EVcache on
m2.xlarge
After - 12 Cassandra on hi1.4xlarge

82. API Operations

83. Five general categories
Retrieving
Write/Update/Remove (all the same op!)
Increment counters
Meta Information
Schema Manipulation
CQL Execution

84. Insertion/Deletion => Mutation
Again: Every mutation is an insert!
- Merge on read
- Sstables are immutable
- Highest timestamp wins

85. CQL
INSERT INTO Hollywood.NerdMovies (user_uuid, fan)
VALUES ('cfd66ccc-d857-4e90-b1e5-df98a3d40cd6', 'johndoe')
USING CONSISTENCY LOCAL_QUORUM AND TTL 86400;

86. Hadoop

87. Using a Client
- Hector
http://hector-client.org
- Astyanax
https://github.com/Netflix/astyanax
- Pelops
https://github.com/s7/scale7-pelops

88. Using a Client → Hector
- Most popular Java client
- In use at very large installations
- A number of tools and utilities built on top
- Very active community
- MIT Licensed

89. Features
- High Level API
- Failover behavior
- High performant connection pool
- JMX counters for management
- Discoverability of new nodes
- Automatic retry of downed hosts
- Suspension of nodes after several timeouts
- Load Balancing: Configurable and extensible
- Locking (Beta)

90. Hector's Architecture

91. vs JDBC
Hector is operation-oriented
Whereas
JDBC is connection-oriented

92. API Abstractions
Templates
Mutator
Thrift

93. ColumnFamilyTemplate
Familiar, type-safe approach
- based on template-method design pattern
- generic: ColumnFamilyTemplate<K,N>
(K is the key type, N the column name type)
ColumnFamilyTemplate template =
new ThriftColumnFamilyTemplate(keyspaceName,
columnFamilyName,
StringSerializer.get(),
StringSerializer.get());
*** (no generics for clarity)

94. ColumnFamilyTemplate
new ThriftColumnFamilyTemplate(
keyspaceName,
columnFamilyName,
StringSerializer.get(),
Key Format
StringSerializer.get());
Column Name Format
- Cassandra calls this a “comparator”
- Remember: defines column order in on-disk format

95. ColumnFamilyTemplate
ColumnFamilyResult<String, String> res =
cft.queryColumns("patricioe");
String value = res.getString("email");
Date startDate = res.getDate(“DateOfBirth”);
Key Format
Column Name Format

96. ColumnFamilyTemplate
Inserting data with ColumnFamilyUpdater
ColumnFamilyUpdater updater = template.createUpdater(”pato");
updater.setString("companyName",”Relateiq");
updater.addKey(”sabina");
updater.setString("companyName",”Globant");
template.update(updater);

97. ColumnFamilyTemplate
Deleting Data with ColumnFamilyTemplate
template.deleteColumn("zznate", "notNeededStuff");
template.deleteColumn("zznate", "somethingElse");
template.deleteColumn("patricioe", "aDifferentColumnName");
...
template.deleteRow(“someuser”);
template.executeBatch();

98. Integrating with existing patterns
Hector Object Mapper -> Apache Gora
https://github.com/hector-client/hector/tree/master/object-mapper
Hector JPA*:
https://github.com/riptano/hector-jpa
Spring IOC
CQL: JDBC Driver and Pool in 1.0!
JdbcTemplate FTW!

99. Development Resources
Hector Documentation (http://hector-client.org)
Cassandra Unit
https://github.com/jsevellec/cassandra-unit
Cassandra Maven Plugin
http://mojo.codehaus.org/cassandra-maven-plugin/
CCM localhost cassandra cluster
https://github.com/pcmanus/ccm
OpsCenter
http://www.datastax.com/products/opscenter
Cassandra AMIs
https://github.com/riptano/CassandraClusterAMI

100. Want to contribute?
git clone git@github.com:hector-client/hector.git

101. Summary
- Take advantage of strengths
- idempotence and asynchronicity are your friends
- If it's not in the API, you are probably doing it wrong
- Seek death is still possible if you model incorrectly
- Try Denormalizing (append-only model ?)

102. Patricio Echagüe
patricioe@gmail.com
@patricioe

103. Credits
Nate McCall
Aaron Morton (http://thelastpickle.com)
Datastax (http://www.datastax.com)
http://www.slideshare.net/mikiobraun/cassandra-an-introduction

104. Additional Resources
DataStax Documentation: http://www.datastax.com/docs
Apache Cassandra project wiki: http://wiki.apache.org/cassandra/
“The Dynamo Paper”
http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
P. Helland. Building on Quicksand
http://arxiv.org/pdf/0909.1788
P. Helland. Life Beyond Distributed Transactions
http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf
S. Anand. “Netflix's Transition to High-Availability Storage Systems”
http://media.amazonwebservices.com/Netflix_Transition_to_a_Key_v3.pdf
“The Megastore Paper”
http://research.google.com/pubs/archive/36971.pdf