The presentation I did for the FrontRange PHP User Group on 3/10/2010.

1. NoSQL Databases Jon Meredith [email_address]

3. NOT a product.

4. NOT a single technology.

6. Mostly created in response to scaling and reliability problems.

7. Huge differences between 'NoSQL' systems – but have elements in common.

9. Object databases

10. Graph databases

12. e-commerce

13. Social networking

17. All shambling under the NoSQL banner.

19. My application needs transactions

20. Data needs to be nicely normalized

21. I have replication for scalabilty/reliability

24. Sets

25. Arrays

28. Upgrade/rollback scripts have to operate on the whole database – could be millions of rows.

29. Doing phased rollouts is hard … the application needs to do work

32. Google's protocol buffers

34. Version objects

37. Every change on the master happens on the slave.

38. Slaves are read-only. Does not scale INSERT, UPDATE, DELETE queries.

39. Application responsible for distributing queries to correct server.

41. Updates travel around the ring

43. Add back in to the ring

45. Replication takes time – there is time lag between the first and last server to see an update.

46. You may not read your writes – not getting aCid properties any more.

52. The application needs to know how to resolve it

54. ...Available...

55. ...Maintainable...

56. with an RDBMs requires large efforts by application developers and operational staff

58. App needs to know data location

59. App needs to handle failover

68. Network partitions common

71. Network routes are flapping

72. Data centers are being destroyed by tornadoes

75. Best seller lists

78. Fault tolerant: Keeps N copies of the data

79. Designed for inconsistency

80. Totally decentralized – nodes 'gossip' state

81. Self-healing

83. Availability

85. Amazon chose A-P over C

87. Read operations (get) require 'R' nodes to respond

88. Write operations (put) require 'W' nodes to respond

89. If R+W > N nodes will read their writes (if no failure)

90. NRW tunes the cluster – typically (3,2,2)

94. Dynamo minimizes with vector clocks

95. Vector Clocks

96. Partitioning

98. Shopping Cart - Conflict Network Failure

99. Shopping Cart - Merge

102. Project Voldemort

104. Google Earth

107. Table indexed by {key,timestamp} and a variable number of sparse columns

108. Columns are grouped into column families. Columns in a family are stored together.

109. Each table is broken into tablets.

110. Tablets are stored on a cluster file system (GFS).

111. BigTable – Column Families Copyright Google

113. Programmers write two functions map() and reduce().

114. Table is mapped, then reduced.

115. Job control system monitors and resubmits.

116. Map/Reduce Source: institutes.lanl.gov

119. CouchDB Map/Reduce

122. http://www.vineetgupta.com/ 2010/01/nosql-databases-part-1-landscape.html

123. So many projects! Dynamo, BigTables, Redis Riak, Voldemort, CouchDb, Peanuts Hadoop/Hbase, Cassandra, Hypertable MongoDb, Terrastore, Scalaris, BerkleyDB MemcacheDB, Dynomite, Neo4J, TokyoCabinet … and more

125. Sparse Column Family

128. Decentralized

130. RESTful HTTP interface

131. Fully distributed

132. Clients for multiple languages

134. Filesystem

137. Key/Value Store with structured values

138. Written in C

139. Memcache-like protocol

141. Engine Yard

142. VideoWiki

144. Operations like increment, decrement, intersection, push, pop

145. In-memory (can be backed by disk)

146. Auto-sharding in client libraries

147. No fault tolerance (coming after 2.0)

148. Example: retwis – Twitter clone in PHP

150. BigTable ColumnFamily data model

151. Dynamo data distribution

152. Written in Java

153. Thrift based interface

155. Twitter

157. Used by Ubuntu One

158. HTTP interface

159. Uses Javascript for indexing/mapreduce

160. Incremental replication

163. Multi-process

164. Replicated

167. Neo4J – Graph Database

168. Peanuts – Yahoo

170. Even range queries are hard for distributed hash systems.

171. No transactions – rules out some classes of applications.

172. Space is still evolving

174. They force you to think about distributed design issues like consistency.

175. Play with them!