SlideShare a Scribd company logo
1 of 37
Download to read offline
Data management for interactive web and mobile applications.




DALE HARVEY




                                                                    1
RELAX




@perezd
What is Couchbase?




       1+1 really does equal 3



                                 3
WHAT IS MEMBASE?




                   4
Deployments Leading Membase




  Leading cloud service (PAAS)    Social game leader – FarmVille,
  provider                        Mafia Wars, Café World
  Over 65,000 hosted              Over 230 million monthly users
  applications                    Membase Server
  Membase Server serving over     is the 500,000 ops-per-second
  1,200 Heroku customers (as of   database behind FarmVille and
  June 10, 2010)                  Café World
                                                                    5
Ad targeting

                            40 milliseconds to come
                            up with an answer.




                        3 profiles, real time campaign
                          statistics




                                      2
                 1
                                    profiles, campaigns
               events

                                                      6
Membase is a distributed database




                             Application user




 Web application server   Web application server   Web application server




                           Membase Servers




                 In the data center                                         On the administrator console



                                                                                                           7
Membase is Simple, Fast, Elastic

               Five minutes or less to a working
               cluster
               • Downloads for Linux and Windows
               • Start with a single node
               • One button press joins nodes to a cluster
               Easy to develop against
               • Just SET and GET – no schema required
               • Drop it in. 10,000+ existing applications
                 already “speak membase” (via memcached)
               • Practically every language and application
                 framework is supported, out of the box
               Easy to manage
               • One-click failover and cluster rebalancing
               • Graphical and programmatic interfaces
               • Configurable alerting



                                                              8
Data is sharded into vBuckets




                                9
Clustering With Membase

                  SET request arrives at          SET acknowledgement
                    KEY’s master server 1   2 returned to application




                           3                                                    3
                                  Listener-Sender

                                  RAM




                                                   membase storage engine
                                                                            4


                                 Disk Disk Disk



                                 Disk Disk Disk


Replica Server 1 for KEY        Master server for KEY                               Replica Server 2 for KEY
                                                                                                          10
Membase is Simple, Fast, Elastic

               Zero-downtime elasticity
               • Spread I/O and data across commodity
                 servers (or VMs)
               • Consistent performance with linear cost
               • Dynamic rebalancing of a live cluster
               All nodes are created equal
               • No special case nodes
               • Any node can replace any other node, online
               • Clone to grow
               Extensible
               • Filtered TAP interface provides hook points
                 for external systems (e.g. full-text search,
                 backup, warehouse)
               • Data bucket – engine API for specialized
                 container types



                                                                11
WHAT IS CouchDB?




                   12
Robust
      JSON
Replication
Map Reduce
 GeoCouch
     HTTP

              13
USE CASES




            14
DOCUMENT STORE
=> GET http://127.0.0.1:5984/mydatabase/
uniqueid
=> PUT http://127.0.0.1:5984/mydatabase/
uniqueid http://127.0.0.1:5984/mydatabase
=> POST
=> DELETE http://127.0.0.1:5984/mydatabase/
uniqueid
{
    “_id”   : “uniqueid”,
    “_rev” : 1234567,
    “title” : “My New Blog Post”,
    “body” : “A quick brown fox .......”,
    “_attachments”: {
      “profile.jpg”: { ...... }
    }
}
                                              15
ROBUST STORAGE ENGINE

Append-Only
Reliable & Concurrent
Hot Backups
Compaction

                          16
INCREMENTAL MAP REDUCE




                         17
PEER-BASED REPLICATION

curl -X POST http://127.0.0.1:5984/_replicate -d
'{
   "source":
     "http://couch.example.com:5984/remote-db",
   "target":
     "local-db"
}'




                                              18
NAILING MOBILE
       • Extremely Robust Storage Engine
          Consumer devices may lose power at any time
       • Lightweight Runtime
          Small install, low memory footprint, easy on
          battery life
       • Offline sync via Replication
          Multi-master synchronization engine
       • GeoSpatial Index
          Mapping and location aware apps
       • HTML5 Application Server
          Easy to add a web view to your mobile data

https://github.com/couchbaselabs/iOS-Couchbase


                                                         19
GEOCOUCH
      CIVICAPI.COM, ETC




Advanced R-Tree Index
                          20
ARCHITECTURE
2.0 Architecture Overview

                           Application



                      Couchbase SDK API


                     MCD                 Couch

                                                 Application Tier



                                                 Elastic Couchbase Server Cluster


         Moxi     Memcached              Couch
                                         Proxy

                  ep_engine

        Cluster
        Manager



                         Couchbase Server


                  vBucket Data       View Data




                                                                                    22
JSON Document Handling

• Per Bucket JSON Document Mode toggle

• JSON Document Mode disabled (or invalid
  JSON)
  – Values are opaque to couch (and in views)
  – Stored as attachments to documents that hold
    metadata
  – Returned values are guaranteed bitwise identical


• JSON Document Mode enabled
  – Values are expected to be valid JSON
  – Stored in document along metadata
                                                       5
  – Returned values may not be bitwise identical but
JSON Document Encodings for memcached
                                                       Couch Document
              Meta Data
      Key:”zdfasf”
      Expiration:43452345            {
      Flags:2352145                       “_id”: ”zdfasf”
                                          “$Expiration”: 43452345,
                                          “$Flags”: 2352145,
                                          “$AttachReason”:”JSON Mode Disabled”, //Other
                Value                reasons “Invalid JSON”, etc.
                                          [“$KeyMangled”: false,] //only included if true
                                           “_attachments”: { “$value”: { … } }
     0x87632f98,0xa76b876d, …!       }




                                                       Couch Document
             Meta Data
      Key:”zdfasf”
      Expiration:43452345
      Flags:2352145              {
                                         “_id”: ”zdfasf”
                                         “$Expiration”: 43452345,
                                         “$Flags”: 2352145,
           JSON Document                 [“$KeyMangled”: false,] //only included if true
                                          “foo”: ”bar”
       {                         }
              “foo”: ”bar”
       }




                                                                                            24
Scatter Gather Map Reduce




• Get next answer from local vBuckets and from
  child nodes to select the next result for a node
• Result includes info on number of vBuckets not
  included
  – Timeouts, Incorrect Design Doc version, etc   8
Scatter Gather Example




                         9
Scatter Gather Example




                         10
Scatter Gather Example




                         11
Scatter Gather Example




                         12
Scatter Gather Example




                         13
Scatter Gather Example




                         14
Scatter Gather Example




                         15
Scatter Gather Example




                         16
Scatter Gather Example




                         17
Scatter Gather Example




                         18
Scatter Gather Example




                         19
Q&A




DALE HARVEY
DALE@COUCHBASE.COM
@DALEHARVEY




                           37

More Related Content

What's hot

Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekMark Kromer
 
Elasticsearch in Netflix
Elasticsearch in NetflixElasticsearch in Netflix
Elasticsearch in NetflixDanny Yuan
 
Migrating to CouchDB
Migrating to CouchDBMigrating to CouchDB
Migrating to CouchDBJohn Wood
 
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인Amazon Web Services Korea
 
Introduction to Spark (Intern Event Presentation)
Introduction to Spark (Intern Event Presentation)Introduction to Spark (Intern Event Presentation)
Introduction to Spark (Intern Event Presentation)Databricks
 
Schema-on-Read vs Schema-on-Write
Schema-on-Read vs Schema-on-WriteSchema-on-Read vs Schema-on-Write
Schema-on-Read vs Schema-on-WriteAmr Awadallah
 
Kafka for Real-Time Replication between Edge and Hybrid Cloud
Kafka for Real-Time Replication between Edge and Hybrid CloudKafka for Real-Time Replication between Edge and Hybrid Cloud
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
 
Azure Data Factory presentation with links
Azure Data Factory presentation with linksAzure Data Factory presentation with links
Azure Data Factory presentation with linksChris Testa-O'Neill
 
Apache Spark Fundamentals
Apache Spark FundamentalsApache Spark Fundamentals
Apache Spark FundamentalsZahra Eskandari
 
Apache Spark Introduction
Apache Spark IntroductionApache Spark Introduction
Apache Spark Introductionsudhakara st
 
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...Simplilearn
 
Azure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdfAzure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdfMaheshPandit16
 
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Eric Bragas
 
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인Amazon Web Services Korea
 
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Edureka!
 
Azure Data Factory
Azure Data FactoryAzure Data Factory
Azure Data FactoryHARIHARAN R
 

What's hot (20)

Azure Data Factory for Azure Data Week
Azure Data Factory for Azure Data WeekAzure Data Factory for Azure Data Week
Azure Data Factory for Azure Data Week
 
Snowflake Overview
Snowflake OverviewSnowflake Overview
Snowflake Overview
 
Elasticsearch in Netflix
Elasticsearch in NetflixElasticsearch in Netflix
Elasticsearch in Netflix
 
Migrating to CouchDB
Migrating to CouchDBMigrating to CouchDB
Migrating to CouchDB
 
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인
Aurora MySQL Backtrack을 이용한 빠른 복구 방법 - 진교선 :: AWS Database Modernization Day 온라인
 
Graph Databases
Graph DatabasesGraph Databases
Graph Databases
 
SQOOP PPT
SQOOP PPTSQOOP PPT
SQOOP PPT
 
Introduction to Spark (Intern Event Presentation)
Introduction to Spark (Intern Event Presentation)Introduction to Spark (Intern Event Presentation)
Introduction to Spark (Intern Event Presentation)
 
Azure Cosmos DB
Azure Cosmos DBAzure Cosmos DB
Azure Cosmos DB
 
Schema-on-Read vs Schema-on-Write
Schema-on-Read vs Schema-on-WriteSchema-on-Read vs Schema-on-Write
Schema-on-Read vs Schema-on-Write
 
Kafka for Real-Time Replication between Edge and Hybrid Cloud
Kafka for Real-Time Replication between Edge and Hybrid CloudKafka for Real-Time Replication between Edge and Hybrid Cloud
Kafka for Real-Time Replication between Edge and Hybrid Cloud
 
Azure Data Factory presentation with links
Azure Data Factory presentation with linksAzure Data Factory presentation with links
Azure Data Factory presentation with links
 
Apache Spark Fundamentals
Apache Spark FundamentalsApache Spark Fundamentals
Apache Spark Fundamentals
 
Apache Spark Introduction
Apache Spark IntroductionApache Spark Introduction
Apache Spark Introduction
 
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...
Apache Spark Architecture | Apache Spark Architecture Explained | Apache Spar...
 
Azure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdfAzure Data Factory Introduction.pdf
Azure Data Factory Introduction.pdf
 
Intro to Azure Data Factory v1
Intro to Azure Data Factory v1Intro to Azure Data Factory v1
Intro to Azure Data Factory v1
 
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인
Amazon Dynamo DB 활용하기 - 강민석 :: AWS Database Modernization Day 온라인
 
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
Azure Data Factory | Moving On-Premise Data to Azure Cloud | Microsoft Azure ...
 
Azure Data Factory
Azure Data FactoryAzure Data Factory
Azure Data Factory
 

Viewers also liked

Ocean base海量结构化数据存储系统 hadoop in china
Ocean base海量结构化数据存储系统 hadoop in chinaOcean base海量结构化数据存储系统 hadoop in china
Ocean base海量结构化数据存储系统 hadoop in chinaknuthocean
 
Couchdb and me
Couchdb and meCouchdb and me
Couchdb and meiammutex
 
Mysql HandleSocket技术在SNS Feed存储中的应用
Mysql HandleSocket技术在SNS Feed存储中的应用Mysql HandleSocket技术在SNS Feed存储中的应用
Mysql HandleSocket技术在SNS Feed存储中的应用iammutex
 
Consistency Models in New Generation Databases
Consistency Models in New Generation DatabasesConsistency Models in New Generation Databases
Consistency Models in New Generation Databasesiammutex
 
8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slideiammutex
 
Consistency in Distributed Systems
Consistency in Distributed SystemsConsistency in Distributed Systems
Consistency in Distributed SystemsShane Johnson
 
SDEC2011 NoSQL Data modelling
SDEC2011 NoSQL Data modellingSDEC2011 NoSQL Data modelling
SDEC2011 NoSQL Data modellingKorea Sdec
 
Big Challenges in Data Modeling: NoSQL and Data Modeling
Big Challenges in Data Modeling: NoSQL and Data ModelingBig Challenges in Data Modeling: NoSQL and Data Modeling
Big Challenges in Data Modeling: NoSQL and Data ModelingDATAVERSITY
 
Thoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency ModelsThoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency Modelsiammutex
 
Boosting Machine Learning with Redis Modules and Spark
Boosting Machine Learning with Redis Modules and SparkBoosting Machine Learning with Redis Modules and Spark
Boosting Machine Learning with Redis Modules and SparkDvir Volk
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDBrogerbodamer
 
Coherence and consistency models in multiprocessor architecture
Coherence and consistency models in multiprocessor architectureCoherence and consistency models in multiprocessor architecture
Coherence and consistency models in multiprocessor architectureUniversity of Pisa
 
Consistency in Distributed Systems
Consistency in Distributed SystemsConsistency in Distributed Systems
Consistency in Distributed SystemsDATAVERSITY
 
Cache coherence problem and its solutions
Cache coherence problem and its solutionsCache coherence problem and its solutions
Cache coherence problem and its solutionsMajid Saleem
 
Benchmarking MongoDB and CouchBase
Benchmarking MongoDB and CouchBaseBenchmarking MongoDB and CouchBase
Benchmarking MongoDB and CouchBaseChristopher Choi
 
Redis modules 101
Redis modules 101Redis modules 101
Redis modules 101Dvir Volk
 

Viewers also liked (20)

Intro To Couch Db
Intro To Couch DbIntro To Couch Db
Intro To Couch Db
 
Ocean base海量结构化数据存储系统 hadoop in china
Ocean base海量结构化数据存储系统 hadoop in chinaOcean base海量结构化数据存储系统 hadoop in china
Ocean base海量结构化数据存储系统 hadoop in china
 
Couchdb and me
Couchdb and meCouchdb and me
Couchdb and me
 
Ooredis
OoredisOoredis
Ooredis
 
Mysql HandleSocket技术在SNS Feed存储中的应用
Mysql HandleSocket技术在SNS Feed存储中的应用Mysql HandleSocket技术在SNS Feed存储中的应用
Mysql HandleSocket技术在SNS Feed存储中的应用
 
Consistency Models in New Generation Databases
Consistency Models in New Generation DatabasesConsistency Models in New Generation Databases
Consistency Models in New Generation Databases
 
8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide8 minute MongoDB tutorial slide
8 minute MongoDB tutorial slide
 
Consistency in Distributed Systems
Consistency in Distributed SystemsConsistency in Distributed Systems
Consistency in Distributed Systems
 
SDEC2011 NoSQL Data modelling
SDEC2011 NoSQL Data modellingSDEC2011 NoSQL Data modelling
SDEC2011 NoSQL Data modelling
 
Big Challenges in Data Modeling: NoSQL and Data Modeling
Big Challenges in Data Modeling: NoSQL and Data ModelingBig Challenges in Data Modeling: NoSQL and Data Modeling
Big Challenges in Data Modeling: NoSQL and Data Modeling
 
skip list
skip listskip list
skip list
 
Cache coherence
Cache coherenceCache coherence
Cache coherence
 
Thoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency ModelsThoughts on Transaction and Consistency Models
Thoughts on Transaction and Consistency Models
 
Boosting Machine Learning with Redis Modules and Spark
Boosting Machine Learning with Redis Modules and SparkBoosting Machine Learning with Redis Modules and Spark
Boosting Machine Learning with Redis Modules and Spark
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDB
 
Coherence and consistency models in multiprocessor architecture
Coherence and consistency models in multiprocessor architectureCoherence and consistency models in multiprocessor architecture
Coherence and consistency models in multiprocessor architecture
 
Consistency in Distributed Systems
Consistency in Distributed SystemsConsistency in Distributed Systems
Consistency in Distributed Systems
 
Cache coherence problem and its solutions
Cache coherence problem and its solutionsCache coherence problem and its solutions
Cache coherence problem and its solutions
 
Benchmarking MongoDB and CouchBase
Benchmarking MongoDB and CouchBaseBenchmarking MongoDB and CouchBase
Benchmarking MongoDB and CouchBase
 
Redis modules 101
Redis modules 101Redis modules 101
Redis modules 101
 

Similar to Couchdb + Membase = Couchbase

Membase Meetup Chicago - january 2011
Membase Meetup Chicago - january 2011Membase Meetup Chicago - january 2011
Membase Meetup Chicago - january 2011Membase
 
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...DATAVERSITY
 
Couchbase b jmeetup
Couchbase b jmeetupCouchbase b jmeetup
Couchbase b jmeetupmysqlops
 
Membase Meetup - Silicon Valley
Membase Meetup - Silicon ValleyMembase Meetup - Silicon Valley
Membase Meetup - Silicon ValleyMembase
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedKorea Sdec
 
MySQL Options in OpenStack
MySQL Options in OpenStackMySQL Options in OpenStack
MySQL Options in OpenStackTesora
 
OpenStack Days East -- MySQL Options in OpenStack
OpenStack Days East -- MySQL Options in OpenStackOpenStack Days East -- MySQL Options in OpenStack
OpenStack Days East -- MySQL Options in OpenStackMatt Lord
 
Membase East Coast Meetups
Membase East Coast MeetupsMembase East Coast Meetups
Membase East Coast MeetupsMembase
 
Hadoop World 2011: Hadoop as a Service in Cloud
Hadoop World 2011: Hadoop as a Service in CloudHadoop World 2011: Hadoop as a Service in Cloud
Hadoop World 2011: Hadoop as a Service in CloudCloudera, Inc.
 
Membase Meetup 2010
Membase Meetup 2010Membase Meetup 2010
Membase Meetup 2010Membase
 
Achieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefAchieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefMatt Ray
 
App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)outstanding59
 
Inside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworldInside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworldRichard McDougall
 
App Cap2956v2 121001194956 Phpapp01 (1)
App Cap2956v2 121001194956 Phpapp01 (1)App Cap2956v2 121001194956 Phpapp01 (1)
App Cap2956v2 121001194956 Phpapp01 (1)outstanding59
 
Best Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopBest Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopDataWorks Summit
 
Implementing High Availability Caching with Memcached
Implementing High Availability Caching with MemcachedImplementing High Availability Caching with Memcached
Implementing High Availability Caching with MemcachedGear6
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisScaleOut Software
 

Similar to Couchdb + Membase = Couchbase (20)

Membase Meetup Chicago - january 2011
Membase Meetup Chicago - january 2011Membase Meetup Chicago - january 2011
Membase Meetup Chicago - january 2011
 
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...
How AOL Advertising Uses NoSQL to Make Millions of Smart Targeting Decisions ...
 
Couchbase b jmeetup
Couchbase b jmeetupCouchbase b jmeetup
Couchbase b jmeetup
 
Membase Meetup - Silicon Valley
Membase Meetup - Silicon ValleyMembase Meetup - Silicon Valley
Membase Meetup - Silicon Valley
 
SDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speedSDEC2011 Using Couchbase for social game scaling and speed
SDEC2011 Using Couchbase for social game scaling and speed
 
MySQL Options in OpenStack
MySQL Options in OpenStackMySQL Options in OpenStack
MySQL Options in OpenStack
 
OpenStack Days East -- MySQL Options in OpenStack
OpenStack Days East -- MySQL Options in OpenStackOpenStack Days East -- MySQL Options in OpenStack
OpenStack Days East -- MySQL Options in OpenStack
 
Membase East Coast Meetups
Membase East Coast MeetupsMembase East Coast Meetups
Membase East Coast Meetups
 
Hadoop World 2011: Hadoop as a Service in Cloud
Hadoop World 2011: Hadoop as a Service in CloudHadoop World 2011: Hadoop as a Service in Cloud
Hadoop World 2011: Hadoop as a Service in Cloud
 
Membase Meetup 2010
Membase Meetup 2010Membase Meetup 2010
Membase Meetup 2010
 
Deep dive into AWS fargate
Deep dive into AWS fargateDeep dive into AWS fargate
Deep dive into AWS fargate
 
Achieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefAchieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with Chef
 
App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)
 
Inside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworldInside the Hadoop Machine @ VMworld
Inside the Hadoop Machine @ VMworld
 
App Cap2956v2 121001194956 Phpapp01 (1)
App Cap2956v2 121001194956 Phpapp01 (1)App Cap2956v2 121001194956 Phpapp01 (1)
App Cap2956v2 121001194956 Phpapp01 (1)
 
Hadoop on VMware
Hadoop on VMwareHadoop on VMware
Hadoop on VMware
 
Best Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopBest Practices for Virtualizing Hadoop
Best Practices for Virtualizing Hadoop
 
Implementing High Availability Caching with Memcached
Implementing High Availability Caching with MemcachedImplementing High Availability Caching with Memcached
Implementing High Availability Caching with Memcached
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data Analysis
 
From 0 to syncing
From 0 to syncingFrom 0 to syncing
From 0 to syncing
 

More from iammutex

Scaling Instagram
Scaling InstagramScaling Instagram
Scaling Instagramiammutex
 
Redis深入浅出
Redis深入浅出Redis深入浅出
Redis深入浅出iammutex
 
深入了解Redis
深入了解Redis深入了解Redis
深入了解Redisiammutex
 
NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析iammutex
 
MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用iammutex
 
Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告iammutex
 
redis 适用场景与实现
redis 适用场景与实现redis 适用场景与实现
redis 适用场景与实现iammutex
 
Introduction to couchdb
Introduction to couchdbIntroduction to couchdb
Introduction to couchdbiammutex
 
What every data programmer needs to know about disks
What every data programmer needs to know about disksWhat every data programmer needs to know about disks
What every data programmer needs to know about disksiammutex
 
redis运维之道
redis运维之道redis运维之道
redis运维之道iammutex
 
Realtime hadoopsigmod2011
Realtime hadoopsigmod2011Realtime hadoopsigmod2011
Realtime hadoopsigmod2011iammutex
 
[译]No sql生态系统
[译]No sql生态系统[译]No sql生态系统
[译]No sql生态系统iammutex
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 
Redis cluster
Redis clusterRedis cluster
Redis clusteriammutex
 
Hadoop introduction berlin buzzwords 2011
Hadoop introduction   berlin buzzwords 2011Hadoop introduction   berlin buzzwords 2011
Hadoop introduction berlin buzzwords 2011iammutex
 
No sql but even less security
No sql but even less securityNo sql but even less security
No sql but even less securityiammutex
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators iammutex
 
MongoDB开发应用实践
MongoDB开发应用实践MongoDB开发应用实践
MongoDB开发应用实践iammutex
 
MongoDB @ SourceForge
MongoDB @ SourceForgeMongoDB @ SourceForge
MongoDB @ SourceForgeiammutex
 

More from iammutex (20)

Scaling Instagram
Scaling InstagramScaling Instagram
Scaling Instagram
 
Redis深入浅出
Redis深入浅出Redis深入浅出
Redis深入浅出
 
深入了解Redis
深入了解Redis深入了解Redis
深入了解Redis
 
NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析NoSQL误用和常见陷阱分析
NoSQL误用和常见陷阱分析
 
MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用MongoDB 在盛大大数据量下的应用
MongoDB 在盛大大数据量下的应用
 
Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告Rethink db&tokudb调研测试报告
Rethink db&tokudb调研测试报告
 
redis 适用场景与实现
redis 适用场景与实现redis 适用场景与实现
redis 适用场景与实现
 
Introduction to couchdb
Introduction to couchdbIntroduction to couchdb
Introduction to couchdb
 
What every data programmer needs to know about disks
What every data programmer needs to know about disksWhat every data programmer needs to know about disks
What every data programmer needs to know about disks
 
Ooredis
OoredisOoredis
Ooredis
 
redis运维之道
redis运维之道redis运维之道
redis运维之道
 
Realtime hadoopsigmod2011
Realtime hadoopsigmod2011Realtime hadoopsigmod2011
Realtime hadoopsigmod2011
 
[译]No sql生态系统
[译]No sql生态系统[译]No sql生态系统
[译]No sql生态系统
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 
Redis cluster
Redis clusterRedis cluster
Redis cluster
 
Hadoop introduction berlin buzzwords 2011
Hadoop introduction   berlin buzzwords 2011Hadoop introduction   berlin buzzwords 2011
Hadoop introduction berlin buzzwords 2011
 
No sql but even less security
No sql but even less securityNo sql but even less security
No sql but even less security
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators
 
MongoDB开发应用实践
MongoDB开发应用实践MongoDB开发应用实践
MongoDB开发应用实践
 
MongoDB @ SourceForge
MongoDB @ SourceForgeMongoDB @ SourceForge
MongoDB @ SourceForge
 

Recently uploaded

RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 

Recently uploaded (20)

RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 

Couchdb + Membase = Couchbase

  • 1. Data management for interactive web and mobile applications. DALE HARVEY 1
  • 3. What is Couchbase? 1+1 really does equal 3 3
  • 5. Deployments Leading Membase Leading cloud service (PAAS) Social game leader – FarmVille, provider Mafia Wars, Café World Over 65,000 hosted Over 230 million monthly users applications Membase Server Membase Server serving over is the 500,000 ops-per-second 1,200 Heroku customers (as of database behind FarmVille and June 10, 2010) Café World 5
  • 6. Ad targeting 40 milliseconds to come up with an answer. 3 profiles, real time campaign statistics 2 1 profiles, campaigns events 6
  • 7. Membase is a distributed database Application user Web application server Web application server Web application server Membase Servers In the data center On the administrator console 7
  • 8. Membase is Simple, Fast, Elastic Five minutes or less to a working cluster • Downloads for Linux and Windows • Start with a single node • One button press joins nodes to a cluster Easy to develop against • Just SET and GET – no schema required • Drop it in. 10,000+ existing applications already “speak membase” (via memcached) • Practically every language and application framework is supported, out of the box Easy to manage • One-click failover and cluster rebalancing • Graphical and programmatic interfaces • Configurable alerting 8
  • 9. Data is sharded into vBuckets 9
  • 10. Clustering With Membase SET request arrives at SET acknowledgement KEY’s master server 1 2 returned to application 3 3 Listener-Sender RAM membase storage engine 4 Disk Disk Disk Disk Disk Disk Replica Server 1 for KEY Master server for KEY Replica Server 2 for KEY 10
  • 11. Membase is Simple, Fast, Elastic Zero-downtime elasticity • Spread I/O and data across commodity servers (or VMs) • Consistent performance with linear cost • Dynamic rebalancing of a live cluster All nodes are created equal • No special case nodes • Any node can replace any other node, online • Clone to grow Extensible • Filtered TAP interface provides hook points for external systems (e.g. full-text search, backup, warehouse) • Data bucket – engine API for specialized container types 11
  • 13. Robust JSON Replication Map Reduce GeoCouch HTTP 13
  • 14. USE CASES 14
  • 15. DOCUMENT STORE => GET http://127.0.0.1:5984/mydatabase/ uniqueid => PUT http://127.0.0.1:5984/mydatabase/ uniqueid http://127.0.0.1:5984/mydatabase => POST => DELETE http://127.0.0.1:5984/mydatabase/ uniqueid { “_id” : “uniqueid”, “_rev” : 1234567, “title” : “My New Blog Post”, “body” : “A quick brown fox .......”, “_attachments”: { “profile.jpg”: { ...... } } } 15
  • 16. ROBUST STORAGE ENGINE Append-Only Reliable & Concurrent Hot Backups Compaction 16
  • 18. PEER-BASED REPLICATION curl -X POST http://127.0.0.1:5984/_replicate -d '{ "source": "http://couch.example.com:5984/remote-db", "target": "local-db" }' 18
  • 19. NAILING MOBILE • Extremely Robust Storage Engine Consumer devices may lose power at any time • Lightweight Runtime Small install, low memory footprint, easy on battery life • Offline sync via Replication Multi-master synchronization engine • GeoSpatial Index Mapping and location aware apps • HTML5 Application Server Easy to add a web view to your mobile data https://github.com/couchbaselabs/iOS-Couchbase 19
  • 20. GEOCOUCH CIVICAPI.COM, ETC Advanced R-Tree Index 20
  • 22. 2.0 Architecture Overview Application Couchbase SDK API MCD Couch Application Tier Elastic Couchbase Server Cluster Moxi Memcached Couch Proxy ep_engine Cluster Manager Couchbase Server vBucket Data View Data 22
  • 23. JSON Document Handling • Per Bucket JSON Document Mode toggle • JSON Document Mode disabled (or invalid JSON) – Values are opaque to couch (and in views) – Stored as attachments to documents that hold metadata – Returned values are guaranteed bitwise identical • JSON Document Mode enabled – Values are expected to be valid JSON – Stored in document along metadata 5 – Returned values may not be bitwise identical but
  • 24. JSON Document Encodings for memcached Couch Document Meta Data Key:”zdfasf” Expiration:43452345 { Flags:2352145 “_id”: ”zdfasf” “$Expiration”: 43452345, “$Flags”: 2352145, “$AttachReason”:”JSON Mode Disabled”, //Other Value reasons “Invalid JSON”, etc. [“$KeyMangled”: false,] //only included if true “_attachments”: { “$value”: { … } } 0x87632f98,0xa76b876d, …! } Couch Document Meta Data Key:”zdfasf” Expiration:43452345 Flags:2352145 { “_id”: ”zdfasf” “$Expiration”: 43452345, “$Flags”: 2352145, JSON Document [“$KeyMangled”: false,] //only included if true “foo”: ”bar” { } “foo”: ”bar” } 24
  • 25. Scatter Gather Map Reduce • Get next answer from local vBuckets and from child nodes to select the next result for a node • Result includes info on number of vBuckets not included – Timeouts, Incorrect Design Doc version, etc 8