SlideShare a Scribd company logo
1 of 34
Download to read offline
From distributed caches
to in-memory data grids
             TechTalk by Max A. Alexejev
                 malexejev@gmail.com
Memory Hierarchy
                     R
                    <1ns
                     L1
               ~4 cycles, ~1ns        Cost
                     L2
              ~10 cycles, ~3ns
                    L3
              ~42 cycles, ~15ns
                   DRAM
                   >65ns

             Flash / SSD / USB
                                       Storage
                                       term
                    HDD

         Tapes, Remote systems, etc



                                  2   Max A. Alexejev
Software caches
 Improve response times by reducing data access latency

 Offload persistent storages

 Only work for IO-bound applications!




                                3                          Max A. Alexejev
Caches and data location

                               Consistency
                                protocol
                                                Shared


   Local                  Remote


                                              Distributed
           Hierarchical        Distribution
                                algorithm


                           4                             Max A. Alexejev
Ok, so how do we grow beyond one node?

 Data replication




                     5            Max A. Alexejev
Pro’s and Con’s of replication

  Pro

  • Best read performance (for local replicated caches)
  • Fault tolerant cache (both local and remote)
  • Can be smart: replicate only part of CRUD cycle

  Con

  • Poor writes performance
  • Additional network load
  • Can scale only vertically: limited by single machine size
  • In case of master-master replication, requires complex consistency protocol



                                          6                                       Max A. Alexejev
Ok, so how do we grow beyond one node?

 Data distribution




                      7           Max A. Alexejev
Pro’s and Con’s of data distribution

  Pro

  • Can scale horizontally beyond single machine size
  • Reads and writes performance scales horizontally

  Con

  • No fault tolerance for cached data
  • Increased latency of reads (due to network round-trip and
    serialization expenses)

                               8                                Max A. Alexejev
What do high-load applications need
from cache?



                Linear      Distributed
      Low
              horizontal
    latency                    cache
              scalability




                        9             Max A. Alexejev
Cache access patterns:                                              Client

     Cache Aside
     For reading data:            For writing data
     1.   Application asks        1.   Application writes
          for some data for a          some new data or
          given key                    updates existing.
                                                                 Cache
     2.   Check the cache         2.   Write it to the
     3.   If data is in the            cache
          cache return it to      3.   Write it to the DB.
          the user
     4.   If data is not in the   Overall:
          cache fetch it from
          the DB, put it in       •    Increases reads
          the cache, return it         performance
          to the user.            •    Offloads DB reads
                                                                          DB
                                  •    Introduces race
                                       conditions for
                                       writes

10                                             Max A. Alexejev
Cache access patterns:                                                          Client

     Read Through
     For reading data:
     1.    Application asks for some data for a given key
     2.    Check the cache
     3.    If data is in the cache return it to the user                     Cache
     4.    If data is not in the cache – cache will invoke fetching
           it from the DB by himself, saving retrieved value and
           returning it to the user.


     Overall:
     •    Reduces reads latency
     •    Offloads read load from underlying storage
     •    May have blocking behavior, thus helping with dog-pile                      DB
          effect
     •    Requires “smarter” cache nodes




11                                                         Max A. Alexejev
Cache access patterns:                                     Client

     Write Through
     For writing data
     1. Application writes some new data or
         updates existing.                              Cache
     2. Write it to the cache
     3. Cache will then synchronously write it
         to the DB.

     Overall:
     • Slightly increases writes latency
                                                                 DB
     • Provides natural invalidation
     • Removes race conditions on writes



12                                    Max A. Alexejev
Cache access patterns:                                                    Client

     Write Behind
     For writing data
     1.    Application writes some new data or updates existing.
     2.    Write it to the cache
           Cache adds writes request to its internal queue.
     3.
                                                                       Cache
     4.    Later, cache asynchronously flushes queue to DB on a
           periodic basis and/or when queue size reaches certain
           limit.


     Overall:
     •    Dramatically reduces writes latency by a price of
          inconsistency window
     •    Provides writes batching
     •    May provide updates deduplication                                     DB




13                                                   Max A. Alexejev
A variety of products on the market…

  Memcached
                       Hazelcast
                                            Cassandra

         GigaSpaces
                                   Redis
                                                        Terracotta
    Oracle
  Coherence           Infinispan
                                           MongoDB

              Riak
                            EhCache                       …


                                   14                            Max A. Alexejev
KV caches     NoSQL        Data Grids

                                                                                   Oracle
                                                      Memcached      Redis
                                                                                 Coherence



                                                       Ehcache     Cassandra      GemFire


     Lets sort em out!                                   …         MongoDB       GigaSpaces

     Some products are really hard to
     sort – like Terracotta in both DSO                               …           GridGain
     and Express modes.
                                                                                  Hazelcast



                                                                                  Infinispan




15                                Max A. Alexejev
Why don’t we have any distributed
in-memory RDBMS?

  Master – MultiSlaves configuration

  • Is, if fact, an example of replication
  • Helps with reads distribution, but does not help with writes
  • Does not scale beyond single master

  Horizontal partitioning (sharding)

  • Helps with reads and writes for datasets with good data affinity
  • Does not work nicely with joins semantics (i.e., there are no
    distributed joins)


                                       16                              Max A. Alexejev
Key-Value caches
•   Memcached and EHCache are good examples to look at

•   Keys and values are arbitrary binary (serializable) entities

•   Basic operations are put(K,V), get(K), replace(K,V), remove(K)

•   May provide group operations like getAll(…) and putAll(…)

•   Some operations provide atomicity guarantees (CAS,
    inc/dec)

                                  17                           Max A. Alexejev
Memcached
     • Developed for LiveJournal in 2003
     • Has client libraries in PHP, Java,
        Ruby, Python and many others
     • Nodes are independent and don’t
        communicate with each other



18                                 Max A. Alexejev
EHCache
     •   Initially named “Easy Hibernate Cache”
     •   Java-centric, mature product with open-
         source and commercial editions
     •   Open-source version provides only
         replication capabilities, distributed
         caching requires commercial license for
         both EHCache and Terracotta TSA



19                                     Max A. Alexejev
NoSQL Systems
 A whole bunch of different products with both persistent and
   non-persistent storage options. Lets call them caches and
   storages, accordingly.

 Built to provide good horizontal scalability

 Try to fill the feature gap between pure KV and full-blown
   RDBMS



                                20                         Max A. Alexejev
 Written in C, supported by
                                                      VMWare

                                                   Client libraries for C, C#, Java,
                                                      Scala, PHP, Erlang, etc

                                                   Single-threaded async impl

                                                   Has configurable persistence
     Case study: Redis
                                                   Works with K-V pairs, where K is a
                                                      string and V may be either number,
     hset users:goku powerlevel 9000                  string or Object (JSON)
     hget users:goku powerlevel                    Provides 5 interfaces for: strings,
                                                      hashes, sorted lists, sets, sorted
                                                      sets

                                                   Supports transactions


21                              Max A. Alexejev
Use cases: Redis
 Good for fixed lists, tagging, ratings, counters, analytics and
   queues (pub-sub messaging)

 Has Master – MultiSlave replication support. Master node is
   currently a SPOF.

 Distributed Redis was named “Redis Cluster” and is currently
   under development



                                22                           Max A. Alexejev
•   Written in Java, developed in
                                             Facebook.
                                         •   Inspired by Amazon Dynamo
                                             replication mechanics, but
                                             uses column-based data
                                             model.
     Case study: Cassandra               •   Good for logs processing,
                                             index storage, voting, jobs
                                             storage etc.
                                         •   Bad for transactional
                                             processing.
                                         •   Want to know more? Ask
                                             Alexey!


23                     Max A. Alexejev
In-Memory Data Grids
New generation of caching products, trying to combine benefits of replicated and
distributed schemes.

                                         24                               Max A. Alexejev
IMDG: Evolution
  Data Grids                        Computational
  • Reliable storage and            Grids
    live data balancing             • Reliable jobs
    among grid nodes                  execution, scheduling
                                      and load balancing




                           Modern
                            IMDG

                             25                               Max A. Alexejev
IMDG: Caching concepts
•   Implements KV cache interface                   •   Live data redistribution when nodes are
                                                        going up or down – no data loss, no
•   Provides indexed search by values                   clients termination

•   Provides reliable distributed locks             •   Supports RT, WT, WB caching patterns
    interface                                           and hierarchical caches (near caching)

•   Caching scheme – partitioned or                 •   Supports atomic computations on grid
    distributed, may be specified per cache             nodes
    or cache service

•   Provides events subscription for entries
    (change notifications)

•   Configurable fault tolerance for
    distributed schemes (HA)

•   Equal data (and read/write load)           26                                       Max A. Alexejev
    distribution among grid nodes
IMDG: Under the hood
     •   All data is split in a number of sections,
         called partitions.
     •   Partition, rather then entry, is an atomic
         unit of data migration when grid
         rebalances. Number of partitions is fixed
         for cluster lifetime.
     •   Indexes are distributed among grid nodes.
     •   Clients may or may not be part of the grid
         cluster.




27                                           Max A. Alexejev
IMDG Under the hood:
     Requests routing
     For get() and put() requests:
     1.     Cluster member, that makes a request, calculates key hash
            code.
     2.     Partition number is calculated using this hash code.
     3.     Node is identified by partition number.
     4.     Request is then routed to identified node, executed, and
            results are sent back to the client member who initiated
            request.


     For filter queries:
     1.     Cluster member initiating requests sends it to all storage
            enabled nodes in the cluster.
     2.     Query is executed on every node using distributed indexes
            and partial results are sent to the requesting member.
     3.     Requesting member merges partial results locally.
     4.     Final result set is returned from filter method.



28                                                         Max A. Alexejev
IMDG: Advanced use-cases
 Messaging

 Map-Reduce calculations

 Cluster-wide singleton

 And more…




                            29   Max A. Alexejev
GC tuning for large grid nodes
 An easy way to go: rolling restarts or storage-enabled cluster
   nodes. Can not be used in any project.

 A complex way to go: fine-tune CMS collector to ensure that
   it will always keep up cleaning garbage concurrently under
   normal production workload.

 An expensive way to go: use OffHeap storages provided by
   some vendors (Oracle, Terracotta) and use direct memory
   buffers available to JVM.

                               30                          Max A. Alexejev
IMDG: Market players
 Oracle Coherence: commercial, free for evaluation use.

 GigaSpaces: commercial.

 GridGain: commercial.

 Hazelcast: open-source.

 Infinispan: open-source.



                              31                           Max A. Alexejev
Terracotta
A company behind EHCache, Quartz and Terracotta Server Array.
Acquired by Software AG.

                                       32                       Max A. Alexejev
Terracotta Server Array
All data is split in a number of sections, called stripes.
Stripes consist of 2 or more Terracotta nodes. One of them is Active node, others have Passive status.
All data is distributed among stripes and replicated inside stripes.

Open Source limitation: only one stripe. Such setup will support HA, but will not distribute cache data. I.e., it is not horizontally scalable.

                                                                               33                                                                 Max A. Alexejev
Max A. Alexejev
QA Session
And thank you for coming!

More Related Content

What's hot

Handle Large Messages In Apache Kafka
Handle Large Messages In Apache KafkaHandle Large Messages In Apache Kafka
Handle Large Messages In Apache KafkaJiangjie Qin
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm Chandler Huang
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudNoritaka Sekiyama
 
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -Yoshiyasu SAEKI
 
Troubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the BeastTroubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the BeastDataWorks Summit
 
Scalable and Available, Patterns for Success
Scalable and Available, Patterns for SuccessScalable and Available, Patterns for Success
Scalable and Available, Patterns for SuccessDerek Collison
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisDvir Volk
 
MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks EDB
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in JavaRuben Badaró
 
Adaptive Query Execution: Speeding Up Spark SQL at Runtime
Adaptive Query Execution: Speeding Up Spark SQL at RuntimeAdaptive Query Execution: Speeding Up Spark SQL at Runtime
Adaptive Query Execution: Speeding Up Spark SQL at RuntimeDatabricks
 
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdfRun Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdfAnya Bida
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & FeaturesDataStax Academy
 
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...Amazon Web Services
 
How to Actually Tune Your Spark Jobs So They Work
How to Actually Tune Your Spark Jobs So They WorkHow to Actually Tune Your Spark Jobs So They Work
How to Actually Tune Your Spark Jobs So They WorkIlya Ganelin
 
Hadoop Security Architecture
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security ArchitectureOwen O'Malley
 
Spark shuffle introduction
Spark shuffle introductionSpark shuffle introduction
Spark shuffle introductioncolorant
 
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)Amy W. Tang
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcachedJurriaan Persyn
 
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in Spark
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in SparkSpark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in Spark
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in SparkBo Yang
 

What's hot (20)

Handle Large Messages In Apache Kafka
Handle Large Messages In Apache KafkaHandle Large Messages In Apache Kafka
Handle Large Messages In Apache Kafka
 
Introduction to Storm
Introduction to Storm Introduction to Storm
Introduction to Storm
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
 
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -
Apache Sparkにおけるメモリ - アプリケーションを落とさないメモリ設計手法 -
 
Troubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the BeastTroubleshooting Kerberos in Hadoop: Taming the Beast
Troubleshooting Kerberos in Hadoop: Taming the Beast
 
Scalable and Available, Patterns for Success
Scalable and Available, Patterns for SuccessScalable and Available, Patterns for Success
Scalable and Available, Patterns for Success
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks MongoDB vs. Postgres Benchmarks
MongoDB vs. Postgres Benchmarks
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Adaptive Query Execution: Speeding Up Spark SQL at Runtime
Adaptive Query Execution: Speeding Up Spark SQL at RuntimeAdaptive Query Execution: Speeding Up Spark SQL at Runtime
Adaptive Query Execution: Speeding Up Spark SQL at Runtime
 
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdfRun Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
Run Apache Spark on Kubernetes in Large Scale_ Challenges and Solutions-2.pdf
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
 
NoSQL
NoSQLNoSQL
NoSQL
 
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303...
 
How to Actually Tune Your Spark Jobs So They Work
How to Actually Tune Your Spark Jobs So They WorkHow to Actually Tune Your Spark Jobs So They Work
How to Actually Tune Your Spark Jobs So They Work
 
Hadoop Security Architecture
Hadoop Security ArchitectureHadoop Security Architecture
Hadoop Security Architecture
 
Spark shuffle introduction
Spark shuffle introductionSpark shuffle introduction
Spark shuffle introduction
 
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in Spark
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in SparkSpark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in Spark
Spark Shuffle Deep Dive (Explained In Depth) - How Shuffle Works in Spark
 

Similar to From distributed caches to in-memory data grids

Jug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsJug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsDavide Carnevali
 
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Lindsey Aitchison
 
5266732.ppt
5266732.ppt5266732.ppt
5266732.ppthothyfa
 
Couchbase - Yet Another Introduction
Couchbase - Yet Another IntroductionCouchbase - Yet Another Introduction
Couchbase - Yet Another IntroductionKelum Senanayake
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesshnkr_rmchndrn
 
Scale your Alfresco Solutions
Scale your Alfresco Solutions Scale your Alfresco Solutions
Scale your Alfresco Solutions Alfresco Software
 
M6d cassandrapresentation
M6d cassandrapresentationM6d cassandrapresentation
M6d cassandrapresentationEdward Capriolo
 
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
 
Scalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetScalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetBala Subra
 
Hw09 Low Latency, Random Reads From Hdfs
Hw09   Low Latency, Random Reads From HdfsHw09   Low Latency, Random Reads From Hdfs
Hw09 Low Latency, Random Reads From HdfsCloudera, Inc.
 
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBeyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBen Stopford
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisScaleOut Software
 
Hadoop enhancements using next gen IA technologies
Hadoop enhancements using next gen IA technologiesHadoop enhancements using next gen IA technologies
Hadoop enhancements using next gen IA technologiesBigdata Meetup Kochi
 
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...Lviv Startup Club
 
Cache and consistency in nosql
Cache and consistency in nosqlCache and consistency in nosql
Cache and consistency in nosqlJoão Gabriel Lima
 
Cache performance considerations
Cache performance considerationsCache performance considerations
Cache performance considerationsSlideshare
 
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalSizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalVigyan Jain
 
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...Ontico
 

Similar to From distributed caches to in-memory data grids (20)

Jug Lugano - Scale over the limits
Jug Lugano - Scale over the limitsJug Lugano - Scale over the limits
Jug Lugano - Scale over the limits
 
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
 
5266732.ppt
5266732.ppt5266732.ppt
5266732.ppt
 
Couchbase - Yet Another Introduction
Couchbase - Yet Another IntroductionCouchbase - Yet Another Introduction
Couchbase - Yet Another Introduction
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
 
Scale your Alfresco Solutions
Scale your Alfresco Solutions Scale your Alfresco Solutions
Scale your Alfresco Solutions
 
M6d cassandrapresentation
M6d cassandrapresentationM6d cassandrapresentation
M6d cassandrapresentation
 
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabIntroduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLab
 
Scalable Resilient Web Services In .Net
Scalable Resilient Web Services In .NetScalable Resilient Web Services In .Net
Scalable Resilient Web Services In .Net
 
Hw09 Low Latency, Random Reads From Hdfs
Hw09   Low Latency, Random Reads From HdfsHw09   Low Latency, Random Reads From Hdfs
Hw09 Low Latency, Random Reads From Hdfs
 
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBeyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear Scalability
 
Using Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data AnalysisUsing Distributed In-Memory Computing for Fast Data Analysis
Using Distributed In-Memory Computing for Fast Data Analysis
 
Hadoop enhancements using next gen IA technologies
Hadoop enhancements using next gen IA technologiesHadoop enhancements using next gen IA technologies
Hadoop enhancements using next gen IA technologies
 
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...
Yaroslav Nedashkovsky - "Data Engineering in Information Security: how to col...
 
Cache and consistency in nosql
Cache and consistency in nosqlCache and consistency in nosql
Cache and consistency in nosql
 
How swift is your Swift - SD.pptx
How swift is your Swift - SD.pptxHow swift is your Swift - SD.pptx
How swift is your Swift - SD.pptx
 
Cache performance considerations
Cache performance considerationsCache performance considerations
Cache performance considerations
 
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalSizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
 
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
 
Apache cassandra
Apache cassandraApache cassandra
Apache cassandra
 

Recently uploaded

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 

Recently uploaded (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 

From distributed caches to in-memory data grids

  • 1. From distributed caches to in-memory data grids TechTalk by Max A. Alexejev malexejev@gmail.com
  • 2. Memory Hierarchy R <1ns L1 ~4 cycles, ~1ns Cost L2 ~10 cycles, ~3ns L3 ~42 cycles, ~15ns DRAM >65ns Flash / SSD / USB Storage term HDD Tapes, Remote systems, etc 2 Max A. Alexejev
  • 3. Software caches  Improve response times by reducing data access latency  Offload persistent storages  Only work for IO-bound applications! 3 Max A. Alexejev
  • 4. Caches and data location Consistency protocol Shared Local Remote Distributed Hierarchical Distribution algorithm 4 Max A. Alexejev
  • 5. Ok, so how do we grow beyond one node?  Data replication 5 Max A. Alexejev
  • 6. Pro’s and Con’s of replication Pro • Best read performance (for local replicated caches) • Fault tolerant cache (both local and remote) • Can be smart: replicate only part of CRUD cycle Con • Poor writes performance • Additional network load • Can scale only vertically: limited by single machine size • In case of master-master replication, requires complex consistency protocol 6 Max A. Alexejev
  • 7. Ok, so how do we grow beyond one node?  Data distribution 7 Max A. Alexejev
  • 8. Pro’s and Con’s of data distribution Pro • Can scale horizontally beyond single machine size • Reads and writes performance scales horizontally Con • No fault tolerance for cached data • Increased latency of reads (due to network round-trip and serialization expenses) 8 Max A. Alexejev
  • 9. What do high-load applications need from cache? Linear Distributed Low horizontal latency cache scalability 9 Max A. Alexejev
  • 10. Cache access patterns: Client Cache Aside For reading data: For writing data 1. Application asks 1. Application writes for some data for a some new data or given key updates existing. Cache 2. Check the cache 2. Write it to the 3. If data is in the cache cache return it to 3. Write it to the DB. the user 4. If data is not in the Overall: cache fetch it from the DB, put it in • Increases reads the cache, return it performance to the user. • Offloads DB reads DB • Introduces race conditions for writes 10 Max A. Alexejev
  • 11. Cache access patterns: Client Read Through For reading data: 1. Application asks for some data for a given key 2. Check the cache 3. If data is in the cache return it to the user Cache 4. If data is not in the cache – cache will invoke fetching it from the DB by himself, saving retrieved value and returning it to the user. Overall: • Reduces reads latency • Offloads read load from underlying storage • May have blocking behavior, thus helping with dog-pile DB effect • Requires “smarter” cache nodes 11 Max A. Alexejev
  • 12. Cache access patterns: Client Write Through For writing data 1. Application writes some new data or updates existing. Cache 2. Write it to the cache 3. Cache will then synchronously write it to the DB. Overall: • Slightly increases writes latency DB • Provides natural invalidation • Removes race conditions on writes 12 Max A. Alexejev
  • 13. Cache access patterns: Client Write Behind For writing data 1. Application writes some new data or updates existing. 2. Write it to the cache Cache adds writes request to its internal queue. 3. Cache 4. Later, cache asynchronously flushes queue to DB on a periodic basis and/or when queue size reaches certain limit. Overall: • Dramatically reduces writes latency by a price of inconsistency window • Provides writes batching • May provide updates deduplication DB 13 Max A. Alexejev
  • 14. A variety of products on the market… Memcached Hazelcast Cassandra GigaSpaces Redis Terracotta Oracle Coherence Infinispan MongoDB Riak EhCache … 14 Max A. Alexejev
  • 15. KV caches NoSQL Data Grids Oracle Memcached Redis Coherence Ehcache Cassandra GemFire Lets sort em out! … MongoDB GigaSpaces Some products are really hard to sort – like Terracotta in both DSO … GridGain and Express modes. Hazelcast Infinispan 15 Max A. Alexejev
  • 16. Why don’t we have any distributed in-memory RDBMS? Master – MultiSlaves configuration • Is, if fact, an example of replication • Helps with reads distribution, but does not help with writes • Does not scale beyond single master Horizontal partitioning (sharding) • Helps with reads and writes for datasets with good data affinity • Does not work nicely with joins semantics (i.e., there are no distributed joins) 16 Max A. Alexejev
  • 17. Key-Value caches • Memcached and EHCache are good examples to look at • Keys and values are arbitrary binary (serializable) entities • Basic operations are put(K,V), get(K), replace(K,V), remove(K) • May provide group operations like getAll(…) and putAll(…) • Some operations provide atomicity guarantees (CAS, inc/dec) 17 Max A. Alexejev
  • 18. Memcached • Developed for LiveJournal in 2003 • Has client libraries in PHP, Java, Ruby, Python and many others • Nodes are independent and don’t communicate with each other 18 Max A. Alexejev
  • 19. EHCache • Initially named “Easy Hibernate Cache” • Java-centric, mature product with open- source and commercial editions • Open-source version provides only replication capabilities, distributed caching requires commercial license for both EHCache and Terracotta TSA 19 Max A. Alexejev
  • 20. NoSQL Systems  A whole bunch of different products with both persistent and non-persistent storage options. Lets call them caches and storages, accordingly.  Built to provide good horizontal scalability  Try to fill the feature gap between pure KV and full-blown RDBMS 20 Max A. Alexejev
  • 21.  Written in C, supported by VMWare  Client libraries for C, C#, Java, Scala, PHP, Erlang, etc  Single-threaded async impl  Has configurable persistence Case study: Redis  Works with K-V pairs, where K is a string and V may be either number, hset users:goku powerlevel 9000 string or Object (JSON) hget users:goku powerlevel  Provides 5 interfaces for: strings, hashes, sorted lists, sets, sorted sets  Supports transactions 21 Max A. Alexejev
  • 22. Use cases: Redis  Good for fixed lists, tagging, ratings, counters, analytics and queues (pub-sub messaging)  Has Master – MultiSlave replication support. Master node is currently a SPOF.  Distributed Redis was named “Redis Cluster” and is currently under development 22 Max A. Alexejev
  • 23. Written in Java, developed in Facebook. • Inspired by Amazon Dynamo replication mechanics, but uses column-based data model. Case study: Cassandra • Good for logs processing, index storage, voting, jobs storage etc. • Bad for transactional processing. • Want to know more? Ask Alexey! 23 Max A. Alexejev
  • 24. In-Memory Data Grids New generation of caching products, trying to combine benefits of replicated and distributed schemes. 24 Max A. Alexejev
  • 25. IMDG: Evolution Data Grids Computational • Reliable storage and Grids live data balancing • Reliable jobs among grid nodes execution, scheduling and load balancing Modern IMDG 25 Max A. Alexejev
  • 26. IMDG: Caching concepts • Implements KV cache interface • Live data redistribution when nodes are going up or down – no data loss, no • Provides indexed search by values clients termination • Provides reliable distributed locks • Supports RT, WT, WB caching patterns interface and hierarchical caches (near caching) • Caching scheme – partitioned or • Supports atomic computations on grid distributed, may be specified per cache nodes or cache service • Provides events subscription for entries (change notifications) • Configurable fault tolerance for distributed schemes (HA) • Equal data (and read/write load) 26 Max A. Alexejev distribution among grid nodes
  • 27. IMDG: Under the hood • All data is split in a number of sections, called partitions. • Partition, rather then entry, is an atomic unit of data migration when grid rebalances. Number of partitions is fixed for cluster lifetime. • Indexes are distributed among grid nodes. • Clients may or may not be part of the grid cluster. 27 Max A. Alexejev
  • 28. IMDG Under the hood: Requests routing For get() and put() requests: 1. Cluster member, that makes a request, calculates key hash code. 2. Partition number is calculated using this hash code. 3. Node is identified by partition number. 4. Request is then routed to identified node, executed, and results are sent back to the client member who initiated request. For filter queries: 1. Cluster member initiating requests sends it to all storage enabled nodes in the cluster. 2. Query is executed on every node using distributed indexes and partial results are sent to the requesting member. 3. Requesting member merges partial results locally. 4. Final result set is returned from filter method. 28 Max A. Alexejev
  • 29. IMDG: Advanced use-cases  Messaging  Map-Reduce calculations  Cluster-wide singleton  And more… 29 Max A. Alexejev
  • 30. GC tuning for large grid nodes  An easy way to go: rolling restarts or storage-enabled cluster nodes. Can not be used in any project.  A complex way to go: fine-tune CMS collector to ensure that it will always keep up cleaning garbage concurrently under normal production workload.  An expensive way to go: use OffHeap storages provided by some vendors (Oracle, Terracotta) and use direct memory buffers available to JVM. 30 Max A. Alexejev
  • 31. IMDG: Market players  Oracle Coherence: commercial, free for evaluation use.  GigaSpaces: commercial.  GridGain: commercial.  Hazelcast: open-source.  Infinispan: open-source. 31 Max A. Alexejev
  • 32. Terracotta A company behind EHCache, Quartz and Terracotta Server Array. Acquired by Software AG. 32 Max A. Alexejev
  • 33. Terracotta Server Array All data is split in a number of sections, called stripes. Stripes consist of 2 or more Terracotta nodes. One of them is Active node, others have Passive status. All data is distributed among stripes and replicated inside stripes. Open Source limitation: only one stripe. Such setup will support HA, but will not distribute cache data. I.e., it is not horizontally scalable. 33 Max A. Alexejev
  • 34. Max A. Alexejev QA Session And thank you for coming!

Editor's Notes

  1. Add pictures
  2. Add pictures
  3. Add pictures
  4. Well, sort em.
  5. Didn’t I forget anything? Master-Master config, 4 example
  6. Fix TBDs
  7. Add picture
  8. Check what Computational Grid really is
  9. Add picture with partitions
  10. TBD
  11. TBD
  12. TBD
  13. Add picture
  14. Add better pic with L1/L2/L3