SlideShare a Scribd company logo

Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons Learned

•
•
Jonas Bonér
Jonas Bonér

The document discusses lessons learned from building scalable, highly concurrent, and fault-tolerant systems, noting that shared mutable state and threads can lead to non-deterministic code, and recommending approaches like dataflow concurrency, actors, and software transactional memory to avoid these issues. It also advocates designing systems to be reactive by using asynchronous messaging and non-blocking I/O, and to be fault-tolerant using failure recovery mechanisms like those enabled by actor models.

Technology
1 of 109
Download to read offline
Building Scalable, Highly Concurrent & Fault-Tolerant Systems: Lessons Learned Jonas Bonér CTO Typesafe Twitter: @jboner
I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Agony I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Agony and Pain I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again lots of Pain I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
Agenda • It’s All Trade-offs • Go Concurrent • Go Reactive • Go Fault-Tolerant • Go Distributed • Go Big
It’s all Trade-offs
Performance vs Scalability
Latency vs Throughput
Availability vs Consistency
Go Concurrent
Shared mutable state
Shared mutable state Together with threads...
Shared mutable state Together with threads... ...leads to
Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to
Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to ...and the root of all EVIL
Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to ...and the root of all EVIL Please, avoid it at all cost
Shared mutable state B L E T U ! A Together with threads... M e!! IM at EVIL ...code that is totally INDETERMINISTIC e t ...leads to s s ...and the root of all U avoid it at all cost Please,
The problem with locks • Locks do not compose • Locks break encapsulation • Taking too few locks • Taking too many locks • Taking the wrong locks • Taking locks in the wrong order • Error recovery is hard
You deserve better tools • Dataflow Concurrency • Actors • Software Transactional Memory (STM) • Agents
Dataflow Concurrency • Deterministic • Declarative • Data-driven • Threads are suspended until data is available • Lazy & On-demand • No difference between: • Concurrent code • Sequential code • Examples: Akka & GPars
Actors •Share NOTHING •Isolated lightweight event-based processes •Each actor has a mailbox (message queue) •Communicates through asynchronous and non-blocking message passing •Location transparent (distributable) •Examples: Akka & Erlang
STM • See the memory as a transactional dataset • Similar to a DB: begin, commit, rollback (ACI) • Transactions are retried upon collision • Rolls back the memory on abort • Transactions can nest and compose • Use STM instead of abusing your database with temporary storage of “scratch” data • Examples: Haskell, Clojure & Scala
Agents • Reactive memory cells (STM Ref) • Send a update function to the Agent, which 1. adds it to an (ordered) queue, to be 2. applied to the Agent asynchronously • Reads are “free”, just dereferences the Ref • Cooperates with STM • Examples: Clojure & Akka
If we could start all over...
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM)
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM) 4. Finally - only if really needed
If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM) 4. Finally - only if really needed • Add Monitors (Locks) and explicit Threads
Go Reactive
Never block • ...unless you really have to • Blocking kills scalability (and performance) • Never sit on resources you don’t use • Use non-blocking IO • Be reactive • How?
Go Async Design for reactive event-driven systems 1. Use asynchronous message passing 2. Use Iteratee-based IO 3. Use push not pull (or poll) • Examples: • Akka or Erlang actors • Play’s reactive Iteratee IO • Node.js or JavaScript Promises • Server-Sent Events or WebSockets • Scala’s Futures library
Go Fault-Tolerant
Failure Recovery in Java/C/C# etc.
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with:
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic • SCATTERED all over the code base
Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control d o • If this thread blows up you are screwed n !! • So you need to do all explicit error handling a ! c r WITHIN this single thread e te • To make things worse - errors do not W et propagate between threads so there is NO WAY OF EVEN FINDING OUT that b something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic • SCATTERED all over the code base
Just Let It Crash
The right way 1. Isolated lightweight processes 2. Supervised processes • Each running process has a supervising process • Errors are sent to the supervisor (asynchronously) • Supervisor manages the failure • Same semantics local as remote • For example the Actor Model solves it nicely
Go Distributed
Performance vs Scalability
How do I know if I have a performance problem?
How do I know if I have a performance problem? If your system is slow for a single user
How do I know if I have a scalability problem?
How do I know if I have a scalability problem? If your system is fast for a single user but slow under heavy load
(Three) Misconceptions about Reliable Distributed Computing - Werner Vogels 1. Transparency is the ultimate goal 2. Automatic object replication is desirable 3. All replicas are equal and deterministic Classic paper: A Note On Distributed Computing - Waldo et. al.
Fallacy 1 Transparent Distributed Computing • Emulating Consistency and Shared Memory in a distributed environment • Distributed Objects • “Sucks like an inverted hurricane” - Martin Fowler • Distributed Transactions • ...don’t get me started...
Fallacy 2 RPC • Emulating synchronous blocking method dispatch - across the network • Ignores: • Latency • Partial failures • General scalability concerns, caching etc. • “Convenience over Correctness” - Steve Vinoski
Instead
Instead Embrace the Network Use it i th w Asynchronous don e Message be d an Passing
Guaranteed Delivery Delivery Semantics • No guarantees • At most once • At least once • Once and only once
It’s all lies.
It’s all lies.
The network is inherently unreliable and there is no such thing as 100% guaranteed delivery It’s all lies.
Guaranteed Delivery
Guaranteed Delivery The question is what to guarantee
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network?
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC?
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue?
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver?
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver? 5. The message is - starting to be processed by the receiver?
Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver? 5. The message is - starting to be processed by the receiver? 6. The message is - has completed processing by the receiver?
Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one
Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one Different USE-CASES Different GUARANTEES Different COSTS
Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one Different USE-CASES Different GUARANTEES Different COSTS For each additional guarantee you add you will either: • decrease performance, throughput or scalability • increase latency
Just
Just Use ACKing
Just Use ACKing and be done with it
Latency vs Throughput
You should strive for maximal throughput with acceptable latency
Go Big
Go Big Data
Big Data Imperative OO programming doesn't cut it • Object-Mathematics Impedance Mismatch • We need functional processing, transformations etc. • Examples: Spark, Crunch/Scrunch, Cascading, Cascalog, Scalding, Scala Parallel Collections • Hadoop have been called the: • “Assembly language of MapReduce programming” • “EJB of our time”
Big Data Batch processing doesn't cut it • Ala Hadoop • We need real-time data processing • Examples: Spark, Storm, S4 etc. • Watch“Why Big Data Needs To Be Functional” by Dean Wampler
Go Big DB
When is a RDBMS not good enough?
Scalingreads to a RDBMS is hard
Scalingwrites to a RDBMS is impossible
Do we really need a RDBMS?
Do we really need a RDBMS? Sometimes...
Do we really need a RDBMS?
Do we really need a RDBMS? But many times we don’t
Atomic Consistent Isolated Durable
Availability vs Consistency
Brewer’s CAP theorem
You can only pick 2 Consistency Availability Partition tolerance At a given point in time
Centralized system • In a centralized system (RDBMS etc.) we don’t have network partitions, e.g. P in CAP • So you get both: Consistency Availability
Distributed system • In a distributed (scalable) system we will have network partitions, e.g. P in CAP • So you get to only pick one: Consistency Availability
Basically Available Soft state Eventually consistent
Think about your data Then think again • When do you need ACID? • When is Eventual Consistency a better fit? • Different kinds of data has different needs • You need full consistency less than you think
How fast is fast enough? • Never guess: Measure, measure and measure • Start by defining a baseline • Where are we now? • Define what is “good enough” - i.e. SLAs • Where do we want to go? • When are we done? • Beware of micro-benchmarks
...or, when can we go for a beer? • Never guess: Measure, measure and measure • Start by defining a baseline • Where are we now? • Define what is “good enough” - i.e. SLAs • Where do we want to go? • When are we done? • Beware of micro-benchmarks
To sum things up... 1. Maximizing a specific metric impacts others • Every strategic decision involves a trade-off • There's no "silver bullet" 2. Applying yesterday's best practices to the problems faced today will lead to: • Waste of resources • Performance and scalability bottlenecks • Unreliable systems
SO
GO
...now home and build yourself Scalable, Highly Concurrent & Fault-Tolerant Systems
Thank You Email: jonas@typesafe.com Web: typesafe.com Twitter: @jboner

Recommended

10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouse10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouserpolat
 
Using ClickHouse for Experimentation
Using ClickHouse for ExperimentationUsing ClickHouse for Experimentation
Using ClickHouse for ExperimentationGleb Kanterov
 
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...DataStax
 
Zookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersZookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersjeetendra mandal
 
Distributed SQL Databases Deconstructed
Distributed SQL Databases DeconstructedDistributed SQL Databases Deconstructed
Distributed SQL Databases DeconstructedYugabyte
 
PostgreSQL replication
PostgreSQL replicationPostgreSQL replication
PostgreSQL replicationNTT DATA OSS Professional Services
 
Fluentd with MySQL
Fluentd with MySQLFluentd with MySQL
Fluentd with MySQLI Goo Lee
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performancePostgreSQL-Consulting
 

Recommended

10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouse10 Good Reasons to Use ClickHouse
10 Good Reasons to Use ClickHouserpolat
 
Using ClickHouse for Experimentation
Using ClickHouse for ExperimentationUsing ClickHouse for Experimentation
Using ClickHouse for ExperimentationGleb Kanterov
 
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...
Replication and Consistency in Cassandra... What Does it All Mean? (Christoph...DataStax
 
Zookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersZookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersjeetendra mandal
 
Distributed SQL Databases Deconstructed
Distributed SQL Databases DeconstructedDistributed SQL Databases Deconstructed
Distributed SQL Databases DeconstructedYugabyte
 
PostgreSQL replication
PostgreSQL replicationPostgreSQL replication
PostgreSQL replicationNTT DATA OSS Professional Services
 
Fluentd with MySQL
Fluentd with MySQLFluentd with MySQL
Fluentd with MySQLI Goo Lee
 
Linux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performanceLinux tuning to improve PostgreSQL performance
Linux tuning to improve PostgreSQL performancePostgreSQL-Consulting
 
ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리confluent
 
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...Altinity Ltd
 
Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )Mydbops
 
Introducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring StatemachineIntroducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring StatemachineVMware Tanzu
 
How is Kafka so Fast?
How is Kafka so Fast?How is Kafka so Fast?
How is Kafka so Fast?Ricardo Paiva
 
Percona XtraDB Cluster
Percona XtraDB ClusterPercona XtraDB Cluster
Percona XtraDB ClusterKenny Gryp
 
Optimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on KubernetesOptimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on KubernetesDinakar Guniguntala
 
Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®confluent
 
MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting Mydbops
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfAlkin Tezuysal
 
Onion Architecture / Clean Architecture
Onion Architecture / Clean ArchitectureOnion Architecture / Clean Architecture
Onion Architecture / Clean ArchitectureAttila Bertók
 
GC free coding in @Java presented @Geecon
GC free coding in @Java presented @GeeconGC free coding in @Java presented @Geecon
GC free coding in @Java presented @GeeconPeter Lawrey
 
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019VMware Tanzu
 
Summary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in TokyoSummary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in TokyoCLOUDIAN KK
 
Dataflow shuffle service
Dataflow shuffle service Dataflow shuffle service
Dataflow shuffle service Yuta Hono
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PGConf APAC
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)Aurimas Mikalauskas
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
 
Load Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesLoad Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesSeveralnines
 
Getting started with postgresql
Getting started with postgresqlGetting started with postgresql
Getting started with postgresqlbotsplash.com
 
Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsNLJUG
 
Akka cluster overview at 010dev
Akka cluster overview at 010devAkka cluster overview at 010dev
Akka cluster overview at 010devRoland Kuhn
 

More Related Content

What's hot

ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리confluent
 
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...Altinity Ltd
 
Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )Mydbops
 
Introducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring StatemachineIntroducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring StatemachineVMware Tanzu
 
How is Kafka so Fast?
How is Kafka so Fast?How is Kafka so Fast?
How is Kafka so Fast?Ricardo Paiva
 
Percona XtraDB Cluster
Percona XtraDB ClusterPercona XtraDB Cluster
Percona XtraDB ClusterKenny Gryp
 
Optimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on KubernetesOptimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on KubernetesDinakar Guniguntala
 
Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®confluent
 
MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting Mydbops
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfAlkin Tezuysal
 
Onion Architecture / Clean Architecture
Onion Architecture / Clean ArchitectureOnion Architecture / Clean Architecture
Onion Architecture / Clean ArchitectureAttila Bertók
 
GC free coding in @Java presented @Geecon
GC free coding in @Java presented @GeeconGC free coding in @Java presented @Geecon
GC free coding in @Java presented @GeeconPeter Lawrey
 
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019VMware Tanzu
 
Summary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in TokyoSummary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in TokyoCLOUDIAN KK
 
Dataflow shuffle service
Dataflow shuffle service Dataflow shuffle service
Dataflow shuffle service Yuta Hono
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PGConf APAC
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)Aurimas Mikalauskas
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Mydbops
 
Load Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesLoad Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesSeveralnines
 
Getting started with postgresql
Getting started with postgresqlGetting started with postgresql
Getting started with postgresqlbotsplash.com
 

What's hot (20)

ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리ksqlDB로 실시간 데이터 변환 및 스트림 처리
ksqlDB로 실시간 데이터 변환 및 스트림 처리
 
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
Introduction to the Mysteries of ClickHouse Replication, By Robert Hodges and...
 
Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )Percona XtraDB Cluster ( Ensure high Availability )
Percona XtraDB Cluster ( Ensure high Availability )
 
Introducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring StatemachineIntroducing Saga Pattern in Microservices with Spring Statemachine
Introducing Saga Pattern in Microservices with Spring Statemachine
 
How is Kafka so Fast?
How is Kafka so Fast?How is Kafka so Fast?
How is Kafka so Fast?
 
Percona XtraDB Cluster
Percona XtraDB ClusterPercona XtraDB Cluster
Percona XtraDB Cluster
 
Optimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on KubernetesOptimizing {Java} Application Performance on Kubernetes
Optimizing {Java} Application Performance on Kubernetes
 
Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®Tips & Tricks for Apache Kafka®
Tips & Tricks for Apache Kafka®
 
MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting MySQL GTID Concepts, Implementation and troubleshooting
MySQL GTID Concepts, Implementation and troubleshooting
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdf
 
Onion Architecture / Clean Architecture
Onion Architecture / Clean ArchitectureOnion Architecture / Clean Architecture
Onion Architecture / Clean Architecture
 
GC free coding in @Java presented @Geecon
GC free coding in @Java presented @GeeconGC free coding in @Java presented @Geecon
GC free coding in @Java presented @Geecon
 
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
Greenplum and Kafka: Real-time Streaming to Greenplum - Greenplum Summit 2019
 
Summary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in TokyoSummary of "Google's Big Table" at nosql summer reading in Tokyo
Summary of "Google's Big Table" at nosql summer reading in Tokyo
 
Dataflow shuffle service
Dataflow shuffle service Dataflow shuffle service
Dataflow shuffle service
 
PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs PostgreSQL WAL for DBAs
PostgreSQL WAL for DBAs
 
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
MySQL Performance Tuning. Part 1: MySQL Configuration (includes MySQL 5.7)
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera ) Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
 
Load Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - SlidesLoad Balancing MySQL with HAProxy - Slides
Load Balancing MySQL with HAProxy - Slides
 
Getting started with postgresql
Getting started with postgresqlGetting started with postgresql
Getting started with postgresql
 

Viewers also liked

Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsNLJUG
 
Akka cluster overview at 010dev
Akka cluster overview at 010devAkka cluster overview at 010dev
Akka cluster overview at 010devRoland Kuhn
 
Slides - Intro to Akka.Cluster
Slides - Intro to Akka.ClusterSlides - Intro to Akka.Cluster
Slides - Intro to Akka.Clusterpetabridge
 
Introduction to Actor Model and Akka
Introduction to Actor Model and AkkaIntroduction to Actor Model and Akka
Introduction to Actor Model and AkkaYung-Lin Ho
 
Introduction to Akka
Introduction to AkkaIntroduction to Akka
Introduction to AkkaJohan Andrén
 
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...Jonas Bonér
 
Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Jonas Bonér
 

Viewers also liked (7)

Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based Applications
 
Akka cluster overview at 010dev
Akka cluster overview at 010devAkka cluster overview at 010dev
Akka cluster overview at 010dev
 
Slides - Intro to Akka.Cluster
Slides - Intro to Akka.ClusterSlides - Intro to Akka.Cluster
Slides - Intro to Akka.Cluster
 
Introduction to Actor Model and Akka
Introduction to Actor Model and AkkaIntroduction to Actor Model and Akka
Introduction to Actor Model and Akka
 
Introduction to Akka
Introduction to AkkaIntroduction to Akka
Introduction to Akka
 
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...
Akka: Simpler Scalability, Fault-Tolerance, Concurrency & Remoting through Ac...
 
Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)
 

More from Jonas Bonér

We are drowning in complexity—can we do better?
We are drowning in complexity—can we do better?We are drowning in complexity—can we do better?
We are drowning in complexity—can we do better?Jonas Bonér
 
Kalix: Tackling the The Cloud to Edge Continuum
Kalix: Tackling the The Cloud to Edge ContinuumKalix: Tackling the The Cloud to Edge Continuum
Kalix: Tackling the The Cloud to Edge ContinuumJonas Bonér
 
The Reactive Principles: Design Principles For Cloud Native Applications
The Reactive Principles: Design Principles For Cloud Native ApplicationsThe Reactive Principles: Design Principles For Cloud Native Applications
The Reactive Principles: Design Principles For Cloud Native ApplicationsJonas Bonér
 
Cloudstate—Towards Stateful Serverless
Cloudstate—Towards Stateful ServerlessCloudstate—Towards Stateful Serverless
Cloudstate—Towards Stateful ServerlessJonas Bonér
 
Designing Events-first Microservices
Designing Events-first MicroservicesDesigning Events-first Microservices
Designing Events-first MicroservicesJonas Bonér
 
How Events Are Reshaping Modern Systems
How Events Are Reshaping Modern SystemsHow Events Are Reshaping Modern Systems
How Events Are Reshaping Modern SystemsJonas Bonér
 
Reactive Microsystems: The Evolution of Microservices at Scale
Reactive Microsystems: The Evolution of Microservices at ScaleReactive Microsystems: The Evolution of Microservices at Scale
Reactive Microsystems: The Evolution of Microservices at ScaleJonas Bonér
 
From Microliths To Microsystems
From Microliths To MicrosystemsFrom Microliths To Microsystems
From Microliths To MicrosystemsJonas Bonér
 
Without Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersWithout Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersJonas Bonér
 
Life Beyond the Illusion of Present
Life Beyond the Illusion of PresentLife Beyond the Illusion of Present
Life Beyond the Illusion of PresentJonas Bonér
 
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven Systems
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven SystemsGo Reactive: Building Responsive, Resilient, Elastic & Message-Driven Systems
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven SystemsJonas Bonér
 
Reactive Supply To Changing Demand
Reactive Supply To Changing DemandReactive Supply To Changing Demand
Reactive Supply To Changing DemandJonas Bonér
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsGo Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsJonas Bonér
 
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspectiveEvent Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspectiveJonas Bonér
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVM
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVMState: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVM
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVMJonas Bonér
 
Pragmatic Real-World Scala (short version)
Pragmatic Real-World Scala (short version)Pragmatic Real-World Scala (short version)
Pragmatic Real-World Scala (short version)Jonas Bonér
 

More from Jonas Bonér (18)

We are drowning in complexity—can we do better?
We are drowning in complexity—can we do better?We are drowning in complexity—can we do better?
We are drowning in complexity—can we do better?
 
Kalix: Tackling the The Cloud to Edge Continuum
Kalix: Tackling the The Cloud to Edge ContinuumKalix: Tackling the The Cloud to Edge Continuum
Kalix: Tackling the The Cloud to Edge Continuum
 
The Reactive Principles: Design Principles For Cloud Native Applications
The Reactive Principles: Design Principles For Cloud Native ApplicationsThe Reactive Principles: Design Principles For Cloud Native Applications
The Reactive Principles: Design Principles For Cloud Native Applications
 
Cloudstate—Towards Stateful Serverless
Cloudstate—Towards Stateful ServerlessCloudstate—Towards Stateful Serverless
Cloudstate—Towards Stateful Serverless
 
Designing Events-first Microservices
Designing Events-first MicroservicesDesigning Events-first Microservices
Designing Events-first Microservices
 
How Events Are Reshaping Modern Systems
How Events Are Reshaping Modern SystemsHow Events Are Reshaping Modern Systems
How Events Are Reshaping Modern Systems
 
Reactive Microsystems: The Evolution of Microservices at Scale
Reactive Microsystems: The Evolution of Microservices at ScaleReactive Microsystems: The Evolution of Microservices at Scale
Reactive Microsystems: The Evolution of Microservices at Scale
 
From Microliths To Microsystems
From Microliths To MicrosystemsFrom Microliths To Microsystems
From Microliths To Microsystems
 
Without Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersWithout Resilience, Nothing Else Matters
Without Resilience, Nothing Else Matters
 
Life Beyond the Illusion of Present
Life Beyond the Illusion of PresentLife Beyond the Illusion of Present
Life Beyond the Illusion of Present
 
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven Systems
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven SystemsGo Reactive: Building Responsive, Resilient, Elastic & Message-Driven Systems
Go Reactive: Building Responsive, Resilient, Elastic & Message-Driven Systems
 
Reactive Supply To Changing Demand
Reactive Supply To Changing DemandReactive Supply To Changing Demand
Reactive Supply To Changing Demand
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsGo Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
 
Introducing Akka
Introducing AkkaIntroducing Akka
Introducing Akka
 
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspectiveEvent Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVM
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVMState: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVM
State: You're Doing It Wrong - Alternative Concurrency Paradigms For The JVM
 
Pragmatic Real-World Scala (short version)
Pragmatic Real-World Scala (short version)Pragmatic Real-World Scala (short version)
Pragmatic Real-World Scala (short version)
 

Recently uploaded

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 

Recently uploaded (20)

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 

Building Scalable, Highly Concurrent & Fault Tolerant Systems - Lessons Learned

  • 1. Building Scalable, Highly Concurrent & Fault-Tolerant Systems: Lessons Learned Jonas Bonér CTO Typesafe Twitter: @jboner
  • 2. I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
  • 3. I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
  • 4. I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Agony I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
  • 5. I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Lessons I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Learned I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again through... I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again Agony and Pain I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again lots of Pain I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again I will never use distributed transactions again
  • 6. Agenda • It’s All Trade-offs • Go Concurrent • Go Reactive • Go Fault-Tolerant • Go Distributed • Go Big
  • 7.
  • 9. Performance vs Scalability
  • 10. Latency vs Throughput
  • 11. Availability vs Consistency
  • 14. Shared mutable state Together with threads...
  • 15. Shared mutable state Together with threads... ...leads to
  • 16. Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to
  • 17. Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to ...and the root of all EVIL
  • 18. Shared mutable state Together with threads... ...code that is totally INDETERMINISTIC ...leads to ...and the root of all EVIL Please, avoid it at all cost
  • 19. Shared mutable state B L E T U ! A Together with threads... M e!! IM at EVIL ...code that is totally INDETERMINISTIC e t ...leads to s s ...and the root of all U avoid it at all cost Please,
  • 20. The problem with locks • Locks do not compose • Locks break encapsulation • Taking too few locks • Taking too many locks • Taking the wrong locks • Taking locks in the wrong order • Error recovery is hard
  • 21. You deserve better tools • Dataflow Concurrency • Actors • Software Transactional Memory (STM) • Agents
  • 22. Dataflow Concurrency • Deterministic • Declarative • Data-driven • Threads are suspended until data is available • Lazy & On-demand • No difference between: • Concurrent code • Sequential code • Examples: Akka & GPars
  • 23. Actors •Share NOTHING •Isolated lightweight event-based processes •Each actor has a mailbox (message queue) •Communicates through asynchronous and non-blocking message passing •Location transparent (distributable) •Examples: Akka & Erlang
  • 24. STM • See the memory as a transactional dataset • Similar to a DB: begin, commit, rollback (ACI) • Transactions are retried upon collision • Rolls back the memory on abort • Transactions can nest and compose • Use STM instead of abusing your database with temporary storage of “scratch” data • Examples: Haskell, Clojure & Scala
  • 25. Agents • Reactive memory cells (STM Ref) • Send a update function to the Agent, which 1. adds it to an (ordered) queue, to be 2. applied to the Agent asynchronously • Reads are “free”, just dereferences the Ref • Cooperates with STM • Examples: Clojure & Akka
  • 26. If we could start all over...
  • 27. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core
  • 28. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming
  • 29. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow
  • 30. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed
  • 31. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming
  • 32. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed
  • 33. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM)
  • 34. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM) 4. Finally - only if really needed
  • 35. If we could start all over... 1. Start with a Deterministic, Declarative & Immutable core • Logic & Functional Programming • Dataflow 2. Add Indeterminism selectively - only where needed • Actor/Agent-based Programming 3. Add Mutability selectively - only where needed • Protected by Transactions (STM) 4. Finally - only if really needed • Add Monitors (Locks) and explicit Threads
  • 37. Never block • ...unless you really have to • Blocking kills scalability (and performance) • Never sit on resources you don’t use • Use non-blocking IO • Be reactive • How?
  • 38. Go Async Design for reactive event-driven systems 1. Use asynchronous message passing 2. Use Iteratee-based IO 3. Use push not pull (or poll) • Examples: • Akka or Erlang actors • Play’s reactive Iteratee IO • Node.js or JavaScript Promises • Server-Sent Events or WebSockets • Scala’s Futures library
  • 40. Failure Recovery in Java/C/C# etc.
  • 41. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control
  • 42. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed
  • 43. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread
  • 44. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed
  • 45. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with:
  • 46. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic
  • 47. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control • If this thread blows up you are screwed • So you need to do all explicit error handling WITHIN this single thread • To make things worse - errors do not propagate between threads so there is NO WAY OF EVEN FINDING OUT that something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic • SCATTERED all over the code base
  • 48. Failure Recovery in Java/C/C# etc. • You are given a SINGLE thread of control d o • If this thread blows up you are screwed n !! • So you need to do all explicit error handling a ! c r WITHIN this single thread e te • To make things worse - errors do not W et propagate between threads so there is NO WAY OF EVEN FINDING OUT that b something have failed • This leads to DEFENSIVE programming with: • Error handling TANGLED with business logic • SCATTERED all over the code base
  • 50.
  • 51. The right way 1. Isolated lightweight processes 2. Supervised processes • Each running process has a supervising process • Errors are sent to the supervisor (asynchronously) • Supervisor manages the failure • Same semantics local as remote • For example the Actor Model solves it nicely
  • 53. Performance vs Scalability
  • 54. How do I know if I have a performance problem?
  • 55. How do I know if I have a performance problem? If your system is slow for a single user
  • 56. How do I know if I have a scalability problem?
  • 57. How do I know if I have a scalability problem? If your system is fast for a single user but slow under heavy load
  • 58. (Three) Misconceptions about Reliable Distributed Computing - Werner Vogels 1. Transparency is the ultimate goal 2. Automatic object replication is desirable 3. All replicas are equal and deterministic Classic paper: A Note On Distributed Computing - Waldo et. al.
  • 59. Fallacy 1 Transparent Distributed Computing • Emulating Consistency and Shared Memory in a distributed environment • Distributed Objects • “Sucks like an inverted hurricane” - Martin Fowler • Distributed Transactions • ...don’t get me started...
  • 60. Fallacy 2 RPC • Emulating synchronous blocking method dispatch - across the network • Ignores: • Latency • Partial failures • General scalability concerns, caching etc. • “Convenience over Correctness” - Steve Vinoski
  • 62. Instead Embrace the Network Use it i th w Asynchronous don e Message be d an Passing
  • 63. Guaranteed Delivery Delivery Semantics • No guarantees • At most once • At least once • Once and only once
  • 66. The network is inherently unreliable and there is no such thing as 100% guaranteed delivery It’s all lies.
  • 68. Guaranteed Delivery The question is what to guarantee
  • 69. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network?
  • 70. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC?
  • 71. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue?
  • 72. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver?
  • 73. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver? 5. The message is - starting to be processed by the receiver?
  • 74. Guaranteed Delivery The question is what to guarantee 1. The message is - sent out on the network? 2. The message is - received by the receiver host’s NIC? 3. The message is - put on the receiver’s queue? 4. The message is - applied to the receiver? 5. The message is - starting to be processed by the receiver? 6. The message is - has completed processing by the receiver?
  • 75. Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one
  • 76. Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one Different USE-CASES Different GUARANTEES Different COSTS
  • 77. Ok, then what to do? 1. Start with 0 guarantees (0 additional cost) 2. Add the guarantees you need - one by one Different USE-CASES Different GUARANTEES Different COSTS For each additional guarantee you add you will either: • decrease performance, throughput or scalability • increase latency
  • 78. Just
  • 80. Just Use ACKing and be done with it
  • 81. Latency vs Throughput
  • 82. You should strive for maximal throughput with acceptable latency
  • 85. Big Data Imperative OO programming doesn't cut it • Object-Mathematics Impedance Mismatch • We need functional processing, transformations etc. • Examples: Spark, Crunch/Scrunch, Cascading, Cascalog, Scalding, Scala Parallel Collections • Hadoop have been called the: • “Assembly language of MapReduce programming” • “EJB of our time”
  • 86. Big Data Batch processing doesn't cut it • Ala Hadoop • We need real-time data processing • Examples: Spark, Storm, S4 etc. • Watch“Why Big Data Needs To Be Functional” by Dean Wampler
  • 88. When is a RDBMS not good enough?
  • 89. Scalingreads to a RDBMS is hard
  • 90. Scalingwrites to a RDBMS is impossible
  • 92. Do we really need a RDBMS? Sometimes...
  • 94. Do we really need a RDBMS? But many times we don’t
  • 96. Availability vs Consistency
  • 98. You can only pick 2 Consistency Availability Partition tolerance At a given point in time
  • 99. Centralized system • In a centralized system (RDBMS etc.) we don’t have network partitions, e.g. P in CAP • So you get both: Consistency Availability
  • 100. Distributed system • In a distributed (scalable) system we will have network partitions, e.g. P in CAP • So you get to only pick one: Consistency Availability
  • 102. Think about your data Then think again • When do you need ACID? • When is Eventual Consistency a better fit? • Different kinds of data has different needs • You need full consistency less than you think
  • 103. How fast is fast enough? • Never guess: Measure, measure and measure • Start by defining a baseline • Where are we now? • Define what is “good enough” - i.e. SLAs • Where do we want to go? • When are we done? • Beware of micro-benchmarks
  • 104. ...or, when can we go for a beer? • Never guess: Measure, measure and measure • Start by defining a baseline • Where are we now? • Define what is “good enough” - i.e. SLAs • Where do we want to go? • When are we done? • Beware of micro-benchmarks
  • 105. To sum things up... 1. Maximizing a specific metric impacts others • Every strategic decision involves a trade-off • There's no "silver bullet" 2. Applying yesterday's best practices to the problems faced today will lead to: • Waste of resources • Performance and scalability bottlenecks • Unreliable systems
  • 106. SO
  • 107. GO
  • 108. ...now home and build yourself Scalable, Highly Concurrent & Fault-Tolerant Systems
  • 109. Thank You Email: jonas@typesafe.com Web: typesafe.com Twitter: @jboner