SlideShare a Scribd company logo
1 of 70
Download to read offline
* Wix Scale / concerns
* polyglot
* Easily change
*performance/scale
* broker
* Opinionated Types
* No explicit commit
Ordered
Unordered
* Each service, metrics like
* First Group topic
REST
* 2nd topic
REST
REST
REST
REST
* distributed
* In memory
DC A
(Kafka Cluster A)
DC B
(Kafka Cluster A)
DC B
(Kafka Cluster A)
DC A
(Kafka Cluster A)
COPIES
IN REAL
TIME
* 3rd topic
How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup
How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup
How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup
How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup
How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup

More Related Content

More from Natan Silnitsky

Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Natan Silnitsky
 
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...Workflow Engines & Event Streaming Brokers - Can they work together? [Current...
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...Natan Silnitsky
 
DevSum - Lessons Learned from 2000 microservices
DevSum - Lessons Learned from 2000 microservicesDevSum - Lessons Learned from 2000 microservices
DevSum - Lessons Learned from 2000 microservicesNatan Silnitsky
 
GeeCon - Lessons Learned from 2000 microservices
GeeCon - Lessons Learned from 2000 microservicesGeeCon - Lessons Learned from 2000 microservices
GeeCon - Lessons Learned from 2000 microservicesNatan Silnitsky
 
Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL
Migrating to Multi Cluster Managed Kafka - ApacheKafkaILMigrating to Multi Cluster Managed Kafka - ApacheKafkaIL
Migrating to Multi Cluster Managed Kafka - ApacheKafkaILNatan Silnitsky
 
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven Microservices
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven MicroservicesWix+Confluent Meetup - Lessons Learned from 2000 Event Driven Microservices
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven MicroservicesNatan Silnitsky
 
BuildStuff - Lessons Learned from 2000 Event Driven Microservices
BuildStuff - Lessons Learned from 2000 Event Driven MicroservicesBuildStuff - Lessons Learned from 2000 Event Driven Microservices
BuildStuff - Lessons Learned from 2000 Event Driven MicroservicesNatan Silnitsky
 
Lessons Learned from 2000 Event Driven Microservices - Reversim
Lessons Learned from 2000 Event Driven Microservices - ReversimLessons Learned from 2000 Event Driven Microservices - Reversim
Lessons Learned from 2000 Event Driven Microservices - ReversimNatan Silnitsky
 
Devoxx Ukraine - Kafka based Global Data Mesh
Devoxx Ukraine - Kafka based Global Data MeshDevoxx Ukraine - Kafka based Global Data Mesh
Devoxx Ukraine - Kafka based Global Data MeshNatan Silnitsky
 
Devoxx UK - Migrating to Multi Cluster Managed Kafka
Devoxx UK - Migrating to Multi Cluster Managed KafkaDevoxx UK - Migrating to Multi Cluster Managed Kafka
Devoxx UK - Migrating to Multi Cluster Managed KafkaNatan Silnitsky
 
Dev Days Europe - Kafka based Global Data Mesh at Wix
Dev Days Europe - Kafka based Global Data Mesh at WixDev Days Europe - Kafka based Global Data Mesh at Wix
Dev Days Europe - Kafka based Global Data Mesh at WixNatan Silnitsky
 
Kafka Summit London - Kafka based Global Data Mesh at Wix
Kafka Summit London - Kafka based Global Data Mesh at WixKafka Summit London - Kafka based Global Data Mesh at Wix
Kafka Summit London - Kafka based Global Data Mesh at WixNatan Silnitsky
 
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative Natan Silnitsky
 
5 Takeaways from Migrating a Library to Scala 3 - Scala Love
5 Takeaways from Migrating a Library to Scala 3 - Scala Love5 Takeaways from Migrating a Library to Scala 3 - Scala Love
5 Takeaways from Migrating a Library to Scala 3 - Scala LoveNatan Silnitsky
 
Migrating to Multi Cluster Managed Kafka - DevopStars 2022
Migrating to Multi Cluster Managed Kafka - DevopStars 2022Migrating to Multi Cluster Managed Kafka - DevopStars 2022
Migrating to Multi Cluster Managed Kafka - DevopStars 2022Natan Silnitsky
 
Open sourcing a successful internal project - Reversim 2021
Open sourcing a successful internal project - Reversim 2021Open sourcing a successful internal project - Reversim 2021
Open sourcing a successful internal project - Reversim 2021Natan Silnitsky
 
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021Natan Silnitsky
 
Advanced Caching Patterns used by 2000 microservices - Code Motion
Advanced Caching Patterns used by 2000 microservices - Code MotionAdvanced Caching Patterns used by 2000 microservices - Code Motion
Advanced Caching Patterns used by 2000 microservices - Code MotionNatan Silnitsky
 
Advanced Caching Patterns used by 2000 microservices - Devoxx Ukraine
Advanced Caching Patterns used by 2000 microservices - Devoxx UkraineAdvanced Caching Patterns used by 2000 microservices - Devoxx Ukraine
Advanced Caching Patterns used by 2000 microservices - Devoxx UkraineNatan Silnitsky
 
Advanced Microservices Caching Patterns - Devoxx UK
Advanced Microservices Caching Patterns - Devoxx UKAdvanced Microservices Caching Patterns - Devoxx UK
Advanced Microservices Caching Patterns - Devoxx UKNatan Silnitsky
 

More from Natan Silnitsky (20)

Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
 
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...Workflow Engines & Event Streaming Brokers - Can they work together? [Current...
Workflow Engines & Event Streaming Brokers - Can they work together? [Current...
 
DevSum - Lessons Learned from 2000 microservices
DevSum - Lessons Learned from 2000 microservicesDevSum - Lessons Learned from 2000 microservices
DevSum - Lessons Learned from 2000 microservices
 
GeeCon - Lessons Learned from 2000 microservices
GeeCon - Lessons Learned from 2000 microservicesGeeCon - Lessons Learned from 2000 microservices
GeeCon - Lessons Learned from 2000 microservices
 
Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL
Migrating to Multi Cluster Managed Kafka - ApacheKafkaILMigrating to Multi Cluster Managed Kafka - ApacheKafkaIL
Migrating to Multi Cluster Managed Kafka - ApacheKafkaIL
 
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven Microservices
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven MicroservicesWix+Confluent Meetup - Lessons Learned from 2000 Event Driven Microservices
Wix+Confluent Meetup - Lessons Learned from 2000 Event Driven Microservices
 
BuildStuff - Lessons Learned from 2000 Event Driven Microservices
BuildStuff - Lessons Learned from 2000 Event Driven MicroservicesBuildStuff - Lessons Learned from 2000 Event Driven Microservices
BuildStuff - Lessons Learned from 2000 Event Driven Microservices
 
Lessons Learned from 2000 Event Driven Microservices - Reversim
Lessons Learned from 2000 Event Driven Microservices - ReversimLessons Learned from 2000 Event Driven Microservices - Reversim
Lessons Learned from 2000 Event Driven Microservices - Reversim
 
Devoxx Ukraine - Kafka based Global Data Mesh
Devoxx Ukraine - Kafka based Global Data MeshDevoxx Ukraine - Kafka based Global Data Mesh
Devoxx Ukraine - Kafka based Global Data Mesh
 
Devoxx UK - Migrating to Multi Cluster Managed Kafka
Devoxx UK - Migrating to Multi Cluster Managed KafkaDevoxx UK - Migrating to Multi Cluster Managed Kafka
Devoxx UK - Migrating to Multi Cluster Managed Kafka
 
Dev Days Europe - Kafka based Global Data Mesh at Wix
Dev Days Europe - Kafka based Global Data Mesh at WixDev Days Europe - Kafka based Global Data Mesh at Wix
Dev Days Europe - Kafka based Global Data Mesh at Wix
 
Kafka Summit London - Kafka based Global Data Mesh at Wix
Kafka Summit London - Kafka based Global Data Mesh at WixKafka Summit London - Kafka based Global Data Mesh at Wix
Kafka Summit London - Kafka based Global Data Mesh at Wix
 
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative
Migrating to Multi Cluster Managed Kafka - Conf42 - CloudNative
 
5 Takeaways from Migrating a Library to Scala 3 - Scala Love
5 Takeaways from Migrating a Library to Scala 3 - Scala Love5 Takeaways from Migrating a Library to Scala 3 - Scala Love
5 Takeaways from Migrating a Library to Scala 3 - Scala Love
 
Migrating to Multi Cluster Managed Kafka - DevopStars 2022
Migrating to Multi Cluster Managed Kafka - DevopStars 2022Migrating to Multi Cluster Managed Kafka - DevopStars 2022
Migrating to Multi Cluster Managed Kafka - DevopStars 2022
 
Open sourcing a successful internal project - Reversim 2021
Open sourcing a successful internal project - Reversim 2021Open sourcing a successful internal project - Reversim 2021
Open sourcing a successful internal project - Reversim 2021
 
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021
How to successfully manage a ZIO fiber’s lifecycle - Functional Scala 2021
 
Advanced Caching Patterns used by 2000 microservices - Code Motion
Advanced Caching Patterns used by 2000 microservices - Code MotionAdvanced Caching Patterns used by 2000 microservices - Code Motion
Advanced Caching Patterns used by 2000 microservices - Code Motion
 
Advanced Caching Patterns used by 2000 microservices - Devoxx Ukraine
Advanced Caching Patterns used by 2000 microservices - Devoxx UkraineAdvanced Caching Patterns used by 2000 microservices - Devoxx Ukraine
Advanced Caching Patterns used by 2000 microservices - Devoxx Ukraine
 
Advanced Microservices Caching Patterns - Devoxx UK
Advanced Microservices Caching Patterns - Devoxx UKAdvanced Microservices Caching Patterns - Devoxx UK
Advanced Microservices Caching Patterns - Devoxx UK
 

Recently uploaded

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 

Recently uploaded (20)

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 

How to build 1000 microservices with Kafka and thrive - Wix @Scale Meetup