At Wix, we have created a universal event-driven programming infrastructure on top of the Kafka message broker.
This infra makes sure messages are eventually successfully consumed and produced no matter what failure it encounters.
In this talk, you will learn about the features we introduced in order to make sure our distributed system can safely handle an ever growing message throughput in a fault tolerant manner.
You will be introduced to such techniques as retry topics, local persistent queues, and cooperative fibers that help make your flows more resilient and performant.
You will also learn how to make this infra work for all programming languages tech stacks with optimal resource manage using the power of Kubernetes and gRPC.
When to use a client library, and when to deploy an external pod (DaemonSet, StatefulSet) or even deploy a sidecar.
54. So, We use Kubernetes’ flexibility
to deploy Greyhound producers and consumers
in different patterns, in order to comply with
different requirements.
@NSilnitsky
55. Wix harnesses Kafka, Kubernetes and gRPC to
achieve a polyglot, fault tolerant, scalable
event-driven distributed system.