Micro-Services RabbitMQ vs Kafka
•Download as PPTX, PDF•
4 likes•289 views
RabbitMQ and Kafka are the most popular brokers, they are often used to integrate microservices.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Micro-Services RabbitMQ vs Kafka
Similar to Micro-Services RabbitMQ vs Kafka (20)
Recently uploaded
Recently uploaded (20)
Micro-Services RabbitMQ vs Kafka
- 1. The Game of Brokers RabbitMQ vs Kafka “The message is coming”
- 2. 2 Agenda ● Introduction ● Demo Live ● Real Use case Questions ● Do you have microservices in production ? ● How many of you know what RabbitMQ and Kafka are?
- 3. Micro-Services, let’s start 3 Service 1 <<<<Service 2 Service 3 Service 4
- 4. 4
- 6. Micro-Services, Full meshed 6 Suppose you have 20 services 380 connections (or more) ([N(N-1)]/2)*2 Complex Topology
- 7. Micro-Services, Full meshed 7 Avoid full meshed connections. if you need it, You should use some service discovering (Consul/ETCD)
- 8. Micro-Services, messaging integration 8 Service 1 Service 3 Broker(s) Service 2 Service 4 msg msg
- 9. Micro-Services, publish and subscribe 9 Producer Queue Consumer 2 Consumer 1
- 10. 10
- 11. Micro-Services, Broker 11 Suppose you have 20 services 20 connections (or more) Easy Topology
- 12. 12 If the service integration crashes
- 13. 13
- 14. 14 RabbitMQ Kafka Open Source Pivotal Apache Beam VM (Erlang/Elixir) JVM (Scala/Java) Distributed message broker Distributed streaming platform
- 15. 15 RabbitMQ Kafka Exchange(s) Queue(s) Partition 1 Partition 2 Partition 3 TopicVirtual Host msg msg msg msg
- 16. 16 Demo Time -let’s order something Event sourcing id = 1 status=new_order what=coat id = 1 status=delivered what=coat name=Jon Snow address=Winterfell
- 17. Pros 17 RabbitMQ Kafka Complex and dynamic routing Very Fast Supports lot of AMQP (others) clients Event-Store (may not need DB) Messages policies (DLX) Handle GB without problems Distributed Remote Calls Streaming Scaling consumers (round-robin) Handle several consumers
- 18. Use cases 18 RabbitMQ Kafka System integration System integration Distributed Task Queue Event-Store Scaling applications Log system
- 19. 19 Before deciding... Yes _must_ have an idea I don’t believe in generic framework. ( if one day I want to switch) Invest time in order to know the tool Don’t create a “spaghetti” design
- 20. OpenStack (Currently) 20 RabbitMQ: Distributed RPC calls RabbitMQ: Events dispatcher (lot of subscribers)
- 21. OpenStack (Thinking/working about) 21 RabbitMQ: Distributed RPC calls Kafka: Events dispatcher (lot of subscribers)
- 22. So, which is the best? 22 RabbitMQ and Kafka solve two different problems. Company knowledge is important ….. Why not both?
- 23. 23 About ME ● Senior Developer @SUSE ● Work on OpenStack middleware modules ● RabbitMQ member/supporter ● Co-Author of RabbitMQ cookbook ● @gsantomaggio /twitter/github/IRC
- 24. No one is born hating another person because of the color of his skin, or his background, or his religion. People must learn to hate, and if they can learn to hate, they can be taught to love, for love comes more naturally to the human heart than its opposite Any questions? You can find me at @gsantomaggio 24
Editor's Notes
- Today we talk about three points: Kafka Vs RabbitMQ How they work Which one is better These 3 point is what I call “the games of throne”
- Short introduction Demo live Real use case Who does think that RabbitMQ is better than Kafka ? Who does think that Kafka is better than RabbitMQ ? If not will explain
- Let’s start speaking about MS integration, let’s suppose you want to start building the application You start to write you first MS, at some point you need to connect it another one Then 3 and 4… so ok, we happy
- So you speak with your team, and decide to link all ( or most of them) services each other Sounds good. Each module must be notified by the change
- Every works fine with few services With lot of services could be a problem
- Complex topology Each module must be notified by the change
- Communication is sync
- Let’s use a service bus, the services are not connected each others but through The communication is asynchronous
- Let’s use a service bus, the services are not connected each others but through The communication is asynchronous Even the consumer is down, the broker stores the messages
- easy topology Single point of failure ( later speak about clustering )
- Oh no again…..
- First is complex Single point of failure that you have to handle
- Main concepts In rabbitmq publish to exchange and consume from a queue, the queue is the unit scale Produce and consume to topic/partition and consume from there RabbitMQ deletes the messages after consumed a message, Kafka does not Explain better the concepts of messages
- Working with event sourcing You have your store with different products, for example clothes, food, books,
- Speak about the consumer group Kafka is fast, stores the messages for long time RMQ distribute the work to more cosumers You cannot remove a partition
- Speak about the consumer group Kafka is fast, stores the messages for long time RMQ distribute the work to more cosumers