Recent releases of RabbitMQ have made the most popular open-source message broker even more rock solid. The team has made it easier to deploy and easier to operate from Day 2 on. As a de facto standard for message-based architectures, this is great news for teams building microservices and other distributed applications. But let’s face it: not everyone is running the latest version and taking advantage of these new stability and operational features. Now the next major version (3.8) is shipping and it has some breakthrough features for reliability and scaling. Are you ready? Join Pivotal’s Michael Klishin and Karl Nilsson, RabbitMQ Software Engineers, as they share what’s new in RabbitMQ 3.8. You’ll learn: ● How making RabbitMQ persistent and fault tolerant is evolving from mirrored queues to quorum queues ● What mixed-version clusters are and how they simplify upgrades ● How RabbitMQ 3.8 continues to make deploying and operating RabbitMQ easier ● What’s available in terms of OAuth 2.0 support, monitoring improvements, and more ● What features are in development for future versions and minor releases Speakers: Michael Klishin, RabbitMQ Software Engineer Karl Nilsson, RabbitMQ Software Engineer

1. © Copyright 2019 Pivotal Software, Inc. All rights Reserved.
Karl Nilsson and Michael Klishin
May 2019
What’s New in RabbitMQ 3.8

2. Cover w/ Image
Agenda
! Introduction
! Quorum Queues
! Simplified Upgrades
! Observability
! OAuth 2.0
! Erlang 22 Support
! 3.8 availability
! Q+A

3. Intro
What’s New in RabbitMQ 3.8

4. Intro
Michael Klishin
! Staff engineer at Pivotal
Who are we?

5. Intro
Karl Nilsson
! Staff engineer at Pivotal
Who are we?

6. Quorum Queues
What’s New in RabbitMQ 3.8

7. Quorum Queues in RabbitMQ 3.8
Mirrored Queues
! Use a variation of Chained Replication
! Suboptimal data transfers
๏ Mostly a linear algorithm
! Does not deal with link failures optimally
Quorum Queues
! Recovery from failures is predictable and well
defined
! Queue sync after failure transfers as little data
as possible
! Passes more resiliency tests
! No ring topology, parallel replication
! Lower bandwidth usage*
! Opens the door to off-site and multi-DC
replication

8. Quorum Queues in RabbitMQ 3.8
Queue Types
! Classic (mirrored) queues
! Quorum queues

9. Quorum Queues in RabbitMQ 3.8
How are Quorum Queues Different?
! Better defined leader election semantics
! Follower promotion is very fast (minimal data transfers) and non-blocking
! Better data safety and better defined consistency model
! All messages are persisted
! Requires the majority of nodes to be online in order to be available
! Favours consistency over availability
! Higher peak memory usage compared to classic mirrored queues
๏ Can be limited via configuration
! Higher latency
๏ Replicates and fsyncs all operations
! Several classic mirrored queue features are missing, some are by design

10. Quorum Queues in RabbitMQ 3.8
Quorum Queues Use Cases
! Durable, persistent, replicated
! For clusters with 3 or more nodes only
! Only when you need the guarantees or recovery features it provides
! Higher latency is acceptable
! Not doing mass fanouts

11. Quorum Queues in RabbitMQ 3.8
Quorum Queues: When Not to Use
! Unless applications do their part to ensure data safety: publisher and consumer confirms,
mandatory publishing
! Very long backlogs
! Queues transient in nature
! Low latency workloads
! Memory constrained systems
๏ At least not with all defaults

12. Quorum Queues in RabbitMQ 3.8
Quorum Queues: Features
! Dead lettering
! Delivery count and poison message handling
! In memory limits
! Only one queue length limit strategy: drop head
! Single Active Consumer (SAC)

13. Quorum Queues in RabbitMQ 3.8
Quorum Queues: Features Under Consideration
! Message TTL
! Queue length limit reject publish

14. Simplified Upgrades
What’s New in RabbitMQ 3.8

15. Simplified Upgrades in RabbitMQ 3.8
Upgrades from 3.6 to 3.7
! Cluster-wide shutdown
! Blue/Green deployment (spin a new cluster,
switch traffic over)
Upgrades from 3.7 to 3.8
! Rolling upgrade (one node at a time)
๏ Feature flags control what new features are
enabled
๏ A feature not present in older versions won’t
be available until all cluster members
support it and it’s been enabled
๏ New step for deployment automation tools:
enable feature flags
๏ Nodes that do not support feature flags
enabled in a cluster cannot join that cluster
! Blue/Green deployment
! Cluster-wide shutdown might still be necessary
in a future version depending on the nature of
changes

16. Observability
What’s New in RabbitMQ 3.8

17. Observability in RabbitMQ 3.8
Observability
! New metrics
๏ Inter-node communication (Erlang distribution) metrics (e.g. to detect bottlenecks)
๏ Unroutable message rate
๏ Empty basic.get (polling) responses
! New built-in plugin that exposes metrics to Prometheus
๏ Long term metric storage
๏ Lower overhead, separate from the system it monitors
! Visualised by Grafana
๏ Pre-built dashboards
๏ Custom dashboards
๏ Powerful tools for aggregation and correlation

19. OAuth 2.0
What’s New in RabbitMQ 3.8

20. OAuth 2.0 Support in RabbitMQ 3.8
OAuth 2.0 Support
! Allows clients use OAuth 2.0/JWT tokens to authenticate and claim permissions
! Implemented as a new plugin
! OAuth 2.0/JWT token scopes that follow naming conventions are translated to RabbitMQ
permissions
! Clients can use any OAuth 2.0 authorization code flow

21. OAuth 2.0 Support in RabbitMQ 3.8
OAuth 2.0 Support
! Management UI will use the authorization code flow
! Officially supported clients can simplify token renewal
! Token renewal for Shovel and Federation plugin is an open-ended question

22. Erlang 22.0 Support
What’s New in RabbitMQ 3.8

23. Erlang 22.0 Support in RabbitMQ 3.8
Erlang 22.0 Support
! New inter-node communication implementation
๏ Improved handling of large payloads, lower latency
! Efficiency improvements
! Experimental TLSv1.3 support
๏ TLSv1.1 and earlier are disabled by default

24. 3.8 Availability
What’s New in RabbitMQ 3.8

25. OAuth 2.0 Support in RabbitMQ 3.8
3.8 Availability
! Est. Q2-Q3 2019
! 3.8.0-beta.4 is available from GitHub and Bintray today
! Documentation on next.rabbitmq.com: Quorum Queues, Feature Flags

26. Questions?
What’s New in RabbitMQ 3.8

27. Thank you!
What’s New in RabbitMQ 3.8

28. Transforming How The World Builds Software
© Copyright 2019 Pivotal Software, Inc. All rights Reserved.