MQTT is the most popular IoT protocol for connecting devices at scale and a modern alternative for lightweight backend (microservice) communication. This session covers everything you need to know about scalable pub/sub communication with MQTT for up to millions of devices and shows the available software options in the (open source) ecosystem. We also address the brand new features of MQTT 5 as well as advanced microservice integration topics.
6. Introduction
• HiveMQ CTO
• Strong background in distributed
and large scale systems
architecture
• OASIS MQTT TC Member
• Author of “The Technical
Foundations of IoT”
• Conference Speaker
• Program committee member for
German and international IoT
conferences
Dominik
Obermaier
@dobermai
9. IOT CHALLENGES
➤ Unreliable communication channels (e.g.
mobile)
➤ Constrained Devices
➤ Low Bandwidth and High Latency
environments
➤ Bi-directional communication required
➤ Security
➤ Instantaneous data exchange
10. HTTP?
➤ Most popular web protocol
➤ Designed for the human web
➤ Request / Response based
➤ Document centric
➤ No Quality of service
➤ Stateless
➤ Text based (binary with HTTP/2)
➤ No Push capabilities
➤ Not possible to get notified when a client
is offline
14. WHAT IS MQTT?
➤ IoT Messaging Protocol
➤ Minimal Overhead
➤ Publish / Subscribe
➤ Easy
➤ Binary
➤ Data Agnostic
➤ Designed for reliable
communication over unreliable
channels
15. USE CASES
➤ Push Communication
➤ Unreliable communication
channels (e.g. mobile)
➤ Constrained Devices
➤ Low Bandwidth and High Latency
environments
➤ Communication from backend to
IoT device
➤ Lightweight backen communication
21. CHALLENGE 1:
HUGE AMOUNT OF TCP CONNECTIONS
➤ MQTT uses persistent
TCP connections
➤ Number of concurrent
TCP connections
limited (hard to scale >
1-2 Million on a single
machine)
➤ “Reconnect Storms”
22. CHALLENGE 2:
SECURITY
➤ TLS for secure communication
➤ TLS handshake is resource
intensive
➤ Brokers accessible over the
Internet
➤ Scalable backend systems for
authentication & authorization
needed
23. CHALLENGE 3:
MQTT IS STATEFUL
➤ MQTT allows sessions
➤ Subscriptions, Queued
Messages, Message Flow
remembered for offline client
➤ Retained Messages for every
topic possible (dynamic
topics!)
24. CHALLENGE 4:
HIGH AVAILABILITY
➤ MQTT Brokers are a single point
of failure
➤ Cloud Environments WILL fail
➤ Bridged Brokers (brokers that are
chained via MQTT) do not
support session migration
25. CHALLENGE 5:
BACKEND CONNECTIVITY
➤ IoT projects are seldom green-field
➤ MQTT brokers must be integrated
into existing infrastructure (DB,
Message Queues, Microservices)
➤ Authentication & Authorization
Backend Applications
➤ Data Ingestion Backends often
need to scale linearly to cope with
incoming data
29. HIVEMQ CLUSTER ADVANTAGES
➤ Elimination of the Single Point
of Failure
➤ Sophisticated message
distribution across cluster
nodes
➤ Clients can connect to any
node to resume their sessions
➤ Linearly scalable
➤ High Availability due to
resilience and fault-tolerance
➤ Zero-Downtime Upgrades
30. HIVEMQ CLUSTER DETAILS
➤ Masterless Architecture
➤ Highly Resilient
➤ Designed for availability
➤ Elastic. You can add and remove nodes at
runtime
➤ Network partitions are supported
➤ Data distribution across cluster nodes
➤ Configurable Replica Count
➤ Self-Healing mechanisms when merging
network partitions
➤ No Zookeeper :-)
Broker 2 Broker 3
Broker 1
MQTT Broker Cluster
34. PROBLEMS AND PAINS
➤ Orchestration is part of the IoT device AND the
backend
➤ Multiple Steps involved in a business
booking on the client and for microservices
➤ Firmware Update needed for every change in
flow. IoT device updates are hard to deploy
➤ No “You build it you run it” for devices
➤ In case of orchestrating service: can’t send data
to client proactively
➤ Internal microservices need to be exposed to
the Internet
➤ Push and pull communication combined
Broker 2 Broker 3
Broker 1
MQTT Broker Cluster