ScaleDay conference

1. 1

2. Why and how we proxy our IoT brokerWhy and how we proxy our IoT broker
connectionsconnections
MQTT at scale
2

3. Clément DECOODT
DevOps - IoT
3

4. IoT?IoT?
4

5. Hello, IoT world!Hello, IoT world!
MQTT 101
IoT platform architecture
More details about the reverse proxy
5

6. MQTTMQTT
Message Queuing Telemetry Transport
Not a message queue!
Publish-Subscribe protocol
6 . 1

7. 25
BrokerClient A
CONNECT
SUBSCRIBE
server/temperature
CONNACK
Client B
PUBLISH
server/temperature
25 °C
PUBLISH
server/temperature
25 °C
25
DISCONNECT

8. Simon A. Eugster [ ],CC BY-SA 4.0 via Wikimedia Commons
6 . 2

9. MQTTMQTT
First specs: 1999
OASIS Standard since 2014
Ultra light protocol
Multiple endpoints: TCP, TLS, WebSocket
6 . 3

10. CLICLI
Subscribing
Publishing
mosquitto_sub -i $deviceid
-h iot.fr-par.scw.cloud
-t $topic
1
2
3
mosquitto_pub -i $deviceid
-h iot.fr-par.scw.cloud
-t $topic
-m $message
1
2
3
4
6 . 4

11. from paho.mqtt import client as mqtt
client = mqtt.Client("electric_meter")
client.connect("iot.fr-par.scw.cloud", 1883)
def on_message(client, userdata, message):
print("Received: ",
str(message.payload.decode("utf-8")))
client.on_message = on_message
client.subscribe("switches")
client.publish("info","Electric meter v0.0.1")
client.loop_start()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
6 . 5

12. IoT StationIoT Station
Episode 1: the hubEpisode 1: the hub
7

13. IoT at ScalewayIoT at Scaleway
Connecting devices together
Give access to cloud resources to devices
Deploy code to the devices
8 . 1

14. IoT at ScalewayIoT at Scaleway
Connecting devices together
8 . 1

15. IoT at ScalewayIoT at Scaleway
Product plans
Shared broker
Dedicated
High availability
8 . 2

16. Platform architecturePlatform architecture
9 . 1

17. ConstraintsConstraints
gRPC server for API
Single MQTT endpoint for all customers
9 . 2

18. Control planeControl plane
Server-Worker architecture
Workers act as state synchronizers
9 . 3

19. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
9 . 4

20. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
9 . 4

21. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking 2 - Wrapping2 - Wrapping
9 . 4

22. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
2 - Wrapping2 - Wrapping
9 . 4

23. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
2 - Wrapping2 - Wrapping
9 . 4

24. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
2 - Wrapping2 - Wrapping
9 . 4

25. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
2 - Wrapping2 - Wrapping
9 . 4

26. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
9 . 4

27. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
9 . 4

28. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
9 . 4

29. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
9 . 4

30. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
Broker-independent
9 . 4

31. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
Broker-independent
9 . 4

32. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
Broker-independent
No service interruption
9 . 4

33. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives
Let's not reinvent the wheel
1 - Forking1 - Forking
Hard to maintain
Broker-dependent
Better access to the internals
2 - Wrapping2 - Wrapping
Separation of concerns
Broker-independent
No service interruption
9 . 4

34. API
Gateway
gRPC API
K8S
Worker
Database
BrokersKubernetes
Broker 1 Broker N
Brokers
(mosquitto)
Homemade
Legend
Open-Source software
Managed product
9 . 5

35. API
Gateway
gRPC API
K8S
Worker
Database
BrokersKubernetes
Broker 1 Broker N
Brokers
(mosquitto)
Homemade
Legend
Open-Source software
Managed product
Load
Balancer
MQTT
Reverse
Proxy
9 . 6

36. API
Gateway
gRPC API
K8S
Worker
Database
BrokersKubernetes
Broker 1 Broker N
Brokers
(mosquitto)
Homemade
Legend
Open-Source software
Managed product
Load
Balancer
MQTT
Reverse
Proxy
TSDB
(InfluxDB)
9 . 7

37. MQTT Proxy internalsMQTT Proxy internals
10

38. Wanted featuresWanted features
Horizontally scalable
TCP and WebSocket endpoints
Device-related metrics
TLS Termination
MQTT parser on-the-fly
11 . 1

39. Pipeline stages explainedPipeline stages explained
11 . 2

40. Pipeline stages explainedPipeline stages explained
1 - WebSocket/TCP endpoint
11 . 2

41. Pipeline stages explainedPipeline stages explained
2 - TLS termination
1 - WebSocket/TCP endpoint
11 . 2

42. Pipeline stages explainedPipeline stages explained
3 - MQTT Packet metadata parsing
1 - WebSocket/TCP endpoint
2 - TLS termination
11 . 2

43. Pipeline stages explainedPipeline stages explained
4 - Device/Hub-related metadata fetching
1 - WebSocket/TCP endpoint
2 - TLS termination
3 - MQTT Packet metadata parsing
11 . 2

44. Pipeline stages explainedPipeline stages explained
5 - Authentication/Authorization
1 - WebSocket/TCP endpoint
2 - TLS termination
3 - MQTT Packet metadata parsing
4 - Device/Hub-related metadata fetching
11 . 2

45. Pipeline stages explainedPipeline stages explained
6 - Optional topic rewriting
1 - WebSocket/TCP endpoint
2 - TLS termination
3 - MQTT Packet metadata parsing
4 - Device/Hub-related metadata fetching
5 - Authentication/Authorization
11 . 2

46. Pipeline stages explainedPipeline stages explained
7 - Billing and metrics reporting
1 - WebSocket/TCP endpoint
2 - TLS termination
3 - MQTT Packet metadata parsing
4 - Device/Hub-related metadata fetching
5 - Authentication/Authorization
6 - Optional topic rewriting
11 . 2

47. Pipeline stages explainedPipeline stages explained
8 - MQTT Packet serialization and transmission
1 - WebSocket/TCP endpoint
2 - TLS termination
3 - MQTT Packet metadata parsing
4 - Device/Hub-related metadata fetching
5 - Authentication/Authorization
6 - Optional topic rewriting
7 - Billing and metrics reporting
11 . 2

48. Wait, how?Wait, how?
11 . 3

49. Dynamic broker discoveryDynamic broker discovery
Finding the right broker at the right timeFinding the right broker at the right time
12 . 1

50. Dynamic broker discoveryDynamic broker discovery
Shared brokers are easy to find
Clustered/Dedicated brokers needs to be contacted somehow
12 . 2

51. Kubernetes DNS service discoveryKubernetes DNS service discovery
1. Give each broker a deterministic DNS record
2. Find the broker ID from the device ID, resolve the record from it
3. Connect, and have fun!
12 . 3

52. Let's meet together!Let's meet together!
From 15h30 - Workshop - Pilotez une ampoule connectée
16h30 - GANYMEDE - Demystifier le buzzword
All day - IoT Stand
13

53. Thank you!Thank you!
Follow us on
Twitter and LinkedIn @Scaleway
14

54. 15

55. Advanced MQTTAdvanced MQTT
https://www.hivemq.com/blog/mqtt-essentials-part-1-introducing-mqtt/
16