Apache Kafka and MQTT are a perfect combination for many IoT use cases. This presentation covers the pros and cons of both technologies. Various use cases across industries, including connected vehicles, manufacturing, mobility services, and smart city are explored. The examples use different architectures, including lightweight edge scenarios, hybrid integrations, and serverless cloud solutions. Blog series with more details here: https://www.kai-waehner.de/blog/2021/03/15/apache-kafka-mqtt-sparkplug-iot-blog-series-part-1-of-5-overview-comparison/

1. Manufacturing 4.0, Connected Vehicles,
Mobility Services, and Smart City
Real Time Data Processing and Analytics powered by Apache Kafka and MQTT
Kai Waehner
Field CTO Confluent
kai.waehner@confluent.io
linkedin.com/in/kaiwaehner
@KaiWaehner
www.confluent.io
www.kai-waehner.de
Dominik Obermaier
CTO HiveMQ
dominik.obermaier@hivemq.com
www.linkedin.com/in/dobermai
@dobermai
www.hivemq.com
1

2. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

3. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

4. Industry Trends - Connected World
4
People on the Internet

5. A Streaming Platform is the Underpinning
of an Event-driven Architecture
Ubiquitous connectivity
Globally scalable platform for all
event producers and consumers
Immediate data access
Data accessible to all consumers in
real time
Single system of record
Persistent storage to enable
reprocessing of past events
Continuous queries
Stream processing capabilities for
in-line data transformation
Microservices
DBs
SaaS apps
Mobile
Customer 360
Real-time
fraud
detection
Data
warehouse
Producers
Consumers
Database
change
Microservices
events
SaaS
data
Customer
experience
s
Streams of real time events
Stream processing
apps

6. Data Movement from Device to Enterprise
Services
SCADA
ERP
Enterprise Data Center
• MQTT Protocol
decouples data
integration bindings
• Bi-directional data
movement between
devices and data center
• Support for unreliable
network
MQTT
Broker
...

7. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

8. Food Value Chain
IoT-Based and Data-Driven
Single source of truth
across the food value chain
(in the factories, and across regions)
Business critical
operations
(tracking, calculations, alerts, …)

9. Daimler - Automotive Manufacturing
Vehicle Diagnostic System (VDS)
• Critical part of manufacturing process that cannot go down
HiveMQ Solution
• Efficient way to share information between test devices on factory
floor and enterprise IT systems
• Test devices behave properly if network connection is dropped and
reconnected
Results
● Rolled out to 24 factories around the world
● 10,000 testing devices connected
● Generating 470 million messages/month
bit.ly/hivemq-daimler

10. Reference Architecture - Manufacturing

11. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

12. ‘My Porsche’
A digital service platform for customers, fans, and enthusiasts
https://medium.com/porschedev

13. Autonomic - Connecting Millions of Cars
Transportation Mobility Cloud (TMC)
= Accessing connected car data and sending remote commands as the
foundation for mobility applications:
driver safety, preventive maintenance, fleet management…
HiveMQ Solution
● High degree of customization
● Scale to millions of connections
● Easy to create a proof of concept
Results
● Millions of cars connected via TMC
● Extensive use of HiveMQ Extension SDK
○ forward all MQTT messages to Kafka
○ messages distributed to different microservices &
applications
bit.ly/hivemq-autonomic

14. Reference Architecture - Connected Vehicles

15. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

16. DB Musterfirma | Vorname Name |
Abteilung | Datum ("Einfügen >
Kopf- und Fußzeile")
22
Customer
timetable
Operational
timetable
Assignments
Railway station
knowledge
Dispositions
Train positions
Matching
Aggregation
Consolidation
Apache
Kafka
Analysis
Railway station
Trains
Mobile Apps
Employees
Deutsche Bahn AG | Reisendeninformation
RI-Plattform
Single Point of Truth

17. ZF - Enabling Next Generation of Mobility
Cloud Ecosystem for Connected Vehicles
• Cloud-agnostic services without vendor lock-in
HiveMQ Solution
● Open extension system for custom extensions
● Flexible deployment model to deploy Connectivity Suite to a public
cloud or on-premise
● Implement authorization and authentication the way they wanted
Results
● Remote diagnostics for technical insight & management of vehicle
performance
● Fleet monitoring
● Secure & reliable middleware between connected vehicles & cloud
services
bit.ly/hivemq-zf

18. Reference Architecture - Mobility Services

19. Other Components
Kafka Ecosystem
(3) Read Data
TensorFlow
I/O
(5) Deploy
Model
(2) Preprocess
Data
(8a)
Alert Car
Real Time
Kafka App
TensorFlow
Serving
gRPC
Car Sensor
HiveMQ
MQTT Broker
MQTT
Connector
Kafka Connect
or
Confluent Proxy
or
HiveMQ Plugin
Kafka
Cluster
Kafka
Connect
Grafana
Elastic
Search
KSQL TensorFlow
(04)
Train
Model
TensorFlow
Real Time
Kafka
Streams
Application
(Java / Scala)
Real Time
Edge
Computing
(C / librdkafka)
TensorFlow
Lite
(1)
Ingest Data (8b)
Alert Driver
(e.g. mobile App)
(6a) Consume
Car Data
(6b)
All Data
(7) Potential
Defect
25

20. https://github.com/kaiwaehner/hivemq-mqtt-tensorflow-kafka-realtime-iot-machine-learning-training-inference
or
http://bit.ly/kafka-mqtt-ml-demo
Try it out in 30 minutes!
Demo 100.000 Connected Cars
(Kafka + MQTT + TensorFlow)
26

21. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

22. NAV (Norwegian Work and Welfare Department):
Life is a Stream of Events
https://www.confluent.io/kafka-summit-sf18/life-is-a-stream-of-events/

23. Berlex - Cloud-based Traffic Control Systems
Portable, cloud-based traffic signals
• Reliable control and monitoring of traffic signals from the cloud
HiveMQ Solution
● Ability to run without TLS
● Support for MQTT 5 shared subscriptions
● Clusters can be configured with ease to achieve the high availability
● Consolidate log files of all cluster nodes onto a central Syslog server
Results
● Successful deployment in Sweden and Norway
planned European expansion
● Real-time information delivery & constant monitoring
● High level of customer satisfaction
bit.ly/hivemq-berlex

24. Reference Architecture - Smart City

25. Agenda
1. Industry Trends - Connected World, Industry 4.0
2. Manufacturing
3. Connected Vehicles
4. Mobility Services
5. Smart City
6. HiveMQ and Confluent

26. Confluent Platform
Fully Managed Cloud Service
Self Managed Software FREEDOM OF CHOICE
COMMITTER-DRIVEN EXPERTISE Partners
Training
Professional
Services
Enterpris
e Support
Apache Kafka
EFFICIENT
OPERATIONS AT SCALE
PRODUCTION-
STAGE PREREQUISITES
UNRESTRICTED
DEVELOPER PRODUCTIVITY
SQL-based Stream Processing
KSQL (ksqlDB)
Rich Pre-built Ecosystem
Connectors | Hub | Schema Registry
Multi-language Development
non-Java clients | REST Proxy
GUI-driven Mgmt & Monitoring
Control Center
Flexible DevOps Automation
Operator | Ansible
Dynamic Performance & Elasticity
Auto Data Balancer | Tiered Storage
Enterprise-grade Security
RBAC | Secrets | Audit logs
Data Compatibility
Schema Registry | Schema Validation
Global Resilience
Multi-Region Clusters | Replicator
Developer Operator Architect
Open Source | Community licensed
PARTNERSHIP
FOR BUSINESS SUCCESS
Complete Engagement Model
Revenue / Cost / Risk Impact
TCO / ROI
Executive Buyer

27. HiveMQ Platform

28. Kai Waehner
Technology Evangelist
kai.waehner@confluent.io
linkedin.com/in/kaiwaehner
@KaiWaehner
www.confluent.io
www.kai-waehner.de
Dominik Obermaier
CTO HiveMQ
dominik.obermaier@hivemq.com
www.linkedin.com/in/dobermai
@dobermai
www.hivemq.com
34
Questions? Feedback?
Please contact us!