3. 3
PROCESS A
BROKER
PROCESS B
PUBLISHER SUBSCRIBER B
SUBSCRIBER A
SUBSCRIBER C
BROKER
What is MQ?
A mechanism that supports message passing
between application processes.
Queuing model
Pub/Sub model
4. 4
PROCESS A PROCESS BBROKER
Queuing model
What is MQ?
supports transporting data between processes asynchronously
- These processes might not be woken at all time.
5. 5
supports publishing data to processes that share the ‘topic’
- There is no need for Publisher to know the existance of Subscriber(s).
Pub/Sub model
What is MQ?
PUBLISHER SUBSCRIBER B
SUBSCRIBER A
SUBSCRIBER C
BROKER
8. SERVICE A SERVICE B
SERVICE E SERVICE D
REQUEST
SERVICE C
8
A system without MQ
Introduction
9. SERVICE A SERVICE B
REQUEST
SERVICE E SERVICE D SERVICE C
DATA
REQUEST ”REQUEST ”
9Introduction
A system without MQ
10. SERVICE A SERVICE B
REQUEST
SERVICE E SERVICE D SERVICE C
DATA
IF THIS IS STOPPED...
WHAT SHOULD IT DO ..?
REQUEST ”REQUEST ”
10Introduction
A system without MQ
11. SERVICE A SERVICE B
REQUEST
SERVICE E SERVICE D SERVICE C
DATA
SERVICE F
A NEW SERVICE IS APPEARED !!
REQUEST ”REQUEST ” WHAT SHOULD IT DO ..??
11Introduction
A system without MQ
13. SERVICE A SERVICE B
MQ
SERVICE CSERVICE E SERVICE D
REQUEST REQUEST
13
A system with MQ
Introduction
14. SERVICE A SERVICE B
MQ
SERVICE E SERVICE D SERVICE C
DATA
REQUEST ”REQUEST ”
REQUEST REQUEST
REQUEST ”
14Introduction
A system with MQ
15. SERVICE A SERVICE B
MQ
SERVICE E SERVICE D SERVICE C
15
DATA
REQUEST REQUEST
REQUEST ”
IF THIS IS STOPPED...
REQUEST ”REQUEST ”
OK, NO PROBLEM
Introduction
A system with MQ
16. SERVICE A SERVICE B
MQ
SERVICE E SERVICE D SERVICE C
16
DATA
REQUEST ”REQUEST ”
REQUEST REQUEST
REQUEST ”
SERVICE F
REQUEST ”
A NEW SERVICE IS APPEARED !!
OK, NO PROBLEM
Introduction
A system with MQ
17. MQ simplify the system model of distributed system.
- It could make a system HA(High Available)
- It could make a system Scalable
17Introduction
A system with MQ
19. Comparison with each protocols and implementations
19
Messaging Pub/Sub
AMQP (RabbitMQ)
MQTT (ActiveMQ)
STOMP (NewtMQ)
Kafka
ZeroMQ
…
Overview
- There are many distinctive protocols/implementations.
21. CASE: SENSU
21
- Scalability for Pub/Sub
- Load leveling for Queuing
(reference:SENSU | SENSU Architecture)
How they are used?
22. CASE: YAHOO! JAPAN
22
Very long queue (many millions of messages) and Scalability
(reference:ストリーム処理プラットフォームにおけるKafka導入事例 #kafkajp)
How they are used?
23. CASE: IBM
23
Less Traffic & Scalability for IoT Cloud Platform
IBM IoT Foundation
(MQTT Frontend)
IBM Bluemix
(PaaS)
APPLICATION
Devices
THE INTERNET
APPLICATION
WEBSPHERE MQ
APPLICATION
MQTT
(reference:IoT クラウドで広がる Bluemix の世界 - IBM Bluemix)
How they are used?
25. 25
There is no MQ that satisfies all the system requirements,
I guess.
What’s the best one ?
26. 26What’s the best one ?
If you only know yourself, but not your opponent,
you may win or may lose.
(reference: The art of war, Sun Tzu)
There is no MQ that satisfies all the system requirements,
I guess.
38. MQTT
38
Lightweight message protocol
Three qualities of service for message delivery
- “At most once", where messages are delivered according to
the best efforts of the operating environment.
- "At least once", where messages are assured to arrive
but duplicates can occur.
- "Exactly once", where message are assured to arrive exactly once.
Feature of MQTT
39. MQTT
39
Lightweight message protocol
- The header-size comparison of MQTT and HTTP.
(publish data to the topic ‘hoge’)
(Fixed Header)
(Variable Header)
(POST data to ‘http://hoge.fuga.com/hoge')
POST /hoge HTTP/1.1n
Host: hoge.fuga.comn
Content-Type: application/octet-streamn
Content-Length: 100n
n
Feature of MQTT
40. MQTT
40
Lightweight message protocol
- The header-size comparison of MQTT and HTTP.
(publish data to the topic ‘hoge’)
(Fixed Header)
(Variable Header)
(POST data to ‘http://hoge.fuga.com/hoge')
POST /hoge HTTP/1.1n
Host: hoge.fuga.comn
Content-Type: application/octet-streamn
Content-Length: 100n
n
10 BYTES 101 BYTES<
Feature of MQTT
42. KAFKA
42
- provide both ordering guarantees and load balancing
- relies on the filesystem for storing and caching messages
(reference: Documentation - Apache Kafka)
Feature of Kafka
44. An example of micro service is
What essentials of MQ for a micro-service is?
A cloud operating system
- that controls large pools of compute, storage,and networking resources
(reference: What is OpenStack? / OpenStack)
45. (reference: OpenStack Docs: Logical architecture)
What essentials of MQ for a micro-service is?
Logical architecture of OpenStack
46. Logical architecture of OpenStack
What essentials of MQ for a micro-service is?
(reference: OpenStack Docs: Logical architecture)
48. What essentials of MQ for a micro-service is?
How OpenStack use MQ?
Use Oslo/Messaging that provides
- RPC for implementing client-server remote procedure calls
- Notify for emitting and handling event notifications
(reference: Oslo/Messaging - OpenStack)
49. What essentials of MQ for a micro-service is?
How OpenStack use MQ?
Use-case of Oslo/Messaging in Nova
(reference: Nova System Architecture)
50. What essentials of MQ for a micro-service is?
What are essentials of MQ
for OpenStack?
51. What essentials of MQ for a micro-service is?
The essentials of MQ for OpenStack are
Order-guarantee
High Availability
Performance
(Scalability)
52. What essentials of MQ for a micro-service is?
The non-essentials of MQ for OpenStack are
Transaction
- increase throughput by compromising message lost on the node
which is failure
Encryption
- performance increase without encoding/decoding processing
54. A yet another brokered-MQ which specialize in throughput
54
(reference: ツチノコブログ)
NewtMQ
Performance oriented MQ
55. NATS
Performance oriented MQ
A simple, high performance messaging system for
cloud native applications and microservices architectures
(reference: About NATS | The best messaging system for native cloud application developement)
60. Conclusion
NATS is better for micro-service applications
- If you can permit these restrictions
There are many MQ with a functional character
- choose it which confirms to your application