Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to jtf2016-mom
Similar to jtf2016-mom (20)
More from Ohyama Hiroyasu
Recently uploaded
Recently uploaded (20)
jtf2016-mom
- 1. MOM Requirement for Message Oriented Middleware
- 2. ▸ MQ (MOM) ▸ ( ) MOM ▸ ( ) MOM 2
- 3. ▸ MQ ▸ MOM 3
- 4. ▸ - - 4
- 5. (OHYAMA HIROYASU) / SOFTWARE ENGINEER ▸ Works Applications (formerly Ariel Networks) > GREE > DMM.com Labo ▸ Onix OS, Jaxon OpenFlow Controller, NewtMQ ▸ OpenFlow (ASCII ) , , ( ) 5
- 6. SO, WHAT IS MQ ? 6
- 7. 7 ▸ Message Passing - Queuing - Pub/Sub WHAT IS MQ ? PROCESS A BROKER PROCESS B PUBLISHER SUBSCRIBER B SUBSCRIBER A SUBSCRIBER C BROKER
- 8. 8 ▸ Message Passing - Queuing ( Message Passing ) WHAT IS MQ ? PROCESS A PROCESS BBROKER
- 9. ▸ Message Passing - Pub/Sub “ ” (Subscriber ) 9 WHAT IS MQ ? PUBLISHER SUBSCRIBER B SUBSCRIBER A SUBSCRIBER C BROKER
- 10. WHY MQ ? 10
- 11. 11
- 12. WHY MQ ? SERVICE A SERVICE B SERVICE E SERVICE D REQUEST SERVICE C ▸ MQ 12
- 13. WHY MQ ? SERVICE A SERVICE B REQUEST SERVICE E SERVICE D SERVICE C DATA REQUEST ”REQUEST ” 13 ▸ MQ
- 14. WHY MQ ? SERVICE A SERVICE B REQUEST SERVICE E SERVICE D SERVICE C DATA IF THIS IS STOPPED... WHAT SHOULD IT DO ..? REQUEST ”REQUEST ” 14 ▸ MQ
- 15. WHY MQ ? SERVICE A SERVICE B REQUEST SERVICE E SERVICE D SERVICE C DATA SERVICE F A NEW SERVICE IS APPEARED !! REQUEST ”REQUEST ” WHAT SHOULD IT DO ..?? 15 ▸ MQ
- 16. 16
- 17. WHY MQ ? SERVICE A SERVICE B MQ SERVICE C ▸ MQ SERVICE E SERVICE D REQUEST REQUEST 17
- 18. WHY MQ ? SERVICE A SERVICE B MQ SERVICE E SERVICE D SERVICE C ▸ MQ DATA REQUEST ”REQUEST ” REQUEST REQUEST REQUEST ” 18
- 19. WHY MQ ? SERVICE A SERVICE B MQ SERVICE E SERVICE D SERVICE C ▸ MQ 19 DATA REQUEST REQUEST REQUEST ” IF THIS IS STOPPED... REQUEST ”REQUEST ” OK, NO PROBLEM
- 20. WHY MQ ? SERVICE A SERVICE B MQ SERVICE E SERVICE D SERVICE C ▸ MQ 20 DATA REQUEST ”REQUEST ” REQUEST REQUEST REQUEST ” SERVICE F REQUEST ” A NEW SERVICE IS APPEARED !! OK, NO PROBLEM
- 21. WHY MQ ? ▸ - High Available - Scalable 21
- 22. ARCHITECTURE 22
- 23. ARCHITECTURE BROKER MQ / BROKER-LESS MQ 23 ▸ Broker MQ ▸ Broker-less MQ
- 24. BROKER MQ / BROKER-LESS MQ 24 ▸ Pros - Scalability - Isolation ▸ Cons - more traffic - more latency ARCHITECTURE ▸ Broker MQ
- 25. BROKER MQ / BROKER-LESS MQ 25 ▸ Pros - less traffic - less latency ▸ Cons - Scalability - Isolation ARCHITECTURE ▸ Broker-less MQ
- 26. OVERVIEW OF MQ (PROTOCOL & IMPLEMENTATION) 26
- 27. OVERVIEW OF MQ OVERVIEW ▸ 27 Messaging Pub/Sub AMQP (RabbitMQ) MQTT (ActiveMQ) × STOMP (NewtMQ) Kafka ZeroMQ …
- 28. 28 WHAT THE BEST MOM IS?
- 29. WHAT THE BEST MOM IS? 29 ( ) MOM ( ) CONCLUSION
- 30. MQ MOM 30 ( ) MOM ( ) CONCLUSION WHAT THE BEST MOM IS?
- 32. 32 AMQP(V0.9.1)
- 38. FEATURE AMQP(V0.9.1) 38 ▸ QUEUE QUEUE QUEUE SUBSCRIBER PUBLISHER SUBSCRIBERQUEUE EXCHANGE EXCHANGE SUBSCRIBER SUBSCRIBER AMQP SERVER AMQP SERVER CRITICAL.* *.SERVICE-A *.SERVICE-B # [ Advanced Pub/Sub ]
- 39. FEATURE AMQP(V0.9.1) 39 ▸ QUEUE QUEUE QUEUE SUBSCRIBER PUBLISHER SUBSCRIBERQUEUE EXCHANGE EXCHANGE SUBSCRIBER SUBSCRIBER AMQP SERVER AMQP SERVER [ Advanced Pub/Sub ] CRITICAL.* *.SERVICE-A *.SERVICE-B #
- 40. FEATURE AMQP(V0.9.1) 40 ▸ QUEUE QUEUE QUEUE SUBSCRIBER PUBLISHER SUBSCRIBERQUEUE EXCHANGE EXCHANGE SUBSCRIBER SUBSCRIBER AMQP SERVER AMQP SERVER INFO.SERVICE-A [ Advanced Pub/Sub ] CRITICAL.* *.SERVICE-A *.SERVICE-B #
- 41. FEATURE AMQP(V0.9.1) 41 ▸ QUEUE QUEUE QUEUE SUBSCRIBER PUBLISHER SUBSCRIBERQUEUE EXCHANGE EXCHANGE SUBSCRIBER SUBSCRIBER AMQP SERVER AMQP SERVER INFO.SERVICE-A INFO.SERVICE-A [ Advanced Pub/Sub ] CRITICAL.* *.SERVICE-A *.SERVICE-B #
- 42. STOMP 42FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol
- 43. 43FEATURE {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ } STOMP ▸ Text-base - Binary-based Protocol - Text-based Protocol
- 44. 44FEATURE {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ } STOMP 00110101101... 00110101101... (ENCODING) TEXT → BINARY ▸ Text-base - Binary-based Protocol - Text-based Protocol
- 45. 45FEATURE STOMP (DECODING) BINARY → TEXT {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ } ▸ Text-base - Binary-based Protocol - Text-based Protocol
- 46. 46FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol STOMP (DECODING) BINARY → TEXT {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ }▸ PROS - TRAFFIC ▸ CONS - CPU COST ( ENCODING / DECODING )
- 47. 47FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol STOMP {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ }
- 48. 48FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol STOMP {‘NAME’: ‘HIRO... {‘NAME’: ‘HIRO... DATA COPY
- 49. 49FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol STOMP {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ } DATA COPY
- 50. 50FEATURE ▸ Text-base - Binary-based Protocol - Text-based Protocol STOMP {‘NAME’ : ‘HIROYASU OHYAMA’, ‘AGE’ : 30, ‘HOBBY’ : ‘AUTO MOBILE’ } DATA COPY ▸ PROS - CPU COST ▸ CONS - TRAFFIC
- 52. MQTT 52FEATURE ▸ - MQTT HTTP (publish data to the topic ‘hoge’) (Fixed Header) (Variable Header) (POST data to ‘http://hoge.fuga.com/hoge') POST /hoge HTTP/1.1n Host: hoge.fuga.comn Content-Type: application/octet-streamn Content-Length: 100n n
- 53. MQTT 53FEATURE ▸ - MQTT HTTP (publish data to the topic ‘hoge’) (Fixed Header) (Variable Header) (POST data to ‘http://hoge.fuga.com/hoge') POST /hoge HTTP/1.1n Host: hoge.fuga.comn Content-Type: application/octet-streamn Content-Length: 100n n 10 BYTES 101 BYTES
- 54. MQTT 54FEATURE ▸ - At-most-once - At-least-once - Exactly-once
- 55. MQTT 55FEATURE ▸ - At-most-once - At-least-once - Exactly-once PUBLISHER MQTT BROKER SUBSCRIBE SUBSCRIBER
- 56. MQTT 56FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBEPUBLISHER SUBSCRIBER
- 57. MQTT 57FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKER SUBSCRIBEPUBLISHER SUBSCRIBER
- 58. SUBSCRIBER MQTT 58FEATURE ▸ - At-most-once - At-least-once - Exactly-once PUBLISHER MQTT BROKER SUBSCRIBE▸ PROS - CPU COST / TRAFFIC ▸ CONS -
- 59. MQTT 59FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKER SUBSCRIBEPUBLISHER SUBSCRIBER
- 60. MQTT 60FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBEPUBLISHER SUBSCRIBER
- 61. MQTT 61FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKER PUBACK SUBSCRIBEPUBLISHPUBLISHER SUBSCRIBER
- 62. MQTT 62FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKER SUBSCRIBEPUBLISH PUBACK PUBLISHER SUBSCRIBER
- 63. MQTT 63FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKER SUBSCRIBEPUBLISH PUBACK PUBLISH PUBLISHER SUBSCRIBER
- 64. SUBSCRIBER MQTT 64FEATURE ▸ - At-most-once - At-least-once - Exactly-once PUBLISHER MQTT BROKER SUBSCRIBEPUBLISH PUBACK PUBLISH ▸ PROS - ▸ CONS - SUBSCRIBER
- 65. MQTT 65FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBEPUBLISHER SUBSCRIBER
- 66. MQTT 66FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBLISHER SUBSCRIBER
- 67. MQTT 67FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL PUBLISHER SUBSCRIBER
- 68. MQTT 68FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL PUBCOMP PUBLISHER SUBSCRIBER
- 69. MQTT 69FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL (1/2) PUBLISHER SUBSCRIBER
- 70. MQTT 70FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL PUBREL (1/2) PUBLISHER SUBSCRIBER
- 71. MQTT 71FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL PUBREL PUBCOMP (1/2) PUBLISHER SUBSCRIBER
- 72. MQTT 72FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE (2/2) PUBREC PUBLISHER SUBSCRIBER
- 73. PUBREC MQTT 73FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE (2/2) PUBLISH PUBLISHER SUBSCRIBER
- 74. PUBREC MQTT 74FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE (2/2) PUBLISH PUBREC PUBREL PUBLISHER SUBSCRIBER
- 75. PUBREC MQTT 75FEATURE ▸ - At-most-once - At-least-once - Exactly-once MQTT BROKERPUBLISH SUBSCRIBE (2/2) PUBLISH PUBREC PUBREL PUBCOMP PUBLISHER SUBSCRIBER
- 76. SUBSCRIBER MQTT 76FEATURE ▸ - At-most-once - At-least-once - Exactly-once PUBLISHER MQTT BROKERPUBLISH SUBSCRIBE PUBREC PUBREL PUBREL PUBCOMP (2/2) ▸ PROS - ▸ CONS - CPU COST / TRAFFIC SUBSCRIBER
- 77. KAFKA 77FEATURE ▸ Scalable - for Availability - for Performance / Capacity
- 78. KAFKA 78FEATURE BAD CASE Queue M1M2M3M4 SUBSCRIBER A SUBSCRIBER B ▸ / strong ordering guarantees Queue
- 79. KAFKA 79FEATURE Queue M3M4 SUBSCRIBER A SUBSCRIBER B M1 M2 BAD CASE ▸ / strong ordering guarantees Queue
- 80. KAFKA 80FEATURE Queue M3 M4 SUBSCRIBER A SUBSCRIBER B M2 BAD CASE ▸ / strong ordering guarantees Queue
- 81. KAFKA 81FEATURE Queue M3 M4 SUBSCRIBER A SUBSCRIBER B M1 M2 BAD CASE M1 > M2 > M3 > M4 ( ) M1 > M3 > M4 > M2 ▸ / strong ordering guarantees Queue
- 82. KAFKA 82FEATURE NORMAL CASE Queue M1M2M3M4 SUBSCRIBER A ▸ / strong ordering guarantees Queue
- 83. KAFKA 83FEATURE Queue M1M2M3M4 SUBSCRIBER A NORMAL CASE ▸ / strong ordering guarantees Queue
- 84. Consumer group B KAFKA 84FEATURE Kafka’s CASE Partition-1 M1M4 Partition-2 M2M3 CONSUMER A CONSUMER B CONSUMER C Consumer group A ▸ / strong ordering guarantees Queue
- 85. Consumer group B KAFKA 85FEATURE Kafka’s CASE Partition-1 M1M4 Partition-2 M2M3 CONSUMER A CONSUMER B CONSUMER C Consumer group A ▸ / strong ordering guarantees Queue Partition Consumer Group Consumer Group
- 86. Consumer group B KAFKA 86FEATURE Kafka’s CASE Partition-1 M1 Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1 M2 M1M4 M2M3 ▸ / strong ordering guarantees Queue
- 87. Consumer group B KAFKA 87FEATURE Kafka’s CASE Partition-1 Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M4M1M4 M2M3 ▸ / strong ordering guarantees Queue M2 M2
- 88. Consumer group B KAFKA 88FEATURE Kafka’s CASE Partition-1 ▸ / strong ordering guarantees Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1M4 M2M3 M4 M2 M2
- 89. Consumer group B KAFKA 89FEATURE Kafka’s CASE Partition-1 ▸ / strong ordering guarantees Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1M4 M2M3 M4 M2 M2
- 90. Consumer group B KAFKA 90FEATURE Kafka’s CASE Partition-1 ▸ / strong ordering guarantees Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1M4 M2M3 M4 M2 M2 “ Queue ”
- 91. Consumer group B KAFKA 91FEATURE Kafka’s CASE Partition-1 ▸ / strong ordering guarantees Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1M4 M2M3 M4 M2 M2 “ Partition ” “ Queue ”
- 92. Consumer group B KAFKA 92FEATURE Kafka’s CASE Partition-1 ▸ / strong ordering guarantees Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M1M4 M2M3 (Partition-1) M1 > M4 (Partition-2) M2 > M3 “ Partition ” “ Queue ” M4 M2 M2
- 93. Consumer group B KAFKA 93FEATURE Kafka’s CASE Partition-1 Partition-2 CONSUMER A CONSUMER B CONSUMER C Consumer group A M4 M4 M3 M3 M1M4 M2M3 Scalable ( COND: Number of Consumer Process ≦ Count of Partition ) ▸ / strong ordering guarantees Queue
- 94. WHAT PURPOSE TO USE ? 94
- 95. WHAT PURPOSE TO USE MOM ? 95 ▸ / Asynchronous Communication ▸ / Isolation ▸ / Load Leveling ▸ / Scalability ▸ / Data Store PURPOSE
- 96. HOW MOM USE ? 96
- 97. CASE: SENSU 97 ▸ Scalability for Pub/Sub ▸ Load leveling for Queuing HOW MOM USE? (Source SENSU | SENSU Architecture)
- 98. CASE: OPENSTACK 98HOW MOM USE? ▸ Scalability and High Availability for RPC over MQ NOVA SCHEDULERNOVA CONDUCTOR NOVA COMPUTENOVA COMPUTENOVA COMPUTENOVA COMPUTE HTTP NOVA APICLIENT RPC (Remote Procedure Call) (see also AMQP and NOVA - OpenStack Docs)
- 99. CASE: IBM 99 ▸ Less Traffic & Scalability for IoT Cloud Platform HOW MOM USE? IBM IoT Foundation (MQTT Frontend) IBM Bluemix (PaaS) APPLICATION Devices THE INTERNET APPLICATION WEBSPHERE MQ APPLICATION MQTT (see also IBM Bluemix - Watson Internet of Things Platform)
- 100. CONCLUSION 100
- 101. CONCLUSION 101 ▸ ( ) MOM (e.g.) (e.g.) (e.g.) (e.g.) … ( ) MOM ( )
- 102. WHAT REQUIRES TO MOM 102 for you ?
- 103. THANK YOU! 103