Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@matteocollina   matteocollina.com  Building a multi-protocol  broker for the Internet of  Things using node.jsNode.js Con...
How many people will owna smartphone by 2014       1.000.000.000
Did you seeit coming   http://www.flickr.com/photos/12738000@N00/360231193/
They didn’t!  ...
Whats next              (hint: its a big number)    50.000.000.000
50.000.000.0001.000.000.000
"Things"
Lets experiment
Goal:   I want to   chat with   my house
>: hi househi matteo>: whats thetemperature?20
>: 4 8 15 16 23 42 If you don’t understand this, you are not a geek!
Enter Hubot              Hubot © 2012 GitHub Inc. All rights
A programmable robotthat is controlled throughchat                      Hubot © 2012 GitHub Inc. All rights
We can supercharge our house with hubot
We can supercharge our house with hubot                  Howmodule.exports = (robot) ->  robot.respond /what’s the tempera...
In order to ask our tempto hubot, we need to: 1. sense the temp 2. get the temp on the web 3. build a web API
We are building an API      In Italy,     “API” means        “bees”                   http://www.flickr.com/photos/thesean...
What do we need tosense my housetemperature
We add a sensor to an Arduino                    TMP                     36                                TMP            ...
We add an ethernet shield to  connect to the Internet                                                          TMP        ...
What protocol do we useto push our temperatureto our API?
HTTP is slow and safe               http://www.flickr.com/photos/clearlyambiguous/48185613/
We need afast, binaryprotocol               http://www.flickr.com/photos/grrphoto/305649629
M2M protocols are         “Things” should interactmostly ad-hoc, and        with our lives, and all theresearchers and    ...
• “things” exposed       • “things” exposed  with binary protocol     to the web• publish/subscribe      • request/respons...
DiscoverQESTqest.me
HTTP Clients           MQTT Clients   QEST                     REST Server          MQTT Server• MQTT broker              ...
state-of-art                  state-of-art                                             QEST-based      QEST-based         ...
QEST : MQTT to REST• retains every message received                     client = PubSubClient(server, 1883, callback);    ...
QEST : REST to MQTT• transform every HTTP PUT received to a MQTT message                            curl -X PUT -d { "payl...
HTTP/MQTT ClientsHow to                          Load Balancerscale         REST Server   MQTT Server           REST Serve...
How muchtime took meto write thefirst versionof QEST?
How much       A day.time took meto write thefirst version    Thanksof QEST?
A day.    • Express          • Socket.io          • node_redis Thanks   • MQTT.js          • Coffeescript
What happens if youwrite a broker in a day?                           (Hint)
Spaghetti code!
How to untangle it?
http://www.flickr.com/photos/adewale_oshineye/2933030620/Refactoring!!!
1.         A single file         can go really         far! Well, the first improvement were two files
Coffee-scriptis awesome for prototyping!
Coffee-scriptis a real pain to debug
2. Extracting a data layer!        HTTP Clients           MQTT Clients     REST Server          MQTT Server               ...
Discover my roots
Ruby Language
Ruby LanguageRuby on Rails
Ruby on Rails       The  ActiveRecord    pattern is   AWESOME!
ActiveRecord1. Best pattern for storing data2. Its main responsibilities are   CRUD operations3. I used that pattern to bu...
Hubot © 2012 GitHub Inc. All rights
3. Extracting an inter process pub/sub library
Discover  Ascoltatorigithub.com/mcollina/ascoltatori
ir     to lta                      • Redisco           is a multi-    • ZeroMQ      process pub/subAs      library backed ...
r     i      to ltaco     var	  ascoltatori	  =	  require(ascoltatori);     //	  you	  can	  use	  RedisAscoltatore,	  Zer...
Memory   Redis    ZeroMQ       RabbitMQ   Mosquitto                    r i                 to        lta100.000 us   co 10...
i r     to ltaco            allowed me toAs            refactor QEST as            just an MQTT            wrapper!
ut ing           s    rib ek         or  nt seco m  Ia            github.com/mcollina/qest            github.com/mcollina/...
DEMO!        Source NASA
Matteo CollinaSoftware Engineer @MavigexPh.D. Student @University of Bolognamatteocollina.com
@matteocollinaThank You! Matteo Collina (matteo.collina2@unibo.it)                                                http://w...
Upcoming SlideShare
Loading in …5
×

Building a multi protocol broker for the internet of things using nodejs

Have you ever wondered how to interconnect your apps with physical things? Have you ever felt that the request/response pattern of HTTP is not enough? What about a binary protocol? In this talk you will discover the internal of the open source QEST broker, a Node.js-based broker for the Internet of Things that implements a classic publish/subscribe pattern, while making it accessible from HTTP and MQTT, an ultra-fast binary protocol.

  • Login to see the comments

Building a multi protocol broker for the internet of things using nodejs

  1. 1. @matteocollina matteocollina.com Building a multi-protocol broker for the Internet of Things using node.jsNode.js Conference - Brescia 2012/11/09
  2. 2. How many people will owna smartphone by 2014 1.000.000.000
  3. 3. Did you seeit coming http://www.flickr.com/photos/12738000@N00/360231193/
  4. 4. They didn’t! ...
  5. 5. Whats next (hint: its a big number) 50.000.000.000
  6. 6. 50.000.000.0001.000.000.000
  7. 7. "Things"
  8. 8. Lets experiment
  9. 9. Goal: I want to chat with my house
  10. 10. >: hi househi matteo>: whats thetemperature?20
  11. 11. >: 4 8 15 16 23 42 If you don’t understand this, you are not a geek!
  12. 12. Enter Hubot Hubot © 2012 GitHub Inc. All rights
  13. 13. A programmable robotthat is controlled throughchat Hubot © 2012 GitHub Inc. All rights
  14. 14. We can supercharge our house with hubot
  15. 15. We can supercharge our house with hubot Howmodule.exports = (robot) -> robot.respond /what’s the temperature?/i, (msg) -> msg.http("http://mchouse.it/temperature") .header("Accept", "application/json") .get() (err, res, body) -> msg.send JSON.parse(body)
  16. 16. In order to ask our tempto hubot, we need to: 1. sense the temp 2. get the temp on the web 3. build a web API
  17. 17. We are building an API In Italy, “API” means “bees” http://www.flickr.com/photos/theseanster93/4056815767
  18. 18. What do we need tosense my housetemperature
  19. 19. We add a sensor to an Arduino TMP 36 TMP 36
  20. 20. We add an ethernet shield to connect to the Internet TMP 36 http://www.flickr.com/photos/snootlab/6052465980/
  21. 21. What protocol do we useto push our temperatureto our API?
  22. 22. HTTP is slow and safe http://www.flickr.com/photos/clearlyambiguous/48185613/
  23. 23. We need afast, binaryprotocol http://www.flickr.com/photos/grrphoto/305649629
  24. 24. M2M protocols are “Things” should interactmostly ad-hoc, and with our lives, and all theresearchers and technology should bebusinesses focus on low built to make them easylevel problems. to use.
  25. 25. • “things” exposed • “things” exposed with binary protocol to the web• publish/subscribe • request/response• topics as the naming • URIs as the system naming system
  26. 26. DiscoverQESTqest.me
  27. 27. HTTP Clients MQTT Clients QEST REST Server MQTT Server• MQTT broker QEST• REST interface• HTTP semantics Data Layer• no QoS• built on node.js and redis Redis
  28. 28. state-of-art state-of-art QEST-based QEST-based approach to IoT apps IoT apps approach to solution to IoT apps solution to IoT apps Web App Web App Web App Bridge Web App QEST Bridge QEST IoT Broker Device IoT Device 3 2 1 0 9 8 7 6 5 4 3 2 1 0 GND SCL AREF SDA 1 1 1 1 DIGITAL RX TX PWM PWM PWM PWM PWM PWM L TX Arduino UNO ON Broker RX 1 ICSP www.arduino.cc RESET IOREF POWER ANALOG IN 3V3 5V Gnd Vin 0 1 2 3 4 5 Device 3 2 1 0 9 8 7 6 5 4 3 2 1 0 GND SCLAREF SDA 1 1 1 1 DIGITAL RX TX PWM PWM PWM PWM PWM PWM L TX RX Arduino UNO ON 1 ICSP www.arduino.cc RESET IOREF POWER ANALOG IN 3V3 5V Gnd Vin 0 1 2 3 4 5 Device
  29. 29. QEST : MQTT to REST• retains every message received client = PubSubClient(server, 1883, callback); client.publish("temp", "30");• every topic has its own URI: /topics/<NAME> curl -H "Accept: txt" http://qest.me/topics/temp
  30. 30. QEST : REST to MQTT• transform every HTTP PUT received to a MQTT message curl -X PUT -d { "payload": 42 } -H "Content-Type: application/json" http://qest.me/topics/temp• devices can listen directly to MQTT topics void callback(char* topic, byte* payload, int length) { ... } PubSubClient(server, 1883, callback); client.subscribe("temp");
  31. 31. HTTP/MQTT ClientsHow to Load Balancerscale REST Server MQTT Server REST Server MQTT Server QEST Data Layer ... QEST Data Layer Redis
  32. 32. How muchtime took meto write thefirst versionof QEST?
  33. 33. How much A day.time took meto write thefirst version Thanksof QEST?
  34. 34. A day. • Express • Socket.io • node_redis Thanks • MQTT.js • Coffeescript
  35. 35. What happens if youwrite a broker in a day? (Hint)
  36. 36. Spaghetti code!
  37. 37. How to untangle it?
  38. 38. http://www.flickr.com/photos/adewale_oshineye/2933030620/Refactoring!!!
  39. 39. 1. A single file can go really far! Well, the first improvement were two files
  40. 40. Coffee-scriptis awesome for prototyping!
  41. 41. Coffee-scriptis a real pain to debug
  42. 42. 2. Extracting a data layer! HTTP Clients MQTT Clients REST Server MQTT Server QEST Data Layer Redis
  43. 43. Discover my roots
  44. 44. Ruby Language
  45. 45. Ruby LanguageRuby on Rails
  46. 46. Ruby on Rails The ActiveRecord pattern is AWESOME!
  47. 47. ActiveRecord1. Best pattern for storing data2. Its main responsibilities are CRUD operations3. I used that pattern to build a cross-process pub/sub system
  48. 48. Hubot © 2012 GitHub Inc. All rights
  49. 49. 3. Extracting an inter process pub/sub library
  50. 50. Discover Ascoltatorigithub.com/mcollina/ascoltatori
  51. 51. ir to lta • Redisco is a multi- • ZeroMQ process pub/subAs library backed by • RabbitMQ • MQTT (Mosquitto) • Memory (EventEmitter)
  52. 52. r i to ltaco var  ascoltatori  =  require(ascoltatori); //  you  can  use  RedisAscoltatore,  ZeromqAscoltatore,As //  RabbitAscoltatore,  MQTTAscoltatore var  ascoltatore  =  new  ascoltatori.MemoryAscoltatore(); ascoltatore.subscribe("hello/*",  function()  {    //  this  will  print  {  0:  "hello/42",  1:  "a  message"  }    console.log(arguments);      process.exit(0); }); ascoltatore.publish("hello/42",  "a  message",  function()  {    console.log("message  published"); });
  53. 53. Memory Redis ZeroMQ RabbitMQ Mosquitto r i to lta100.000 us co 10.000 usAs 1.000 us 100 us 10 us 1 us 1 10 100 1000 Listeners
  54. 54. i r to ltaco allowed me toAs refactor QEST as just an MQTT wrapper!
  55. 55. ut ing s rib ek or nt seco m Ia github.com/mcollina/qest github.com/mcollina/ascoltatori
  56. 56. DEMO! Source NASA
  57. 57. Matteo CollinaSoftware Engineer @MavigexPh.D. Student @University of Bolognamatteocollina.com
  58. 58. @matteocollinaThank You! Matteo Collina (matteo.collina2@unibo.it) http://www.flickr.com/photos/axel-d/479627824/

×