Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A RESTful API for Controlling Dynamic Streaming Topologies

604 views

Published on

A RESTful API for Controlling Dynamic Streaming Topologies, presented at the WS-REST 2014 Workshop.

Published in: Software, Technology, Business
  • Login to see the comments

  • Be the first to like this

A RESTful API for Controlling Dynamic Streaming Topologies

  1. 1. A RESTful API for Controlling Dynamic Streaming Topologies Masiar Babazadeh! University of Lugano Switzerland @masiarb Cesare Pautasso! University of Lugano Switzerland @pautasso 1
  2. 2. 2 F CP F F
  3. 3. 3 P
  4. 4. 4 F
  5. 5. 5 C
  6. 6. 6 P C
  7. 7. 7
  8. 8. 8 Web Liquid Streams
  9. 9. 8 Web Liquid Streams
  10. 10. 8 Web Liquid Streams
  11. 11. 9
  12. 12. 9
  13. 13. 9
  14. 14. 9
  15. 15. 9
  16. 16. 9
  17. 17. 9
  18. 18. 10
  19. 19. x 11 F CP F F WebSocket W ebRTC WebSocket ZeroM Q ZeroMQ
  20. 20. x 11 F CP F F WebSocket W ebRTC WebSocket ZeroM Q ZeroMQ
  21. 21. x 11 F CP F F WebSocket W ebRTC WebSocket ZeroM Q ZeroMQ
  22. 22. 12 F Workers
  23. 23. 13 F Operator Elasticity
  24. 24. 13 F Operator Elasticity
  25. 25. 13 F Operator Elasticity
  26. 26. 13 F Operator Elasticity
  27. 27. 13 F Operator Elasticity
  28. 28. 13 F Operator Elasticity
  29. 29. 13 F Operator Elasticity
  30. 30. 14 F Fault Tolerance
  31. 31. 14 F Fault Tolerance
  32. 32. 14 F Fault Tolerance
  33. 33. 14 F Fault Tolerance
  34. 34. 15 F CP F FX
  35. 35. 15 F CP F FX
  36. 36. 15 F CP F
  37. 37. 15 F CP F F
  38. 38. 16 Operator Migration F CP F
  39. 39. 16 Operator Migration F C P F
  40. 40. 17 Peer Overloading F CP F
  41. 41. 17 Peer Overloading F CP F
  42. 42. 17 Peer Overloading F CP F C
  43. 43. 17 Peer Overloading F CP F C
  44. 44. 17 Peer Overloading F CP F C
  45. 45. 17 Peer Overloading F CP F C
  46. 46. 18 API WLS Runtime P C
  47. 47. 19 RPC
  48. 48. 20 REST
  49. 49. 21 Operator Tight Coupling
  50. 50. 22 Operator
  51. 51. 22 Operator
  52. 52. 22 Operator
  53. 53. 22 Operator
  54. 54. 22 Operator
  55. 55. 23 Independent Evolution Operator
  56. 56. 23 Independent Evolution Operator
  57. 57. 23 Independent Evolution Operator
  58. 58. 24 Operator Error Detection & Recovery
  59. 59. 24 Operator Error Detection & Recovery operators[0].getWorkers();
  60. 60. 24 Operator Error Detection & Recovery Exception
  61. 61. 24 Operator Error Detection & Recovery GET …/workers
  62. 62. 24 Operator Error Detection & Recovery “request timeout”
  63. 63. 24 Operator Error Detection & Recovery {! ! “error” : …! }
  64. 64. 24 Operator Error Detection & Recovery GET …/workers
  65. 65. 24 Operator Error Detection & Recovery
  66. 66. 25 Representations { "topology": { "id": "example", "operators": […], "bindings": […] } } <topology> <id>example</id> <operators> <operator>…</operator> … </operators> <bindings> <binding>…</binding> … </bindings> </topology>
  67. 67. 2626 F CP F F
  68. 68. 2626 F CP F F Operators
  69. 69. 2626 F CP F F Bindings
  70. 70. 2626 F CP F F Workers
  71. 71. 2626 F CP F F Peers
  72. 72. 2626 F CP F F Topologies
  73. 73. 27 Hypermedia Data Model
  74. 74. 28 Peer Discovery C F GET /! GET /peers! POST /peers! GET /peers/:pid! DELETE /peers/:pid
  75. 75. 29 Peer Representation {! "peer": ! {! ! ! ! ! ! "id": "example",! ! ! ! ! ! "operators": […],! ! ! ! ! ! "topologies": […],! ! ! ! ! ! "CPU": [0, 50, 0] ! ! ! ! }! } GET /peers/example
  76. 76. 30 Topologies GET /topologies! POST /topologies! GET /topologies/:tid! PUT /topologies/:tid! DELETE /topologies/:tid F CP F
  77. 77. 31 Topology Representation { "topology": { "id": "example", "operators": […], "bindings": […] } } GET /topologies/example
  78. 78. 32 F Operators GET /operators! GET /topologies/:tid/operators! GET /topologies/:tid/operators/:oid! PUT /topologies/:tid/operators/:oid! PATCH /topologies/:tid/operators/:oid/script! PATCH /topologies/:tid/operators/:oid/bindings! DELETE /topologies/:tid/operators/:oid
  79. 79. 33 {! "operators" : [! {! "topology" : "example",! "id" : "prod",! "workers" : […],! "href" : “…/topologies/example/operators/prod”,! "peer" : "http://IP:port/",! "replicas" : [! ! "http://IP2:port2/topologies/example/operators/prod"! ! ! ] ! ! },! ]! } Operator Representation GET /topologies/example/operators
  80. 80. 34 “topology” : { “operators: […], "bindings": [ { "from": “…/topologies/example/operators/a”, "to": “…/topologies/example/operators/b”, "type": "round_robin" }, { "from": “…/topologies/example/operators/b”, "to": “…/topologies/example/operators/c”, "type": "broadcast" } ] } Binding Representation
  81. 81. 35 Workers GET ! /topologies/:tid/operators/:oid/workers! GET ! /topologies/:tid/operators/:oid/workers/:wid! POST ! /topologies/:tid/operators/:oid/workers! DELETE ! /topologies/:tid/operators/:oid/workers/:wid
  82. 82. 36 “operators“ : [! !{! ! "topology" : "example",! "id" : "prod",
 ! "workers" : [! ! ! {! ! ! ! "id" : "0",! ! ! ! "href" : “…/topologies/example/operators/prod/workers/0”,! ! ! ! "operator" : “…/topologies/example/operators/prod”,! ! ! ! "uptime" : 3600,! ! ! ! "messages" : 42,! ! ! ! "req-res-ratio" : 1.5! ! ! }, …! ! ],! ! …! ! }! ] Worker Representation
  83. 83. 37
  84. 84. 3838 F Load Balancing
  85. 85. 3838 F Load Balancing
  86. 86. 3838 F GET /topologies/:tid/operators/:oid/workers Load Balancing
  87. 87. 3838 F “workers”: [
 ! {! ! ! ! "id" : "0",! ! ! ! "href" : …,,! ! ! ! "operator" : …,! ! ! ! "uptime" : 3600,! ! ! ! "messages" : 42,! ! ! ! "req-res-ratio" : 1.5”,! ! ! },! !…! ]Load Balancing
  88. 88. 3838 F POST /topologies/:tid/operators/:oid/workers Load Balancing
  89. 89. 3838 F Load Balancing
  90. 90. 3838 F Load Balancing
  91. 91. 3939 F Fault Tolerance
  92. 92. 3939 F Fault Tolerance
  93. 93. 3939 F GET /topologies/:tid/operators/:oid/workers Fault Tolerance
  94. 94. 3939 F “workers”: [
 ! {! ! ! ! "id" : "0",! ! ! ! "href" : “…/prod/workers/0”,! ! ! ! …! ! ! },! {! ! ! ! “id” : “1”,! ! ! ! “href” : …,! ! ! ! “error”: “request timeout”! ! ! }! ] Fault Tolerance
  95. 95. 3939 F POST /topologies/:tid/operators/:oid/workers Fault Tolerance
  96. 96. 3939 F Fault Tolerance
  97. 97. 3939 F DELETE /topologies/:tid/operators/:oid/workers/:wid Fault Tolerance
  98. 98. 3939 F Fault Tolerance
  99. 99. 40 F CP F Fault Tolerance
  100. 100. 40 F CP F Fault Tolerance
  101. 101. 40 F CP F GET /topologies/:tid/operators Fault Tolerance
  102. 102. 40 F CP F “operators”: [
 ! {! ! ! ! …! ! ! },! {! ! ! “id” : “C”,! ! ! “href” : …,! ! ! “error” : ! ! ! “request timeout”! ! ! }! ]Fault Tolerance
  103. 103. 40 F CP F PUT /topologies/:tid/operators/C Fault Tolerance
  104. 104. PATCH /topologies/:tid/operators/bindings 40 F CP F C Fault Tolerance
  105. 105. 40 F CP F C DELETE /topologies/:tid! /operators/C Fault Tolerance
  106. 106. 40 F P F C Fault Tolerance
  107. 107. 41 F CP F Fault Tolerance
  108. 108. 41 F CP F Fault Tolerance
  109. 109. 41 F CP F GET /peers/:pid Fault Tolerance
  110. 110. connection timeout 41 F CP F Fault Tolerance
  111. 111. PUT /topologies/:tid/operators/… 41 F CP F Fault Tolerance
  112. 112. 41 F CP F CF Fault Tolerance
  113. 113. PATCH /topologies/:tid/operators/bindings 41 F CP F CF Fault Tolerance
  114. 114. 41 F CP F CF Fault Tolerance
  115. 115. 41 F CP F CF Fault Tolerance
  116. 116. DELETE /topologies/:tid! /operators/… 41 F CP F CF Fault Tolerance
  117. 117. 41 F P CF Fault Tolerance
  118. 118. 42 F CP F Peer Overloading
  119. 119. 42 F CP F GET /peers/:pid Peer Overloading
  120. 120. 42 F CP F { ! ! “peer”: 
 ! {! ! ! ! …! ! ! ! “CPU” : ! ! ! ! [100,100,100] ! }! }Peer Overloading
  121. 121. PUT /topologies/:tid/operators/C 42 F CP F Peer Overloading
  122. 122. PATCH /topologies/:tid/operators/bindings 42 F CP F C Peer Overloading
  123. 123. 42 F CP F C Peer Overloading
  124. 124. 42 F CP F C Peer Overloading
  125. 125. 43 F CP F Leaving Peer
  126. 126. 43 F CP F DELETE /peers/IP:port Leaving Peer
  127. 127. 43 F CP F Leaving Peer
  128. 128. 43 F CP F PUT /topologies/:tid/operators/… Leaving Peer
  129. 129. 43 F CP F CF PATCH /topologies/:tid/operators/bindings Leaving Peer
  130. 130. 43 F CP F CF DELETE /topologies/:tid! /operators/… Leaving Peer
  131. 131. 43 F P CF Leaving Peer
  132. 132. 43 F P CF Leaving Peer
  133. 133. 44
  134. 134. 44
  135. 135. 44
  136. 136. 44
  137. 137. 44
  138. 138. 44
  139. 139. A RESTful API for Controlling Dynamic Streaming Topologies Masiar Babazadeh! University of Lugano Switzerland @masiarb Cesare Pautasso! University of Lugano Switzerland @pautasso 45 http://design.inf.usi.ch/

×