Recommended
Recommended
More Related Content
What's hot
What's hot (8)
Similar to Microservice Protocols of Interaction
Similar to Microservice Protocols of Interaction (20)
Recently uploaded
Recently uploaded (20)
Microservice Protocols of Interaction
- 1. Microservice Protocols of Interaction Todd L. Montgomery @toddlmontgomery
- 2. About me…
- 3. What is a Protocol? Why should we care!?
- 4. @toddlmontgomery pro·to·col noun ˈprō-tə-ˌkȯl, -ˌkōl, -ˌkäl, -kəl ... 3 b : a set of conventions governing the treatment and especially the formatting of data in an electronic communications system <network protocols> ... 3 a : a code prescribing strict adherence to correct etiquette and precedence (as in diplomatic exchange and in the military services) <a breach of protocol>
- 5. Protocols of Interaction Wire Protocol, Method Calls, Shared Memory Interactions, etc.
- 7. Forced Decoupling via an “Asynchronous, Binary Boundary”
- 9. The truth is… Protocols can and do Couple
- 10. Protocols of Interaction are quite important! Protocols of Interaction Matter!
- 11. The Environment
- 12. Networks, and especially the Internet, are Hostile Environments
- 14. Data can be lost, duplicated, and re-ordered!!
- 15. TCP connections can… be closed unexpectedly end in an unknown state be intercepted by idiots, er Proxies
- 16. Duplicated Re-Ordered Lost Which means Data over TCP* might be… * - When connections are re-established
- 17. Don’t assume the network is reliable https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing
- 18. Case Studies
- 19. Case Study 1 Loose Ordering
- 20. @toddlmontgomery Sync Requests & Responses Request Request Request Response Response Response Throughput limited by Round-Trip Time (RTT)!
- 21. @toddlmontgomery Async Requests & Responses Request Request Request Response Response Response Throughput less limited by Round-Trip Time!
- 22. @toddlmontgomery Async Requests & Responses Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2
- 23. Aside…
- 24. Ordering is an Illusion!!
- 25. Compiler can re-order Runtime can re-order CPU can re-order
- 26. Ordering has to be imposed!
- 27. @toddlmontgomery Async Requests & Responses Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2
- 28. @toddlmontgomery Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2 Ordering
- 29. @toddlmontgomery Correlation! Request 0 Request 1 Request 2 Response 0 Response 1 Response 2 (Valid) Re-Ordering (one of many)
- 30. @toddlmontgomery Handling the Unexpected Request 0 Response 1 Invalid, Drop We only know of 0. 1 is unknown!
- 31. SCTP HTTP/2 (SPDY) … most OSI Layer 4 protocols
- 32. Case Study 2 Can you hear me now? Timeouts & Retries
- 36. @toddlmontgomery ACK Processing … Avoid Spurious Retransmits Retransmit Original TimeoutInterval
- 37. @toddlmontgomery Interval = N x “typical” RTT Account for processing delay X TimeoutInterval “Average”
- 38. @toddlmontgomery Measure! But very “noisy”? RTTMeasurement Variances in processing, transmission, etc.
- 39. TCP Retransmit Timeout (RTO) Err = M - A A <- A + gErr D <- D + h(|Err| - D) RTO = A + 4D M = measurement, A = smoothed average, D = smoothed mean deviation, g and h = gain constants (0 to 1)
- 40. TCP Retransmit Timeout (RTO) Err = M - A A <- A + gErr D <- D + h(|Err| - D) RTO = A + 4D Do you measure on a Retransmit? NO!
- 41. @toddlmontgomery Does processing twice hurt? X Original ACK Retrans Process Once Process Twice TimeoutInterval
- 42. @toddlmontgomery Are Original & Retransmit treated the same? X Original ACK Retrans Process Once Process Twice TimeoutInterval
- 44. Case Study 3 What I Need! When I Need It! “Lifetime” Management
- 45. “Managing” Application Working Set or Service Liveness
- 46. Caching Algorithms LRU, MRU, PLRU, RR, SLRU, LFU, … “Liveness” is essential
- 48. @toddlmontgomery Request ACK Service A is Alive! Service B is Alive! Service A Service B Consequence of Processing
- 49. @toddlmontgomery Keepalive Keepalive Service A is Alive! Service B is Alive! Service A Service B Absence of Processing
- 50. RIP Route Deletion Step 0 - route info broadcast @30 seconds Step 1 (3 min) - Set Distance to Infinity (16) Step 2 (+1 min) - Delete Route Aside… RIP… aptly named
- 51. Aeron Driver Keepalive Time of Last Activity = Shared Variable Doesn’t need to be a message…
- 52. @toddlmontgomery Bye Bye Service A is gone! Service B is gone! Service A Service B Optimization, but insufficient with arbitrary failures
- 53. Liveness often exists across transient connectivity
- 54. So… Don’t conflate transport state with liveness! Like TCP connection state
- 55. Dead TCP connection != Dead Service
- 56. Live TCP connection != Live Service
- 58. Case Study 4 Elasti-What? Self-Similar Behavior
- 59. Request X Request X Request X Request X, X, X Multiple same/similar requests at the same time Response X, X, X
- 61. 1, 2, 3 1, 2, 3 1, 2, 3 1, 2, 3 Non-correlated loss X X X
- 62. NAK 1, 2, 3 NAK 2 NAK 1 NAK 3 Request individual lost data Retransmit 1, 2, 3
- 63. 1, 2, 3 1, 2, 3 1, 2, 3 1, 2, 3 Temporally/Spatially Correlated Loss X X X
- 64. NAK 2 NAK 2 NAK 2 NAK 2, 2, 2 Multiple requests for same data Retransmit 2, 2, 2
- 65. Request 2 Request 2 Request 2 Request 2, 2, 2 It’s a generic problem!
- 66. Request 2 Request 2 Request 2 Request 2, 2, 2 Overloading Responder & Network
- 68. Request 2 Publish Requests Don’t Immediately Request, Listen first Timeout! Request 2 Request 2 Suppress Request
- 69. Request 2 How long to wait & listen for? Timeout! Request 2 Request 2 Suppress Request
- 70. Statistics to the Rescue!
- 71. SRM Backoff RandomBackoff = [C1, C1+C2] * 1-way delay Random is more than good enough
- 72. Request 2 Request 2 Request 2, 2 Must also shed duplicates on the responder Response 2, 2 Shed second “Request 2” if too soon X X
- 74. Case Study 5 Hey, Slow Down! Flow (& Congestion) Control
- 76. @toddlmontgomery Data Data Data ACK ACK ACK Throughput = Data Length / RTT RTT Stop-And-Wait Flow Control
- 77. Delay Bandwidth BDP = (Byte / sec) * sec = Bytes BDP (Buffer)
- 78. @toddlmontgomery Data ACK RTT Throughput = N * Data Length / RTT … N Data “Blobs”
- 79. So… How big is N? This is surprisingly hard to answer
- 80. It depends…
- 81. Big… but Don’t overflow receiver Don’t overflow “network”
- 82. TCP Flow Control Receiver advertises N
- 83. TCP Congestion Control Sender probes for network N
- 84. TCP Sender min(Receiver N, Network N) Only go as fast as Network & Receiver
- 85. ReactiveStreams Subscriber uses explicit request(N) Publisher assumes best case http://www.reactive-streams.org/
- 86. Takeaways!
- 87. Protocols of interaction are important & can be tremendously impactful for better or worse…
- 88. @toddlmontgomery Questions? • IETF http://www.ietf.org/ • Aeron https://github.com/real-logic/Aeron • Twitter @toddlmontgomery Thank You!