Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Next Generation Client APIs in Envoy Mobile

488 views

Published on

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2x0Fav8.

Jose Nino guides the audience through the journey of Mobile APIs at Lyft. He focuses on how the team has reaped the benefits of API generation to experiment with the network transport layer. He also discusses recent developments the team has made with Envoy Mobile and the roadmap ahead. Filmed at qconlondon.com.

Jose Nino works as a Software Engineer at Lyft.

Published in: Technology
  • Login to see the comments

Next Generation Client APIs in Envoy Mobile

  1. 1. Jose Nino @junr03 Next Generation Client APIs with Envoy Mobile QCon London - March 2020 Some slides built in collaboration with: @goaway & @rebello95
  2. 2. InfoQ.com: News & Community Site Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/ • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com
  4. 4. Server Networking Networking Client @rebello95
  5. 5. Server Networking Networking Client @junr03 API
  6. 6. Server Networking Networking Server @junr03 Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking
  7. 7. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking
  8. 8. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking �� �� �� �� �� ��
  9. 9. Jose Nino @junr03 Infrastructure Engineer ~4 years @
  10. 10. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking �� �� �� �� �� ��
  11. 11. Server Networking Networking Client @junr03 API
  12. 12. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 ~3 years in Server Networking
  13. 13. Server-side Interface Definition Language (IDL) Service A “server” @junr03 #service_a.proto service ServiceA { rpc UpdateA(ARequest) returns (AResponse) { option (http.http_options).path = "/service_a"; option (http.http_options).method = "POST"; } } Service B “client”
  14. 14. ● Network Proxy driven by Open-source (OSS) community ● Common substrate for network behavior What is ? @junr03 Sidecar Service B “client” Sidecar Service A “server”
  15. 15. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party Strongly-typed APIs and a Universal Network Primitive @junr03
  16. 16. (circa early 2019) The End @junr03
  17. 17. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party�� Strongly-typed APIs and a Universal Network Primitive @junr03
  18. 18. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03
  19. 19. Consistency ? ✓ Performance ? ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03
  20. 20. Storage Edge Sidecar Service B Sidecar Service A 3rd-party Clients as another node in the mesh Mobile Client @goaway @junr03 API
  21. 21. Next generation Client APIs with Envoy Mobile Mobile Client @junr03 API �� We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). ~1 year in Client Networking
  22. 22. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Shape
  23. 23. Early API workflow at Lyft Tech spec handwritten.swift handwritten.kt handwritten.py/.go @rebello95
  24. 24. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95
  25. 25. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95
  26. 26. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03 ��
  27. 27. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03
  28. 28. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } if (android): else: Programming errors @rebello95
  29. 29. ● Handwritten code led to mistakes/typos ● No visibility into deserialization failures at compile time Problems @rebello95 @junr03
  30. 30. YAML schema integration @rebello95
  31. 31. paths: /foo: post: parameters: - in: body schema: $ref: '#/definitions/FooRequest' responses: 200: schema: $ref: '#/definitions/FooResponse' YAML definitions @rebello95
  32. 32. ● Manual code generation for Android ● No integration on iOS ● No single source of truth YAML results @rebello95
  33. 33. Consistency @rebello95 @junr03 We needed a source of truth that provided guarantees about the shape of any given API.
  34. 34. Envisioned workflow Tech spec IDL repo foo_api.swift foo_api.kt foo_api.py/.go foo_api.proto @rebello95
  35. 35. ● Platform-neutral IDL for defining and serializing strongly-typed APIs. ● JSON support (migration) ● Already used server-side (consistency) Why protobuf? @rebello95 @junr03
  36. 36. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift @rebello95
  37. 37. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @rebello95
  38. 38. Using generated APIs in Swift @rebello95
  39. 39. // Modules/FooV1API/FooAPI.swift public struct FooRequest: Codable, Equatable { … } public struct FooResponse: Codable, Equatable { public let id: Int64 public let firstName: String } Generated models @rebello95
  40. 40. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { // Abstracted implementation } Generated APIs @rebello95
  41. 41. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { 🌟 // Abstracted implementation 🌟 (hint hint wink wink!) } Generated APIs @rebello95 @junr03
  42. 42. // Modules/FooV1APIMock/FooAPI.swift open class FooMockAPI: FooAPI { open lazy var mockUpdateFoo: (FooRequest) -> Result<FooResponse, FooAPIError> = { … } open func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> { … } } Generated mocks @rebello95
  43. 43. ● Single source of truth for all APIs ● Consistent, generated code on all platforms ● Highly testable ● Underlying transport fully abstracted 🌟 565958 Results @rebello95
  44. 44. Performance @rebello95 @junr03 Experimenting with payload encoding
  45. 45. Client Service JSON request Existing system JSON response @rebello95
  46. 46. Client Service JSON request Content-Type: json Accept: x-protobuf,json Content negotiation Middleware JSON > Protobuf Protobuf request Content-Type: x-protobuf Protobuf response Content-Type: x-protobuf Middleware No change Protobuf response Content-Type: x-protobuf @rebello95
  47. 47. JSON Protobuf @rebello95
  48. 48. ● 40%+ reduction in payload sizes ● Faster response times ● Higher success rates ● Transparent to engineers Wins from protobuf @rebello95
  49. 49. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
  50. 50. Extensibility @junr03 Using protobuf annotations to declare API behavior
  51. 51. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95 @junr03
  52. 52. Service Envoy URLSession iOS OkHTTP Android @rebello95
  53. 53. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
  54. 54. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Transport
  55. 55. Service Envoy Library X Client @rebello95
  56. 56. Clients as another node in the mesh Storage Edge Sidecar Service B Sidecar Service A 3rd-party Client @goaway @junr03
  57. 57. Standardizing infrastructure @goaway @junr03
  58. 58. Why is world domination transport standardization useful? ● Write once, deploy everywhere ● Common tooling for common problems ● Reduce cognitive load @goaway @junr03
  59. 59. Envoy Mobile to the rescue! ● v0.1: proof-of-concept ● v0.2: laying the foundation ● v0.3: first production-ready release @goaway @junr03
  60. 60. Envoy as a Library @junr03 How did we turn a network proxy into a networking library?
  61. 61. Build System: bazel @goaway @junr03 ● Cross-platform support ● Used by Envoy
  62. 62. //library/common:c_types //library/kotlin:android_framework //library/swift:ios_framework Layered Architecture //:ios_dist //:android_dist //library/common:main_interface @goaway @junr03 //library/common/... @envoy//source/... Platform (iOS/Android) Bridge (C) Native (C++/Envoy)
  63. 63. How to run a process in an app? picture of an engine (a very fast one) @goaway @junr03
  64. 64. Threading contexts Application Threads Envoy Main Thread Callback Threads @goaway @junr03
  65. 65. Library Matrix Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy) Application Threads Envoy Main Thread Callback Threads @goaway @junr03
  66. 66. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
  67. 67. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Engine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
  68. 68. Server Envoy Envoy Main Thread Worker Threads Envoy Dispatcher Http Manager Requests @goaway @junr03
  69. 69. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - using Envoy Constructs
  70. 70. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - starting a stream
  71. 71. Library Lifecycle - dispatching a stream Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
  72. 72. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!
  73. 73. Callbacks Library Lifecycle - callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
  74. 74. Library Lifecycle - platform callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
  75. 75. Platform (iOS/Android) Overall View Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher Canceled @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Canceled
  76. 76. Platform Code let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() val envoy = EnvoyClientBuilder( Domain("api.envoyproxy.io")) .addLogLevel(LogLevel.WARN) .addStatsFlushSeconds(60) ... .build() @goaway @junr03
  77. 77. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03
  78. 78. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03
  79. 79. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03
  80. 80. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03
  81. 81. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> ... } .onData { data -> // Deserialize message data here } ... @goaway @junr03
  82. 82. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> // Product logic } .onData { data -> // Product logic } ... @goaway @junr03
  83. 83. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓
  84. 84. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @goaway @junr03 NSUrl OkHttp
  85. 85. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library Client @goaway @junr03
  86. 86. Ecosystem @junr03 Shape + Transport = Ecosystem!
  87. 87. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ @junr03
  88. 88. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.
  89. 89. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95
  90. 90. Service Envoy URLSession iOS OkHTTP Android @rebello95 @junr03 �� �� ��
  91. 91. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.
  92. 92. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!
  93. 93. Envoy filters Compression Filter gzip request Mobile Proxy Compression Filter Unzip payload 🥳 EdgeClient @junr03
  94. 94. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)
  95. 95. Deferred Requests @junr03 Deferred Request Filter Mobile Proxy Client ❌ Service ❌ Deferred Request Filter Mobile Proxy Client ✅ Service ✅ 🥳 �� ��
  96. 96. Request Signing @junr03 Client Request Signature Filter sign request Mobile Proxy Request Signature Filter Verify signature Edge 🕵‍♀
  97. 97. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)
  98. 98. Observability @junr03 True end-to-end insight
  99. 99. Observability ts(envoy_mobile.cluster.api.upstream_rq.count) ts(envoy_edge.cluster.*.upstream_rq.count) @goaway @junr03
  100. 100. Time-series Metrics Metrics Service @goaway @junr03
  101. 101. Dashboards! ts(envoy_edge...) @goaway @junr03 ts(envoy_mobile...)
  102. 102. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Extensibility ✓ ✓ Performance ✓ ✓ Resilience ✓ ✓ Security ✓ ✓ Observability ✓ ✓
  103. 103. Onwards! ● Additional functionality in filters ● Protocol Experimentation with QUIC ● OSS Code Generators: complete OSS ecosystem for model-based APIs. @junr03
  104. 104. Next Generation Client APIs @junr03 Model-based APIs defined in a strongly-typed IDL so platform engineers can iterate on behavior using a common transport layer.
  105. 105. Alpha Beta Production Experiment @Lyft! @goaway @junr03
  106. 106. envoy-mobile.github.io @junr03 Join us!
  107. 107. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/

×