Next Generation Client APIs in Envoy Mobile

1. Jose Nino @junr03 Next Generation Client APIs with Envoy Mobile QCon London - March 2020 Some slides built in collaboration with: @goaway & @rebello95

2. InfoQ.com: News & Community Site Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/ • Over 1,000,000 software developers, architects and CTOs read the site worldwide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week

3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com

4. Server Networking Networking Client @rebello95

5. Server Networking Networking Client @junr03 API

6. Server Networking Networking Server @junr03 Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking

7. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking

8. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking ��

9. Jose Nino @junr03 Infrastructure Engineer ~4 years @

10. Service Networking Networking Server @junr03 API Server Networking Service Networking Server Networking Service Networking Service Networking Service Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Server Networking Service Networking Server Networking Networking Server Networking Server Networking Server Networking ��

11. Server Networking Networking Client @junr03 API

12. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 ~3 years in Server Networking

13. Server-side Interface Deﬁnition Language (IDL) Service A “server” @junr03 #service_a.proto service ServiceA { rpc UpdateA(ARequest) returns (AResponse) { option (http.http_options).path = "/service_a"; option (http.http_options).method = "POST"; } } Service B “client”

14. ● Network Proxy driven by Open-source (OSS) community ● Common substrate for network behavior What is ? @junr03 Sidecar Service B “client” Sidecar Service A “server”

15. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party Strongly-typed APIs and a Universal Network Primitive @junr03

16. (circa early 2019) The End @junr03

17. Storage EdgeMobile Client Sidecar Service B Sidecar Service A 3rd-party�� Strongly-typed APIs and a Universal Network Primitive @junr03

18. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03

19. Consistency ? ✓ Performance ? ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03

20. Storage Edge Sidecar Service B Sidecar Service A 3rd-party Clients as another node in the mesh Mobile Client @goaway @junr03 API

21. Next generation Client APIs with Envoy Mobile Mobile Client @junr03 API �� We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). ~1 year in Client Networking

22. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Shape

23. Early API workﬂow at Lyft Tech spec handwritten.swift handwritten.kt handwritten.py/.go @rebello95

24. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95

25. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95

26. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03 ��

27. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } Programming errors @rebello95 @junr03

28. iOS Server { “key_a”: “hello” } Android { “kye_a”: “hello” } if (android): else: Programming errors @rebello95

29. ● Handwritten code led to mistakes/typos ● No visibility into deserialization failures at compile time Problems @rebello95 @junr03

30. YAML schema integration @rebello95

31. paths: /foo: post: parameters: - in: body schema: $ref: '#/definitions/FooRequest' responses: 200: schema: $ref: '#/definitions/FooResponse' YAML deﬁnitions @rebello95

32. ● Manual code generation for Android ● No integration on iOS ● No single source of truth YAML results @rebello95

33. Consistency @rebello95 @junr03 We needed a source of truth that provided guarantees about the shape of any given API.

34. Envisioned workﬂow Tech spec IDL repo foo_api.swift foo_api.kt foo_api.py/.go foo_api.proto @rebello95

35. ● Platform-neutral IDL for deﬁning and serializing strongly-typed APIs. ● JSON support (migration) ● Already used server-side (consistency) Why protobuf? @rebello95 @junr03

36. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift @rebello95

37. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @rebello95

38. Using generated APIs in Swift @rebello95

39. // Modules/FooV1API/FooAPI.swift public struct FooRequest: Codable, Equatable { … } public struct FooResponse: Codable, Equatable { public let id: Int64 public let firstName: String } Generated models @rebello95

40. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { // Abstracted implementation } Generated APIs @rebello95

41. // Modules/FooV1API/FooAPI.swift import RxSwift public protocol FooAPI { func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> } public struct FooClientAPI: FooAPI { 🌟 // Abstracted implementation 🌟 (hint hint wink wink!) } Generated APIs @rebello95 @junr03

42. // Modules/FooV1APIMock/FooAPI.swift open class FooMockAPI: FooAPI { open lazy var mockUpdateFoo: (FooRequest) -> Result<FooResponse, FooAPIError> = { … } open func updateFoo(_ request: FooRequest) -> Single<Result<FooResponse, FooAPIError>> { … } } Generated mocks @rebello95

43. ● Single source of truth for all APIs ● Consistent, generated code on all platforms ● Highly testable ● Underlying transport fully abstracted 🌟 565958 Results @rebello95

44. Performance @rebello95 @junr03 Experimenting with payload encoding

45. Client Service JSON request Existing system JSON response @rebello95

46. Client Service JSON request Content-Type: json Accept: x-protobuf,json Content negotiation Middleware JSON > Protobuf Protobuf request Content-Type: x-protobuf Protobuf response Content-Type: x-protobuf Middleware No change Protobuf response Content-Type: x-protobuf @rebello95

47. JSON Protobuf @rebello95

48. ● 40%+ reduction in payload sizes ● Faster response times ● Higher success rates ● Transparent to engineers Wins from protobuf @rebello95

49. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓

50. Extensibility @junr03 Using protobuf annotations to declare API behavior

51. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95 @junr03

52. Service Envoy URLSession iOS OkHTTP Android @rebello95

53. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓

54. We recognized two important dimensions in API management: what data we send (shape), and how we send it (transport). @junr03 API Transport

55. Service Envoy Library X Client @rebello95

56. Clients as another node in the mesh Storage Edge Sidecar Service B Sidecar Service A 3rd-party Client @goaway @junr03

57. Standardizing infrastructure @goaway @junr03

58. Why is world domination transport standardization useful? ● Write once, deploy everywhere ● Common tooling for common problems ● Reduce cognitive load @goaway @junr03

59. Envoy Mobile to the rescue! ● v0.1: proof-of-concept ● v0.2: laying the foundation ● v0.3: ﬁrst production-ready release @goaway @junr03

60. Envoy as a Library @junr03 How did we turn a network proxy into a networking library?

61. Build System: bazel @goaway @junr03 ● Cross-platform support ● Used by Envoy

62. //library/common:c_types //library/kotlin:android_framework //library/swift:ios_framework Layered Architecture //:ios_dist //:android_dist //library/common:main_interface @goaway @junr03 //library/common/... @envoy//source/... Platform (iOS/Android) Bridge (C) Native (C++/Envoy)

63. How to run a process in an app? picture of an engine (a very fast one) @goaway @junr03

64. Threading contexts Application Threads Envoy Main Thread Callback Threads @goaway @junr03

65. Library Matrix Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy) Application Threads Envoy Main Thread Callback Threads @goaway @junr03

66. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

67. Library Lifecycle - Running Envoy Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Engine @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

68. Server Envoy Envoy Main Thread Worker Threads Envoy Dispatcher Http Manager Requests @goaway @junr03

69. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - using Envoy Constructs

70. Platform (iOS/Android) Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Library Lifecycle - starting a stream

71. Library Lifecycle - dispatching a stream Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

72. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!

73. Callbacks Library Lifecycle - callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

74. Library Lifecycle - platform callbacks Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher @goaway @junr03 Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

75. Platform (iOS/Android) Overall View Application Threads Envoy Main Thread Callback Threads EnvoyEngine EnvoyRunner Dispatcher Http Manager HttpStream Callbacks Lambdas Dispatcher Canceled @goaway @junr03 Bridge (C types/bindings) Native (C++/Envoy) Canceled

76. Platform Code let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() val envoy = EnvoyClientBuilder( Domain("api.envoyproxy.io")) .addLogLevel(LogLevel.WARN) .addStatsFlushSeconds(60) ... .build() @goaway @junr03

77. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03

78. Build an Engine let envoy = try EnvoyClientBuilder(domain: "api.envoyproxy.io") .addLogLevel(.warn) .addStatsFlushSeconds(60) .build() @goaway @junr03

79. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03

80. Build a Request let request = RequestBuilder(path: "/pb.api.v1.Foo/GetBar") .addHeader(name: "x-custom-header", value: "foobar") .build() @goaway @junr03

81. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> ... } .onData { data -> // Deserialize message data here } ... @goaway @junr03

82. Build a Response Handler let handler = ResponseHandler() .onHeaders { headers, status, _ -> // Product logic } .onData { data -> // Product logic } ... @goaway @junr03

83. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Performance ✓ ✓ Extensibility 🤔 ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓

84. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library @goaway @junr03 NSUrl OkHttp

85. IDL pipeline /api/v1/foo.proto Generators +7 others foo.go foo.py foo.kt foo.swift FooV1API library foo-v1-api library Client @goaway @junr03

86. Ecosystem @junr03 Shape + Transport = Ecosystem!

87. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? Consistency ✓ ✓ Performance ✓ ✓ Extensibility ? ✓ Resilience ? ✓ Security ? ✓ Observability ? ✓ @junr03

88. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.

89. service Foo { rpc GetFoo(FooRequest) returns (FooResponse) { option (http.http_options).path = "/foo/receive"; option (http.http_options).method = "GET"; option (http.http_options).client_poll_ms = 5000; option (http.http_options).gzip_enabled = true; } } Declarative APIs @rebello95

90. Service Envoy URLSession iOS OkHTTP Android @rebello95 @junr03 ��

91. Extensibility @junr03 Using protobuf annotations to declare API behavior and Envoy Mobile to enact consistent behavior.

92. Http Manager Http Filters @junr03 L7 Filter L7 FilterHttp Filter Dispatcher ● Foundational extension point ● Enact behavior ● Provide the guarantees we want!

93. Envoy ﬁlters Compression Filter gzip request Mobile Proxy Compression Filter Unzip payload 🥳 EdgeClient @junr03

94. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)

95. Deferred Requests @junr03 Deferred Request Filter Mobile Proxy Client ❌ Service ❌ Deferred Request Filter Mobile Proxy Client ✅ Service ✅ 🥳 ��

96. Request Signing @junr03 Client Request Signature Filter sign request Mobile Proxy Request Signature Filter Verify signature Edge 🕵‍♀

97. Advanced transport @junr03 ● Dynamic Forward Proxy ✅ ● Compression 🚧 ● Deferred Requests 🕒 (Resilience) ● OAuth 🕒 (Security) ● Request Signing 🕒 (Security) ● Network condition 🕒 (Resilience)

98. Observability @junr03 True end-to-end insight

99. Observability ts(envoy_mobile.cluster.api.upstream_rq.count) ts(envoy_edge.cluster.*.upstream_rq.count) @goaway @junr03

100. Time-series Metrics Metrics Service @goaway @junr03

101. Dashboards! ts(envoy_edge...) @goaway @junr03 ts(envoy_mobile...)

102. Three 9s of reliability at the server-side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. What do we want from our Client APIs? @junr03 Consistency ✓ ✓ Extensibility ✓ ✓ Performance ✓ ✓ Resilience ✓ ✓ Security ✓ ✓ Observability ✓ ✓

103. Onwards! ● Additional functionality in ﬁlters ● Protocol Experimentation with QUIC ● OSS Code Generators: complete OSS ecosystem for model-based APIs. @junr03

104. Next Generation Client APIs @junr03 Model-based APIs deﬁned in a strongly-typed IDL so platform engineers can iterate on behavior using a common transport layer.

105. Alpha Beta Production Experiment @Lyft! @goaway @junr03

106. envoy-mobile.github.io @junr03 Join us!

107. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ api-envoy/

Next Generation Client APIs in Envoy Mobile

C4Media

Next Generation Client APIs in Envoy Mobile