Distributed Reactive Services with Reactor & Spring with Stéphane Maldini at SpringOne Tour 2019

1. Multi-Service Reactive Streams using
RSocket, Reactor, and Spring
Stephane Maldini, Project Reactor Lead
@smaldini

2. 2
No, I do not have a medium blog
=+
@smaldini

3. “The network is the unsolved problem of cloud, and open source. We need
the network to be a first-class citizen of a cloud system.”
-Daniel Berg, distinguished engineer at IBM Cloud
Communication in the Cloud is a Challenge
3

4. “A standard approach to connecting and communicating with microservices is
needed. Standardization and uniformity simplifies management.”
-Tony Lock, distinguished analyst at Freeform Dynamics
Communication in the Cloud is a Challenge
4

5. What does RSocket bring on the table ?
5

6. 6
Open-source application-layer network protocol

7. 7
Designed to simplify communication between services
Open-source application-layer network protocol

8. 8
Built for Modern Cloud applications
Designed to simplify communication between services
Open-source application-layer network protocol

9. What is a Modern Cloud application?
9

10. What is a Modern Cloud application?
1010
Deployment topologies
Platforms/frameworks
Protocols/interaction models

11. Axis 1: Deployment Topologies
11
Datacenter/private cloudMultiple public cloud providers

12. Axis 2: Platforms/Frameworks
12
Multiple frameworksMultiple orchestration platforms

13. Axis 3: Protocols/Interaction Models
13

14. Axis 3: Protocols/Interaction Models
14

15. Axis 3: Protocols/Interaction Models
15
Request-reply

16. Axis 3: Protocols/Interaction Models
16
Request-reply

17. Axis 3: Protocols/Interaction Models
17
Request-reply Streaming

18. Axis 3: Protocols/Interaction Models
18
Request-reply Streaming

19. Axis 3: Protocols/Interaction Models
19
Request-reply Streaming RPC

20. Axis 3: Protocols/Interaction Models
20
Request-reply Streaming RPC

21. Axis 3: Protocols/Interaction Models
21
Request-reply Streaming RPC
Peer-to-peerReal-time

22. Can We Do Something About This Mess?
22
Request-reply Streaming RPC
Peer-to-peerReal-time

23. Things We Can’t Do Anything About
23
It’s tempting to address complexity by imposing uniformity
of infrastructure. This is costly and constraining, and
doesn’t address any of the difficulties in building reliable
distributed applications.
Deployment Topology

24. Things We Can’t Do Anything About
24
Frameworks and languages have their place, we want to
be open to new technologies and allow developers the
freedom to choose the right tools for the job.
Proliferation of development frameworks

25. Things We Can Do Something About
25
Define a standard protocol that enables us to use this
small finite set of interactions across any combination of
infrastructure, language, framework, so they don’t have to
be implemented, reimplemented and implemented again!

26. http://rsocket.io

27. Java API
public interface RSocket {
Mono<Payload> requestResponse(Payload payload);
Mono<Void> fireAndForget(Payload payload);
Flux<Payload> requestStream(Payload payload);
Flux<Payload> requestChannel(Flux<Payload> payloads);
}

28. Java API
public interface RSocket {
Mono<Payload> requestResponse(Payload payload);
Mono<Void> fireAndForget(Payload payload);
Flux<Payload> requestStream(Payload payload);
Flux<Payload> requestChannel(Flux<Payload> payloads);
}

29. Interaction Models – Request-Response
Mono<Payload> resp = client.requestResponse(requestPayload)
• Standard Request-Response semantics
• Likely to represent the majority of requests for the foreseeable future
• Even this obvious interaction model surpasses HTTP because it is
asynchronous and multiplexed
• Request with account number, respond with account balance

30. Interaction Models – Fire-and-Forget
Mono<Void> resp = client.fireAndForget(requestPayload)
• An optimization of Request-Response when a response isn't necessary
• Significant efficiencies
• Networking (no ack)
• Client/Server processing (immediate release of resources)
• Non-critical event logging

31. Interaction Models – Request-Stream
Flux<Payload> resp = client.requestStream(requestPayload)
• Analogous to Request-Response returning a collection
• The collection is streamed back instead of queuing until complete
• RequestN semantics mean data is not materialized until ready to send
• Request with account number, respond with real-time stream of account
transactions

32. Interaction Models – Channel
Flux<Payload> out = client.requestChannel(Flux<Payload> in)
• A bi-directional stream of messages in both directions
• An unstructured channel allows arbitrary interaction models
• Request burst of initial state, listen for subsequent updates, client
updates subscription without starting new connection
• Request with account number, respond with real-time stream of account
transactions, update subscription to filter certain transaction types,
respond with filtered real-time stream of account transactions

33. Message Driven Binary Protocol
• Requester-Responder interaction is broken down into frames that
encapsulate messages
• The framing is binary (not human readable like JSON or XML)
• Massive efficiencies for machine-to-machine communication
• Downsides only manifest rarely and can be mitigated with tooling
• Payloads are bags of bytes
• Can be JSON or XML (it's all just 1's and 0’s)

35. request/reply
request/void (fire&forget)
request/stream
stream/stream (channel)
4 defined interaction models

36. request/replyrequest/reply

37. request/reply request/reply

38. request/reply request/replyrequest/reply

39. request/reply request/replyrequest/reply

40. request/reply request/replyrequest/reply
🔎multiplexed

41. Multiplexed
• Connections that are only used for a single request are massively
inefficient (HTTP 1.0)
• Pipelining (ordering requests and responses sequentially) is a naive
attempt solving the issue, but results in head-of-line blocking (HTTP 1.1)
• Multiplexing solves the issue by annotating each message on the
connection with a stream id that partitions the connection into multiple
"logical streams"

42. 🔎transport agnostic
e.g. Websocket

43. request/streamrequest/stream

44. request/streamrequest/stream

45. request/stream
🔎 🔎
bidirectional

46. Bi-Directional
• Many protocols (notably not TCP) have a distinction between the client
and server for the lifetime of a connection
• This division means that one side of the connection must initiate all
requests, and the other side must initiate all responses
• Even more flexible protocols like HTTP/2 do not fully drop the distinction
• Servers cannot start an unrequested stream of data to the client
• Once a client initiates a connection to a server, both parties can be
requestors or responders to a logical stream

47. 2
per-message
flow-control

48. 2 2
per-message
flow-control

49. 2
per-message
flow-control

50. 00
per-message
flow-control

51. 0
per-message
flow-control

52. Reactive Streams Back Pressure
Network protocols generally send a single request, and receive an
arbitrarily large response in return
There is nothing to stop the responder (or even the requestor) from sending an
arbitrarily large amount of data and overwhelming the receiver
In cases where TCP back pressure throttles the responder, queues fill with large
amounts of un-transferred data

53. Reactive Streams (pull-push) back pressure ensures that data is only
materialized and transferred when receiver is ready to process it

54. Without you counting bytes manually

55. 20 0

56. 302 0
21

57. 3 2
2
2

58. 3 22

59. 23 2
3

60. 3 2
3
❌

61. 3 22

62. 3 2
Resumption
22

63. 3 23
3
Resumption

64. 3 33
3
Resumption

65. Resumption/Resumability
• Starting as a client-to-edge-server protocol highlighted a common failing of
existing options
• Clients on unstable connections would often drop and need to re-establish
current state
• Led to inefficiencies in both network traffic and data-center compute
• Resumability allows both parties in a "logical connection" to identify
themselves on reconnection
• On Resumption both parties handshake about the last frame received and all
missed frames are re-transmitted
• Frame caching to support is not defined by spec so it can be very flexible
X

67. language agnostic

68. compose with no semantics loss
🔎
🔎
🔎
ws
tcp
udp

69. RSocket Protocol
TCP WebSocket Aeron/UDPHTTP/2
Protobuf JSON Custom Binary
RPC-style Messaging
Java JavaScript C++ Go Flow

70. Using the RSocket API

71. Raw Client
RSocket client =
RSocketFactory.connect()
.transport(TcpClientTransport.create("1.2.3.4", 80))
.start()
.block();

72. Raw Client
RSocket client =
RSocketFactory.connect()
.transport(TcpClientTransport.create("1.2.3.4", 80))
.start()
.block();
client.requestResponse(new DefaultPayload(…))
.doOnComplete(() -> System.out.println(“hooray”)
.subscribe();

73. Raw Client
RSocket client =
RSocketFactory.connect()
.transport(TcpClientTransport.create("1.2.3.4", 80))
.start()
.block();
client.requestResponse(new DefaultPayload(…))
.doOnComplete(() -> System.out.println(“hooray”)
.subscribe();
Censored ByteBuffer creation

74. Too Low Level ?
Shift to the next gear with existing tech built on RSocket
😱

75. Building applications with RSocket API
RSocket is Programming Model Agnostic

76. Building applications with RSocket API
RSocket is Programming Model Agnostic
The RSocket interface is a serviceable contract

77. Building applications with RSocket API
A building block that other programming models could build upon
RSocket is Programming Model Agnostic
The RSocket interface is a serviceable contract

78. RSocket Protocol
TCP WebSocket Aeron/UDPHTTP/2
Protobuf JSON Custom Binary
RPC-style Messaging
Java JavaScript C++ Go Flow

79. RPC-style (Contract Driven)
service RecordsService {
rpc records (RecordsRequest) returns (stream Record) {}
}

80. RPC-style (Contract Driven)
service RecordsService {
rpc records (RecordsRequest) returns (stream Record) {}
}
RecordsServiceClient rankingService =
new RecordsServiceClient(rsocket);
recordsService.records(RecordsRequest.newBuilder()
.setMaxResults(16)
.build())
.subscribe(record -> System.out.println(record));

81. RPC-style (Contract Driven)
service RecordsService {
rpc records (RecordsRequest) returns (stream Record) {}
}
RecordsServiceClient rankingService =
new RecordsServiceClient(rsocket);
recordsService.records(RecordsRequest.newBuilder()
.setMaxResults(16)
.build())
.subscribe(record -> System.out.println(record));
You still need to manage this part

82. RPC-style (Contract Driven)
service RecordsService {
rpc records (RecordsRequest) returns (stream Record) {}
}

83. RPC-style (Contract Driven)
service RecordsService {
rpc records (RecordsRequest) returns (stream Record) {}
}
let recordServiceClient = new RecordsServiceClient(rsocket);
let req = new RecordRequest();
req.setMaxResults(16);
recordServiceClient.records(req)
.subscribe();

84. Messaging-Style
@Controller class TestHandler{
@MessageMapping("/canonical")
Mono<String> handleCanonical(String payload) {
return Mono.delay(Duration.ofMillis(10))
.map(l -> createResponse(payload));
}
@MessageMapping("/async-arg")
Mono<String> handleMonoArg(Mono<String> payload) {
return payload.map(ServerResponderApp::createResponse);
}
}

85. Messaging-Style
@Controller class TestHandler{
@MessageMapping("/canonical")
Mono<String> handleCanonical(String payload) {
return Mono.delay(Duration.ofMillis(10))
.map(l -> createResponse(payload));
}
@MessageMapping("/async-arg")
Mono<String> handleMonoArg(Mono<String> payload) {
return payload.map(ServerResponderApp::createResponse);
}
}
Routing Information : RSocket Extension

86. Messaging-Style
@Controller class TestHandler{
@MessageMapping("/canonical")
Mono<String> handleCanonical(String payload) {
return Mono.delay(Duration.ofMillis(10))
.map(l -> createResponse(payload));
}
@MessageMapping("/async-arg")
Mono<String> handleMonoArg(Mono<String> payload) {
return payload.map(ServerResponderApp::createResponse);
}
}
Out Of The Box - Encoding/Decoding

87. Messaging-Style
@Controller class TestHandler{
@MessageMapping("/canonical")
Mono<String> handleCanonical(String payload) {
return Mono.delay(Duration.ofMillis(10))
.map(l -> createResponse(payload));
}
@MessageMapping("/async-arg")
Mono<String> handleMonoArg(Mono<String> payload) {
return payload.map(ServerResponderApp::createResponse);
}
}
Adapting to the right interaction model, e.g. Request-Response

88. 80
Spring Framework 5.2
&
Spring Boot 2.2

89. 81
demo
https://github.com/reactor/head-first-reactive-with-spring-and-reactor/tree/rsocket

90. Blocking Apps with REST
over HTTP
WebFlux Apps with REST
over HTTP
WebFlux Apps with
RSocket

91. Spring support to expect soon
Spring Cloud Gateway
Spring Config Server
Spring Security
Spring Integration
Micrometer
Steeltoe
Spring Cloud Sleuth

92. > Stay Connected.
https://projectreactor.io
http://rsocket.io

93. Additional Protocol Features

94. Metadata and Data in Frames
• Each Frame has an optional metadata payload
• The metadata payload has a MIME-Type but is otherwise unstructured
• Very flexible
• Can be used to carry metadata about the data payload
• Can be used to carry metadata in order to decode the payload
• Generally means that payloads can be heterogenous and each message
decoded uniquely

95. Fragmentation
• Payload frames have no maximum size
• The protocol is well suited to serving large payloads
• Still Images (Facebook), Video (Netflix)
• Both TCP MTUs and reliability on slow connections lead towards smaller
payloads
• Fragmentation provides a way to continue to reason about "logical
frames" while ensuring that individual payloads are smaller
• Applied transparently, after enforcement of RequestN semantics

96. Cancellation
• All interaction types support cancellation
• Cancellation is a signal by the requestor that any inflight processing
should be terminated eagerly and aggressively
• An obvious requirement for Request-Stream and Channel
• But useful even in Request-Response where the response can be
expensive to generate
• Early termination can lead to significant improvement in efficiency

97. Leasing
• Reactive back pressure ensures that a responder (or either party in a
Channel) cannot overwhelm the receiver
• This does not prevent a requestor from overwhelming a responder
• This commonly happens in server-to-server environments where
throughput is high
• Leasing enables responders to signal capacity to requestors
• This signal is useful for client-side load-balancing
• Without preventing server-side load-balancing

98. > Stay Connected.
https://projectreactor.io
http://rsocket.io