In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined. The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine. Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.

1. Reactive Streams 1.0.0
and why you should care
Dr. Roland Kuhn
@rolandkuhn — Akka Tech Lead

2. Reactive Streams 1.0.0

3. Philosophical Background
3
“You cannot step twice into the same stream.
For as you are stepping in,
other waters are ever flowing on to you.”
— Heraclitus

4. What is a Stream?
• ephemeral flow of data
• possibly unbounded in size
• focused on describing transformation
• can be formed into processing networks
4

5. Reactive Traits

6. The Problem:
Getting Data across an
Asynchronous Boundary
without running OutOfMemory

7. Enter Reactive Streams
7
“Reactive Streams is an initiative to provide a
standard for asynchronous stream processing with
non-blocking back pressure on the JVM.”
— reactive-streams.org

8. Participants
• Engineers from (among others):
• Netflix
• Pivotal
• Red Hat
• Typesafe
• Individuals like Doug Lea and Todd Montgomery
8

9. Motivation
• all participants face the same basic problem
• all are building tools for their community
• a common solution benefits everybody
• interoperability to make best use of efforts
• proposal to include in JDK9 (j.u.c.Flow)
9

10. Recipe for Success
• minimal interfaces
• rigorous specification of semantics
• full TCK for verification of implementation
• complete freedom for many idiomatic APIs
10

11. Supply and Demand
• data items flow downstream
• demand flows upstream
• data items flow only when there is demand
• recipient is in control of incoming data rate
• data in flight is bounded by signaled demand
11
Publisher Subscriber
data
demand

12. Dynamic Push–Pull
• “push” behavior when consumer is faster
• “pull” behavior when producer is faster
• switches automatically between these
• batching demand allows batching data
12
Publisher Subscriber
data
demand

13. Back-Pressure is Contagious
• C is slow
• B must slow down
• A must slow down
13
CA B

14. Back-Pressure can be Propagated
• TCP for example has it built-in
14
CA B
networkhosts

15. The Meat
15
trait Publisher[T] {
def subscribe(sub: Subscriber[T]): Unit
}
trait Subscription {
def request(n: Int): Unit
def cancel(): Unit
}
trait Subscriber[T] {
def onSubscribe(s: Subscription): Unit
def onNext(e: T): Unit
def onError(t: Throwable): Unit
def onComplete(): Unit
}

16. How does it Connect?
16
SubscriberPublisher
subscribe
onSubscribeSubscription

17. How does it Flow?
17
SubscriberPublisher
request
onNextElements
request
onNextElements
request

18. How does it Complete?
18
SubscriberPublisher
request
onNextElements
onComplete

19. How does it Fail?
19
SubscriberPublisher
request
onNextElements
onError

20. Reactive Streams
• asynchronous non-blocking data flow
• asynchronous non-blocking demand flow
• minimal coordination and contention
• message passing allows for distribution across
applications, nodes, CPUs, threads, actors, …
20
http://reactiveEstreams.org/

21. What is it good for?

22. Streams are Ubiquitous
• Streaming SIMD Extensions (SSE) — Intel,1999
• ingesting, transforming and emitting data
• requests & responses flowing through a system
• streams are graphical and intuitive
22

23. Moving Bulk Data over the Network
• content streaming (e.g. video or audio)
• data storage / backup
• data replication between data centers
23

24. Mobile Devices
• streams of updates from the server
• streams of user commands from the client
• bad connection quality—need for back-pressure
24

25. Internet of Things
• ingesting large numbers of low-rate streams
• conflating or extrapolating data
• aggregating many streams
• streaming towards the fleet of devices
25

26. Realtime Data Analysis
• real-time business intelligence
• complex event processing
• temporal correlations (e.g. credit card fraud)
26

27. The Ecosystem

28. RxJava
• «AnAPIforasynchronousprogrammingwith
observablestreams» — reactivex.io
• inspiration for Reactive Streams interfaces
• RxJavaReactiveStreams provides RS compliant
wrapper for Observable
28
io.reactivex:rxjava-reactive-streams:1.0.0

29. Project Reactor
• «afoundationallibraryforbuildingreactivefast-data
applicationsontheJVM» — projectreactor.io
• Stream directly implements Publisher and offers a
rich set of transformations
29
io.projectreactor:reactor-stream:2.0.3.RELEASE

30. Akka Streams
• compose any kind of stream processing topology
from immutable and reusable blueprints
• creates RS compliant Publisher/Subscriber when
materializing a flow graph into Actors
• high-level flow graph DSL
30
com.typesafe.akka:akka-stream-experimental:1.0-RC3

31. ratpack
• «RatpackisasetofJavalibrariesthatfacilitatefast,
efficient,evolvableandwelltestedHTTP
applications.» — ratpack.io
• accepts Publishers of streams to be sent to the
client
• chunked responses, server-sent events, websockets
31
io.ratpack:ratpack-core:0.9.17

32. vert.x 3.0
• «Vert.xisalightweight,highperformanceapplication
platformfortheJVMthat'sdesignedformodern
mobile,web,andenterpriseapplications.» — vertx.io
• Reactive Streams Integration Module
(https://github.com/vert-x3/vertx-reactive-streams)
32
io.vertx:vertx-reactive-streams:3.0.0-milestone6

33. Slick 3.0
• «FunctionalRelationalMappingforScala»—
slick.typesafe.com
• database result sets can be streamed, returning a
Publisher instead of a collection
• no streaming inserts yet
33
com.typesafe.slick:slick_2.11:3.0.0

34. Kafka
• «ApacheKafkaispublish-subscribemessaging
rethoughtasadistributedcommitlog.» —
kafka.apache.org
• Reactive Kafka (by Softwaremill):
consumer/producer are RS Subscriber/Publisher
34
com.softwaremill:reactive-kafka_2.11:0.6.0

35. AMQP
• «Tobecomethestandardprotocolforinteroperability
betweenallmessagingmiddleware» — amqp.org
• https://github.com/ScalaConsultants/reactive-rabbit
• fully asynchronous API
• consumer/publisher are RS Subscriber/Publisher
35
io.scalac:reactive-rabbit_2.11:1.0.0

36. MongoDB
• «TheofficialMongoDBReactiveStreamsJavaDriver,
providingasynchronousstreamprocessingwithnon-
blockingbackpressureforMongoDB.»—
http://mongodb.github.io/mongo-java-driver-
reactivestreams/
• all queries return Publishers
• no streaming inserts yet
36
org.mongodb:mongodb-driver-reactivestreams:1.0.1

37. … and more are coming (TBC)
• Cassandra
• Spark
• Riak
37

38. Reactive Streams in the
Typesafe Reactive Platform

39. 39
O/S-level network stack
Java NIO (JDK)
Akka IO
Akka HTTP Core
Akka HTTP
application level
Akka Streams Database
Driver
Slick 3.0
Play Framework ReactiveStreamsIntegration: preview available

40. Streaming Database Results to the Client
40
import DefaultJsonProtocol._
implicit val denormOrderFormat = jsonFormat5(DenormalizedOrder.apply)
val db = Database.forConfig("reportingDB")
private def getFromDb(userId: Int): Publisher[DenormalizedOrder] =
db.stream(denormalizedOrders.filter(_.userId === userId).result)
Http().bindAndHandle(
(get & path("orders" / IntNumber)) { userId =>
val pub =
Source(getFromDb(userId))
.transform(() => new ToJsonArray)
complete(HttpEntity.Chunked.fromData(`application/json`, pub))
},
"localhost", 8080)

41. REACTIVE PLATFORM
Full Lifecycle Support for Play, Akka, Scala and Spark
Give your project a boost with Reactive Platform:
• Monitor Message-Driven Apps
• Resolve Network Partitions Decisively
• Integrate Easily with Legacy Systems
• Eliminate Incompatibility & Security Risks
• Protect Apps Against Abuse
• Expert Support from Dedicated Product Teams
Enjoy learning? See about the availability of
on-site training for Scala, Akka, Play and Spark!
Learn more about our offersCONTACT US

42. ©Typesafe 2015 – All Rights Reserved