This document outlines the agenda for Season 2 Episode 2 of an event. The episode will include:
1. Lightning talks on various Reactive and Rx topics from different speakers
2. Guest speakers Jake Wharton from Square, Matt Ingenthron from Couchbase, and Will Sargent from Typesafe
3. More details on RxJava and composable reactive functions
31. Tuning for an acceptable latency
distribution
client: ab
server CPU
usage varies
with load
32. RxNetty vs Tomcat
Event loop model more efficient than thread pools:
● Reduces thread CPU overheads
● Reduced thread lock contention
● Better work scheduling: platform chooses, instead of the
kernel scheduling threads
● Warmer CPU caches and memory locality on NUMA:
event worker threads can have better CPU affinity
33. RxNetty Micro Benchmarking
Server: AWS c3.xlarge, tuned to reduce perturbations
Client: Multi-threaded (wrk -t 4 -c 100/400), 1 min warmup
● Sufficient load to exhaust CPU capacity.
Active Benchmarking Methodology:
● Tools: sysstat, perf_events, SystemTap, flame graphs, …
● USE Method, static performance tuning, etc.
● Target analyzed during benchmark confirm target results
and identify (and tune) true limiters.
35. Hello World
● All platforms are <100us for Hello World, so their CPU
overheads are already negligible for this workload.
○ Consider application compute times of >1ms
● Hence testing “Hello Netflix” as well: a basic Netflix
service Hello World with dependency requests.
● Although, Hello World max latency (not pictured) was
still much higher for Tomcat...
36. Visualizing Tomcat Overheads
● Flame graph of
Java CPU time
for Hello World
benchmark
● Thread and I/O
management
overheads
37. Visualizing RxNetty Overheads
● Flame graph of Java CPU
time for Hello World
● Time in read() and write()
(doing I/O; ie, doing “work”)
dominates
● Flame graphs also studied
of kernel and user-time
● Many improvements found
and fixed
38. Blog post forthcoming… Keywords:
● Oracle JDK 1.8.0 (others tried); Xmx & Xms; different
GCs and settings, eg -XX:+UseConcMarkSweepGC
● Apache-tomcat 7.0.54: tested BIO, NIO, APR;
maxThreads=150 or 512; maxQueueSize=up to 400;
acceptCount=up to 100; disabled access log, ...
● Node.js 0.6.12 with clustering (faster than 0.10/0.11)
● vert.x 2.1 with -instances 4
● Java profilers: Google lightweight java profiler; JFR; …
● perf_events; SystemTap; flame graphs, ...
40. Netflix IPC Stack (1.0)
A
p
a
c
h
e
H
T
T
P
C
l
i
e
n
t
Eureka (Service Registry)
Server (Karyon)
Apache
Tomcat
Client
H
y
s
t
r
i
x
E
V
C
a
c
h
e
Ribbon
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin ConsoleHTTP
Eureka Integration
Registration
Fetch Registry
A Blocking Architecture
41. Netflix IPC Stack (2.0)
Client (Ribbon 2.0)
Eureka (Service Registry)
Server (Karyon)
Ribbon Transport
Load
Balancing
Eureka
Integration
Metrics
(Servo)
Bootstrapping (Governator)
Metrics (Servo)
Admin Console
HTTP
Eureka Integration
Registration
Fetch Registry
Ribbon
Hystrix
EVCache
R
x
N
e
t
t
y
RxNetty
UDP
TCP
WebSockets
SSE
A Completely Reactive
Architecture
42. “We want an extremely performant IPC library”
Heard this before?
51. Karyon 2.0
● Adds Netflix constructs (governator, eureka, archaius,
etc.) on RxNetty.
● Provides extensions (servlet, jersey, etc.) to RxNetty.
● More palatable to application developers.
52. Example
@ArchaiusBootstrap
@KaryonBootstrap(name = "hello-netflix-oss")
@Modules(include = {HelloNossApp.KaryonJerseyModuleImpl.class, KaryonWebAdminModule.class, KaryonEurekaModule.class})
public final class HelloNossApp {
public static class KaryonJerseyModuleImpl extends KaryonJerseyModule {
protected void configure() {
super.configure();
bind(AuthenticationService.class).to(AuthenticationServiceImpl.class);
}
public int serverPort() { return 8888; }
public int shutdownPort() { return 8899; }
public void configureInterceptors(GovernatorHttpInterceptorSupport<ByteBuf, ByteBuf> interceptorSupport) {
interceptorSupport.forUri("/hello").interceptIn(AuthInterceptor.class);
}
}
}
54. ● Tendency to become “fat” with boilerplate
code
● Hystrix integration
● Go “async”
● “Declare” a client, rather than “write” a client
Ribbon 2.0 - Motivation
56. Ribbon 2.0 - New “ribbon” module
● Template support for building HTTP request
● Rx style non-blocking and blocking APIs
● Hystrix built-in for fault tolerance
● Pluggable cache access with EVCache
implementation
● Annotation based client creation as an
alternative
58. Ribbon 2: Template Example
HttpResourceGroup movieService = Ribbon.createHttpResourceGroup("movieService");
HttpRequestTemplate<ByteBuf> template1 =
movieService.newRequestTemplate("recommendationsByUserId", ByteBuf.class)
.withMethod("GET")
.withUriTemplate("/users/{userId}/recommendations")
.withFallbackProvider(new RecommendationServiceFallbackHandler());
RibbonRequest<ByteBuf> request = template1.requestBuilder()
.withRequestProperty("userId", “user1”).build();
Observable<ByteBuf> result = request.toObservable(); // non blocking
ByteBuf result = request.execute(); // blocking
59. Ribbon 2.0 - Other New Features
● Ribbon’s async module - “ribbon-transport”
○ HTTP, TCP and UDP clients on top of RxNetty with
load balancing capability
● New load balancing features
○ Command pattern load balancing APIs implemented
with Rx for easy integration for third party client
○ Shuffle sharding server list for fault isolation
91. Motivations with Couchbase Java
➢ Simplify ...
➢ Even *more* performance
➢ Build for the JVM, not for the Java language
○ Have Scala, JRuby already being worked on
92. History of Concurrency
Core Java component, spymemcached, has
used Futures since 2006 or so.
➢ Always been ahead of most other database
like clients, however…
➢ Difficult for developers to code against
○ Dumb benchmarks turn into FAQs
96. What we’re Building On
➢ A set of new components
○ RxJava (both internal and public interface)
○ Netty (internal only)
○ LMAX Disruptor
97. RxJava for Couchbase
➢ Will be part of the client’s public API
○ Helps us support Java 8, but is also backward
compatible
■ Generally get a great Java 8 experience for free!
➢ Also using RxJava extensively internally
○ Breaking the project into core and language façade
helps build toward Scala and others
98. Getting Referenced Elements
client.asyncGet("posts::1").addListener(future -> {
if (future.isDone() && !future.isCancelled()) {
String content = (String) future.get();
List<String> ids =
comments_ids_from_json(content);
int i = 0;
for (String id : ids) {
if (i++ == 5) {
break;
}
client.asyncGet(id).addListener(future1 -> {
if (future1.isDone() && !future1.isCancelled()) {
System.out.println(future1.get());
}
});
}
}
});
99. Getting Referenced Elements Rx Way
bucket
.get("posts::1")
.map(document -> document.content().getArray("comments"))
.flatMap(comments -> Observable.from(comments.toList()))
.take(5)
.filter(o -> o instanceof String)
.flatMap(o -> bucket.get((String) o))
.subscribe(doc -> System.out.println(doc.content()));
107. Akka with Futures
Akka to Future:
val myFuture = actorRef ? message
Future to Akka:
futureStuff pipeTo actorRef
108. Assuming a Services based API
val fooFuture = fooService.get(1)
val barFuture = barService.get(2)
val quuxFuture = quuxService.get(3)
...Simple, right?
109. Not so fast
val fooFuture = fooService.get(1)
val barFuture = barService.get(foo.barId)
val quuxFuture = quuxService.get(bar.quuxId)
...wat.
110. FLATMAP THAT
fooService.get(1).map { maybeFoo =>
maybeFoo.flatMap { foo =>
val barFuture = barService.get(foo.barId)
// do the same for bar & quux
}
}
111. For comprehensions!
for {
maybeFoo <- fooService.get(1)
} yield {
for { // Option / Future does not compose
foo <- maybeFoo
} yield ...
}
112. Scala Async!
async {
val maybeFoo = await(fooService.get(1))
maybeFoo.flatMap { foo =>
val bar = await(barService.get(2))
}
}
113. Scala Async Sadness
COMPILE ERROR.
“await must not be used inside a closure nested
within an async block” == no flatMap of that.
114. Monads don’t compose
But before reaching for readers/transformers:
● Use small methods
● Short circuit
● Use loan patterns