Introduction to the series More than ever before, organizations ask us questions like What does Reactive really mean and why should we care? What does “Going Reactive” mean for our Development and Operations teams? What are the business drivers and market forces that we should be aware of, and how can my organization start the journey towards modernizing our existing applications and infrastructure? From Jonas Bonér, Viktor Klang and Konrad Malawski, co-author of the Reactive Manifesto and contributors to the Reactive Streams initiative, we have set out to produce a three-part “Reactive Revealed” series to give an interactive opportunity for developers, architects and executives alike to get started with understanding Reactive from the beginning, providing an overview of the technological foundation behind it all. Reactive Revealed 1/3: Async NIO, Back-pressure and Message-driven vs Event-driven In this first of three webinars with live Q/A, we explore the underlying technologies and approach of Reactive systems. Akka Team member and Reactive Streams contributor Konrad Malawski will detail what you should know about: The differences and trade-offs between synchronous blocking I/O and asynchronous non-blocking I/O, what these mean for execution efficiency as well as code maintainability. The difference between event-driven and message-driven approaches in terms of design considerations in the small and the large as well as the costs and benefits of either way. The role of flow control in Reactive systems, how it impacts service latency and throughput and which technologies are available to help with implementing it. In the next webinar with Viktor Klang, we review the role of elasticity, scalability and location transparency. In the third and final webinar in the series, with Jonas Bonér, we continue with learning about resiliency, errors vs failures, isolation (and containment), delegation, and replication in Reactive systems.

1. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Part #1
Message-driven
Asynchronous
Non-blocking I/O
and Back-pressure

2. Part #2
Elasticity
Scalability
Location Transparency
Part #3
Resiliency
Failures vs Errors
Isolation & Containment
Delegation & Replication
with Viktor Klang with Jonas Bonér
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

3. Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK
(we’re renaming soon!)
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

4. High Performance Software
Development
For the majority of the time,
high performance software development
is not about compiler hacks and bit twiddling.
It is about fundamental design principles that are
key to doing any effective software development.
Martin Thompson
practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

5. Agenda
• Reactive Manifesto & our focus today
• Asynchronous vs. Synchronous
• Messages compared to Events
• Patterns enabled by Asynchronous
Messaging
• Keeping things Non-blocking: Scheduling
• Non-blocking I/O: IO, AIO, NIO, Zero
• Asynchronous Non-blocking Back-pressure
• Wrapping up and Q/A
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

6. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

7. Initial version published 2 years ago.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

8. Initial version published 2 years ago.
A “not only Typesafe” initiative:
Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

9. Initial version published 2 years ago.
A “not only Typesafe” initiative:
Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson
Defines a tech agnostic shared vocabulary.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

10. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

11. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

12. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Asynchronous

13. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

14. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

15. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

16. Sync / Async Basics
Implicitly back-pressures the sender,
since it can only send M2 after M1 finished.
(we’ll discuss back-pressure in depth shortly)
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

17. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

18. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

19. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

20. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

21. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

22. Sync / Async Basics
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

23. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Messages compared to Events

24. Messages compared to Events
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Events are not addressed to a specific receiver.
They’re just “facts”, that get published to topics.

25. Messages compared to Events
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Event-based systems tend to
focus on addressable event sources.

26. Messages compared to Events
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
A Message can contain an Event as it’s
payload,
i.e. “listener ! SeatReserved()”.
Messages always have a specific
recipient.
Be it Actor or Topic or something else.

27. Messages compared to Events
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Message-based systems tend to
focus on addressable recipients.

28. Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski
Load management

29. Async enables transparent load
balancing
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

30. Async enables transparent load
balancing
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

31. Async enables transparent load
balancing
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

32. Async enables transparent load
balancing
Response latency grows since more and
more messages pile up in the mailbox…
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

33. Async enables transparent load
balancing
Response latency grows since more and
more messages pile up in the mailbox…
We can:
A. back-pressure the Publisher
B. add more workers!
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

34. Async enables transparent load
balancing
Can add workers
dynamically.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

35. Non-blocking
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

36. Async where it matters: Scheduling

37. Async where it matters: Scheduling

38. Async where it matters: Scheduling

39. Async where it matters: Scheduling

40. Async where it matters: Scheduling

41. Async where it matters: Scheduling

42. Async where it matters: Scheduling

43. Async where it matters: Scheduling

44. Async where it matters: Scheduling

45. Scheduling (notice they grey sync call)

46. Scheduling (notice they grey sync call)

47. Scheduling (now with Async db call)

48. Scheduling (now with Async db call)

49. Scheduling (now with Async db call)

50. Scheduling (now with Async db call)

51. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

52. IO / AIO
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

53. IO / AIO / NIO
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

54. IO / AIO / NIO / Zero
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

55. Interruption!
CPU: User Mode / Kernel Mode
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

56. Kernels and CPUs
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

57. Kernels and CPUs
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

58. Kernels and CPUs
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

59. Kernels and CPUs
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

60. Kernels and CPUs
http://wiki.osdev.org/Context_SwitchingReactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

61. Kernels and CPUs
[…] switching from user-level to kernel-level
on a (2.8 GHz) P4 is 1348 cycles.
[…] Counting actual time, the P4 takes 481ns […]
http://wiki.osdev.org/Context_SwitchingReactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

62. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

63. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

64. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

65. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

66. I / O
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

67. I / O
“Don’t worry.
It only gets worse!”
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

68. I / O
“Don’t worry.
It only gets worse!”
Same data in 3 buffers!4 mode switches!
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

69. Asynchronous I / O [Linux]
Linux AIO = JVM NIO
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

70. Asynchronous I / O [Linux]
NewIO… since 2004!
(No-one calls it “new” any more)
Linux AIO = JVM NIO
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

71. Asynchronous I / O [Linux]
Less time wasted waiting.
Same amount of buffer copies.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

72. ZeroCopy = sendfile [Linux]
“Work smarter.
Not harder.”
http://fourhourworkweek.com/Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

73. ZeroCopy = sendfile [Linux]
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

74. ZeroCopy = sendfile [Linux]
Data never leaves kernel mode!
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

75. Delegating failures
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

76. Delegate Failures
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

77. Delegate Failures
Backup Requests
using
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

78. Backup requests
A technique for fighting “long tail latencies”.
By issuing duplicated work, when SLA seems in danger.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

79. Backup requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

80. Backup requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

81. Backup requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

82. Backup requests - send
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

83. Backup requests - send
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

84. Backup requests - send
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

85. Backup requests
Avg Std dev 95%ile 99%ile 99.9%ile
No backups 33 ms 1524 ms 24 ms 52 ms 994 ms
After 10ms 14 ms 4 ms 20 ms 23 ms 50 ms
After 50ms 16 ms 12 ms 57 ms 63 ms 68 ms
Jeff Dean - Achieving Rapid Response Times in Large Online Services
Peter Bailis - Doing Redundant Work to Speed Up Distributed Queries
Akka - Krzysztof Janosz @ Akkathon, Kraków - TailChoppingRouter (docs, pr)
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

86. Lesson learned:
Backup requests allow
trade-off increased load
for decreased latency.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

87. Asynchronous Back-pressure
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

88. Back-pressure
&
Combined Requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

89. Combined requests & back-pressure
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

90. Combined requests & back-pressure
No no no…!
Not THAT Back-pressure!
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

91. Combined requests & back-pressure
THAT kind of back-pressure:
www.reactive-streams.org Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

92. Combined requests & back-pressure
THAT kind of back-pressure:
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawskiwww.reactive-streams.org

93. Combined requests & back-pressure
THAT kind of back-pressure:
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawskiwww.reactive-streams.org

94. Combined requests & back-pressure
THAT kind of back-pressure:
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawskiwww.reactive-streams.org

95. Combined requests
A technique for avoiding duplicated work.
By aggregating requests, possibly increasing latency.
“Wat? Why would I increase latency!?”
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

96. Combined requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

97. Combined requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

98. Combined requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

99. Combined requests
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

100. Lesson learned:
Back-pressure saves systems from overload.
Combined requests trade higher latency,
for less work for the downstream.
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

101. Wrapping up
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

102. Wrapping up
• provides a shared vocabulary
• a “ubiquitous language“, for Reactive Systems
• is not re-discovering the wheel, instead just solidifies proven patterns
• is technology independent
The Reactive Manifesto
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

103. Wrapping up
• are loosely coupled by design
• therefore can be elastic and resilient
• allow certain patterns to be used which otherwise wouldn’t be possible
Asynchronous and Message-driven systems:
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

104. Links
• akka.io
• reactive-streams.org
• akka-user
• Gil Tene - How NOT to measure latency, 2013
• Jeff Dean @ Velocity 2014
• Alan Bateman, Jeanfrancois Arcand (Sun) Async IO Tips @
JavaOne
• http://linux.die.net/man/2/select
• http://linux.die.net/man/2/poll
• http://linux.die.net/man/4/epoll
• giltene/jHiccup
• Linux Journal: ZeroCopy I, Dragan Stancevis 2013
• Last slide car picture: http://actu-moteurs.com/sprint/gt-tour/jean-
philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

105. Links
• http://wiki.osdev.org/Context_Switching
• CppCon: Herb Sutter "Lock-Free Programming (or, Juggling Razor Blades)"
• http://www.infoq.com/presentations/reactive-services-scale
• Gil Tene’s HdrHistogram.org
• http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
• Rob Pike - Concurrency is NOT Parallelism (video)
• Brendan Gregg - Systems Performance: Enterprise and the Cloud (book)
• http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-
capture.html
• Jeff Dean, Luiz Andre Barroso - The Tail at Scale (whitepaper, ACM)
• http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-
when-more-machines-equal.html
• http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-
system-architects/
• Marcus Lagergren - Oracle JRockit: The Definitive Guide (book)
• http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free-
concurrent.html
• Handling of Asynchronous Events - http://www.win.tue.nl/~aeb/linux/lk/lk-
12.html
• http://www.kegel.com/c10k.html
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

106. Links
• www.reactivemanifesto.org/
• Seriously the only right way to micro benchmark on the JVM:
• JMH openjdk.java.net/projects/code-tools/jmh/
• JMH for Scala: https://github.com/ktoso/sbt-jmh
• http://www.ibm.com/developerworks/library/l-async/
• http://lse.sourceforge.net/io/aio.html
• https://code.google.com/p/kernel/wiki/AIOUserGuide
• ShmooCon: C10M - Defending the Internet At Scale (Robert Graham)
• http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html#.VO6E11PF8SM
•User-level threads....... with threads. - Paul Turner @ Linux Plumbers Conf
2013
•https://www.facebook.com/themainstreetpiggyboys/photos/a.1390784047896
753.1073741829.1390594984582326/1423592294615928/?type=1&theater
for the Rollin’ Cuy on last slide
Reactive Revealed: Message Driven
with Konrad `@ktosopl` Malawski

107. Go Reactive
More interesting stuff…
HOW TO VIDEO
Managing Library
Dependencies with
Play, sbt & Activator
WATCH NOW
HAVE QUESTIONS?
Get in touch with
Typesafe today!
CONTACT US
SUCCESS STORY
How Walmart Canada
Saved Black Friday
with Akka, Play and
Scala
READ NOW

108. EXPERT TRAINING
Delivered on-site for Akka, Spark, Scala and Play
Help is just a click away. Get in touch
with Typesafe about our training courses.
• Intro Workshop to Apache Spark
• Fast Track & Advanced Scala
• Fast Track to Akka with Java or
Scala
• Fast Track to Play with Java or Scala
• Advanced Akka with Java or Scala
Ask us about local trainings available by
24 Typesafe partners in 14 countries
around the world.
CONTACT US Learn more about on-site training

109. ©Typesafe 2015 – All Rights Reserved©Typesafe 2015 – All Rights Reserved