Overview of why (and how) we have adopted a functional reactive approach to solve problems of scale @ AOL, in particular within AOL Advertising's forecasting platform.

1. ReactiveProgramming @ Scale

2. Paul Downey
● Director of UI & APIs for demand advertising
● Based in Dublin, Ireland
● Email: paul.downey@teamaol.com
John McClean
● Systems Architect Demand Side Forecasting
● Based in Dublin, Ireland.
● Twitter: cyclops_aol
● Medium: https://medium.com/@johnmcclean
● Email: john.mcclean@teamaol.com
About Speakers

3. A
A
A
P
P
P
Exchange

4. RTB Ad Cycle
100 ms
20,000,000,000+

5. ● Responsive
● Scale
● Resilient
● Capacity to process lots of data
System Requirements

7. Moore’s Law
7

8. Race Conditions
Deadlocks
Contention
Non-Deterministic State
Thread Starvation
Livelocks

9. Amadahl’s Law
9

10. Concurrency in
Practice.

11. Java Concurrency In Theory
11

12. Java Concurrency In Practice
12

13. Why is it like this?
13

14. Why is it like this?
14

15. Can we fix this?
15

16. What is Reactive?
16

17. What is Reactive?
17
Functional Reactive Programming?
Actor based concurrency?

18. Functional Reactive
Programming.

19. Is it something new?
19
1997

20. Big in 1997?
20

21. Is it something new?
21

22. Well established on the front end
22

23. What is it?

24. Functional programming?
24

25. Functional programming
25
Less buggy code
Easier to parallelise
Easier to optimise

26. Reactive programming
26
Reactive pipeline manages flow
What not How

27. Reactive programming
27
What to do
Change each element by multiplying by 100
Remove all elements over 550
Print out each remaining element

28. Reactive programming
28
Composition
Change each element by multiplying by 100
Remove all elements over 550
Print out each remaining element
map(e -> e*100)
forEach(System.out::println)
filter(e->e<551)

29. Reactive programming
29
.map(e -> e*100)
.forEach(System.out::println)
.filter(e->e<551)
Example
ReactiveSeq.of(6,5,1,2)

30. A Reactive Pipeline
30
first stage second stage terminal stage

31. A Reactive Pipeline
31
first stage
(map)
second stage
(filter)
terminal stage
(forEach)

32. A Reactive Pipeline
32
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
Data is pushed through the pipeline
500
2
1
5 600

33. Forecasting

34. Forecasting
34
Campaign Planning and Optimization
Query data across petabytes data
Accuracy & Responsiveness

35. Price / Volume Curve

36. Every Day We Process
36
Bid Request
Records
Impression Records Viewability Records
20B 2B 2B

37. Indexing Data
37

38. Reactive
Indexing

39. What we need to do

40. A Reactive Pipeline
40
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
Data is pushed through the pipeline
500
2
1
5 600

41. Scaling Up
41
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
core 1
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
core 2
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
core 3
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
core 4

42. Scaling Up for I/O
42
first stage
(map)
second stage
(filter)
terminal stage
(forEach)
future 1
future 2
future 3
future 4
future 5
future 6
future 7
future 8
future 9
future
10 future
11

43. Performance Differences
43

44. What would the
code look like?

45. Sequential Stream
45
.map(e -> e*100)
.forEach(System.out::println)
.filter(e->e<551)
Example
ReactiveSeq.of(6,5,1,2)
.futureOperations(executor)

46. Async I/O
46
.map(e -> e*100)
.filter(e->e<551)
Example
FutureW.of(()->loadData(6))
//non-blocking, FutureW executes asynchronously
.forEach(System.out::println)

47. Parallel Async Stream
47
.map(this::loadData)
.forEach(System.out::println)
.filter(e->e<551)
Example
new LazyReact(100).of(6,5,1,2)

48. What we need to do

49. Reactive Indexing Architecture
49

50. What have we
learnt?

51. Conclusions
51

52. Conclusions
52

53. Conclusions
53

54. Thank You.