Introduction to RFX for Backend Developer Reactive Function (X) - the Open Source Framework for solving Fast Data Problem

1. Introduction to RFX for
Backend Developer
Reactive Function (X)
the Open Source Framework for solving Fast Data Problem
and reacting to the World with Deep Learning
By Triều Nguyễn, the creator of RFX
http://mc2ads.com (Reactive Big Data Lab)
λ(x)

2. 2008: Java Developer, develop Social Trading Network for a
startup (Yopco)
2011: joined FPT Online, software engineer, worked in banbe.
net social network and VnExpress Mobile Restful API
2012: backend engineer at Greengar Studios
12/2012 to now - back to FPT Online, lead engineer,
developed new version Data Analytics Platform (ad-network
eclick.vn and VnExpress News)
Introduction about myself

3. 1. What is Rfx ?
2. Inception and Ideas
3. How Rfx was born
4. Why is Rfx ?
a. from big picture view (business)
b. from business view
c. from specific problems
5. Concepts and architecture: The BIG picture
6. Coding and tutorials from practical problems
7. Resources for self-studying
Contents of this talk

4. ● A framework for reactive real-time big fast data
● A collection of Open Source Tools
● The mission of RFX
→ “BUILD digital data-driven brain for every company in the
World”
What is RFX or Reactive Function X ?

5. INCEPTION and Ideas
Ideas when I was student, internship at DRD,
non-profit Organization
More info at http://activefunctor.blogspot.com

6. Challenges for today computing
Ask bigger questions

7. http://singularityhub.com/2014/04/20/new-imaging-method-shows-young-neurons-making-connections-exchanging-information/

8. Complex brain network topology

9. Digital Brain Topology

10. Think about
this module

11. Why Rfx ?
● Ideas since 2007 (from Haskell and Actor model theory)
● R&D and Deployed in Production since 2013
● Open Source: Apache License, Version 2.0
● Full Stack: Front-end and Back-end
● Apply Agile for Analytics and Data Science
● Apply Reactive Lambda Architecture
● Really fast and near-real-time processing
● Tested with 1.000.000 logs / second (1 million in 1 second)
● Simple development model for big data developer

12. What problems can be solved with Rfx

13. Domain (in business) where Rfx can be used
● real-time data analytics for digital marketing, advertising
● hospital systems
● personal banking system
● financial institution to detect frauds
● manufacturing plant
● airline systems
● online trading system
● emergency control system
● manufacturing plant management system
● road tolling system detects
● social networking site

14. Paper: http://vialab.science.uoit.ca/textvis2011/papers/textvis%202011-rohrdantz.pdf

15. Problem: How to monitor Mobile Web Performance and react
to slow response time
http://sixrevisions.com/mobile/pay-attention-to-mobile-web-performance

16. Luggage management system, events are produced by the check-in process
and by the various radio-frequency identification (RFID) readers, which emit
events about the movement of the luggage in the system. The events
generated by the event processing system are consumed by the luggage
control system itself, by airport staff, or even by the passengers themselves.
Problem: Monitoring sensor data and real-time security checking

17. Actor
User, Mobile,
Browser, ...
Reactive Lambda Architecture
System Rfx-
Topology
data + context + metadata
useful (data + relationship)
Database
NoSQL
1. Actor → System
2. System → Database
3. Database → System
4. System → Actor

18. Concepts
● Each user, who uses the services and creates data, is the
actor in system
● Actor is the source of all events (aka: logs), (click, reading
news, sending message to friends, playing games, ...)
● Functor (aka: neuron) is a computing object, used for
storing, processing data and emitting results to subscribed
functors
● Topology is the directed graph, define how functors that
are connected with stream data and process data

19. Philosophy and the Architecture

21. There are 3 demos, from simple to advanced user story
User story 1: Counting Real-time URL Pageview
User story 2: Monitoring Social Media Statistics
User story 3: Social Ranking for Recommendation Engine

22. User Story
Domain problem: Reactive Real-time Marketing
User story’s details:
1. User does read news from a website
→ tracking user activities (pageview, time on site)
2. User does login with Facebook Account
3. User clicks on like Facebook button
→ tracking what user liked, commented
4. The marketer/data analyst should see the trending most
read article in real-time
● → Personalized articles for the reader
● → Native advertising in real-time

24. Demo user story 1: Counting Real-time URL Pageview
Input:
1. The pageview logs from HTTP
Output:
1. The total number of page-view
2. The total number of page-view per hour
3. The total number of page-view per minute
4. The total number of page-view per second
5. The total number of page-view for URL

26. Demo user story 2: Monitoring Social Media Statistics
Input:
1. The pageview logs from HTTP
Output:
The social media statistics from:
1. Facebook: Like, Share, Comment
2. Twitter: Tweet Count
3. LinkedIn: Share Count
4. Geolocation heat-map report

28. Demo user story 3:
Social Ranking for Recommendation Engine
Input:
● Data: the URL of article
● Context: where (User's Location), when (time visit), from
where (referer url)
● Metadata: keywords, category of article
Output:
Real-time Statistics about pageview, social media statistics
(Share, Like, Comment), recommended articles
The list of articles are ranked by:
● most liked and same category
● most viewed and same category
● most liked, same category and near user's location

30. Reference Resources
Main website for Rfx: http://www.mc2ads.com
Ideas:
● http://journal.frontiersin.org/Journal/10.3389/fninf.2010.00112/full
● http://singularityhub.com/2014/04/20/new-imaging-method-shows-young-neurons-making-
connections-exchanging-information
● http://en.wikipedia.org/wiki/Actor_model_theory
● http://java.dzone.com/articles/introduction-event-processing
● http://www.technologyreview.com/featuredstory/526501/brain-mapping
● http://www.technologyreview.com/featuredstory/513696/deep-learning
Apache Storm: http://storm.incubator.apache.org
Apache Kafka: http://kafka.apache.org
In-memory NoSQL: http://redis.io
Deep Learning for Java: http://deeplearning4j.org
Distributed processing with Actor Model: http://akka.io
Papers:
● Real-Time Visualization of Streaming Text Data
● Hypernetworks for the Science of. Complex Systems
Main Blogs: http://www.mc2ads.org

31. The end and thank you
https://github.com/mc2ads/rfx
http://www.mc2ads.org
λ(x)