08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Assaf Araki – Real Time Analytics at Scale
1. REAL time
Analytics AT
SCALE
SMART DATA PIPES For THE
INTERNET OF THINGS
Assaf Araki, Big Data Analytics Architect
Big Data Analytics, Intel
2. Intro to Big Data
Analytics @ Intel People (+100)
Data
Scientists
Management
Big Data
Developers
Analytics
PMs
13%
41%
9%
37%
CONTRIBUTION TO Data Center Group
CONTRIBUTION TO INTEL
Operations
MISSIO
N
#1 Operational excellence
#2 Help Intel win area of
Intelligent machines
VISION
Analytics is a
competitive
advantage for Intel
Industry / Academy
Technical due-diligence
assessment for Intel Capital
Benchmark with startups
Academy Collaborations
Assist Intel Sales & Marketing
DESIGN
Cut validations time-to-market
MANUFACTURI
NGReduce test cost
SALES &
MARKETINGIncrease sales through analytics
Stream
Analytics
Cloud
Parkinson
Research
Machine
Learning
Strategy
4. Use case : The Parkinson Disease
research
44
CLINICAL TRIALS
Create and Validate
Algorithms & Measures
POPULATION STUDY
Generate insights
Using Big data analytics
8. Smart Ingestion
characteristics
Personalized
Easy to use
Smart Data
Pipe
• Per single device or user
• Maintain state and required data for ML
• Easily subscribe to any Stream
• Use familiar development Languages (Java, Scala)
• Developers focus on logic development
• Apply analytics on the Stream
• Trigger actions (close the feedback loop) in timely manner
Scalability
• Linear scalability (scale Out)
• Extremely High concurrencies
• High Throughput
Fault
Tolerance• No Single point of failure
• Seamless recovery
• Persistent
9. Smart Data Ingestion – High level
overview
9
Device
Device
Device
Device
Scalable, Persistent Broker Processing, Stream
Analytics
10. What is Akka?
• Micro-service(Actor) oriented.
• Message Driven
• Lock-free
• Location-transparent
• High performance
• Fault Tolerant
• Scales linearly
11. Stream Processing - the Akka
way…
11
Each actor is a small peace of Java or Scala
code performing its role
A set of actors creates a topology which is
responsible for device’s data stream
processing
A single Akka node may have millions of
concurrent actors handling different streams
and operations
Change
detection
Automatic
change
detection
time rules
matcher
Detect & raise
alert for
matched rules
Sleep
quality
calculating
users’ sleep
quality
Tremor
detection
Tremor
detection based
on devices’
Aggregator
Aggregation
(50hz to
minutes / hours)
Sample Parkinson Disease re
Subscriber Parser Aggregator
HBase
Writer
Analytics
Manager
Change
Detection
UnZip
Real Time
Rules
Sleep
Quality
13. • Core OS & Docker containers enable portability and ease of deployment anywhere
• Enables the flexibility of choosing a set of desired containers based on a given use case
requirements
Easy Portability With Docker &
Core OS
Preconfigured containers ready to be loaded
14. • IoT data Ingestion goes beyond moving the data into the cloud
• We have deployed a scalable and fault tolerance, multi-protocol pipeline that
enables stream Analytics
• Stream Analytics platform is leveraged for Other IoT projects
Summary
The Internet of Things (IoT) is creating unprecedented business opportunities for both individuals and organizations.
The story
The name of the man in the picture on the left is Andy Grove and he is one of Intel’s founders and has Parkinson (PD)
The story begins when he reads and article in the NY times about Big Data and decides to start a project within Intel related to PD and Big Data
He contacts Michael J fox foundation and then decides to start a joint effort together
The idea is to elaborate Internet of things, wearable's technology and big data platforms to assist PD research
PD
Neurodegenerative disease, movement disorder symptoms
Existing treatment are mainly for quality of life improvements and not for curing
~6M patients, ~1M in the US and ~5M in the rest of the globe
Life expectancy: ~10-15 years
1 out 100 over the age of 60 is a PD patient
No Test and no Progression markers
On this slide the focus should be on the patient reported capabilities and the configurable data collection strategies.
For the patient reported explain the Medication reminder and reporting capabilities which helps us track patients compliance, learn abour medication effect on the motor symptoms and this while providing value to the patients
The Objective measures part is covered later on in the PPT.
In the Other section talk about the ability to configure which sensorial data to use for each cohort of users
Quick review of PD solution layers as a use case of IoT platform
Batch Layer based on Spark
Storage layer using Hadoop, HBase & MySQL for Metadata
Powerful, scalable ingestion layer based on Akka & Kafka
A dynamic stream analytics layer based on Akka actor system framework
Scalable Service layer providing set of APIs for registration & data extraction out of the platform
UI layer – the only layer in this diagram which is unique to PD solution – using Pebble watch and Android application to collect data and interact with patients
You can note that 5 out of the presented 6 layers (excluding the UI layer) are part of the IoT platform and can be used for similar products / verticals
Multi-protocol pipeline built over AKKA & KAFKA
KAKFA is a fast, scalable, durable & distributed messaging system - high-throughput, low-latency platform for handling real-time data feeds.
AKKA is an Actor based framework allowing high concurrency, distributedand resilient based on events / messaging
This layer is responsible for:
Pulling messages
Parse & Process
Concurrent & controlled write
Writing correct concurrent, fault-tolerant and scalable applications is hard.
Akka uses the Actor Model to raise the abstraction level and provide a better platform to build correct concurrent and scalable applications.
Can support millions of concurrent actors handling different streams which is a good fit to IoT characteristics.
We use Akka for:
Processing messages
Near Real-time rules
Change detection at the device level
Docker is an open-source project that automates the deployment of applications inside software containers
CoreOS is an open source lightweight operating system based on the Linux kernel and designed for providing infrastructure to clustered deployments
Change Detection – Single (Kolmogorov-Smirnov) & Multi sensor ( Under patent )
Anomaly Detection
Periodicity
Stream classification