SlideShare a Scribd company logo
1 of 16
Download to read offline
Deep recurrent neural network
for sequence learning in Spark
Yves MABIALA
THALES
Outline
• Thales & Big Data
• On the difficulty of Sequence Learning
• Deep Learning for Sequence Learning
• Spark implementation of Deep Learning
• Use cases
– Predictive maintenance
– NLP
Thales & Big Data
Thales systems produce a huge quantity of data
Transportation systems (ticketing, supervision, …)
Security (radar traces, network logs, …)
Satellite (photos, videos, …)
which is often
Massive
Heterogeneous
Extremely dynamic
and where understanding the dynamics of the monitored phenomena
is mandatory Sequence Learning
What is sequence learning ?
Sequence learning refers to a set of ML tasks where a model has
to either deal with sequences as input, produce sequences as
output or both
Goal : Understand the dynamic of a sequence to
– Classify
– Predict
– Model
Typical applications
– Text
• Classify texts (sentiment analysis)
• Generate textual description of images (image captioning)
– Video
• Video classification
– Speech
• Speech to text
How is it typically handled ?
Taking into account the dynamic is difficult
– Often people do not bother
• E.g. text analysis using bag of word (one hot encoding)
– Problem for certain tasks such as sentiment classification (order of the words is important)
– Or use popular statistical approaches
• (Hidden) Markov model for prediction (and classification)
– Shortterm dependency (order 1) : 𝑃(𝑋$ = 𝑥 (𝑋$'( = 	𝑥$'(,… , 𝑋$', = 𝑥$',)⁄ ) = 𝑃(𝑋$ = 𝑥$ 𝑋$'( =	 𝑥$'()⁄
• Autoregressive approaches for time series forecasting
The chair is red 1 0 1 1 0 0 0 0
The cat is on a chair
The cat is young 1 1 0 0 1 1 0 0
1 1 1 0 0 1 1 1
The is chair red young cat on a
Link with artificial neural network ?
Artificial neural network is a set of statistical models inspired from the brain
– Transforms the input by applying at each layer (non linear) functions
– More layers equals more capabilities (≥ 2	hidden layers : Deep Learning)
• From manual features building to feature learning
Set of transformation and activation operations
– Affine : 𝒀 = 𝑾 𝒕
𝑿 + 𝒃, sigmoid activation :
𝟏
𝟏8𝐞𝐱𝐩	('𝑿)
, tanh activation : 𝒀 = 𝐭𝐚𝐧𝐡	( 𝑿)
• Only affine + activation layers = multi layer perceptron (available in Spark ML since 1.5.0)
– Convolutional : Apply a spatial convolution on the 1D/2D input (signal, image, …) : 𝐘 = 	𝒄𝒐𝒏𝒗 𝑿, 𝑾 + 𝒃
• Learns spatial features used for classification (images) , prediction
– Recurrent : Introduces a recurrent part to learn dependencies between observations (features related to
the dynamic)
Objective
– Find the best weights W to minimize the difference between the predicted output and the desired one
(using back-propagation algorithm)
input
hidden
layers
output
Able to cope with varying size sequences either at the input or at the output
Recurrent Neural Network basics
One to many
(fixedsize input,
sequence output)
e.g. Image captioning
Many to many
(sequence input to sequence
output)
e.g. Speech to text
Many to one
(sequence input to fixedsize
output)
e.g. Text classification
Artificial neural networks with one or more recurrent layers
Classical neural network Recurrent neural network
𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌
𝒀 𝒌
𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌
𝒀 𝒌 = 𝒇(𝑾 𝒕 𝑿 𝒌 + 𝑯𝒀 𝒌'𝟏)
𝑿 𝒌𝑿
𝒀 𝒌 = 𝒇(𝑾 𝒕 𝑿 𝒌)
𝒀
Unrolled through time
𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌
𝑿
𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌
𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌
𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌
𝒀
On the difficulty of training recurrent networks
RNNs are (were) known to be difficult to learn
– More weights and more computational steps
• More computationally expensive (accelerator needed for matrix ops : Blas or GPU)
• More data needed to converge (scalability over Big Data architectures : Spark)
– Theano, Tensor Flow, Caffe do not have distributed versions
– Unable to learn long range dependencies (Graves & Al 2014)
• At a given time t, RNN does not remember the observations before 𝑋J',
⇒ New RNN architectures with memory preservation (more context)
𝑍$ = 𝑓 𝑊N
O
𝑋$ + 𝐻N 𝑌$'(
𝑅$ = 𝑓(𝑊S
O
𝑋$ + 𝐻S 𝑌$'()
𝐻T$ = tanh(𝑊YJZ[
O
𝑋$ + 𝑈 𝑌$'(	o	𝑅$ ) 	
𝑌$	 = 1 − 𝑍$ 𝑌$'( + 𝑍$ 𝐻T$
LSTM GRU
Recurrent neural networks in Spark
Spark implementation of DL algorithms (data parallel)
– All the needed blocks
• Affine, convolutional, recurrent layers (Simple and GRU)
• Sigmoid, tanh, reLU activations
• SGD, rmsprop, adadelta optimizers
– CPU (and GPU backend)
– Fully compatible with existing DL library in Spark ML
Performance
– On 6 nodes cluster (CPU)
• 5.46 average speedup (some communication overhead)
– About the same speedup as MLP in Spark ML
Driver
Worker 1
Worker 2
Worker 3
Resulting gradients (2)
Model broadcast (1)
Use case 1 : predictive maintenance (1)
Context
– Thales and its clients build systems in different domains
• Transportation (ticketing, controlling)
• Defense (radar)
• Satellites
– Need better and more accurate maintenance services
• From planned maintenance (every x days) to an alert maintenance
• From expert detection to automatic failure prediction
• From whole subsystem changes to more localized reparations
Goal
– Detect early signs of a (sub)system failure using data coming
from sensors monitoring the health of a system (HUMS)
Use case 1 : predictive maintenance (2)
Example on a real system
– 20 sensors (20 values every 5 minutes), label (failure or not)
– Take 3 hours of data and predict the probability of failure in the next hour (fully
customizable)
Learning using MLLIB
Use case 1 : predictive maintenance (3)
Recurrent net learning
Impact of recurrent nets
– Logistic regression
• 70% detection with 70% accuracy
– Recurrent Neural Network
• 85% detection with 75% accuracy
Use case 2 : Sentiment analysis (1)
Context
– Social network analysis application developed at Thales (Twitter, Facebook,
blogs, forums)
• Analyze both the content of the texts and the relations (texts, actors)
– Multiple (big data) analysis
• Actor community detection
• Text clustering (themes)
• …
Focus on
– Sentiment analysis on the collected texts
• Classify texts based on their sentiment
Use case 2 : Sentiment analysis (2)
Learning dataset
– Sentiment140 + Kaggle challenge (1.5M labeled tweets)
– 50% positives, 50% negatives
Compare Bag of words + classifier approaches (Naïve Bayes, SVM, logistic
regression) versus RNN
Use case 2 : Sentiment analysis (3)
NB SVM
Log
Reg
NeuralNet
(perceptron)
RNN (GRU)
100 61.4 58.4 58.4 55.6 NA
1 000 70.6 70.6 70.6 70.8 68.1
10 000 75.4 75.1 75.4 76.1 72.3
100 000 78.1 76.6 76.9 78.5 79.2
700 000 80 78.3 78.3 80 84.1
Results
40
45
50
55
60
65
70
75
80
85
90 NB
SVM
LogReg
NeuralNet
RNN
(GRU)
The end…
THANK YOU !

More Related Content

What's hot

KubeVirt (Kubernetes and Cloud Native Toronto)
KubeVirt (Kubernetes and Cloud Native Toronto)KubeVirt (Kubernetes and Cloud Native Toronto)
KubeVirt (Kubernetes and Cloud Native Toronto)Stephen Gordon
 
Seminar Report on Honeypot
Seminar Report on HoneypotSeminar Report on Honeypot
Seminar Report on HoneypotAmit Poonia
 
Traditional Firewall vs. Next Generation Firewall
Traditional Firewall vs. Next Generation FirewallTraditional Firewall vs. Next Generation Firewall
Traditional Firewall vs. Next Generation Firewall美兰 曾
 
OpenShift Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud
OpenShift  Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud OpenShift  Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud
OpenShift Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud Hidetsugu Sugiyama
 
Modern Network Security Issue and Challenge
Modern Network Security Issue and ChallengeModern Network Security Issue and Challenge
Modern Network Security Issue and ChallengeIkhtiar Khan Sohan
 
[cb22] Tales of 5G hacking by Karsten Nohl
[cb22] Tales of 5G hacking by Karsten Nohl[cb22] Tales of 5G hacking by Karsten Nohl
[cb22] Tales of 5G hacking by Karsten NohlCODE BLUE
 
Next generation firewall(ngfw)feature and benefits
Next generation firewall(ngfw)feature and benefitsNext generation firewall(ngfw)feature and benefits
Next generation firewall(ngfw)feature and benefitsAnthony Daniel
 
Opinion – 5G Reality Check: Speeds
Opinion – 5G Reality Check: SpeedsOpinion – 5G Reality Check: Speeds
Opinion – 5G Reality Check: Speeds3G4G
 
Introduction to SDN
Introduction to SDNIntroduction to SDN
Introduction to SDNAPNIC
 
How to build high performance 5G networks with vRAN and O-RAN
How to build high performance 5G networks with vRAN and O-RANHow to build high performance 5G networks with vRAN and O-RAN
How to build high performance 5G networks with vRAN and O-RANQualcomm Research
 
Firewall Architecture
Firewall Architecture Firewall Architecture
Firewall Architecture Yovan Chandel
 
5G Security Briefing
5G Security Briefing5G Security Briefing
5G Security Briefing3G4G
 
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...Amazon Web Services
 

What's hot (20)

KubeVirt (Kubernetes and Cloud Native Toronto)
KubeVirt (Kubernetes and Cloud Native Toronto)KubeVirt (Kubernetes and Cloud Native Toronto)
KubeVirt (Kubernetes and Cloud Native Toronto)
 
Seminar Report on Honeypot
Seminar Report on HoneypotSeminar Report on Honeypot
Seminar Report on Honeypot
 
Traditional Firewall vs. Next Generation Firewall
Traditional Firewall vs. Next Generation FirewallTraditional Firewall vs. Next Generation Firewall
Traditional Firewall vs. Next Generation Firewall
 
OpenShift Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud
OpenShift  Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud OpenShift  Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud
OpenShift Kubernetes Native Infrastructure for 5GC and Telco Edge Cloud
 
Modern Network Security Issue and Challenge
Modern Network Security Issue and ChallengeModern Network Security Issue and Challenge
Modern Network Security Issue and Challenge
 
Honeypots
HoneypotsHoneypots
Honeypots
 
IronPort
IronPortIronPort
IronPort
 
[cb22] Tales of 5G hacking by Karsten Nohl
[cb22] Tales of 5G hacking by Karsten Nohl[cb22] Tales of 5G hacking by Karsten Nohl
[cb22] Tales of 5G hacking by Karsten Nohl
 
Next generation firewall(ngfw)feature and benefits
Next generation firewall(ngfw)feature and benefitsNext generation firewall(ngfw)feature and benefits
Next generation firewall(ngfw)feature and benefits
 
Opinion – 5G Reality Check: Speeds
Opinion – 5G Reality Check: SpeedsOpinion – 5G Reality Check: Speeds
Opinion – 5G Reality Check: Speeds
 
Pave the Golden Path On Your Internal Platform
Pave the Golden Path On Your Internal PlatformPave the Golden Path On Your Internal Platform
Pave the Golden Path On Your Internal Platform
 
NFV Tutorial
NFV TutorialNFV Tutorial
NFV Tutorial
 
NMAP
NMAPNMAP
NMAP
 
Introduction to SDN
Introduction to SDNIntroduction to SDN
Introduction to SDN
 
Intro to Kubernetes
Intro to KubernetesIntro to Kubernetes
Intro to Kubernetes
 
How to build high performance 5G networks with vRAN and O-RAN
How to build high performance 5G networks with vRAN and O-RANHow to build high performance 5G networks with vRAN and O-RAN
How to build high performance 5G networks with vRAN and O-RAN
 
Firewall Architecture
Firewall Architecture Firewall Architecture
Firewall Architecture
 
5G Security Briefing
5G Security Briefing5G Security Briefing
5G Security Briefing
 
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...
Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1) ...
 
Epc cups overview
Epc cups overviewEpc cups overview
Epc cups overview
 

Similar to Deep Recurrent Neural Networks for Sequence Learning in Spark by Yves Mabiala

Deep learning from a novice perspective
Deep learning from a novice perspectiveDeep learning from a novice perspective
Deep learning from a novice perspectiveAnirban Santara
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learningJunaid Bhat
 
Deep recurrent neutral networks for Sequence Learning in Spark
Deep recurrent neutral networks for Sequence Learning in SparkDeep recurrent neutral networks for Sequence Learning in Spark
Deep recurrent neutral networks for Sequence Learning in SparkDataWorks Summit/Hadoop Summit
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & OpportunityiTrain
 
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 Separating Hype from Reality in Deep Learning with Sameer Farooqui Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer FarooquiDatabricks
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning ApplicationsNVIDIA Taiwan
 
Using Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsUsing Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsGreg Makowski
 
Automatic Attendace using convolutional neural network Face Recognition
Automatic Attendace using convolutional neural network Face RecognitionAutomatic Attendace using convolutional neural network Face Recognition
Automatic Attendace using convolutional neural network Face Recognitionvatsal199567
 
Nural network ER. Abhishek k. upadhyay
Nural network ER. Abhishek  k. upadhyayNural network ER. Abhishek  k. upadhyay
Nural network ER. Abhishek k. upadhyayabhishek upadhyay
 
Deep learning: the future of recommendations
Deep learning: the future of recommendationsDeep learning: the future of recommendations
Deep learning: the future of recommendationsBalázs Hidasi
 
Big Sky Earth 2018 Introduction to machine learning
Big Sky Earth 2018 Introduction to machine learningBig Sky Earth 2018 Introduction to machine learning
Big Sky Earth 2018 Introduction to machine learningJulien TREGUER
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learningAmr Rashed
 
Deep Learning Sample Class (Jon Lederman)
Deep Learning Sample Class (Jon Lederman)Deep Learning Sample Class (Jon Lederman)
Deep Learning Sample Class (Jon Lederman)Jon Lederman
 
deepnet-lourentzou.ppt
deepnet-lourentzou.pptdeepnet-lourentzou.ppt
deepnet-lourentzou.pptyang947066
 
Recurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRURecurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRUananth
 
Temporal Hypermap Theory and Application
Temporal Hypermap Theory and ApplicationTemporal Hypermap Theory and Application
Temporal Hypermap Theory and ApplicationAbel Nyamapfene
 
Hardware Implementation of Cascade SVM
Hardware Implementation of Cascade SVMHardware Implementation of Cascade SVM
Hardware Implementation of Cascade SVMQian Wang
 
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Myungyon Kim
 

Similar to Deep Recurrent Neural Networks for Sequence Learning in Spark by Yves Mabiala (20)

Deep learning from a novice perspective
Deep learning from a novice perspectiveDeep learning from a novice perspective
Deep learning from a novice perspective
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Deep recurrent neutral networks for Sequence Learning in Spark
Deep recurrent neutral networks for Sequence Learning in SparkDeep recurrent neutral networks for Sequence Learning in Spark
Deep recurrent neutral networks for Sequence Learning in Spark
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
 
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 Separating Hype from Reality in Deep Learning with Sameer Farooqui Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer Farooqui
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
 
Using Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical ApplicationsUsing Deep Learning to do Real-Time Scoring in Practical Applications
Using Deep Learning to do Real-Time Scoring in Practical Applications
 
Automatic Attendace using convolutional neural network Face Recognition
Automatic Attendace using convolutional neural network Face RecognitionAutomatic Attendace using convolutional neural network Face Recognition
Automatic Attendace using convolutional neural network Face Recognition
 
Nural network ER. Abhishek k. upadhyay
Nural network ER. Abhishek  k. upadhyayNural network ER. Abhishek  k. upadhyay
Nural network ER. Abhishek k. upadhyay
 
Deep learning: the future of recommendations
Deep learning: the future of recommendationsDeep learning: the future of recommendations
Deep learning: the future of recommendations
 
Big Sky Earth 2018 Introduction to machine learning
Big Sky Earth 2018 Introduction to machine learningBig Sky Earth 2018 Introduction to machine learning
Big Sky Earth 2018 Introduction to machine learning
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Deep Learning Sample Class (Jon Lederman)
Deep Learning Sample Class (Jon Lederman)Deep Learning Sample Class (Jon Lederman)
Deep Learning Sample Class (Jon Lederman)
 
deepnet-lourentzou.ppt
deepnet-lourentzou.pptdeepnet-lourentzou.ppt
deepnet-lourentzou.ppt
 
Recurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRURecurrent Neural Networks, LSTM and GRU
Recurrent Neural Networks, LSTM and GRU
 
Temporal Hypermap Theory and Application
Temporal Hypermap Theory and ApplicationTemporal Hypermap Theory and Application
Temporal Hypermap Theory and Application
 
Deep learning (2)
Deep learning (2)Deep learning (2)
Deep learning (2)
 
Hardware Implementation of Cascade SVM
Hardware Implementation of Cascade SVMHardware Implementation of Cascade SVM
Hardware Implementation of Cascade SVM
 
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
 

More from Spark Summit

FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang Spark Summit
 
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...Spark Summit
 
Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang Wu
Apache Spark Structured Streaming Helps Smart Manufacturing with  Xiaochang WuApache Spark Structured Streaming Helps Smart Manufacturing with  Xiaochang Wu
Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang WuSpark Summit
 
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data  with Ramya RaghavendraImproving Traffic Prediction Using Weather Data  with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data with Ramya RaghavendraSpark Summit
 
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...Spark Summit
 
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...Spark Summit
 
Apache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingApache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingSpark Summit
 
Apache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingApache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingSpark Summit
 
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...Spark Summit
 
Next CERN Accelerator Logging Service with Jakub Wozniak
Next CERN Accelerator Logging Service with Jakub WozniakNext CERN Accelerator Logging Service with Jakub Wozniak
Next CERN Accelerator Logging Service with Jakub WozniakSpark Summit
 
Powering a Startup with Apache Spark with Kevin Kim
Powering a Startup with Apache Spark with Kevin KimPowering a Startup with Apache Spark with Kevin Kim
Powering a Startup with Apache Spark with Kevin KimSpark Summit
 
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya RaghavendraImproving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya RaghavendraSpark Summit
 
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...Spark Summit
 
How Nielsen Utilized Databricks for Large-Scale Research and Development with...
How Nielsen Utilized Databricks for Large-Scale Research and Development with...How Nielsen Utilized Databricks for Large-Scale Research and Development with...
How Nielsen Utilized Databricks for Large-Scale Research and Development with...Spark Summit
 
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...Spark Summit
 
Goal Based Data Production with Sim Simeonov
Goal Based Data Production with Sim SimeonovGoal Based Data Production with Sim Simeonov
Goal Based Data Production with Sim SimeonovSpark Summit
 
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...Spark Summit
 
Getting Ready to Use Redis with Apache Spark with Dvir Volk
Getting Ready to Use Redis with Apache Spark with Dvir VolkGetting Ready to Use Redis with Apache Spark with Dvir Volk
Getting Ready to Use Redis with Apache Spark with Dvir VolkSpark Summit
 
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...Deduplication and Author-Disambiguation of Streaming Records via Supervised M...
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...Spark Summit
 
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...Spark Summit
 

More from Spark Summit (20)

FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang
 
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...
VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...
 
Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang Wu
Apache Spark Structured Streaming Helps Smart Manufacturing with  Xiaochang WuApache Spark Structured Streaming Helps Smart Manufacturing with  Xiaochang Wu
Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang Wu
 
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data  with Ramya RaghavendraImproving Traffic Prediction Using Weather Data  with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
 
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...
A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...
 
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...
No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...
 
Apache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingApache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim Dowling
 
Apache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim DowlingApache Spark and Tensorflow as a Service with Jim Dowling
Apache Spark and Tensorflow as a Service with Jim Dowling
 
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...
MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...
 
Next CERN Accelerator Logging Service with Jakub Wozniak
Next CERN Accelerator Logging Service with Jakub WozniakNext CERN Accelerator Logging Service with Jakub Wozniak
Next CERN Accelerator Logging Service with Jakub Wozniak
 
Powering a Startup with Apache Spark with Kevin Kim
Powering a Startup with Apache Spark with Kevin KimPowering a Startup with Apache Spark with Kevin Kim
Powering a Startup with Apache Spark with Kevin Kim
 
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya RaghavendraImproving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
 
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...
Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...
 
How Nielsen Utilized Databricks for Large-Scale Research and Development with...
How Nielsen Utilized Databricks for Large-Scale Research and Development with...How Nielsen Utilized Databricks for Large-Scale Research and Development with...
How Nielsen Utilized Databricks for Large-Scale Research and Development with...
 
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...
Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...
 
Goal Based Data Production with Sim Simeonov
Goal Based Data Production with Sim SimeonovGoal Based Data Production with Sim Simeonov
Goal Based Data Production with Sim Simeonov
 
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...
Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...
 
Getting Ready to Use Redis with Apache Spark with Dvir Volk
Getting Ready to Use Redis with Apache Spark with Dvir VolkGetting Ready to Use Redis with Apache Spark with Dvir Volk
Getting Ready to Use Redis with Apache Spark with Dvir Volk
 
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...Deduplication and Author-Disambiguation of Streaming Records via Supervised M...
Deduplication and Author-Disambiguation of Streaming Records via Supervised M...
 
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...
 

Recently uploaded

Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 

Recently uploaded (17)

Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 

Deep Recurrent Neural Networks for Sequence Learning in Spark by Yves Mabiala

  • 1. Deep recurrent neural network for sequence learning in Spark Yves MABIALA THALES
  • 2. Outline • Thales & Big Data • On the difficulty of Sequence Learning • Deep Learning for Sequence Learning • Spark implementation of Deep Learning • Use cases – Predictive maintenance – NLP
  • 3. Thales & Big Data Thales systems produce a huge quantity of data Transportation systems (ticketing, supervision, …) Security (radar traces, network logs, …) Satellite (photos, videos, …) which is often Massive Heterogeneous Extremely dynamic and where understanding the dynamics of the monitored phenomena is mandatory Sequence Learning
  • 4. What is sequence learning ? Sequence learning refers to a set of ML tasks where a model has to either deal with sequences as input, produce sequences as output or both Goal : Understand the dynamic of a sequence to – Classify – Predict – Model Typical applications – Text • Classify texts (sentiment analysis) • Generate textual description of images (image captioning) – Video • Video classification – Speech • Speech to text
  • 5. How is it typically handled ? Taking into account the dynamic is difficult – Often people do not bother • E.g. text analysis using bag of word (one hot encoding) – Problem for certain tasks such as sentiment classification (order of the words is important) – Or use popular statistical approaches • (Hidden) Markov model for prediction (and classification) – Shortterm dependency (order 1) : 𝑃(𝑋$ = 𝑥 (𝑋$'( = 𝑥$'(,… , 𝑋$', = 𝑥$',)⁄ ) = 𝑃(𝑋$ = 𝑥$ 𝑋$'( = 𝑥$'()⁄ • Autoregressive approaches for time series forecasting The chair is red 1 0 1 1 0 0 0 0 The cat is on a chair The cat is young 1 1 0 0 1 1 0 0 1 1 1 0 0 1 1 1 The is chair red young cat on a
  • 6. Link with artificial neural network ? Artificial neural network is a set of statistical models inspired from the brain – Transforms the input by applying at each layer (non linear) functions – More layers equals more capabilities (≥ 2 hidden layers : Deep Learning) • From manual features building to feature learning Set of transformation and activation operations – Affine : 𝒀 = 𝑾 𝒕 𝑿 + 𝒃, sigmoid activation : 𝟏 𝟏8𝐞𝐱𝐩 ('𝑿) , tanh activation : 𝒀 = 𝐭𝐚𝐧𝐡 ( 𝑿) • Only affine + activation layers = multi layer perceptron (available in Spark ML since 1.5.0) – Convolutional : Apply a spatial convolution on the 1D/2D input (signal, image, …) : 𝐘 = 𝒄𝒐𝒏𝒗 𝑿, 𝑾 + 𝒃 • Learns spatial features used for classification (images) , prediction – Recurrent : Introduces a recurrent part to learn dependencies between observations (features related to the dynamic) Objective – Find the best weights W to minimize the difference between the predicted output and the desired one (using back-propagation algorithm) input hidden layers output
  • 7. Able to cope with varying size sequences either at the input or at the output Recurrent Neural Network basics One to many (fixedsize input, sequence output) e.g. Image captioning Many to many (sequence input to sequence output) e.g. Speech to text Many to one (sequence input to fixedsize output) e.g. Text classification Artificial neural networks with one or more recurrent layers Classical neural network Recurrent neural network 𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌 𝒀 𝒌 𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌 𝒀 𝒌 = 𝒇(𝑾 𝒕 𝑿 𝒌 + 𝑯𝒀 𝒌'𝟏) 𝑿 𝒌𝑿 𝒀 𝒌 = 𝒇(𝑾 𝒕 𝑿 𝒌) 𝒀 Unrolled through time 𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌 𝑿 𝒀 𝒌'𝟑 𝒀 𝒌'𝟐 𝒀 𝒌'𝟏 𝒀 𝒌 𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌 𝑿 𝒌'𝟑 𝑿 𝒌'𝟐 𝑿 𝒌'𝟏 𝑿 𝒌 𝒀
  • 8. On the difficulty of training recurrent networks RNNs are (were) known to be difficult to learn – More weights and more computational steps • More computationally expensive (accelerator needed for matrix ops : Blas or GPU) • More data needed to converge (scalability over Big Data architectures : Spark) – Theano, Tensor Flow, Caffe do not have distributed versions – Unable to learn long range dependencies (Graves & Al 2014) • At a given time t, RNN does not remember the observations before 𝑋J', ⇒ New RNN architectures with memory preservation (more context) 𝑍$ = 𝑓 𝑊N O 𝑋$ + 𝐻N 𝑌$'( 𝑅$ = 𝑓(𝑊S O 𝑋$ + 𝐻S 𝑌$'() 𝐻T$ = tanh(𝑊YJZ[ O 𝑋$ + 𝑈 𝑌$'( o 𝑅$ ) 𝑌$ = 1 − 𝑍$ 𝑌$'( + 𝑍$ 𝐻T$ LSTM GRU
  • 9. Recurrent neural networks in Spark Spark implementation of DL algorithms (data parallel) – All the needed blocks • Affine, convolutional, recurrent layers (Simple and GRU) • Sigmoid, tanh, reLU activations • SGD, rmsprop, adadelta optimizers – CPU (and GPU backend) – Fully compatible with existing DL library in Spark ML Performance – On 6 nodes cluster (CPU) • 5.46 average speedup (some communication overhead) – About the same speedup as MLP in Spark ML Driver Worker 1 Worker 2 Worker 3 Resulting gradients (2) Model broadcast (1)
  • 10. Use case 1 : predictive maintenance (1) Context – Thales and its clients build systems in different domains • Transportation (ticketing, controlling) • Defense (radar) • Satellites – Need better and more accurate maintenance services • From planned maintenance (every x days) to an alert maintenance • From expert detection to automatic failure prediction • From whole subsystem changes to more localized reparations Goal – Detect early signs of a (sub)system failure using data coming from sensors monitoring the health of a system (HUMS)
  • 11. Use case 1 : predictive maintenance (2) Example on a real system – 20 sensors (20 values every 5 minutes), label (failure or not) – Take 3 hours of data and predict the probability of failure in the next hour (fully customizable) Learning using MLLIB
  • 12. Use case 1 : predictive maintenance (3) Recurrent net learning Impact of recurrent nets – Logistic regression • 70% detection with 70% accuracy – Recurrent Neural Network • 85% detection with 75% accuracy
  • 13. Use case 2 : Sentiment analysis (1) Context – Social network analysis application developed at Thales (Twitter, Facebook, blogs, forums) • Analyze both the content of the texts and the relations (texts, actors) – Multiple (big data) analysis • Actor community detection • Text clustering (themes) • … Focus on – Sentiment analysis on the collected texts • Classify texts based on their sentiment
  • 14. Use case 2 : Sentiment analysis (2) Learning dataset – Sentiment140 + Kaggle challenge (1.5M labeled tweets) – 50% positives, 50% negatives Compare Bag of words + classifier approaches (Naïve Bayes, SVM, logistic regression) versus RNN
  • 15. Use case 2 : Sentiment analysis (3) NB SVM Log Reg NeuralNet (perceptron) RNN (GRU) 100 61.4 58.4 58.4 55.6 NA 1 000 70.6 70.6 70.6 70.8 68.1 10 000 75.4 75.1 75.4 76.1 72.3 100 000 78.1 76.6 76.9 78.5 79.2 700 000 80 78.3 78.3 80 84.1 Results 40 45 50 55 60 65 70 75 80 85 90 NB SVM LogReg NeuralNet RNN (GRU)