15. Time domain analysis
Statistical features
Very limited point of view
Geometrical features
Overcomplex algorithmic heuristics
Decompositions
Oversimplified econometric POV
“ARIMA”-like models
Just autocorrelation on different lags
Multilayer perceptrons
Universal approximation theorem
Designed with non-linearity
Convolutional neural networks
Can learn arbitrary local geometrical patterns
17. Frequency domain analysis
Fourier transform
Very limited point of view
Losing information that varies over time
Wavelet transform
Fixed wavelet family
Convolutional neural networks
Convolution theorem for frequency analysis
Learnable and extendable kernel family
Keeps information over time
19. State space models
Hidden Markov Models
Difficult to train, not for high-dimensional data
No long-term dependencies
Dynamical factor models
Just a vector autoregression?
Kalman filters
Requires a model of the system
Recurrent neural networks
Designed to learn long-range dependencies
Designed to deal with high-dimensional data
Hierarchical state space
Non-linearity
20. State space models
Hidden Markov Models
Difficult to train, not for high-dimensional data
No long-term dependencies
Dynamical factor models
Just a vector autoregression?
Kalman filters
Requires a model of the system
Recurrent neural networks
Truncated implementation
Difficult to optimize
Slow in inference
Autoregressive networks
Connection to autoregressive models
Still long-range context
Faster and more accurate
25. Pattern matching
“Normal” distances
Don’t work with time series
Dynamic time warping
Computationally difficult
Metric learning
Requires a lot of customization
Autoencoders
Flexibility in choosing encoding scheme
Fully unsupervised
After training just forward pass to get embedding
Siamese networks
Learn pattern matching or clustering directly
26. Anomaly detection
Density-based methods
Don’t really work with time series
Correlation-based methods
Assuming linear properties
Fit “ARIMA”, check for residuals
Depends on a simple model
Autoencoders
Flexibility in choosing encoding scheme
Fully unsupervised
Just need to adjust the thresholds
Generative adversarial networks
Get a generative model “for free”
Use discriminator as anomaly detector
28. Simulation and generation
Mathematical models
…
Sequence2sequence schemes
Flexibility in choosing encoding scheme
Variational autoencoders
Object manipulation via disentangled representations
Generative adversarial networks
State of the art results at the moment
Neural ODEs
Naturally model dynamical systems with arbitrary precision
34. - Time series and signals are everywhere
- To all “classical” approaches there are “neural” alternatives
- CNN or autoregressive CNN is a baseline
- Try unsupervised learning for better embedding space
- Simulation with VAEs and GANs is amazing!
- Try to combine your hand-crafted features with DL
Takeaways
35. Open for collaborations :)
Facebook / Instagram @rachnogstyle
Medium @alexrachnog
Linkedin Alexandr Honchar
36. Resources
* Is Deep Learning the Final Frontier and the End of Signal Processing - Panel Discussion at Technion
https://www.youtube.com/watch?v=LZnAFO5gkOQ&t=9s
* Stanford ECG: https://stanfordmlgroup.github.io/projects/ecg/
* Groceries sales forecasting: https://www.kaggle.com/c/favorita-grocery-sales-forecasting
* Wikipedia traffic forecasting: https://www.kaggle.com/c/web-traffic-time-series-forecasting
* Forecasting at Uber: https://eng.uber.com/tag/forecasting/
* DeepMind WaveNet: https://deepmind.com/blog/wavenet-generative-model-raw-audio/
* EEG2Thoughts: https://medium.com/@justlv/using-ai-to-read-your-thoughts-with-keras-and-an-
eeg-sensor-167ace32e84a
* ECG interpretation: https://medium.com/mawi-band/how-ai-based-arrhythmia-detector-can-
explain-its-decisions-b4f433faa4a2
* Replacing mathematical models with NNs in biosignal analysis: https://medium.com/mawi-band/
towards-ai-based-only-biosignal-analysis-pipeline-39e6e31244a6
* Statistical and Machine Learning forecasting methods: Concerns and ways forward: http://
journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889
* Time-series Extreme Event Forecasting with Neural Networks at Uber: http://roseyu.com/time-
series-workshop/submissions/TSW2017_paper_3.pdf