pyBCN meetup speech slides 20.09.18

1. DEEP LEARNING
for time series
Alex Honchar | Mawi Solutions | Solutions architect, consultant

2. • 5 years in ML area | Ukraine, Russia, Italy, USA
• AI Solution architect | Mawi Solutions, ECG analysis
• AI consultant | self employed, finance and gaming
ABOUT ME

3. • Time series in the wild
• Classical approaches
• Deep learning
• Takeaways
OUTLINE

4. TS IN THE WILD 🦁
• Planet Earth

5. TS IN THE WILD 🦁
• Planet Earth
• Civilization delights

6. TS IN THE WILD 🦁
• Planet Earth
• Civilization delights
• Human beings

7. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
https://github.com/blue-yonder/tsfresh

8. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
https://github.com/blue-yonder/tsfresh

9. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
https://pypi.org/project/fastdtw/

10. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
- ML

11. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
- ML
• Regression
- ARIMA models
http://www.statsmodels.org/

12. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
- ML
• Regression
- ARMA models
- smoothing / decomposition
https://pypi.org/project/stldecompose/

13. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
- ML
• Regression
- ARMA models
- smoothing / decomposition
- nonlinear dynamics

14. CLASSICAL APPROACHES 🔬
• Classification
- time domain analysis
- frequency domain analysis
- nearest neighbors
- ML
• Regression
- ARMA models
- smoothing / decomposition
- nonlinear dynamics
- ML

15. DEEP LEARNING 🧠
TCE conference, 2014

16. DEEP LEARNING 🧠
• RNN

17. DEEP LEARNING 🧠
• RNN
1.Theoretical infinite memory
2.Multistep prediction ability
3.Don't work in parallel
4.Difficult to optimize
5.Slow in inference
6.Truncated implementation
7.Doubtful superior performance

18. DEEP LEARNING 🧠
• RNN
• CNN

19. DEEP LEARNING 🧠
• RNN
• CNN
• RNN + CNN

20. DEEP LEARNING 🧠
• RNN
• CNN
• RNN + CNN

21. DEEP LEARNING 🧠
• RNN
• CNN
• RNN + CNN
• Autoregressive CNN

22. DEEP LEARNING 🧠
• RNN
• CNN
• RNN + CNN
• Autoregressive CNN
• Other tasks

23. DEEP LEARNING 🧠
• RNN
• CNN
• RNN + CNN
• Autoregressive CNN
• Other tasks

24. HYBRID SOLUTIONS 🐙

25. TAKEAWAYS 📚
• There are dozens of features to feed classic ML with
• Deep learning is eating signal processing
• Autoregressive CNN > CNN > RNN
• AEs and GANs are useful as well
• Try to combine things!

26. Home reading
1. When Recurrent Models Don't Need To Be Recurrent
2. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
3. Deep residual learning for image recognition
4. WaveNet: A generative model for raw audio
5. DEEP TEMPORAL CLUSTERING: FULLY UNSUPERVISED LEARNING OF TIME-DOMAIN FEATURES
6. REAL-VALUED (MEDICAL) TIME SERIES GENERATION WITH RECURRENT CONDITIONAL GANS
7. Time-series Extreme Event Forecasting with Neural Networks at Uber
FB: @rachnogstyle
IG: @rachnogstyle
MEDIUM: @alexrachnog