Existing models for prediction either focus on capturing non linearity of the relation between predictors and target or time series proprieties . Hybridization of ANN and ARIMA can be a useful method to model both aspects in time series prediction

1. Mohamed Baddar
Senior Data Scientist at Careem
Networks GmbH
mohamed.baddar@careem.com
Hybrid Linear and Non-Linear models for time
series prediction - MarketPlace case study
1

2. 2
• Problem Statement
• Notations
• Motivation
• Background
• Hybrid Model
• Marketplace Case Study : Supply Prediction for P2P ride sharing
• Model Pitfalls and possible improvements
• Questions
Agenda

3. • Customer side objective : Reliable ride sharing service
• Reliability means , whenever a customer asks for ride , he finds a captain
• Captain side :High utilization
• Captains receives requests immediately after he declares himself “free”
• Core task to achieve this objective is to predict supply (number of free captains) and demand (number
of bookings), at each location and time instances
• If a significant gap between supply and demand found, we can fill it by increasing supply
• One way is to apply surge to incentify captains to move to areas in hours where this gap is
expected to happen
• We need to be proactive by predicting the problem and acting before it happen
• Supply prediction can help significantly in that problem
Problem Statement (MarketPlace Objective)
3

4. Notation
Y Supply
X Surge (Peak) types
T Time patterns (trend , seasonality)
TS Time Series
ARIMA Autoregressive Integrated Moving Average : model for time series forecasting
NN Neural Networks
NL Non Linear Model
L Linear Model
E White Noise Error
4

5. Currently implemented algorithms
• TS Forecasting : ES,ARIMA : Focus on TS patterns , ARIMA with
covariates assume linear relationship between X and Y
• Machine Learning model : CART, NN capture non linearity
between external factors and predicted quantity but don’t focus
on TS patterns
One possible solution ? Hybridization
• Hybrid model that captures both non linearities between Y and X
and time series patterns T
• Inspired by how ARIMA with covariates is designed
Motivation
5

6. • Non linear model , output of each layer
is a combination of the set of function in
previous layers. Function are
categorized into, propagation, activation
and output functions
• Parameters
• Number of hidden layers
• Number of neurons in each layer
• (+) Capture complex non linear patterns
• (-) slow training , non interpretable
Neural Networks
6

7. ARMA (p,q)
• Quantity modeled as linear function of previous values
and fit errors
• Data must be stationary (mean and variance don’t
change over time)
• More complex models are used to capture seasonality
• Combined with Regression to capture effect of external
factors on Time series
• (+) Capture Time series patterns , ARMA structure and
seasonality
• (-) assume linear relationships
ARIMA
7

8. Data is not (weak) stationary if mean
and variance vary over time
* Differencing
Stationarity and differencing
1-Input Data
2-Log Transformation
Fig 3
8
3- Seasonal and Lag differencing

9. Hybrid Model (PoC)
NL
(NN) L(ARIMA)
NL.Fitted
L.Fitted
Y,X
E
Y = NL(X) + L(X) + E
* T-D : Transformation and differencing
* Applied for Y and X to preserve
Interpretability
T-D
9

10. Transform and Stationarize (Y,X) via log transformation and differencing if necessary
NL_M = NULL// Non Linear Model
L_M = NULL // Linear Model
RMSE = Inf
L = 0’s // Assume initial Linear components as 0’s
while(less than max iterations AND delta(RMSE) > threshold)
NL = Y - L
NL_M (Y~X) <- build NN from NL data
K = Y - NL_M_fitted // Remainder from NL_M
L_M = build ARMA(p,q) given K
//If L_M is not NULL model (for ex. p,q both = 0) then hybridization was actually needed
Y.fitted = NL.fitted + L.fitted //assuming mean of E is zero, white noise
E = Y-fitted
sanity check => E is white noise
//updating phase
RMSE = RMSE_calc(E)
Calculate delta(RMSE)
L = L_M_fitted
* RMSE conversion means NL and L models becomes stable and converges
Hybrid Model Building (PoC)
10

11. Marketplace Case Study (1)
11

12. Data Description
• Data is partitioned by Zone (For example Berlin Mitte , Dubai Al Barsha)
• For Each zone , data is aggregated on time granularity level (hour, 15 or 30 min)
• A time series is create for the supply level for this zone at this time window
• Hybrid model is applied to model supply relation with time and surge. Also to predict future values with
difference surge values. It works like a what-if analysis tool
MarketPlace Case Study (3)
12

13. • If needed , seasonal differencing (frequency = 4*24) then Lag1 differencing
• Neural network with backpropagation training is used as a non-linear model
Y <- NN(dow,Hour,Minute,Surge_1,Surge_2)
Y = (average) number of captains , in zone and time window
Dow => day of week
Hour => factor with 24 levels
minute => 4 level factor : 0,15,30,45
Surge_1,2 : different types of Surges (peak)
• Number of neurons per level = 10 , 1 level
• ARMA model for linear model max p, q = 5
• Model applied for each zone in each city
• Maximum number of Hybrid model iterations = 3
Model implementation
13

14. In-sample data performance
• E = Y - Y.fitted
Y.fitted = NN.fitted +L.fitted
• NN build with 1 layer with 10 neurons
• Sample ARMA model, for one of the dataset :
ARMA(4,2)
• Accuracy, On 5 Zones :
• Average RMSE for NN only = 39
• Average RMSE for NN+ARIMA = 32
• Improvement = approx. 18%
Model Accuracy and diagnostics
14
White noise

15. Remaining Work after first POC
• On Algorithmic side
• Formal verification of the hybridization method
• Experimenting other NL models (CART, GBM, RF) and L Models for TS (ARIMAX, Transfer
functions)
• Further analysis on algorithm convergence
• Explore modification of core NN optimization to adapt with error AR and MA patterns
• On Implementation
• For R neuralnet packages, sometimes NN fails to build , building algorithm doesn’t converge.
• Scaling method for more zones, doing experimentation on more datasets
• On accuracy measures
• Cross validation for NN and rolling origin for ARIMA (as a kind of unit testing)
• Cross validation and rolling origin for the hybrid model
This still WIP !
15

16. 16
• Apply LSTM RNN for time series prediction
• More complex NN
• More complex TS models
• Estimator variance, stability , multiple initial models to avoid local optima
• Revisiet algorithm convergence
Feedback from audience

17. Questions
17

18. We are Hiring !
18

19. Shukran! Thank you! Danke Schön
19