JDemetra+ is a tool for seasonal adjustment (SA) developed by the National Bank of Belgium (NBB) in cooperation with the Deutsche Bundesbank and Eurostat in accordance with the Guidelines of the European Statistical System (ESS). User support, training and methodological development is provided by the devoted Centre of Excellence on Seasonal Adjustment coordinated by INSEE, the French National Statistical Office.
JDemetra+ has been officially recommended, since February 2015, to the members of the ESS and the European System of Central Banks as software for seasonal and calendar adjustment of official statistics.
1. 1
JDemetra+
Java tool for Seasonal Adjustment
Dominique Ladiray
INSEE, France
dominique.ladiray@insee.fr
Dario Buono
Eurostat, European Commission
Dario.Buono@ec.europa.eu
@darbuo
Shared Tools for Computing with Data in Official Statistics
STS 043 Monday 17 July, Room A 1.17 14:00-15:40
2. Outline
• Time Series and Seasonality
• What is JDEMETRA+
• Some History
• Software Architecture
• Seasonal adjustment methods
• Some examples
• The Seasonal Adjustment Centre of Excellence
• Users Support
3. What is a Time Series?
A Time Series is a sequence of measures of a given phenomenon
taken at regular time intervals such as hourly, daily, weekly,
monthly, quarterly, annually, or every so many years
..with at least 3 usual components
• Trend/Cycle: the long term evolution of the series
• Seasonal pattern: regular fluctuations observed during the year
• Irregular: residual and random fluctuations
4. Cause of Seasonality
Seasonality and Climate: variations of the weather/climate (seasons!)
Seasonality and Institutions: social habits or to the administrative rules
Indirect Seasonality: Seasonality that affects other sectors
Why Seasonal Adjustment?
To improve comparability: Over time & Across space
Business cycle analysis
Reduce noise to facilitate economic reading
6. Some Maths : decomposition models
Usual Additive and Multiplicative Models
More components: Outliers, Calendar Effects
7. What is JDEMETRA+?
JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment
• developed by National Bank of Belgium and Bundesbank
• supported by EUROSTAT and the European Central Bank
Trend identification
Outliers treatment
Estimation of missing values
Calendar Adjustment
ARIMA modelling
Benchmarking
JDemetra+ provides a Java implementations of TRAMO-SEATS and of X12-ARIMA. It
is based on the NetBeans platform, is developed under the EUPL license.
7
8. Some Historical Milestones
8
2002
Demetra
• Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98).
• Integration of original software in a user-friendly application.
• Lack of sufficient product development and handling of errors as a result of a loss of technical
knowledge about software.
2010
Demetra+
• Developed in cooperation between Eurostat and the National Bank of Belgium.
• Enables the implementation of the ESS Guidelines on SA.
• Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12-
ARIMA.
• Includes complex technical solutions. Uses .NET technology and can be used only under Windows.
2015
JDemetra+
•Fortran codes re-written in JAVA.
•Open source, platform independent.
•Extensible graphical interface, based on the NetBeans platform (plugins).
•Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11
part.
9. Anno Nobili 2015
ESS Guidelines on Seasonal Adjustment
Introduced in 2009 and revised in 2015
http://ec.europa.eu/eurostat/documents/
3859598/6830795/KS-GQ-15-001-EN-
N.pdf
List of options:
• A (Recommended)
• B (Acceptable)
• C ( to be avoided)
9
10. Official Release of JD+ 2.0.0
Since the 2 of February 2015 JD+ is the
official software to be used for Seasonal
Adjustment within the European
Statistical System for data to be used for
Official Statistics
Official joint ECB/Eurostat
Methodological Note published at:
http://www.cros-
portal.eu/content/official-release-
jdemetra-software-be-used-seasonal-
adjustment
10
11. Software layout
A rich graphical
application (end-users)
dedicated to Seasonal
Adjustment
11
12. Software layout
But in fact an advanced
Java toolkit for time
series processing (End
users, production, IT-
teams, researchers)
12
13. JDemetra+ characteristics
Flexibility
• Encompasses the leading SA algorithms and can evolve independently
Versatility
• Can be used in a rich graphical interface and/or be integrated in other.
Reusability of modules the other circumstances:
• Plug-in for temporal disaggregation
• Outliers detection, estimation of missing values, Arima forecasts
Extensibility
• Additional plug-ins and modules do not change the core engines.
• Efficient process of large datasets through:
• JWSAcruncher, command line application that allows calling JDemetra+ from other
applications;
Web services and direct call to Java libraries.
Open source
13
22. The Seasonal Adjustment Centre of Excellence
• A “joint venture” between Eurostat, NSIs and CBs
• SACE: Belgium, France, Italy, Latvia, Portugal, UK
• Partners: Eurostat, BBk, OECD, ECB, IMF
• SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania,
Slovakia, Slovenia, Spain
• 3-year contract: 03/10/2016 – 02/10/2019
22
23. On the Agenda
• Knowledge sharing and Dissemination
• Support to SA practitioners
• User Group, Helpdesk, Documentation
• Pre- release validation testing (JD+ 2.2.0 in 2017)
• Plug-ins
• Benchmarking, Quality, Analysis of revisions, Weekly and Daily data
• Training, coaching and consultancy.
23
24. Who is using JD+?
A difficult question: Downloads are anonymous
Helpdesk
Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary,
Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus,
Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland,
Turkey
But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal,
Tunisia, UEMOA etc.
24
25. 25
User Support and Documentation
The latest JDemetra+ released version can be downloaded at
https://github.com/jdemetra/jdemetra-app/releases
Java SE 8 or later versions are required. Modules, code and developers documentation and
GitHub
• https://github.com/jdemetra for the official modules
• https://github.com/nbbrd for NBB resources
All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference
Manual) can be found here.
In addition, beyond seasonal adjustment, the following prototype plug-ins are available for:
• Temporal Disaggregation and Benchmarking
• Quality/ Validation reporting
• Revision Analysis
• Nowcasting
• Using JDEMETRA+ with R
26. JDEMETRA+ Helpdesk and trainings
More info about the SACE
http://ec.europa.eu/eurostat/cros/content/seasonal-adjustment-
centre-excellence_en
For your Helpdesk queries
http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustment-
helpdesk_en
For training opportunities visit
http://ec.europa.eu/eurostat/web/ess/about-us/estp
27. Why using JDEMETRA+?
1. Implementation of ESS Guidelines on SA
2. Open source (JAVA)
3. Documented
4. Maintenance ensured (GITHUB)
5. Both x-12 and TRAMO/Seats
6. User Friendly
7. Users/Developers community (you will never walk alone!)
8. Helpdesk support
9. Trainings available
10. Free
28. …by the why who was Demetra?
In Greek mythology it was believed that Demetra was the
goddess of corn, grain, harvest, agriculture, and of fertility
in general. She made the crops grow each year so was
intimately associated with the seasons.
Her daughter Persephone was abducted by Hades to be his
wife in the underworld. In her anger at her daughter's loss,
Demetra caused plants to die and the land to become
desolate.
Zeus, alarmed for the arid earth, sought for Persephone's
return. However, because she had eaten while in the
underworld, Hades had a claim on her. Therefore, it was
decreed that Persephone would spend four months each
year in the underworld.
During these months Demetra would grieve for her
daughter's absence, withdrawing her gifts from the world,
creating winter. Her return brought the spring.
30. Some more info about the architecture and
the State Space Framework
30
31. Algorithmic libraries (jtstoolkit)
31
Matrix computation
Basic data handling
Complex, polynomials
Linear filters
Function optimization
Time series, calendars, regression variables...
Basic statistics
Utilities...
Basic econometrics
Arima, Ucarima
VAR,
Dynamic factor
model
Seats
X11
State space framework
Arima modelling
RegArima
Tramo
Seasonal
adjustment
Structural models...
Benchmarking,
temporal
disaggregation
32. REGARIMA modelling
• Common definitions for Calendar variables,
outliers, intervention variables, user variables...
• Algorithms for likelihood estimation
• Kalman filter (Tramo-like),
• Ansley algorithm (Cholesky on banded matrix)
• (modified) Ljung-Box algorithm (X12-like)
• Equivalent results, different performances
• JD+ uses Kalman filter
• Up to 4 x faster than Ljung-Box
• Ansley in specific cases (outliers detection)
• Optimization procedure
• Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use
slightly different variants.
Model building
(Reg. variables)
Estimation of the model
(likelihood, residuals)
Estimation of the
parameters (by ML)
33. Automatic model identification
• Independent blocks (dynamically
modifiable)
• Specific implementation for Tramo-Seats,
X12
Example: X12 modelling with outliers detection from Tramo
Log/level
Pre-test (seasonality...)
Calendar effects...
Outliers detection
Final estimation
Arima (diff. / Arma)
Models comparison
Model validation
34. Algorithms for signal extraction in JD+
• Wiener-Kolmogorov filters
• Burman's algorithm, Maravall's analysis framework (Seats)
• Kalman smoother
• Koopman's initialization procedure (disturbance or ordinary smoother)
• Matrix computation
• McElroy 's formulae
• Can be applied to any (valid) UCARIMA model
• Results
• Estimates: identical
• Standard deviations: WK approach yields negligible differences (exception: quasi-
unit roots in MA polynomial → large differences)
34
35. State space framework
• Key solution for:
• REGARIMA estimation
• Signal extraction (Kalman smoother)
• Alternative time series modelling (for SA or not)
• Structural models...
• Benchmarking
• Cholette (including multi-variate extension)
• Temporal disaggregation
• Chow-Lin, Fernandez...
• Multi-variate models
• VAR, dynamic factor models, SUTSE…
• JD+ provides an advanced implementation of SSF
36. State space framework (II)
Models
Atomic models:
• Generic (time invariant or
not)
• Ar(i)ma
• Ucarima
• Basic structural
• White noise
• Random walk
• ...
Derived models:
• Composite
• Regression variables
• Aggregation constraints
• ...
Algorithms
Filtering:
• Ordinary filter
• Fast filter (Chandrasekhar)
• Array filter (Kailath...)
Diffuse initialization:
• Koopman
• Square root
• Ad hoc
Smoothing:
• Ordinary
• Disturbance
• Fixed point
Others:
• Univariate handling of
multi-variate models
• Augmented Kalman filter
(for reg. model)
• Extended Kalman filter
(for non linear models)
• ...
Likelihood evaluation:
Prediction error decomposition