SlideShare a Scribd company logo
1 of 36
1
JDemetra+
Java tool for Seasonal Adjustment
Dominique Ladiray
INSEE, France
dominique.ladiray@insee.fr
Dario Buono
Eurostat, European Commission
Dario.Buono@ec.europa.eu
@darbuo
Shared Tools for Computing with Data in Official Statistics
STS 043 Monday 17 July, Room A 1.17 14:00-15:40
Outline
• Time Series and Seasonality
• What is JDEMETRA+
• Some History
• Software Architecture
• Seasonal adjustment methods
• Some examples
• The Seasonal Adjustment Centre of Excellence
• Users Support
What is a Time Series?
A Time Series is a sequence of measures of a given phenomenon
taken at regular time intervals such as hourly, daily, weekly,
monthly, quarterly, annually, or every so many years
..with at least 3 usual components
• Trend/Cycle: the long term evolution of the series
• Seasonal pattern: regular fluctuations observed during the year
• Irregular: residual and random fluctuations
Cause of Seasonality
Seasonality and Climate: variations of the weather/climate (seasons!)
Seasonality and Institutions: social habits or to the administrative rules
Indirect Seasonality: Seasonality that affects other sectors
Why Seasonal Adjustment?
To improve comparability: Over time & Across space
Business cycle analysis
Reduce noise to facilitate economic reading
It is about smoothing..
5
Some Maths : decomposition models
Usual Additive and Multiplicative Models
More components: Outliers, Calendar Effects
What is JDEMETRA+?
JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment
• developed by National Bank of Belgium and Bundesbank
• supported by EUROSTAT and the European Central Bank
Trend identification
Outliers treatment
Estimation of missing values
Calendar Adjustment
ARIMA modelling
Benchmarking
JDemetra+ provides a Java implementations of TRAMO-SEATS and of X12-ARIMA. It
is based on the NetBeans platform, is developed under the EUPL license.
7
Some Historical Milestones
8
2002
Demetra
• Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98).
• Integration of original software in a user-friendly application.
• Lack of sufficient product development and handling of errors as a result of a loss of technical
knowledge about software.
2010
Demetra+
• Developed in cooperation between Eurostat and the National Bank of Belgium.
• Enables the implementation of the ESS Guidelines on SA.
• Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12-
ARIMA.
• Includes complex technical solutions. Uses .NET technology and can be used only under Windows.
2015
JDemetra+
•Fortran codes re-written in JAVA.
•Open source, platform independent.
•Extensible graphical interface, based on the NetBeans platform (plugins).
•Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11
part.
Anno Nobili 2015
ESS Guidelines on Seasonal Adjustment
Introduced in 2009 and revised in 2015
http://ec.europa.eu/eurostat/documents/
3859598/6830795/KS-GQ-15-001-EN-
N.pdf
List of options:
• A (Recommended)
• B (Acceptable)
• C ( to be avoided)
9
Official Release of JD+ 2.0.0
Since the 2 of February 2015 JD+ is the
official software to be used for Seasonal
Adjustment within the European
Statistical System for data to be used for
Official Statistics
Official joint ECB/Eurostat
Methodological Note published at:
http://www.cros-
portal.eu/content/official-release-
jdemetra-software-be-used-seasonal-
adjustment
10
Software layout
A rich graphical
application (end-users)
dedicated to Seasonal
Adjustment
11
Software layout
But in fact an advanced
Java toolkit for time
series processing (End
users, production, IT-
teams, researchers)
12
JDemetra+ characteristics
Flexibility
• Encompasses the leading SA algorithms and can evolve independently
Versatility
• Can be used in a rich graphical interface and/or be integrated in other.
Reusability of modules the other circumstances:
• Plug-in for temporal disaggregation
• Outliers detection, estimation of missing values, Arima forecasts
Extensibility
• Additional plug-ins and modules do not change the core engines.
• Efficient process of large datasets through:
• JWSAcruncher, command line application that allows calling JDemetra+ from other
applications;
Web services and direct call to Java libraries.
Open source
13
Architecture
14
In house
developments
JTsToolkit
Core algorithms
Peripheral modules
External
packages
JDemetra+ plug-insNetBeans
Third party
plug-ins
Jdemetra-core
Jdemetra-app
Cruncher
Seasonal Adjustment Methods
SA methods
Generic modules:
•Analysis
•Seasonality tests
•Revision analysis
•Sliding spans
•I/O (common xml schema)
•Graphical components
•Charts
•SI ratios...
Tramo-Seats, X12-Arima...
Model-based decomposition
(canonical decomposition,
structural models...)
Signal extraction tools:
•Estimation
•Analysis
•Graphical components
Specific
modules
(X11...)
Other filters
(X11...)
REGARIMA pre-processing
REGARIMA modules:
•Common model
•Estimation tools
•Automatic modelling routines
•Analysis tools (residuals,
forecasts...)
•Graphical components
Input output
Graphs and diagnostics
Distributions
Trend/Seasonal Components with Forecasts
Forecast of original series
Plug-ins for Temporal Disaggregation
The Seasonal Adjustment Centre of Excellence
• A “joint venture” between Eurostat, NSIs and CBs
• SACE: Belgium, France, Italy, Latvia, Portugal, UK
• Partners: Eurostat, BBk, OECD, ECB, IMF
• SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania,
Slovakia, Slovenia, Spain
• 3-year contract: 03/10/2016 – 02/10/2019
22
On the Agenda
• Knowledge sharing and Dissemination
• Support to SA practitioners
• User Group, Helpdesk, Documentation
• Pre- release validation testing (JD+ 2.2.0 in 2017)
• Plug-ins
• Benchmarking, Quality, Analysis of revisions, Weekly and Daily data
• Training, coaching and consultancy.
23
Who is using JD+?
A difficult question: Downloads are anonymous
Helpdesk
Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary,
Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus,
Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland,
Turkey
But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal,
Tunisia, UEMOA etc.
24
25
User Support and Documentation
The latest JDemetra+ released version can be downloaded at
https://github.com/jdemetra/jdemetra-app/releases
Java SE 8 or later versions are required. Modules, code and developers documentation and
GitHub
• https://github.com/jdemetra for the official modules
• https://github.com/nbbrd for NBB resources
All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference
Manual) can be found here.
In addition, beyond seasonal adjustment, the following prototype plug-ins are available for:
• Temporal Disaggregation and Benchmarking
• Quality/ Validation reporting
• Revision Analysis
• Nowcasting
• Using JDEMETRA+ with R
JDEMETRA+ Helpdesk and trainings
More info about the SACE
http://ec.europa.eu/eurostat/cros/content/seasonal-adjustment-
centre-excellence_en
For your Helpdesk queries
http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustment-
helpdesk_en
For training opportunities visit
http://ec.europa.eu/eurostat/web/ess/about-us/estp
Why using JDEMETRA+?
1. Implementation of ESS Guidelines on SA
2. Open source (JAVA)
3. Documented
4. Maintenance ensured (GITHUB)
5. Both x-12 and TRAMO/Seats
6. User Friendly
7. Users/Developers community (you will never walk alone!)
8. Helpdesk support
9. Trainings available
10. Free
…by the why who was Demetra?
In Greek mythology it was believed that Demetra was the
goddess of corn, grain, harvest, agriculture, and of fertility
in general. She made the crops grow each year so was
intimately associated with the seasons.
Her daughter Persephone was abducted by Hades to be his
wife in the underworld. In her anger at her daughter's loss,
Demetra caused plants to die and the land to become
desolate.
Zeus, alarmed for the arid earth, sought for Persephone's
return. However, because she had eaten while in the
underworld, Hades had a claim on her. Therefore, it was
decreed that Persephone would spend four months each
year in the underworld.
During these months Demetra would grieve for her
daughter's absence, withdrawing her gifts from the world,
creating winter. Her return brought the spring.
29
Dominique Ladiray
INSEE, France
dominique.ladiray@insee.fr
Thank you for your attention
Dario Buono
Eurostat, European Commission
Dario.Buono@ec.europa.eu
@darbuo
Some more info about the architecture and
the State Space Framework
30
Algorithmic libraries (jtstoolkit)
31
Matrix computation
Basic data handling
Complex, polynomials
Linear filters
Function optimization
Time series, calendars, regression variables...
Basic statistics
Utilities...
Basic econometrics
Arima, Ucarima
VAR,
Dynamic factor
model
Seats
X11
State space framework
Arima modelling
RegArima
Tramo
Seasonal
adjustment
Structural models...
Benchmarking,
temporal
disaggregation
REGARIMA modelling
• Common definitions for Calendar variables,
outliers, intervention variables, user variables...
• Algorithms for likelihood estimation
• Kalman filter (Tramo-like),
• Ansley algorithm (Cholesky on banded matrix)
• (modified) Ljung-Box algorithm (X12-like)
• Equivalent results, different performances
• JD+ uses Kalman filter
• Up to 4 x faster than Ljung-Box
• Ansley in specific cases (outliers detection)
• Optimization procedure
• Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use
slightly different variants.
Model building
(Reg. variables)
Estimation of the model
(likelihood, residuals)
Estimation of the
parameters (by ML)
Automatic model identification
• Independent blocks (dynamically
modifiable)
• Specific implementation for Tramo-Seats,
X12
Example: X12 modelling with outliers detection from Tramo
Log/level
Pre-test (seasonality...)
Calendar effects...
Outliers detection
Final estimation
Arima (diff. / Arma)
Models comparison
Model validation
Algorithms for signal extraction in JD+
• Wiener-Kolmogorov filters
• Burman's algorithm, Maravall's analysis framework (Seats)
• Kalman smoother
• Koopman's initialization procedure (disturbance or ordinary smoother)
• Matrix computation
• McElroy 's formulae
• Can be applied to any (valid) UCARIMA model
• Results
• Estimates: identical
• Standard deviations: WK approach yields negligible differences (exception: quasi-
unit roots in MA polynomial → large differences)
34
State space framework
• Key solution for:
• REGARIMA estimation
• Signal extraction (Kalman smoother)
• Alternative time series modelling (for SA or not)
• Structural models...
• Benchmarking
• Cholette (including multi-variate extension)
• Temporal disaggregation
• Chow-Lin, Fernandez...
• Multi-variate models
• VAR, dynamic factor models, SUTSE…
• JD+ provides an advanced implementation of SSF
State space framework (II)
Models
Atomic models:
• Generic (time invariant or
not)
• Ar(i)ma
• Ucarima
• Basic structural
• White noise
• Random walk
• ...
Derived models:
• Composite
• Regression variables
• Aggregation constraints
• ...
Algorithms
Filtering:
• Ordinary filter
• Fast filter (Chandrasekhar)
• Array filter (Kailath...)
Diffuse initialization:
• Koopman
• Square root
• Ad hoc
Smoothing:
• Ordinary
• Disturbance
• Fixed point
Others:
• Univariate handling of
multi-variate models
• Augmented Kalman filter
(for reg. model)
• Extended Kalman filter
(for non linear models)
• ...
Likelihood evaluation:
Prediction error decomposition

More Related Content

Similar to JDemetra+ Java Tool for Seasonal Adjustment

apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenazi
apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenaziapidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenazi
apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenaziapidays
 
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...Dario Buono
 
Copernicus Climate Change Service - An introduction
Copernicus Climate Change Service -  An introductionCopernicus Climate Change Service -  An introduction
Copernicus Climate Change Service - An introductionCopernicus ECMWF
 
FIspace Infodays Poznan 18 Sep 2013
FIspace Infodays Poznan 18 Sep 2013FIspace Infodays Poznan 18 Sep 2013
FIspace Infodays Poznan 18 Sep 2013Sjaak Wolfert
 
EUDAT CDI Architecture
EUDAT CDI ArchitectureEUDAT CDI Architecture
EUDAT CDI ArchitectureEUDAT
 
BigDataEurope - Empowering Communities with Data Technologies
BigDataEurope - Empowering Communities with Data TechnologiesBigDataEurope - Empowering Communities with Data Technologies
BigDataEurope - Empowering Communities with Data TechnologiesSemantic Web Company
 
General Presentation European Data Portal
General Presentation European Data PortalGeneral Presentation European Data Portal
General Presentation European Data PortalEuropeanDataPortal
 
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, Rome
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, RomeCountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, Rome
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, RomeFAO
 
TAUS MT Showcase, MT@EC for European public administrations and online servic...
TAUS MT Showcase, MT@EC for European public administrations and online servic...TAUS MT Showcase, MT@EC for European public administrations and online servic...
TAUS MT Showcase, MT@EC for European public administrations and online servic...TAUS - The Language Data Network
 
Data management plans – EUDAT Best practices and case study | www.eudat.eu
Data management plans – EUDAT Best practices and case study | www.eudat.euData management plans – EUDAT Best practices and case study | www.eudat.eu
Data management plans – EUDAT Best practices and case study | www.eudat.euEUDAT
 
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Sitra / Hyvinvointi
 
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZ
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZGogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZ
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZUniversity of Sydney
 
Communication activities by Armstrong
Communication activities by ArmstrongCommunication activities by Armstrong
Communication activities by ArmstrongCopernicus ECMWF
 
Introduction to: Big Data Europe Project
Introduction to: Big Data Europe Project Introduction to: Big Data Europe Project
Introduction to: Big Data Europe Project Martin Kaltenböck
 

Similar to JDemetra+ Java Tool for Seasonal Adjustment (20)

OECD/DAC - Results Community October 2018 Workshop - Results approaches for t...
OECD/DAC - Results Community October 2018 Workshop - Results approaches for t...OECD/DAC - Results Community October 2018 Workshop - Results approaches for t...
OECD/DAC - Results Community October 2018 Workshop - Results approaches for t...
 
apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenazi
apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenaziapidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenazi
apidays LIVE Paris 2021 - Boavitza, Year 2 by Laurent Eskenazi
 
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...
Eurostat tools for benchmarking and seasonal adjustment j_demetra+ and jecotr...
 
Copernicus Climate Change Service - An introduction
Copernicus Climate Change Service -  An introductionCopernicus Climate Change Service -  An introduction
Copernicus Climate Change Service - An introduction
 
Linked Data for Cross-Domain Decision-making in Tourism
Linked Data for Cross-Domain Decision-making in TourismLinked Data for Cross-Domain Decision-making in Tourism
Linked Data for Cross-Domain Decision-making in Tourism
 
B08 A44s Wp6i En
B08 A44s Wp6i EnB08 A44s Wp6i En
B08 A44s Wp6i En
 
FIspace Infodays Poznan 18 Sep 2013
FIspace Infodays Poznan 18 Sep 2013FIspace Infodays Poznan 18 Sep 2013
FIspace Infodays Poznan 18 Sep 2013
 
8 european drought observatory micale
8 european drought observatory micale8 european drought observatory micale
8 european drought observatory micale
 
EUDAT CDI Architecture
EUDAT CDI ArchitectureEUDAT CDI Architecture
EUDAT CDI Architecture
 
Étude 2018 Arthur D. Little sur l'économie des Télécoms (EN)
Étude 2018 Arthur D. Little sur l'économie des Télécoms (EN)Étude 2018 Arthur D. Little sur l'économie des Télécoms (EN)
Étude 2018 Arthur D. Little sur l'économie des Télécoms (EN)
 
BigDataEurope - Empowering Communities with Data Technologies
BigDataEurope - Empowering Communities with Data TechnologiesBigDataEurope - Empowering Communities with Data Technologies
BigDataEurope - Empowering Communities with Data Technologies
 
General Presentation European Data Portal
General Presentation European Data PortalGeneral Presentation European Data Portal
General Presentation European Data Portal
 
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, Rome
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, RomeCountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, Rome
CountrySTAT / FENIX Follow-up actions 25 – 26 June 2015 FAO HQ, Rome
 
TAUS MT Showcase, MT@EC for European public administrations and online servic...
TAUS MT Showcase, MT@EC for European public administrations and online servic...TAUS MT Showcase, MT@EC for European public administrations and online servic...
TAUS MT Showcase, MT@EC for European public administrations and online servic...
 
Data management plans – EUDAT Best practices and case study | www.eudat.eu
Data management plans – EUDAT Best practices and case study | www.eudat.euData management plans – EUDAT Best practices and case study | www.eudat.eu
Data management plans – EUDAT Best practices and case study | www.eudat.eu
 
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
 
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZ
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZGogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZ
GogginbeyonddigitaldividetalkPPDD2015ScotsdaleAZ
 
Communication activities by Armstrong
Communication activities by ArmstrongCommunication activities by Armstrong
Communication activities by Armstrong
 
Fourth IDMP CEE workshop: European Drought Observatory (EDO) by Fabio Micale
Fourth IDMP CEE workshop: European Drought Observatory (EDO) by Fabio MicaleFourth IDMP CEE workshop: European Drought Observatory (EDO) by Fabio Micale
Fourth IDMP CEE workshop: European Drought Observatory (EDO) by Fabio Micale
 
Introduction to: Big Data Europe Project
Introduction to: Big Data Europe Project Introduction to: Big Data Europe Project
Introduction to: Big Data Europe Project
 

More from Dario Buono

Big Data and Nowcasting
Big Data and NowcastingBig Data and Nowcasting
Big Data and NowcastingDario Buono
 
Reporting uncertainties - too much information?
Reporting uncertainties - too much information?Reporting uncertainties - too much information?
Reporting uncertainties - too much information?Dario Buono
 
Skills for the new generation of statisticians
Skills for the new generation of statisticians Skills for the new generation of statisticians
Skills for the new generation of statisticians Dario Buono
 
Big data and macroeconomic nowcasting from data access to modelling
Big data and macroeconomic nowcasting from data access to modellingBig data and macroeconomic nowcasting from data access to modelling
Big data and macroeconomic nowcasting from data access to modellingDario Buono
 
Big Data Analysis: The curse of dimensionality in official statistics
Big Data Analysis: The curse of dimensionality in official statisticsBig Data Analysis: The curse of dimensionality in official statistics
Big Data Analysis: The curse of dimensionality in official statisticsDario Buono
 
Physics4Stats & BMI vs. QoL
Physics4Stats & BMI vs. QoLPhysics4Stats & BMI vs. QoL
Physics4Stats & BMI vs. QoLDario Buono
 
Methodological network and strategy
Methodological network and strategy Methodological network and strategy
Methodological network and strategy Dario Buono
 
Safebook quality grading
Safebook quality gradingSafebook quality grading
Safebook quality gradingDario Buono
 
MIP: Analysis of metadata and data revisions
MIP: Analysis of metadata and data revisionsMIP: Analysis of metadata and data revisions
MIP: Analysis of metadata and data revisionsDario Buono
 
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...New innovative 3 way anova a-priori test for direct vs. indirect approach in ...
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...Dario Buono
 
Detecting outliers at the end of the series using forecast intervals
Detecting outliers at the end of the series using forecast intervals Detecting outliers at the end of the series using forecast intervals
Detecting outliers at the end of the series using forecast intervals Dario Buono
 
1 out of 20 scenarios
1 out of 20 scenarios1 out of 20 scenarios
1 out of 20 scenariosDario Buono
 
Eurostat methodological skills staff survey lesson learned final
Eurostat methodological skills staff survey lesson learned finalEurostat methodological skills staff survey lesson learned final
Eurostat methodological skills staff survey lesson learned finalDario Buono
 
Reliability of estimates in socio-demographic groups with small samples
Reliability of estimates in socio-demographic groups with small samplesReliability of estimates in socio-demographic groups with small samples
Reliability of estimates in socio-demographic groups with small samplesDario Buono
 

More from Dario Buono (14)

Big Data and Nowcasting
Big Data and NowcastingBig Data and Nowcasting
Big Data and Nowcasting
 
Reporting uncertainties - too much information?
Reporting uncertainties - too much information?Reporting uncertainties - too much information?
Reporting uncertainties - too much information?
 
Skills for the new generation of statisticians
Skills for the new generation of statisticians Skills for the new generation of statisticians
Skills for the new generation of statisticians
 
Big data and macroeconomic nowcasting from data access to modelling
Big data and macroeconomic nowcasting from data access to modellingBig data and macroeconomic nowcasting from data access to modelling
Big data and macroeconomic nowcasting from data access to modelling
 
Big Data Analysis: The curse of dimensionality in official statistics
Big Data Analysis: The curse of dimensionality in official statisticsBig Data Analysis: The curse of dimensionality in official statistics
Big Data Analysis: The curse of dimensionality in official statistics
 
Physics4Stats & BMI vs. QoL
Physics4Stats & BMI vs. QoLPhysics4Stats & BMI vs. QoL
Physics4Stats & BMI vs. QoL
 
Methodological network and strategy
Methodological network and strategy Methodological network and strategy
Methodological network and strategy
 
Safebook quality grading
Safebook quality gradingSafebook quality grading
Safebook quality grading
 
MIP: Analysis of metadata and data revisions
MIP: Analysis of metadata and data revisionsMIP: Analysis of metadata and data revisions
MIP: Analysis of metadata and data revisions
 
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...New innovative 3 way anova a-priori test for direct vs. indirect approach in ...
New innovative 3 way anova a-priori test for direct vs. indirect approach in ...
 
Detecting outliers at the end of the series using forecast intervals
Detecting outliers at the end of the series using forecast intervals Detecting outliers at the end of the series using forecast intervals
Detecting outliers at the end of the series using forecast intervals
 
1 out of 20 scenarios
1 out of 20 scenarios1 out of 20 scenarios
1 out of 20 scenarios
 
Eurostat methodological skills staff survey lesson learned final
Eurostat methodological skills staff survey lesson learned finalEurostat methodological skills staff survey lesson learned final
Eurostat methodological skills staff survey lesson learned final
 
Reliability of estimates in socio-demographic groups with small samples
Reliability of estimates in socio-demographic groups with small samplesReliability of estimates in socio-demographic groups with small samples
Reliability of estimates in socio-demographic groups with small samples
 

Recently uploaded

Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSINGmarianagonzalez07
 
RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.natarajan8993
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024thyngster
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...Amil Baba Dawood bangali
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...Boston Institute of Analytics
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanMYRABACSAFRA2
 
While-For-loop in python used in college
While-For-loop in python used in collegeWhile-For-loop in python used in college
While-For-loop in python used in collegessuser7a7cd61
 
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...Biometric Authentication: The Evolution, Applications, Benefits and Challenge...
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...GQ Research
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...dajasot375
 
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort servicejennyeacort
 
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理e4aez8ss
 
Machine learning classification ppt.ppt
Machine learning classification  ppt.pptMachine learning classification  ppt.ppt
Machine learning classification ppt.pptamreenkhanum0307
 
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptx
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptxNLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptx
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptxBoston Institute of Analytics
 
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...Florian Roscheck
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfBoston Institute of Analytics
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFAAndrei Kaleshka
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceSapana Sha
 

Recently uploaded (20)

Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
2006_GasProcessing_HB (1).pdf HYDROCARBON PROCESSING
 
RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.RABBIT: A CLI tool for identifying bots based on their GitHub events.
RABBIT: A CLI tool for identifying bots based on their GitHub events.
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
 
Identifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population MeanIdentifying Appropriate Test Statistics Involving Population Mean
Identifying Appropriate Test Statistics Involving Population Mean
 
While-For-loop in python used in college
While-For-loop in python used in collegeWhile-For-loop in python used in college
While-For-loop in python used in college
 
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...Biometric Authentication: The Evolution, Applications, Benefits and Challenge...
Biometric Authentication: The Evolution, Applications, Benefits and Challenge...
 
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
 
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
Indian Call Girls in Abu Dhabi O5286O24O8 Call Girls in Abu Dhabi By Independ...
 
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
9711147426✨Call In girls Gurgaon Sector 31. SCO 25 escort service
 
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
 
Machine learning classification ppt.ppt
Machine learning classification  ppt.pptMachine learning classification  ppt.ppt
Machine learning classification ppt.ppt
 
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptx
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptxNLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptx
NLP Project PPT: Flipkart Product Reviews through NLP Data Science.pptx
 
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFA
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts Service
 

JDemetra+ Java Tool for Seasonal Adjustment

  • 1. 1 JDemetra+ Java tool for Seasonal Adjustment Dominique Ladiray INSEE, France dominique.ladiray@insee.fr Dario Buono Eurostat, European Commission Dario.Buono@ec.europa.eu @darbuo Shared Tools for Computing with Data in Official Statistics STS 043 Monday 17 July, Room A 1.17 14:00-15:40
  • 2. Outline • Time Series and Seasonality • What is JDEMETRA+ • Some History • Software Architecture • Seasonal adjustment methods • Some examples • The Seasonal Adjustment Centre of Excellence • Users Support
  • 3. What is a Time Series? A Time Series is a sequence of measures of a given phenomenon taken at regular time intervals such as hourly, daily, weekly, monthly, quarterly, annually, or every so many years ..with at least 3 usual components • Trend/Cycle: the long term evolution of the series • Seasonal pattern: regular fluctuations observed during the year • Irregular: residual and random fluctuations
  • 4. Cause of Seasonality Seasonality and Climate: variations of the weather/climate (seasons!) Seasonality and Institutions: social habits or to the administrative rules Indirect Seasonality: Seasonality that affects other sectors Why Seasonal Adjustment? To improve comparability: Over time & Across space Business cycle analysis Reduce noise to facilitate economic reading
  • 5. It is about smoothing.. 5
  • 6. Some Maths : decomposition models Usual Additive and Multiplicative Models More components: Outliers, Calendar Effects
  • 7. What is JDEMETRA+? JDEMETRA+ is an Time Series Econometric tool for Seasonal Adjustment • developed by National Bank of Belgium and Bundesbank • supported by EUROSTAT and the European Central Bank Trend identification Outliers treatment Estimation of missing values Calendar Adjustment ARIMA modelling Benchmarking JDemetra+ provides a Java implementations of TRAMO-SEATS and of X12-ARIMA. It is based on the NetBeans platform, is developed under the EUPL license. 7
  • 8. Some Historical Milestones 8 2002 Demetra • Program to compare X-12-ARIMA and TRAMO/SEATS (1997/98). • Integration of original software in a user-friendly application. • Lack of sufficient product development and handling of errors as a result of a loss of technical knowledge about software. 2010 Demetra+ • Developed in cooperation between Eurostat and the National Bank of Belgium. • Enables the implementation of the ESS Guidelines on SA. • Provides graphical interface and common input/output diagnostics for TRAMO/SEATS and X-12- ARIMA. • Includes complex technical solutions. Uses .NET technology and can be used only under Windows. 2015 JDemetra+ •Fortran codes re-written in JAVA. •Open source, platform independent. •Extensible graphical interface, based on the NetBeans platform (plugins). •Developed by the National Bank of Belgium, supported by the Deutsche Bundesbank for the X-11 part.
  • 9. Anno Nobili 2015 ESS Guidelines on Seasonal Adjustment Introduced in 2009 and revised in 2015 http://ec.europa.eu/eurostat/documents/ 3859598/6830795/KS-GQ-15-001-EN- N.pdf List of options: • A (Recommended) • B (Acceptable) • C ( to be avoided) 9
  • 10. Official Release of JD+ 2.0.0 Since the 2 of February 2015 JD+ is the official software to be used for Seasonal Adjustment within the European Statistical System for data to be used for Official Statistics Official joint ECB/Eurostat Methodological Note published at: http://www.cros- portal.eu/content/official-release- jdemetra-software-be-used-seasonal- adjustment 10
  • 11. Software layout A rich graphical application (end-users) dedicated to Seasonal Adjustment 11
  • 12. Software layout But in fact an advanced Java toolkit for time series processing (End users, production, IT- teams, researchers) 12
  • 13. JDemetra+ characteristics Flexibility • Encompasses the leading SA algorithms and can evolve independently Versatility • Can be used in a rich graphical interface and/or be integrated in other. Reusability of modules the other circumstances: • Plug-in for temporal disaggregation • Outliers detection, estimation of missing values, Arima forecasts Extensibility • Additional plug-ins and modules do not change the core engines. • Efficient process of large datasets through: • JWSAcruncher, command line application that allows calling JDemetra+ from other applications; Web services and direct call to Java libraries. Open source 13
  • 14. Architecture 14 In house developments JTsToolkit Core algorithms Peripheral modules External packages JDemetra+ plug-insNetBeans Third party plug-ins Jdemetra-core Jdemetra-app Cruncher
  • 15. Seasonal Adjustment Methods SA methods Generic modules: •Analysis •Seasonality tests •Revision analysis •Sliding spans •I/O (common xml schema) •Graphical components •Charts •SI ratios... Tramo-Seats, X12-Arima... Model-based decomposition (canonical decomposition, structural models...) Signal extraction tools: •Estimation •Analysis •Graphical components Specific modules (X11...) Other filters (X11...) REGARIMA pre-processing REGARIMA modules: •Common model •Estimation tools •Automatic modelling routines •Analysis tools (residuals, forecasts...) •Graphical components
  • 21. Plug-ins for Temporal Disaggregation
  • 22. The Seasonal Adjustment Centre of Excellence • A “joint venture” between Eurostat, NSIs and CBs • SACE: Belgium, France, Italy, Latvia, Portugal, UK • Partners: Eurostat, BBk, OECD, ECB, IMF • SAUG: Partners + Denmark, Finland, Hungary, Lithuania, Luxembourg, Romania, Slovakia, Slovenia, Spain • 3-year contract: 03/10/2016 – 02/10/2019 22
  • 23. On the Agenda • Knowledge sharing and Dissemination • Support to SA practitioners • User Group, Helpdesk, Documentation • Pre- release validation testing (JD+ 2.2.0 in 2017) • Plug-ins • Benchmarking, Quality, Analysis of revisions, Weekly and Daily data • Training, coaching and consultancy. 23
  • 24. Who is using JD+? A difficult question: Downloads are anonymous Helpdesk Latvia, Ireland, Czech Republic, Fyrom, Finland, Germany, Greece, Hungary, Portugal, Spain, France, Italy, Malta, Serbia, Luxembourg, Romania, Cyprus, Austria, Denmark, Iceland, Netherlands, Norway, Slovenia, Sweden, Switzerland, Turkey But many other users: ECB, Eurostat, IMF, Algeria, Cameroon, Maroc, Senegal, Tunisia, UEMOA etc. 24
  • 25. 25 User Support and Documentation The latest JDemetra+ released version can be downloaded at https://github.com/jdemetra/jdemetra-app/releases Java SE 8 or later versions are required. Modules, code and developers documentation and GitHub • https://github.com/jdemetra for the official modules • https://github.com/nbbrd for NBB resources All user documentation (JDemetra+ Quick Start, JDemetra+ User Guide, JDemetra+ Reference Manual) can be found here. In addition, beyond seasonal adjustment, the following prototype plug-ins are available for: • Temporal Disaggregation and Benchmarking • Quality/ Validation reporting • Revision Analysis • Nowcasting • Using JDEMETRA+ with R
  • 26. JDEMETRA+ Helpdesk and trainings More info about the SACE http://ec.europa.eu/eurostat/cros/content/seasonal-adjustment- centre-excellence_en For your Helpdesk queries http://ec.europa.eu/eurostat/cros/content/ess-seasonal-adjustment- helpdesk_en For training opportunities visit http://ec.europa.eu/eurostat/web/ess/about-us/estp
  • 27. Why using JDEMETRA+? 1. Implementation of ESS Guidelines on SA 2. Open source (JAVA) 3. Documented 4. Maintenance ensured (GITHUB) 5. Both x-12 and TRAMO/Seats 6. User Friendly 7. Users/Developers community (you will never walk alone!) 8. Helpdesk support 9. Trainings available 10. Free
  • 28. …by the why who was Demetra? In Greek mythology it was believed that Demetra was the goddess of corn, grain, harvest, agriculture, and of fertility in general. She made the crops grow each year so was intimately associated with the seasons. Her daughter Persephone was abducted by Hades to be his wife in the underworld. In her anger at her daughter's loss, Demetra caused plants to die and the land to become desolate. Zeus, alarmed for the arid earth, sought for Persephone's return. However, because she had eaten while in the underworld, Hades had a claim on her. Therefore, it was decreed that Persephone would spend four months each year in the underworld. During these months Demetra would grieve for her daughter's absence, withdrawing her gifts from the world, creating winter. Her return brought the spring.
  • 29. 29 Dominique Ladiray INSEE, France dominique.ladiray@insee.fr Thank you for your attention Dario Buono Eurostat, European Commission Dario.Buono@ec.europa.eu @darbuo
  • 30. Some more info about the architecture and the State Space Framework 30
  • 31. Algorithmic libraries (jtstoolkit) 31 Matrix computation Basic data handling Complex, polynomials Linear filters Function optimization Time series, calendars, regression variables... Basic statistics Utilities... Basic econometrics Arima, Ucarima VAR, Dynamic factor model Seats X11 State space framework Arima modelling RegArima Tramo Seasonal adjustment Structural models... Benchmarking, temporal disaggregation
  • 32. REGARIMA modelling • Common definitions for Calendar variables, outliers, intervention variables, user variables... • Algorithms for likelihood estimation • Kalman filter (Tramo-like), • Ansley algorithm (Cholesky on banded matrix) • (modified) Ljung-Box algorithm (X12-like) • Equivalent results, different performances • JD+ uses Kalman filter • Up to 4 x faster than Ljung-Box • Ansley in specific cases (outliers detection) • Optimization procedure • Levenberg-Marquardt. Tramo-Seats, X12 and JD+ use slightly different variants. Model building (Reg. variables) Estimation of the model (likelihood, residuals) Estimation of the parameters (by ML)
  • 33. Automatic model identification • Independent blocks (dynamically modifiable) • Specific implementation for Tramo-Seats, X12 Example: X12 modelling with outliers detection from Tramo Log/level Pre-test (seasonality...) Calendar effects... Outliers detection Final estimation Arima (diff. / Arma) Models comparison Model validation
  • 34. Algorithms for signal extraction in JD+ • Wiener-Kolmogorov filters • Burman's algorithm, Maravall's analysis framework (Seats) • Kalman smoother • Koopman's initialization procedure (disturbance or ordinary smoother) • Matrix computation • McElroy 's formulae • Can be applied to any (valid) UCARIMA model • Results • Estimates: identical • Standard deviations: WK approach yields negligible differences (exception: quasi- unit roots in MA polynomial → large differences) 34
  • 35. State space framework • Key solution for: • REGARIMA estimation • Signal extraction (Kalman smoother) • Alternative time series modelling (for SA or not) • Structural models... • Benchmarking • Cholette (including multi-variate extension) • Temporal disaggregation • Chow-Lin, Fernandez... • Multi-variate models • VAR, dynamic factor models, SUTSE… • JD+ provides an advanced implementation of SSF
  • 36. State space framework (II) Models Atomic models: • Generic (time invariant or not) • Ar(i)ma • Ucarima • Basic structural • White noise • Random walk • ... Derived models: • Composite • Regression variables • Aggregation constraints • ... Algorithms Filtering: • Ordinary filter • Fast filter (Chandrasekhar) • Array filter (Kailath...) Diffuse initialization: • Koopman • Square root • Ad hoc Smoothing: • Ordinary • Disturbance • Fixed point Others: • Univariate handling of multi-variate models • Augmented Kalman filter (for reg. model) • Extended Kalman filter (for non linear models) • ... Likelihood evaluation: Prediction error decomposition