Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DSD-INT 2020 Web based online Forecast Verification Tool - Zijderveld

13 views

Published on

Presentation by Annette Zijderveld, Rijkswaterstaat, at the Delft-FEWS International User Days 2020, during Delft Software Days - Edition 2020. Thursday, 5 November 2020.

Published in: Software
  • Be the first to comment

  • Be the first to like this

DSD-INT 2020 Web based online Forecast Verification Tool - Zijderveld

  1. 1. 11 Delft-FEWS User Days 5th November 2020 Web-based online Forecast Verification Tool Annette Zijderveld Water Management Center of RWS Peter Gijsbers Deltares
  2. 2. 2 Outline (1) WHY do we want this tool ? (2) WHAT will the tool do? (3) HOW is it build? (4) HOW will it look like? (5) Want to join?
  3. 3. 3 WHY • Water management centre of the Netherlands responsible for daily forecasts of water levels, discharge, tides, waves and currents for all water systems • Higher uncertainties in extreme events Drought High Water Environmental issues
  4. 4. 4 WHY • 7 different Delft-FEWS based operational systems running, 30+ different models based on 10+ different Meteo products • Different groups of forecasters engaged @RWS, some groups daily, some groups only during storm surge/ flooding situations
  5. 5. 5 WHY • Models are changing over the years, characteristics might change unnoticed, and extreme events can make models behave very differently • Validation and verification reports necessary, but by definition often outdated • Lot’s of handwork required to keep track – we can do this better! • In operation we want to have easy access not only to model accuracy information, but also reliability information of the whole operational system (process).
  6. 6. Verification procedures WHY (1) manually compute verification metrics from recreated hindcasts (2) manually compute verification metrics from archived observations and forecasts (3) continuous computation and archiving of predefined verification metrics (4) continuous detection and archiving of observation-forecast pairings, compute verification metrics at moment needed (5) continuous computation and archiving daily verification metric increments, aggregate verification statistics as needed 1 2 3 4 5 flexibility cost(effort,time,volume)
  7. 7. 7 WHAT • ARCHIVE all information and make it available for web services! • Show statistical parameters on ensemble forecast quality • Show run time estimates of all scheduled model runs – flag if runs are being delayed • Show availability and quality information of all observed parameters/ locations will the tool do? • Show statistical parameters of deterministic model runs and forecasts (2020)
  8. 8. 8 WHAT • Show statistical quality parameters of model performance by using conditional filtering • ‘Verification-as-a-service’ by calculating on-the-fly statistical parameters on demand • Make it very easy to keep track of our forecast quality – under all conditions and for all models and systems (without digging in excel each time …) will be possible in future?
  9. 9. Verification procedures WHY (1) manually compute verification metrics from recreated hindcasts (2) manually compute verification metrics from archived observations and forecasts (3) continuous computation and archiving of predefined verification metrics (4) continuous detection and archiving of observation-forecast pairings, compute verification metrics at moment needed (5) continuous computation and archiving daily verification metric increments, aggregate verification statistics as needed 1 2 3 4 5 flexibility cost(effort,time,volume)
  10. 10. 10 HOW current manual process (2) Ensemble Verification System verification metrics (images) PI-Webservice forecast ts observed ts Import database Fews Configuration Python scriptsMatroos Archive
  11. 11. 11 HOW automation being implemented (3) verification metrics (ts) PI-Webservice Performance Module External Storage NetCDF Archive (Matroos 2.0) FSS Archive Export bias rmse stddev Web visualisation metrics ts
  12. 12. 12 HOW automation development being considered (3) EVS-wrapper verification metrics (ts, xml) PI-Webservice General Adapter External Storage NetCDF Archive (Matroos 2.0) FSS Archive Export bias rmse stddev brier prob.skill sc. cont.ranked prob.ss rel.oper.char. EVS Web visualisation metrics ts, other xml
  13. 13. 13 HOW future automation to be investigated (4) EVS-wrapper obs-forecast pairings (‘ts’) PI-Webservice General Adapter External Storage NetCDF Archive (Matroos 2.0) FSS Archive Export EVS metrics calculator ? Web visualisation Transformation
  14. 14. 14 HOW future automation to be investigated (5) EVS-wrapper verification metric increments (ts, xml) PI-Webservice General Adapter External Storage NetCDF Archive (Matroos 2.0) FSS Archive Export bias rmse stddev brier prob.skill sc. cont.ranked prob.ss rel.oper.char. EVS metrics aggregator Web visualisation
  15. 15. • Live demo • Prototype 15 HOW will it look like?
  16. 16. 16 → Lead time Performance matrix →Locations Parameter RMSE shown in matrix for different locations/ leadtimes and 1 source Axis and parameters flexible to choose by user
  17. 17. 17 →RMSEonwaterlevel Statistics → Models / leadtime Parameter RMSE shown in graphic for different models/ lead-time for 1 location Axis and parameters flexible to choose by user
  18. 18. 18 →Date Availability model runs →Modelrunsincl.meteoforcing Model runs delayed? Can I wait for the next run before I will issue the warning? →Time Information on model run
  19. 19. 19 Availability of observations for data assimilation Observation are checked that are essential for data assimilation in the North Sea model
  20. 20. • What are you doing with verifications and why? • How could we help each other with this development? • Any ideas about what we are missing? 20 Want to join? We love to hear from you!
  21. 21. 21 contact Annette Zijderveld Water Management Center of RWS Annette.zijderveld@rws.nl Peter Gijsbers Deltares Peter.gijsbers@deltares.nl Please contact us if you want to have a live demo!

×