Seminario técnico de las becas de formación de posgraduados de Aemet celebrado el 3 de junio de 2013. Proyecto: Métodos avanzados de post-proceso y presentación de productos de Sistemas de Predicción por Conjuntos para el Corto Plaz

Amo-Baladrón, Arancha

English

Agencia Estatal de Meteorología

Python interface for object-oriented  verification methods: SAL Arancha Amo Baladrón Proyect 13: Postprocess advanced methods and presentacion of products of ensemble predictions for the short range  Development and Applications Department  (Madrid) Tutor: Carlos Santos Burguete 03/06/2013 AEMET, Agencia Estatal de Meteorología 1 OUTLINE  • New verification methods. Motivation. • SAL (Structure-Amplitude-Location). • Software design: python interface + fortran90 • Verified periods. • Summary.      03/06/2013 AEMET, Agencia Estatal de Meteorología 2 TRADITIONAL Vs. OBJECT-ORIENTED VERIFICATION METHODS   Traditional methods:  •Measures-oriented: matching forecast grid to an observation grid or set of points.       Counts of fc-obs pairs (y/n)  contingency table  verification scores (RMSE,POD,FAR,CSI,…).  •No diagnostic information. •No consistent with visual evaluation of forecast. •Insensitive to differences in location, shape, and timing errors   double penalty problem hit rate=0 but A is better!!! hit rate > 0 E right shape ??? OBSERVATIONS LOW RESOLUTION MODEL HIGH RESOLUTION MODEL Which forecast would you rather use? MSE, S, A, L = 0   perfect forecast  03/06/2013 AEMET, Agencia Estatal de Meteorología 3 DOUBLE PENALTY PROBLEM MSE = 6.8  S = 0.5 A = - 0.3 L = 0.08 MSE = 8.9   S = 0.07 A = 0.07 L = 0.03 RMSE Oct 2011 Jan 2013 ECMWF T1279  (16km) HARMONIE  (2.5km) OBSERVATIONS 15 Jan 2012 03/06/2013 AEMET, Agencia Estatal de Meteorología 4 NEW VERIFICATION METHODS:  SAL (Structure-Amplitude-Longitude)  •Object-oriented verification methods are especially suitable for:               high-resolution forecasts               highly localized and episodic phenomena  rainfall •Based on the identification of coherent and contiguous entities with characteristic attributes   SAL does not compare attribute values of individual objects searching for matchs in obs and fc fields (e.g. MODE), but…  •Derives attribute values of individual objects. •Averages these values for obs and fc fields separately. •Calculates the difference between averaged values of the obs and fc fields.  S-shape and size      A-total amount of pcp       L-pcp distribution 03/06/2013 AEMET, Agencia Estatal de Meteorología 5 1 ) Upscale observations to model grid 2 ) Search of objects in forecast and observation fields       separately:           Thresholding: subjetive task (visual inspection)                Rmax=P95                 f=1/15           Clustering: selection of grid points belonging to an           object (8 neighbors). 3 ) Object properties:      Rn →  total pcp of the object      xn  →  MC       Vn →  scaled volume or total pcp of the object normalized                   by its max value 4 ) Field properties:      D → domain-average pcp value in the field       V → weighted mean of objects scaled volume in the field        x →  field MC      r →  weighted mean of the distance between objects MCs              and the field MC.       d → max distance between two points in the field  5 ) SAL parameters: normalized differences of field properties.      Structure  → size and shape of pcp objects       Amplitude → accuracy of the total amount of pcp       Location   → accuracy of the pcp distribution and relative                                positions of objects in the field. SAL: PROCEDURE  S, A, L Observations Forecast Upscaling AEMET termopluvio network (3000 stations) 03/06/2013 AEMET, Agencia Estatal de Meteorología 6 SSDM dataBase AEMET pluvioTermo  Network SAL SAL plot Objects maps CRONTAB monthly retrieval lag: 4 months  filesAdmin ssdm models  leadTimes 006, 030 model grbFile 24hAccumPcp 030 - 006 Csv  Geo conversion obs. geoFile  24hAccumPcp observations obsHour 07:00 upscaled  obs. grbFile UPS Grid template (model file) repeat N days python fortran90 verification │ PyDESIGN  03/06/2013 AEMET, Agencia Estatal de Meteorología 7 Code doc. Requirements Brief SAL description Example of usage CODE DOCUMENTATION SAL PLOT 03/06/2013 AEMET, Agencia Estatal de Meteorología 8 A single verification Model forecasts ... MEDIAN STRUCTURE AMPLITUDE IQR LOCATION False Alarms Missing Events False Alarms  Corner 03/06/2013 AEMET, Agencia Estatal de Meteorología 9 FORECAST  OBSERVATION PCP MAPS FOR EACH VERIFICATION  OBJECT MAPS date          coMOx  coMOy  coOBx  coOBy …  ME(bias) MSE  20111001  346.65  144.47   337.07  174.37 …  0.006    1.034 20111002  309.40  112.80   308.45  213.23 …  -0.031   0.600 20111003  118.58  248.71   346.98  248.85 … -0.001    0.001 20111004  227.01  186.42  102.23   298.55 …  0.015    0.071 20111005  229.13  311.49    87.69   344.59 … -0.000    0.007  20111006  214.25  328.49  225.82   331.65 … -0.004    2.144 … SAL RESULTS SAL RESULTS FILE 03/06/2013 AEMET, Agencia Estatal de Meteorología 10 S = 1.3 A = 1.0 L = 0.3 S = 0.09 A = -0.09 L = 0.05         S = 1.3 A = -0.8 L = 0.5     S = -1.3 A = -1.5 L = 0.5      MODEL OBSERVATION HARMONIE (2.5km) Oct 2011 Jan 2013 ECMWF T1279 (0.125º) HNR (0.05º) HARMONIE (2.5km) VERIFIED PERIODS 03/06/2013 AEMET, Agencia Estatal de Meteorología 11 2007 2010 ECMWF T799 (0.25º) ONR (0.16º) HNR (0.05º) XXXII Jornadas Científicas  de la AME GRID INTERPOLATION  HARMONIE (2.5km) S = 0.008 A = 0.03 L = 0.12  S = -0.09 A = 0.03 L = 0.13 Oct2011-Jan2013 S changes slightly  objects in model field are smoothed due to interpolation  03/06/2013 AEMET, Agencia Estatal de Meteorología 12 REGULAR LATLON GRID LAMBERT GRID MODEL COMPARISON S =  0.006 A =  0.04 L =  0.12 HARMONIE (2.5km) ECMWF T1279 (0.125º) S = 0.8 A = 0.14 L = 0.10  S = 0.5    A = 0.5 L = 0.17 HNR (0.05º) Oct2011-Jan2013 - S  improvement at higher resolutions - A  not correlated with resolution.  - L  without significant variability with resolution 03/06/2013 AEMET, Agencia Estatal de Meteorología 13 1 False Alarm 1 Missing Event MODEL COMPARISON S =  -0.11 A =  0.12 L =  0.21 HARMONIE (2.5km) CE T1279 (0.125º) S = 0.9 A = 0.22 L = 0.25  S = 0.7    A = 0.8 L = 0.25 HNR (0.05º) Oct2011-Jan2013: subdomain CVBM SAL parameters present higher values but same tendencies as in the whole domain 03/06/2013 AEMET, Agencia Estatal de Meteorología 14 17 False Alarms 13 Missing Events 13 FA 0 ME  10 FA 14  ME Seasonal variability 2011/12 HARMONIE (2.5 km)  03/06/2013 AEMET, Agencia Estatal de Meteorología 15 MAM JJA SON DJF MAM JJA SON DJF HARMONIE (2.5 km) S A L -0.03 0.11 0.11 -0.06 0.3 0.14 0.002 0.09 0.12 0.06 -0.07 0.14 HNR (0.05º) S A L 0.4 0.6 0.14 1.1 0.9 0.20 0.7 0.4 0.15 0.5 0.6 0.14 ECMWF T1279 (0.125º) S A L 0.8 0.24 0.10 1.0 0.5 0.10 0.8 0.009 0.11 0.9 0.20 0.14 • SAL software framework finished:      - Upscaling observations to grid models.      - Retrieving forecast fields from SSDM AEMET database.      - Providing SAL plots + objects maps for single verifications.      - Supports Lambert conformal projection (gribApi 10.0) and rotated grids.      - Allowing to generate SAL results for selected subdomains.      - Code documentated with Sphinx.  • Comparison of forecast quality using SAL method:       - 2007-2010: HIRLAM (HNR 0.05º,ONR 0.16º) and ECMWF T799 (25 km).      - Oct2011-Jan2013: HNR, ECMWF T1279 (16 km), and HARMONIE (2.5 km).  • SAL solves double penalty problem and provides quantitative, detailed, and explicit information about different aspects of forecast performance; allowing fair comparisons between models of different resolutions.      03/06/2013 AEMET, Agencia Estatal de Meteorología 16 SUMMARY 03/06/2013 AEMET, Agencia Estatal de Meteorología 17 CONTRIBUTIONS      • Oral contribution: Nuevos metodos de verificacion de las predicciones numericas de precipitacion: metodo SAL aplicado a la mesoescala. A. Amo and C. Santos.  XXXII Jornadas Científicas de la Asociación Meteorológica Española (AME).       28-30 de Mayo de 2012. Alcobendas, Madrid.  •    Influence of targeted observations on short-term forecasts of high-impact weather      events in the Mediterranean.      J. Campins, B. Navascués, C. Santos, and A. Amo  •    Long-term verification of HIRLAM and ECMWF forecasts over Southern Europe.      History and perspectives of Numerical Weather Prediction at AEMET.      B. Navascués et al. Atmospheric Research 125-126 (2013) 20–33              THANK YOU   BONUS   03/06/2013 AEMET, Agencia Estatal de Meteorología 0 • AMPLITUDE      A     Normalized difference of the domain-average pcp values between obs and fc fields.     Measure of quantitative accuracy of the total amount of pcp in the region.               A[-2,2]                 A = 0 total agreement                                           A > 0 model overestimates                                           A < 0 model underestimates     • LOCATION   L=L1+L2      L1 Normalized distance between the CM of the obs/fc pcp fields               First order indication of the accuracy of the pcp distribution        L2  Takes into account the average distance between the CM of the                 total pcp field and infividual pcp objects  relative positions of objects in the field.      L[0,2]  with L = 0  CM and average distance objects-CM are equal in obs and fc fields                                  AEMET, Agencia Estatal de Meteorología 1 SAL: measured aspects of forescast quality (I)  Not sensitive to  rotation around CM 03/06/2013 AEMET, Agencia Estatal de Meteorología 2 SAL: measured aspects of forescast quality (II)  • STRUCTURE     S      Compare the volume of the normalized pcp objects.       Information about size and shape of objects.       Individual object volume  total pcp of the object normalized by its max value.       A weighted mean of all objects pcp volume is calculated for obs and fc fields.                              S  normalized difference between obs and fc weighted mean volumes.       S[-2,2]       S >> 0   model predicts widespread pcp but observations show small convective events       S << 0   model predicts small and/or picked pcp objects compare to observations        03/06/2013 Seasonal variability 2011/12 HARMONIE (2.5 km)  MAM JJA SON DJF S = -0.03  A =  0.11 L =  0.11 S = -0.06  A =  0.3 L =  0.14 S = 0.002  A = 0.09 L = 0.12 S =   0.06  A = - 0.07 L =   0.14 03/06/2013 AEMET, Agencia Estatal de Meteorología 3 Seasonal variability 2011/12 ECMWF T1279 (0.125º)  MAM JJA SON DJF S = 0.8  A = 0.24 L = 0.10  S = 1.0   A = 0.5  L = 0.10  S = 0.8   A = 0.009 L = 0.11 S = 0.9    A = 0.20 L = 0.14   03/06/2013 AEMET, Agencia Estatal de Meteorología 4 Seasonal variability 2011/12 HNR (0.05º)  MAM JJA SON DJF S = 0.4   A = 0.6 L = 0.14 S = 1.1  A = 0.9  L = 0.20  S = 0.7   A = 0.4 L = 0.15 S = 0.5    A = 0.6 L = 0.14   03/06/2013 AEMET, Agencia Estatal de Meteorología 5 

Python interface for object-oriented verification methods: SAL

https://repositorio.aemet.es/bitstream/20.500.11765/3286/3/S_4_AranzazuAmo_2013.pdf

Python interface for object-oriented verification methods: SAL

Abstract

Similar works

Full text

Available Versions

Agencia Estatal de Meteorología