Evaluating G2G for use in Rapid Response Catchments: Final Report

Flood impacts can be severe for rapid response catchments (RRCs). Providing targeted flood warnings is challenging using existing methodologies and on account of the typical absence of river flow gauging. The Pitt Review of the Summer 2007 floods recognised the need for new alert procedures for RRCs able to exploit the new distributed flood forecasting capability being progressed from research into operations. Work on the G2G (Grid-to-Grid) distributed hydrological model was accelerated into operational practice to support 5-day countrywide flood outlooks, a major recommendation of the Pitt Review. The present study aims to explore the potential of G2G to support more frequent and detailed alerts relevant to flood warning in RRCs. Integral to this study is the use of emerging rainfall forecast products, in deterministic and ensemble form, which allow the lead-time of G2G flow forecasts to be extended and given an uncertainty context. This Report sets down the overall scope of the project, provides an introduction to G2G by way of background and then reports on the outcomes of the R&D study. This includes extensive preparatory work on collating historical datasets to support G2G model assessment, both relating to hydrometry and new rainfall forecast products. A framework is developed for assessing G2G in both simulation-mode and forecast-mode (as a function of lead-time) targeted at the RRC requirement. Relevant to the requirement is the RRC Register of points and areas of interest compiled by the Environment Agency, and the characteristics of RRCs (occurring in isolation or in combination): small catchment area, urban/sub-urban land-cover and steep slopes. The assessment framework is first applied assuming perfect knowledge of rainfall observations for past and future times, so as not to confound the analysis with errors from rainfall forecasts. Variability of performance measures across groups of sites is summarised through box and whisker plots, groups being differentiated on size of catchment area and nature of G2G run (simulation, and with the addition of state updating and flow insertion in turn). Skill scores judge how well the model performs in detecting a flood event exceeding a flow threshold, taken as the median annual flood (as an indicator of bankfull flow exceedance for natural channels) and fractional multipliers of it. The skill scores include POD (Probability of Detection) and FAR (False Alarm Ratio). Performance maps of R2 Efficiency, indicating the variability in the observations accounted for by the model, are used to portray the spatial variability of G2G accuracy across the country. G2G performance in small catchments, relevant to the RRC requirement, is best over South West, North East and North West regions; also median performance appears robust from one year to the next. Larger catchments benefit most in forecast-mode from flow insertion, whilst smaller headwater catchments gain particularly from ARMA (AutoRegressive Moving Average) error-prediction. An assessment is made of using deterministic rainfall forecasts from NWP UKV - the Numerical Weather Prediction UK Variable Resolution form of the Met Office Unified Model - in a full emulation of G2G in real-time, and using foreknowledge of rainfall observations as a reference baseline. Forecast quality can deteriorate strongly beyond 12 hours, especially for smaller catchments, whilst for some locations good performance is maintained even for long lead-times. Diagnostic analysis reveals that the UKV rainfall forecasts have patterns of overestimation in some lowland areas (e.g. over London) and leeward of high elevation areas (e.g. north and south Pennines). Overall performance is better in Scotland although there is evidence of UKV overestimating rainfall near the coast at Edinburgh and Elgin in the north. The assessment framework is extended to include rainfall forecast ensembles and probabilistic flood forecasting, using a combination of case-study and longer-term analyses. Blended Ensemble rainfall forecasts are assessed in two forms: forecasts out to 24 hours updated 4 times a day, and nowcasts out to 7 hours updated every 15 minutes. The 24 hour forecasts generally perform well as input to G2G in the case studies, the G2G flow forecasts typically signalling a flood peak 12 to 18 hours in advance and ahead of any observed response for small catchments. New regional summary map displays of the probability of flow threshold exceedances over a forecast horizon, and for increasing levels of severity, are developed to highlight evolving hotspots of flood risk over time. The first ever continuous assessment of G2G probability flow forecasts is reported using national maps of probabilistic skill scores - Relative Operating Characteristic (ROC) Skill Score and Brier Skill Score (BSS) - to spatially assess their performance. It is noted that the short periods available for assessment - a 7½ month period over England & Wales and 4 ½ months over Scotland - limit the analyses to low return period flow thresholds. Half the median (2-year) flood is used although a regional pooled analysis allows some assessment up to 5-year. The G2G probability forecast assessed is the probability of the chosen flow threshold being exceeded at any time over the forecast horizon (taken to be 24 hours). Comparison of these scores when applied to deterministic and probabilistic forecasts from G2G provides strong evidence of the value of G2G ensemble forecasts as an indicator of flood risk over Britain. Noticeably poorer performance indicated by the BSS across Scotland is in part attributed to the short, summer-dominated assessment period. Operational tools available to FFC and the SFFS for using G2G flow ensembles are reviewed and options for improvement identified drawing on the experience and findings of the study. This leads to identifying some work of an operational nature for consideration in Phase 3 of the project. The report closes with a summary of project achievements grouped thematically, a set of recommendations both of a general nature and specific to FFC and SFFS needs, and finally some proposals for consideration under Phase 3 of the G2G for Rapid Response Catchments project. Some key benefits arising from the project are summarised below. • Evidence has been produced that shows G2G has good skill in providing strategic forecasts for RRCs. The evidence is stratified by catchment type (area, urbanisation, headwater), form of forecast (simulation or forecast mode) and nature of rainfall input (raingauge, deterministic forecast, ensemble forecast). • Strong evidence has been presented on the advantage of using an ensemble rainfall forecast as input to G2G to obtain a probabilistic flood forecast for an RRC, relative to an approach where only a single deterministic rainfall and flood forecast is obtained. This indicates better guidance can be given on forecast flood risk for RRCs, improving the level of service provision for such catchments which are currently not well served. • An improved G2G model configuration, exploiting gauged flows from 912 sites and including new locally calibrated parameters, has been delivered and made operational for the FFC with England & Wales coverage. The benefit is improved operational flood forecast accuracy. For Scotland, an enhanced configuration will be delivered to SFFS in Spring 2014. • Detailed recommendations on how the visual presentation of G2G ensemble results could be improved are set down in this report. When further developed and implemented, these will prove of benefit to the preparation of Flood Guidance Statements issued by FFC and the SFFS across Britain

Surface Water Flooding Component for NHP HIM: Phase 1 Report

Following recent UK floods (summer 2007 and Cornwall 2010) and the UK Government’s Pitt Review, forecasting and warning of surface water flooding (SWF) has received much attention. To support effective mitigation actions there is a growing demand for more robust, accurate and timely forecast and alert information on surface water flooding and its impacts at local, regional and national scales. The Natural Hazards Partnership (NHP) aims to provide co-ordinated information on natural hazards from across UK government departments, agencies, trading funds and public sector research establishments. Under NHP, a Hazard Impact Model (HIM) framework is being developed and SWF is one of three initial hazards being trialled. This report on the SWF HIM outlines the plan of work and reports on Phase 1, which developed a proof-of-concept of a new approach against a background review of existing operational capabilities. In contrast to rainfall threshold based methods in present use, the new NHP prototype SWF HIM approach builds on the dynamic gridded surface runoff estimates from the Grid-to-Grid (G2G) hydrological model already employed for operational fluvial flood forecasting across Britain. Methodologies for generating dynamic maps of the possible impact are derived using national datasets of population, infrastructure, property and transport. An initial prototype approach is explained and illustrated through a case study. This is then further developed to exploit detailed information contained in the updated Flood Map for Surface Water (uFMfSW), and derived offline using design storm “effective rainfalls” as input to a surface water inundation model configured at 2m resolution. Maximum depth, velocity and hazard on a 2m grid for the Critical Storm Duration (CSD) are available for three return periods, and these are linked to impacts using the national datasets. The link to G2G surface runoff (at 1km resolution) is established by equating this to the “effective rainfall” (at 250m resolution) which facilitates a link to inundation information (at 2m resolution) and the assessment of impact at this scale. The prototype SWF HIM has assumed a fixed CSD of 3 hours as this information was not readily available within the uFMfSW. This assumption was investigated by obtaining the CSD dataset for one tile encompassing Newcastle: this showed 1 hour to be more typical, especially for higher return periods. However, the variable nature of CSD across 1, 3 and 6 hours suggests that ideally this should be accommodated in a revision of the prototype approach. If the resulting impact calculations prove too onerous, then a compromise to consider is to employ a fixed CSD of 1 hour instead of 3 hours. Four end-to-end case studies using the prototype SWF HIM are developed, the first employing foreknowledge of radar rainfall observations as deterministic input to the G2G hydrological model. The remaining three use blended ensemble rainfall forecasts as input to G2G as an offline close emulation of operational system conditions. This allows SWF hazard and impacts to be assessed - in a probabilistic, risk-based context aligned to the Flood Risk Matrix - when used in combination with the uFMfSW information and an Impact Library developed by the Health and Safety Laboratory. A comparative assessment, although not completely “like-for-like”, with the FFC’s operational SWF Decision Support Tool served to provide initial evidence on the potential advantages of the new prototype SWF HIM approach. A perspective on future developments is given, including planning for a real-time end-to-end trial using computer platforms employed by the Flood Forecasting Centre

Understanding the performance of flood forecasting models for investment and incident management: Interim Report - SC130006/R1.

Understanding the performance of flood forecasting models - operated in real-time by the Environment Agency, Natural Resources Wales and the Flood Forecasting Centre - is essential to their informed use for flood guidance across England & Wales. It is also essential to guide future investment in flood incident management. This Interim Report presents progress on a project which aims to design a methodology for assessing flood forecasting models. The methodology is to be implemented to assess local forecast models operated as regional systems alongside the G2G distributed national forecast model configured to have coverage over England & Wales. The scope of work is outlined and project activities reviewed. Activities include an Inception Planning Workshop and engagement with supplier consultants of local forecast model reports on delivery of the underpinning datasets for use in the project. Review of a draft Framework and Proforma for assessment, proposed by the EA, provided useful guidance on requirements but identified the need for a fresh approach to implementation. This would focus on the collation of datasets of observed and forecast flow/level time-series, and not the performance measures themselves. The resulting database could then be subject to a consistent performance analysis producing agreed measures and outputs on performance, and open to change and addition to meet future requirements for assessment. Consideration of the forms of performance assessment to apply are discussed and reported on as an ongoing activity. This work aims to design and trial the overall performance assessment procedure, to agree the approach with the Project Board and then apply it to all datasets once received. The statistical methodology for performance assessment to be used is outlined, initially focussing on the calculation of categorical skill scores using an “all (available) forecast” approach rather than one that is solely “event-based”. A prototype form of Performance Summary applicable to a given model and site - including site/model information, performance metric and event-hydrograph displays, and performance grading – is developed and trialled as a proposed way forward. Some remaining issues are raised for discussion. The Interim Report ends with a summary of achievements to date and an outline of Next Steps

Understanding the performance of flood forecasting models for investment and incident management: Final Report - SC130006/R2.

Understanding the performance of flood forecasting models - operated in real-time by the Environment Agency, Natural Resources Wales and the national Flood Forecasting Centre - is crucial to their informed use for flood guidance across England & Wales. It is also essential to guide future strategic investment in flood incident management. This report presents the results from the first nationwide analysis of flood forecasting model performance across implementations by local centres of the National Flood Forecasting System (NFFS). It considers regional and model-type differences and presents an overview of the current forecasting capability of models in operational use. Previous forecast performance studies of local models were performed on a grouped-catchment or regional basis and not necessarily using a consistent assessment framework. Spatial analysis of flood forecasting model performance in this report is based on Wales and the English geographical regions that align to the old Environment Agency region names. The report also extends the forecast model performance information available for the Grid-to-Grid (G2G) model, a distributed grid-based hydrological model with rainfall-runoff and flow routing elements. G2G is implemented within the NFFS for the Flood Forecasting Centre as an area-wide national model across England & Wales. The G2G model forecasts are compared with those from the local models which span a variety of model-types: rainfall-runoff models of conceptual and transfer function form, and channel flow routing models of hydrological and hydrodynamic form. The approach taken to performance assessment has been to gather “raw” data (river flow observations, flow forecasts and historical simulation of flows) from previous local model performance studies. While there are significant regional differences in how these data were gathered and in the methodologies used to generate flood forecast model outputs, collation of the underlying “project” datasets has allowed standardisation of the methods of assessment used here. This report presents the background to a flood forecasting model “Performance Summary”, as a template for reporting performance at any site from a given model-type, including the underlying performance measures employed. The Performance Summary takes the form of one A4 page for each model at each site, and contains a variety of different performance measures and graphical displays. Just under 1,800 Performance Summary pages have been produced for those working in an operational setting or in strategic planning. The results contained in the Performance Summary for each site and model combination have been brought together and used as the basis of a national analysis and summary. This constitutes an extensive national evidence base of model performance, stratified by model-type, model-group, geographical region and lead-time. Where there is a choice of model forecast, it also includes information on comparative performance. Recommendations and important findings • There is a need for standardisation across local flood forecasting model performance (FFMP) methodologies. For example, models should use the same rainfall scenarios and the model output time-interval should be standardised to 15 minutes. • The creation of a national FFMP database, with a well-defined submission format and quality control, is urged. All new FFMP studies should be required to provide the project dataset in this standard format. • There is also a need for models that perform better at longer lead-times, particularly for the South West and parts of the North West where Physically Realisable Transfer Function (PRTF) models are used. • On a regional basis, the North East and North West have the strongest forecasting performance. Anglian, Thames and Southern regions do less well. This demarcation is to be expected because of the much more challenging hydrological conditions (flatter catchments, groundwater, river management) in the south and east of England. • Compounded regional and model differences mean that it is not generally possible to know whether one local model outperforms another. Nevertheless, the best performing models revealed by this study are usually the Probability Distributed Model (PDM), the extended Kinematic Wave (KW) model and the hydrodynamic river model ISIS. • As would be expected, the national G2G model performs less well than local models at a number of sites. It also performs better than some local models and for some regions especially at longer lead-times. • Comparison of G2G with local models on a model-by-model basis shows huge variability. For example, on average PDM models outperform G2G, but there are many sites for which the opposite is true. • Local models require at-site calibration and provide forecasts only for this site when they are of the rainfall-runoff type and for locations along a gauged river reach when of the river model type. G2G provides forecasts everywhere across its model domain (the non-tidal river basins of England and Wales) at a 1km grid resolution. In addition, G2G gives a spatially coherent picture of flood evolution, in contrast to the gauging station specific forecasts produced by local rainfall-runoff models. • The Performance Summary provides, on a single A4 page, a concise summary of model performance along with hydrometric details for a given site and model. It should be made available to operational users of the NFFS via the tooltip functionality of NFFS map displays. • An extensive national evidence base of model performance has been created, stratified by model type, model group, geographical region and lead-time. It also includes information on comparative performance where a choice of model forecast exists. This evidence base is seen as of particular value for strategic planning relating to investment in flood forecasting models. • Accessing, viewing and assessing the wealth of model performance information in appropriate ways is challenging and is dependent on the intended use, for example, in supporting real-time decision-making or in guiding offline strategic investment decisions. Interactive and easily accessible methods of viewing the performance information should be considered, such as that offered by the prototype FFMP web portal being developed by the Centre for Ecology & Hydrology. • The Performance Summary framework was designed to be readily refreshed to include new datasets from consultants as they become available or are commissioned. Recommendations are made to make this process more efficient and the model assessments more meaningful and useful, both in incident management and for guiding strategic investment in flood forecasting models