Multi-agent system for flood forecasting in Tropical River Basin

It is well known, the problems related to the generation of floods, their control, and management, have been treated with traditional hydrologic modeling tools focused on the study and the analysis of the precipitation-runoff relationship, a physical process which is driven by the hydrological cycle and the climate regime and that is directly proportional to the generation of floodwaters. Within the hydrological discipline, they classify these traditional modeling tools according to three principal groups, being the first group defined as trial-and-error models (e.g., "black-models"), the second group are the conceptual models, which are categorized in three main sub-groups as "lumped", "semi-lumped" and "semi-distributed", according to the special distribution, and finally, models that are based on physical processes, known as "white-box models" are the so-called "distributed-models". On the other hand, in engineering applications, there are two types of models used in streamflow forecasting, and which are classified concerning the type of measurements and variables required as "physically based models", as well as "data-driven models". The Physically oriented prototypes present an in-depth account of the dynamics related to the physical aspects that occur internally among the different systems of a given hydrographic basin. However, aside from being laborious to implement, they rely thoroughly on mathematical algorithms, and an understanding of these interactions requires the abstraction of mathematical concepts and the conceptualization of the physical processes that are intertwined among these systems. Besides, models determined by data necessitates an a-priori understanding of the physical laws controlling the process within the system, and they are bound to mathematical formulations, which require a lot of numeric information for field adjustments. Therefore, these models are remarkably different from each other because of their needs for data, and their interpretation of physical phenomena. Although there is considerable progress in hydrologic modeling for flood forecasting, several significant setbacks remain unresolved, given the stochastic nature of the hydrological phenomena, is the challenge to implement user-friendly, re-usable, robust, and reliable forecasting systems, the amount of uncertainty they must deal with when trying to solve the flood forecasting problem. However, in the past decades, with the growing environment and development of the artificial intelligence (AI) field, some researchers have seldomly attempted to deal with the stochastic nature of hydrologic events with the application of some of these techniques. Given the setbacks to hydrologic flood forecasting previously described this thesis research aims to integrate the physics-based hydrologic, hydraulic, and data-driven models under the paradigm of Multi-agent Systems for flood forecasting by designing and developing a multi-agent system (MAS) framework for flood forecasting events within the scope of tropical watersheds. With the emergence of the agent technologies, the "agent-based modeling" and "multiagent systems" simulation methods have provided applications for some areas of hydro base management like flood protection, planning, control, management, mitigation, and forecasting to combat the shocks produced by floods on society; however, all these focused on evacuation drills, and the latter not aimed at the tropical river basin, whose hydrological regime is extremely unique. In this catchment modeling environment approach, it was applied the multi-agent systems approach as a surrogate of the conventional hydrologic model to build a system that operates at the catchment level displayed with hydrometric stations, that use the data from hydrometric sensors networks (e.g., rainfall, river stage, river flow) captured, stored and administered by an organization of interacting agents whose main aim is to perform flow forecasting and awareness, and in so doing enhance the policy-making process at the watershed level. Section one of this document surveys the status of the current research in hydrologic modeling for the flood forecasting task. It is a journey through the background of related concerns to the hydrological process, flood ontologies, management, and forecasting. The section covers, to a certain extent, the techniques, methods, and theoretical aspects and methods of hydrological modeling and their types, from the conventional models to the present-day artificial intelligence prototypes, making special emphasis on the multi-agent systems, as most recent modeling methodology in the hydrological sciences. However, it is also underlined here that the section does not contribute to an all-inclusive revision, rather its purpose is to serve as a framework for this sort of work and a path to underline the significant aspects of the works. In section two of the document, it is detailed the conceptual framework for the suggested Multiagent system in support of flood forecasting. To accomplish this task, several works need to be carried out such as the sketching and implementation of the system’s framework with the (Belief-Desire-Intention model) architecture for flood forecasting events within the concept of the tropical river basin. Contributions of this proposed architecture are the replacement of the conventional hydrologic modeling with the use of multi-agent systems, which makes it quick for hydrometric time-series data administration and modeling of the precipitation-runoff process which conveys to flood in a river course. Another advantage is the user-friendly environment provided by the proposed multi-agent system platform graphical interface, the real-time generation of graphs, charts, and monitors with the information on the immediate event taking place in the catchment, which makes it easy for the viewer with some or no background in data analysis and their interpretation to get a visual idea of the information at hand regarding the flood awareness. The required agents developed in this multi-agent system modeling framework for flood forecasting have been trained, tested, and validated under a series of experimental tasks, using the hydrometric series information of rainfall, river stage, and streamflow data collected by the hydrometric sensor agents from the hydrometric sensors.Como se sabe, los problemas relacionados con la generación de inundaciones, su control y manejo, han sido tratados con herramientas tradicionales de modelado hidrológico enfocados al estudio y análisis de la relación precipitación-escorrentía, proceso físico que es impulsado por el ciclo hidrológico y el régimen climático y este esta directamente proporcional a la generación de crecidas. Dentro de la disciplina hidrológica, clasifican estas herramientas de modelado tradicionales en tres grupos principales, siendo el primer grupo el de modelos empíricos (modelos de caja negra), modelos conceptuales (o agrupados, semi-agrupados o semi-distribuidos) dependiendo de la distribución espacial y, por último, los basados en la física, modelos de proceso (o "modelos de caja blanca", y/o distribuidos). En este sentido, clasifican las aplicaciones de predicción de caudal fluvial en la ingeniería de recursos hídricos en dos tipos con respecto a los valores y parámetros que requieren en: modelos de procesos basados en la física y la categoría de modelos impulsados por datos. Los modelos basados en la física proporcionan una descripción detallada de la dinámica relacionada con los aspectos físicos que ocurren internamente entre los diferentes sistemas de una cuenca hidrográfica determinada. Sin embargo, aparte de ser complejos de implementar, se basan completamente en algoritmos matemáticos, y la comprensión de estas interacciones requiere la abstracción de conceptos matemáticos y la conceptualización de los procesos físicos que se entrelazan entre estos sistemas. Además, los modelos impulsados por datos no requieren conocimiento de los procesos físicos que gobiernan, sino que se basan únicamente en ecuaciones empíricas que necesitan una gran cantidad de datos y requieren calibración de los datos en el sitio. Los dos modelos difieren significativamente debido a sus requisitos de datos y de cómo expresan los fenómenos físicos. La elaboración de modelos hidrológicos para el pronóstico de inundaciones ha dado grandes pasos, pero siguen sin resolverse algunos contratiempos importantes, dada la naturaleza estocástica de los fenómenos hidrológicos, es el desafío de implementar sistemas de pronóstico fáciles de usar, reutilizables, robustos y confiables, la cantidad de incertidumbre que deben afrontar al intentar resolver el problema de la predicción de inundaciones. Sin embargo, en las últimas décadas, con el entorno creciente y el desarrollo del campo de la inteligencia artificial (IA), algunos investigadores rara vez han intentado abordar la naturaleza estocástica de los eventos hidrológicos con la aplicación de algunas de estas técnicas. Dados los contratiempos en el pronóstico de inundaciones hidrológicas descritos anteriormente, esta investigación de tesis tiene como objetivo integrar los modelos hidrológicos, basados en la física, hidráulicos e impulsados por datos bajo el paradigma de Sistemas de múltiples agentes para el pronóstico de inundaciones por medio del bosquejo y desarrollo del marco de trabajo del sistema multi-agente (MAS) para los eventos de predicción de inundaciones en el contexto de cuenca hidrográfica tropical. Con la aparición de las tecnologías de agentes, se han emprendido algunos enfoques de simulación recientes en la investigación hidrológica con modelos basados en agentes y sistema multi-agente, principalmente en alerta por inundaciones, seguridad y planificación de inundaciones, control y gestión de inundaciones y pronóstico de inundaciones, todos estos enfocado a simulacros de evacuación, y este último no dirigido a la cuenca tropical, cuyo régimen hidrológico es extremadamente único. En este enfoque de entorno de modelado de cuencas, se aplican los enfoques de sistemas multi-agente como un sustituto del modelado hidrológico convencional para construir un sistema que opera a nivel de cuenca con estaciones hidrométricas desplegadas, que utilizan los datos de redes de sensores hidrométricos (por ejemplo, lluvia , nivel del río, caudal del río) capturado, almacenado y administrado por una organización de agentes interactuantes cuyo objetivo principal es realizar pronósticos de caudal y concientización para mejorar las capacidades de soporte en la formulación de políticas a nivel de cuenca hidrográfica. La primera sección de este documento analiza el estado del arte sobre la investigación actual en modelos hidrológicos para la tarea de pronóstico de inundaciones. Es un viaje a través de los antecedentes preocupantes relacionadas con el proceso hidrológico, las ontologías de inundaciones, la gestión y la predicción. El apartado abarca, en cierta medida, las técnicas, métodos y aspectos teóricos y métodos del modelado hidrológico y sus tipologías, desde los modelos convencionales hasta los prototipos de inteligencia artificial actuales, haciendo hincapié en los sistemas multi-agente, como un enfoque de simulación reciente en la investigación hidrológica. Sin embargo, se destaca que esta sección no contribuye a una revisión integral, sino que su propósito es servir de marco para este tipo de trabajos y una guía para subrayar los aspectos significativos de los trabajos. En la sección dos del documento, se detalla el marco de trabajo propuesto para el sistema multi-agente para el pronóstico de inundaciones. Los trabajos realizados comprendieron el diseño y desarrollo del marco de trabajo del sistema multi-agente con la arquitectura (modelo Creencia-Deseo-Intención) para la predicción de eventos de crecidas dentro del concepto de cuenca hidrográfica tropical. Las contribuciones de esta arquitectura propuesta son el reemplazo del modelado hidrológico convencional con el uso de sistemas multi-agente, lo que agiliza la administración de las series de tiempo de datos hidrométricos y el modelado del proceso de precipitación-escorrentía que conduce a la inundación en el curso de un río. Otra ventaja es el entorno amigable proporcionado por la interfaz gráfica de la plataforma del sistema multi-agente propuesto, la generación en tiempo real de gráficos, cuadros y monitores con la información sobre el evento inmediato que tiene lugar en la cuenca, lo que lo hace fácil para el espectador con algo o sin experiencia en análisis de datos y su interpretación para tener una idea visual de la información disponible con respecto a la cognición de las inundaciones. Los agentes necesarios desarrollados en este marco de modelado de sistemas multi-agente para el pronóstico de inundaciones han sido entrenados, probados y validados en una serie de tareas experimentales, utilizando la información de la serie hidrométrica de datos de lluvia, nivel del río y flujo del curso de agua recolectados por los agentes sensores hidrométricos de los sensores hidrométricos de campo.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: María Araceli Sanchis de Miguel.- Secretario: Juan Gómez Romero.- Vocal: Juan Carlos Corrale

Explainable deep learning for insights in El Ni\~no and river flows

The El Ni\~no Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to describe a complex phenomenon and translating DL-based ENSO predictions to river flow predictions. Here we show that eXplainable DL (XDL) methods, based on saliency maps, can extract interpretable predictive information contained in global SST and discover SST information regions and dependence structures relevant for river flows which, in tandem with climate network constructions, enable improved predictive understanding. Our results reveal additional information content in global SST beyond ENSO indices, develop understanding of how SSTs influence river flows, and generate improved river flow prediction, including uncertainty estimation. Observations, reanalysis data, and earth system model simulations are used to demonstrate the value of the XDL-CN based methods for future interannual and decadal scale climate projections

A Machine Learning Approach for Improving Near-Real-Time Satellite-Based Rainfall Estimates by Integrating Soil Moisture

Near-real-time (NRT) satellite-based rainfall estimates (SREs) are a viable option for flood/drought monitoring. However, SREs have often been associated with complex and nonlinear errors. One way to enhance the quality of SREs is to use soil moisture information. Few studies have indicated that soil moisture information can be used to improve the quality of SREs. Nowadays, satellite-based soil moisture products are becoming available at desired spatial and temporal resolutions on an NRT basis. Hence, this study proposes an integrated approach to improve NRT SRE accuracy by combining it with NRT soil moisture through a nonlinear support vector machine-based regression (SVR) model. To test this novel approach, Ashti catchment, a sub-basin of Godavari river basin, India, is chosen. Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA)-based NRT SRE 3B42RT and Advanced Scatterometer-derived NRT soil moisture are considered in the present study. The performance of the 3B42RT and the corrected product are assessed using different statistical measures such as correlation coeffcient (CC), bias, and root mean square error (RMSE), for the monsoon seasons of 2012–2015. A detailed spatial analysis of these measures and their variability across different rainfall intensity classes are also presented. Overall, the results revealed significant improvement in the corrected product compared to 3B42RT (except CC) across the catchment. Particularly, for light and moderate rainfall classes, the corrected product showed the highest improvement (except CC). On the other hand, the corrected product showed limited performance for the heavy rainfall class. These results demonstrate that the proposed approach has potential to enhance the quality of NRT SRE through the use of NRT satellite-based soil moisture estimates

Quantifying ENSO impacts at the basin scale using the Iterative Input variable Selection algorithm

Medium-to-long range streamflow predictions provide a key assistance in anticipating hydro- climatic adverse events and prompting effective adaptation measures. In this context, recent modelling efforts have been dedicated to seasonal and inter-annual predictions based on the teleconnection between at-site hydrological processes and large-scale, low-frequency climate fluctuations, such as El Nino Southern Oscillation (ENSO). This work proposes a novel procedure for first detecting the impact of ENSO on hydro-meteorological processes at the basin scale, and then quantitatively assessing the potential of ENSO indexes for building medium-to-long range streamflow prediction models. Core of this procedure is the adoption of the Iterative Input variable Selection (IIS) algorithm, which is employed to find the most relevant determinants of streamflow variability and derive predictive models based on the selected inputs. The procedure is tested on two different case studies, the Columbia River (US) and the Williams River (Australia), whose sensitivity to ENSO fluctuations has been documented in previous studies. Results show that IIS outcomes for both case studies are consistent with the results of previous analyses conducted with state-of-the-art detection methods, and that ENSO indexes can effectively be used in both regions to enhance the accuracy of streamflow prediction models

EVALUATING THE PERFORMANCE OF PROCESS-BASED AND MACHINE LEARNING MODELS FOR RAINFALL-RUNOFF SIMULATION WITH APPLICATION OF SATELLITE AND RADAR PRECIPITATION PRODUCTS

Hydrology Modeling using HEC-HMS (Hydrological Engineering Centre-Hydrologic Modeling System) is accepted globally for event-based or continuous simulation of the rainfall-runoff operation. Similarly, Machine learning is a fast-growing discipline that offers numerous alternatives suitable for hydrology research\u27s high demands and limitations. Conventional and process-based models such as HEC-HMS are typically created at specific spatiotemporal scales and do not easily fit the diversified and complex input parameters. Therefore, in this research, the effectiveness of Random Forest, a machine learning model, was compared with HEC-HMS for the rainfall-runoff process. In addition, Point gauge observations have historically been the primary source of the necessary rainfall data for hydrologic models. However, point gauge observation does not provide accurate information on rainfall\u27s spatial and temporal variability, which is vital for hydrological models. Therefore, this study also evaluates the performance of satellite and radar precipitation products for hydrological analysis. The results revealed that integrated Machine Learning and physical-based model could provide more confidence in rainfall-runoff and flood depth prediction. Similarly, the study revealed that radar data performance was superior to the gauging station\u27s rainfall data for the hydrologic analysis in large watersheds. The discussions in this research will encourage researchers and system managers to improve current rainfall-runoff simulation models by application of Machine learning and radar rainfall data

Flood Hydrograph Modelıng Studıes By Usıng Gıs And Hec-Hms For Dakar, Senegal

Senegal’in başkenti olan Dakar İli sıcaklık değerlerinin yüksek olduğu yarı kurak bir iklime sahiptir. Dakar’ın yağış sezonu Temmuz-Ekim ayları arasındadır. Bu kısa periyotta yağan yağışlar, yerleşim bölgelerinde tahribata, can ve mal kayıplarına sebep olmaktadır. Tahribatın ve kayıpların asıl nedeni ani yağışlardan ziyade taşkın yataklarında inşa edilen konutlar ve yerleşim birimleridir. Doğal taşkın alanlarının tahrip edilmesi neticesinde taşkın kaynaklı felaketler oluşmaktadır. Taşkın kaynaklı felaketleri önlemek için doğal taşkın alanlarında koruma ve hafifletme çalışmaları yapılmalıdır. Bu çalışmada, Dakar havzası için hidrolojik modelleme çalışmaları yapılarak taşkın hidrografları oluşturulması amaçlanmıştır. Havza sınırlarının ve drenaj hatlarının belirlenmesinde Mekânsal Hidrolojik Modelleme Uzantısı (HEC-GeoHMS) yazılımı, hidrolojik modelleme çalışmalarında ise Hidrolojik Modelleme Sistemi (HEC-HMS) yazılımı kullanılmıştır. Çalışma neticesinde elde edilen hidrograflar yorumlanıp neticelerine yer verilecektir. Bunun yanı sıra model parametrelerinin Dakar taşkın önleme çalışmalarında kullanılabilir nitelikte olması amaçlamıştır

Hybrid machine learning model based on feature decomposition and entropy optimization for higher accuracy flood forecasting

The advancement of machine learning model has widely been adopted to provide flood forecast. However, the model must deal with the challenges to determine the most important features to be used in in flood forecast with high-dimensional non-linear time series when involving data from various stations. Decomposition of time-series data such as empirical mode decomposition, ensemble empirical mode decomposition and discrete wavelet transform are widely used for optimization of input; however, they have been done for single dimension time-series data which are unable to determine relationships between data in high dimensional time series.  In this study, hybrid machine learning models are developed based on this feature decomposition to forecast the monthly water level using monthly rainfall data. Rainfall data from eight stations in Kelantan River Basin are used in the hybrid model. To effectively select the best rainfall data from the multi-stations that provide higher accuracy, these rainfall data are analyzed with entropy called Mutual Information that measure the uncertainty of random variables from various stations. Mutual Information act as optimization method helps the researcher to select the appropriate features to score higher accuracy of the model. The experimental evaluations proved that the hybrid machine learning model based on the feature decomposition and ranked by Mutual Information can increase the accuracy of water level forecasting.  This outcome will help the authorities in managing the risk of flood and helping people in the evacuation process as an early warning can be assigned and disseminate to the citizen

Massive Spatiotemporal Watershed Hydrological Storm Event Response Model (MHSERM) with Time-Lapsed NEXRAD Radar Feed

Correctly and efficiently estimating hydrological responses corresponding to a specific storm event at the streams in a watershed is the main goal of any sound water resource management strategy. Methods for calculating a stream flow hydrograph at the selected streams typically require a great deal of spatial and temporal watershed data such as geomorphological data, soil survey, landcover, precipitation data, and stream network information to name a few. However, extracting and preprocessing such data for estimation and analysis is a hugely time-consuming task, especially for a watershed with hundreds of streams and lakes and complicated landcover and soil characteristics. To deal with the complexity, traditional models have to simplify the watershed and the streams network, use average values for each subcatchment, and then indirectly validate the model by adjusting the parameters through calibration and verification. To obviate such difficulties, and to better utilize the new, high precision spatial/temporal data, a new massive spatiotemporal watershed hydrological storm event response model (MHSERM) was developed and implemented on ESRI ArcMap platform. Different from other hydrological modeling systems, the MHSERM calculated the rainfall run off at a resolution of finer grids that reflects high precision spatial/temporal data characteristics of the watershed, not at conventional catchment or subcatchment scales, and that can simulate the variations of terrain, vegetation and soil far more accurately. The MHSERM provides a framework to utilize the USGS DEM and Landcover data, NRCS SSURGO and STATSGO soil data and National Hydrology Dataset (NHD) by handling millions of elements (grids) and thousands of streams in a real watershed and utilizing the Spatiotemporal NEXRAD precipitation data for each grid in pseudo real-time. Specifically, the MHSERM model has the following new functionalities: (1) Grid the watershed on the basis of high precision data like USGS DEM and Landcover data, NRCS SSURGO and STATSGO soil data, e.g., at a 30 meter by 30 meter resolution; (2) Delineate catchments based on the USGS National Digital Elevation Model (DEM) and the stream network data of the National Hydrography Dataset (NHD); (3) Establish the stream network and routing sequence for a watershed with hundreds of streams and lakes extracted from the National Hydrography Dataset (NHD) either in a supervised or unsupervised manner; (4) Utilize the NCDC NEXRAD precipitation data as spatial and temporal input, and extract the precipitation data for each grid; (5) Calculate the overland runoff volume, flow path and slope to the stream for each grid; (6) Dynamically estimates time of concentration to the stream for each interval, and only for the grids with rainfall excess, not for the whole catchment; (7) Deal with different hydrologic conditions (Good, Fair, Poor) for landcover data and different Antecedent Moisture Condition (AMC); (8) Process single or a series of storm events automatically; thus, the MHSERM model is capable of simulating both discrete and continuous storm events; (9) Calculate the temporal flow rate (i.e., hydrograph) for all the streams in the stream network within the watershed, save them to a database for further analysis and evaluation of various what-if scenarios and BMP designs. In MHSERM model, the SCS Curve number method is used for calculating overland flow runoff volume, and the Muskingum-Cunge method is used for flow routing of the stream network