Groundwater level forecasting with artificial neural networks: A comparison of long short-term memory (LSTM), convolutional neural networks (CNNs), and non-linear autoregressive networks with exogenous input (NARX)

It is now well established to use shallow artificial neural networks (ANNs) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep-learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, shallow recurrent networks frequently seem to be excluded from the study design due to the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANNs, namely non-linear autoregressive networks with exogenous input (NARX) and popular state-of-the-art DL techniques such as long short-term memory (LSTM) and convolutional neural networks (CNNs). We compare the performance on both sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. Further, we also investigate the data dependency in terms of time series length of the different ANN architectures. For seq2val forecasts, NARX models on average perform best; however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL techniques especially when only small amounts of training data are available, where they can clearly outperform LSTMs and CNNs; however, LSTMs and CNNs might perform substantially better with a larger dataset, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though

Deep Learning based assessment of groundwater level development in Germany until 2100

Clear signs of climate stress on groundwater resources have been observed in recent years even in generally water-rich regions such as Germany. Severe droughts, resulting in decreased groundwater recharge, led to declining groundwater levels in many regions and even local drinking water shortages have occurred in past summers. We investigate how climate change will directly influence the groundwater resources in Germany until the year 2100. For this purpose, we use a machine learning groundwater level forecasting framework, based on Convolutional Neural Networks, which has already proven its suitability in modelling groundwater levels. We predict groundwater levels on more than 120 wells distributed over the entire area of Germany that showed strong reactions to meteorological signals in the past. The inputs are derived from the RCP8.5 scenario of six climate models, pre-selected and pre-processed by the German Meteorological Service, thus representing large parts of the range of the expected change in the next 80 years. Our models are based on precipitation and temperature and are carefully evaluated in the past and only wells with models reaching high forecasting skill scores are included in our study. We only consider natural climate change effects based on meteorological changes, while highly uncertain human factors, such as increased groundwater abstraction or irrigation effects, remain unconsidered due to a lack of reliable input data. We can show significant (p<0.05) declining groundwater levels for a large majority of the considered wells, however, at the same time we interestingly observe the opposite behaviour for a small portion of the considered locations. Further, we show mostly strong increasing variability, thus an increasing number of extreme groundwater events. The spatial patterns of all observed changes reveal stronger decreasing groundwater levels especially in the northern and eastern part of Germany, emphasizing the already existing decreasing trends in these region

On the challenges of global entity-aware deep learning models for groundwater level prediction

The application of machine learning (ML) including deep learning models in hydrogeology to model and predict groundwater level in monitoring wells has gained some traction in recent years. Currently, the dominant model class is the so-called single-well model, where one model is trained for each well separately. However, recent developments in neighbouring disciplines including hydrology (rainfall–runoff modelling) have shown that global models, being able to incorporate data of several wells, may have advantages. These models are often called “entity-aware models“, as they usually rely on static data to differentiate the entities, i.e. groundwater wells in hydrogeology or catchments in surface hydrology. We test two kinds of static information to characterize the groundwater wells in a global, entity-aware deep learning model set-up: first, environmental features that are continuously available and thus theoretically enable spatial generalization (regionalization), and second, time-series features that are derived from the past time series at the respective well. Moreover, we test random integer features as entity information for comparison. We use a published dataset of 108 groundwater wells in Germany, and evaluate the performance of the models in terms of Nash–Sutcliffe efficiency (NSE) in an in-sample and an out-of-sample setting, representing temporal and spatial generalization. Our results show that entity-aware models work well with a mean performance of NSE >0.8 in an in-sample setting, thus being comparable to, or even outperforming, single-well models. However, they do not generalize well spatially in an out-of-sample setting (mean NSE <0.7, i.e. lower than a global model without entity information). Strikingly, all model variants, regardless of the type of static features used, basically perform equally well both in- and out-of-sample. The conclusion is that the model in fact does not show entity awareness, but uses static features merely as unique identifiers, raising the research question of how to properly establish entity awareness in deep learning models. Potential future avenues lie in bigger datasets, as the relatively small number of wells in the dataset might not be enough to take full advantage of global models. Also, more research is needed to find meaningful static features for ML in hydrogeology

Groundwater Level Forecasting with Artificial Neural Networks: A Comparison of LSTM, CNN and NARX

It is now well established to use shallow artificial neural networks (ANN) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, especially shallow recurrent networks frequently seem to be excluded from the study design despite the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANN namely nonlinear autoregressive networks with exogenous inputs (NARX), and popular state-of-the-art DL-techniques such as long short-term memory (LSTM) and convolutional neural networks (CNN). We compare both the performance on sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period, while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. We observe that for seq2val forecasts NARX models on average perform best, however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL-models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL-techniques; however, LSTMs and CNNs might perform substantially better with a larger data set, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though

Feature-based Groundwater Hydrograph Clustering Using Unsupervised Self-Organizing Map-Ensembles

Hydrograph clustering helps to identify dynamic patterns within aquifers systems, an important foundation of characterizing groundwater systems and their influences, which is necessary to effectively manage groundwater resources. We develope an unsupervised modeling approach to characterize and cluster hydrographs on regional scale according to their dynamics. We apply feature-based clustering to improve the exploitation of heterogeneous datasets, explore the usefulness of existing features and propose new features specifically useful to describe groundwater hydrographs. The clustering itself is based on a powerful combination of Self-Organizing Maps with a modified DS2L-Algorithm, which automatically derives the cluster number but also allows to influence the level of detail of the clustering. We further develop a framework that combines these methods with ensemble modeling, internal cluster validation indices, resampling and consensus voting to finally obtain a robust clustering result and remove arbitrariness from the feature selection process. Further we propose a measure to sort hydrographs within clusters, useful for both interpretability and visualization. We test the framework with weekly data from the Upper Rhine Graben System, using more than 1800 hydrographs from a period of 30 years (1986-2016). The results show that our approach is adaptively capable of identifying homogeneous groups of hydrograph dynamics. The resulting clusters show both spatially known and unknown patterns, some of which correspond clearly to external controlling factors, such as intensive groundwater management in the northern part of the test area. This framework is easily transferable to other regions and, by adapting the describing features, also to other time series-clustering applications

Deep learning shows declining groundwater levels in Germany until 2100 due to climate change

In this study we investigate how climate change will directly influence the groundwater resources in Germany during the 21(st) century. We apply a machine learning groundwater level prediction approach based on convolutional neural networks to 118 sites well distributed over Germany to assess the groundwater level development under different RCP scenarios (2.6, 4.5, 8.5). We consider only direct meteorological inputs, while highly uncertain anthropogenic factors such as groundwater extractions are excluded. While less pronounced and fewer significant trends can be found under RCP2.6 and RCP4.5, we detect significantly declining trends of groundwater levels for most of the sites under RCP8.5, revealing a spatial pattern of stronger decreases, especially in the northern and eastern part of Germany, emphasizing already existing decreasing trends in these regions. We can further show an increased variability and longer periods of low groundwater levels during the annual cycle towards the end of the century

A low-dimensional hillslope-based catchment model for layered groundwater flow : conceptual development, testing, and application

La prévision des débits d'étiage est une question importante dans la gestion des bassins versants. Pendant les périodes de basses eaux, l'écoulement de base peut devenir une composante majeure des débits en rivière. Dans les modèles hydrologiques, la composante de l'écoulement souterrain est souvent très simplifiée. Ceci découle le plus souvent d'une caractérisation insuffisante des aquifères et les temps de calculs prohibitifs des modèles intégrant de manière complète l'écoulement souterrain. Cette thèse développe un modèle représentant les écoulements souterrains peu profonds et profonds qui nécessitent peu de paramètres et dont les calculs sont efficaces et fiables. Le modèle proposé peut être utilisé à la place d'un modèle d'écoulement souterrain en différences finies ou en éléments finis, mais il a été conçu dans le but d'être incorporé à un modèle hydrologique de bassin versant. Le modèle d'écoulement transitoire de versant "hillslope-storage Boussinesq" (hsB) (Troch et al., 2003, Paniconi et al., 2003) est sélectionné pour représenter l'écoulement souterrain peu profond à l'échelle locale. Le modèle hsB est couplé avec le modèle GLOW d'écoulement permanent basé sur la méthode des éléments analytiques (EA) (Strack, 1989; Haitjema, 1995) qui représente l'écoulement régional profond 2D horizontal. L'approche de couplage utilisée nécessite d'intégrer un terme de percolation à la base du modèle hsB. Ce terme de percolation est représenté par un flux vertical de Darcy à travers un aquitard hypothétique séparant le versant local de l'aquifère 2D régional. Afin de mieux comprendre les facteurs contrôlant la percolation, les facteurs tels que la géométrie du versant, l'inclinaison de la base, les propriétés des aquifères et les conditions limites sont évalués à l'aide d'un modèle 3D basé sur l'équation de Richards (Paniconi et Putti, 1994 ; Camporese et al., 2010). Les observations principales de cette analyse sont: i) l'eau peut circuler vers le bas ou vers le haut entre les aquifères de peu profonds et profonds et séparer les versants en trois zones distinctes: une zone de flux descendant à l'amont, une zone de flux descendant à l'aval et une zone de transition entre les deux; ii) l'inclinaison des versants et leur géométrie déterminent la partition des flux échangés. Ces résultats sont utilisés dans la mise en œuvre du couplage entre les modèles hsB et EA, où chaque versant et l'aquifère sous-jacent sont subdivisés en trois zones de percolation constante. Le modèle hsB/EA est testé sur différents types de versants et sur un bassin hypothétique formé de deux versants convergeant vers un cours d'eau. La comparaison avec les résultats d'un modèle numérique 3D basé sur l'équation de Richards et utilisé comme référence démontrent: i) que les charges hydrauliques, les taux de percolation et les débits aux exutoires sont généralement bien simulés; ii) de meilleurs résultats ont été obtenus pour les versants peu ou très peu inclinés aux géométries uniformes et divergentes; iii) les débits cumulés sont simulés de manière convenable pour le bassin hypothétique. Les écarts entre le modèle de référence sont attribués au fait que la zone non-saturée n'est pas représentée dans le modèle hsB et à l'hypothèse de Dupuit-Forchheimer du modèle EA qui néglige l'écoulement vertical dans la nappe profonde. Le fait que le modèle EA ne permette pas de simuler l'écoulement en régime transitoire est une autre limitation du modèle hsB/EA (e.g. Kuhlman et Neuman, 2009). Une application du modèle hsB/EA sur un bassin versant de 30 km2 situé dans la région de Covey Hill au sud du Québec a été réalisée. Dans cette application du modèle hsB/EA, les débits de base sont assez bien reproduits à l'exutoire du bassin versant pendant les périodes d'étiage. Toutefois, des écarts importants sont observés au cours des débits de pointe. Les erreurs sur les charges sont également non négligeables dans les zones où un gradient vertical a été observé. Ces erreurs peuvent être attribuées en parties aux limitations du modèle développé. \ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : hydrologie des versants, percolation, équation de Boussinesq, équation de Richards, éléments analytiques, eaux souterraines stratifié, modèle coupl

Assessing groundwater level modelling using a 1-D convolutional neural network (CNN): linking model performances to geospatial and time series features

Groundwater level (GWL) forecasting with machine learning has been widely studied due to its generally accurate results and low input data requirements. Furthermore, machine learning models for this purpose can be set up and trained quickly compared to the effort required for process-based numerical models. Despite demonstrating high performance at specific locations, applying the same model architecture to multiple sites across a regional area can lead to varying accuracies. The reasons behind this discrepancy in model performance have been scarcely examined in previous studies. Here, we explore the relationship between model performance and the geospatial and time series features of the sites. Using precipitation (P) and temperature (T) as predictors, we model monthly groundwater levels at approximately 500 observation wells in Lower Saxony, Germany, applying a 1-D convolutional neural network (CNN) with a fixed architecture and hyperparameters tuned for each time series individually. The GWL observations range from 21 to 71 years, resulting in variable test and training dataset time ranges. The performances are evaluated against selected geospatial characteristics (e.g. land cover, distance to waterworks, and leaf area index) and time series features (e.g. autocorrelation, flat spots, and number of peaks) using Pearson correlation coefficients. Results indicate that model performance is negatively influenced at sites near waterworks and densely vegetated areas. Longer subsequences of GWL measurements above or below the mean negatively impact the model accuracy. Besides, GWL time series containing more irregular patterns and with a higher number of peaks might lead to higher model performances, possibly due to a closer link with precipitation dynamics. As deep learning models are known to be black-box models missing the understanding of physical processes, our work provides new insights into how geospatial and time series features link to the input–output relationship of a GWL forecasting model.</p