Artificial immune system based wastewater parameter estimation

The basis of a wastewater treatment system is to achieve the desired characteristics of the wastewater treatment process. An estimation of the obtained wastewater treatment characteristics provides the information needed to set up the current process steps, and it is important to have an optimum treatment. In this study, an artificial immune system (AIS) structure is developed to estimate important wastewater output parameters such as pH, DBO, DQO, and SS for the first time. The proposed AIS models are based on the clonal selection principle, and the dataset is provided from the University of California Irvine (UCI) Machine Learning Library. The current dataset is analyzed by principal component analysis (PCA) to obtain maximum system performance. As a result of the simulation, the output parameters are successfully predicted using the AIS model with real data. The classifier's performance ratios are studied separately using the coefficient of determination (R-2) and the mean squared error of prediction (MSEP), and their rates are given in this study

Forecasting of uv-vis spectrometry time series for online water quality monitoring in operating urban sewer systems

El monitoreo de contaminantes en sistemas de saneamiento urbano es generalmente realizado por medio de campañas de muestreo, las muestras deben ser transportadas, almacenadas y analizadas en laboratorio. Sin embargo, los desarrollos en óptica y electrónica han permitido su fusión y aplicación en la espectrometría UV-Vis. Los sensores UV-Vis tienen como propósito determinar la dinámica de las cargas de materia orgánica (Demanda Química de Oxigeno DQO y Demanda Bioquímica de Oxigeno DBO5), nitratos, nitritos y Sólidos Suspendidos Totales (SST). Adicionalmente a los métodos aplicados para la calibración de los sensores y el análisis las series de tiempo de los espectros de absorbancias UV-Vis, es necesario desarrollar métodos de pronóstico con el fin de ser utilizada en control de monitoreo en línea en tiempo real. La información proveniente de los datos recolectados puede ser utilizada para la toma de decisiones y en aplicaciones de control de tiempo real. Realizar pronósticos es importante en procesos de toma de decisiones. Por lo tanto, el objetivo de este trabajo de investigación fue desarrollar uno o varios métodos de pronóstico que puedan ser aplicados a series de tiempo de espectrometría UV-Vis para el monitoreo en línea de la calidad de agua en sistemas urbanos de saneamiento en operación. Cinco series de tiempo de absorbancia UV-Vis obtenidas en línea en diferentes sitios fueron utilizadas, con un total de 5705 espectros de absorbancia UV-Vis: cuatro sitios experimentales en Colombia (Planta de Tratamiento de Aguas Residuales (PTAR) El-Salitre, PTAR San Fernando, Estación Elevadora de Gibraltar y un Humedal Construido/Tanque de Almacenamiento) y un sitio en Austria (Graz-West R05 Catchment outlet). El proceso propuesto completo consta de etapas a ser aplicadas a las series de tiempo de absorbancia UV-Vis y son: (i) entradas, series de tiempo de absorbancia UV-Vis,(ii) pre-procesamiento de las series de tiempo, análisis de outliers, completar los valores ausentes y reducción de la dimensionalidad,y (iii) procedimientos de pronóstico y evaluación de los resultados. La metodología propuesta fue aplicada a la series de tiempo con diferentes características (absorbancia), esta consiste del enventaneo Winsorising como paso para la remoción de outliers y la aplicación de la transformada discreta de Fourier (DFT) para reemplazar valores ausentes. Los nuevos valores reemplazando o los outliers o los valores ausentes presentan la misma o al menos la misma forma de la serie de tiempo original, permitiendo una visión macro en la coherencia de la serie de tiempo. La reducción de la dimensionalidad en las series de tiempo de absorbancia multivariadas permite obtener menor número de variables a ser procesadas: el análisis por componentes principales (PCA) como transformación lineal captura más del 97% de la variabilidad en cada serie de tiempo (en un rango de una a seis, dependiendo del comportamiento de la series de tiempo absorbancia) y el proceso de Clustering (k-means) combinado con cadenas de Markov. Los procedimientos de pronóstico basados en señales periódicas como la DFT, Chebyshev, Legendre y Regresión Polinomial fueron aplicados y estos pueden capturar el comportamiento dinámico de las series de tiempo. Algunas técnicas de aprendizaje de máquina fueron probadas y fue posible capturar el comportamiento de las series de tiempo en la etapa de calibración, los valores de pronóstico pueden seguir el comportamiento general comparado con los valores observados (excepto ANFIS, GA y Filtro de Kalman). Por lo tanto, ANN y SVM tiene buen rendimiento de pronóstico para la primer parte del horizonte de pronóstico (2 horas). La evaluación de cada metodología de pronóstico fue realizada utilizando cuatro indicadores estadísticos tales como porcentaje absoluto de error (APE), incertidumbre extendida (EU), conjunto de valores dentro del intervalo de confianza (CI) y suma de valores de incertidumbre extendida más el conjunto de valores dentro del intervalo de confianza. El rendimiento de los indicadores provee información acerca de los resultados de pronóstico multivariado con el fin de estimar y evaluar los tiempos de pronóstico para cierta metodología de pronóstico y determinar cuál metodología de pronóstico es mejor adaptada a diferentes rangos de longitudes de onda (espectros de absorbancia) para cada serie de tiempo de absorbancia UV-Vis en cada sitio de estudio. Los resultados en la comparación de las diferentes metodologías de pronóstico, resaltan que no es posible obtener la mejor metodología de pronóstico, porque todas las metodologías de pronóstico propuestas podrían generar un amplio número de valores que permitirán complementar cada una con las otras para diferentes pasos de tiempo de pronóstico y en diferentes rangos del espectro (UV y/o Vis). Por lo tanto, es propuesto un sistema híbrido que es basado en siete metodologías de pronóstico. Así, los valores de los espectros de absorbancia pronosticados fueron transformados a los correspondientes indicadores de calidad de agua (WQI) para utilización en la práctica. Los resultados de pronóstico multivariado presentan valores bajos de APE comparados con los resultados de pronóstico univariado utilizando directamente los valores WQI observados. Estos resultados, probablemente, son obtenidos porque el pronóstico multivariado incluye la correlación presente en todo el rango de los espectros de absorbancia (se captura de forma completa o al menos gran parte de la variabilidad de las series de tiempo),una longitud de onda interfiere con otra u otras longitudes de onda. Finalmente, los resultados obtenidos para el humedal construido/tanque de almacenamiento presentan que es posible obtener apreciables resultados de pronóstico en términos de tiempos de detección para eventos de lluvia. Adicionalmente, la inclusión de variables como escorrentía (nivel de agua para este caso) mejora substancialmente los resultados de pronóstico de la calidad del agua. El monitoreo de contaminantes en sistemas de saneamiento urbano es generalmente realizado por medio de campañas de muestreo, las muestras deben ser transportadas, almacenadas y analizadas en laboratorio. Sin embargo, los desarrollos en óptica y electrónica han permitido su fusión y aplicación en la espectrometría UV-Vis. Los sensores UV-Vis tienen como propósito determinar la dinámica de las cargas de materia orgánica (Demanda Química de Oxigeno DQO y Demanda Bioquímica de Oxigeno DBO5), nitratos, nitritos y Sólidos Suspendidos Totales (SST). Adicionalmente a los métodos aplicados para la calibración de los sensores y el análisis las series de tiempo de los espectros de absorbancias UV-Vis, es necesario desarrollar métodos de pronóstico con el fin de ser utilizada en control de monitoreo en línea en tiempo real. La información proveniente de los datos recolectados puede ser utilizada para la toma de decisiones y en aplicaciones de control de tiempo real. Realizar pronósticos es importante en procesos de toma de decisiones. Por lo tanto, el objetivo de este trabajo de investigación fue desarrollar uno o varios métodos de pronóstico que puedan ser aplicados a series de tiempo de espectrometría UV-Vis para el monitoreo en línea de la calidad de agua en sistemas urbanos de saneamiento en operación. Cinco series de tiempo de absorbancia UV-Vis obtenidas en línea en diferentes sitios fueron utilizadas, con un total de 5705 espectros de absorbancia UV-Vis: cuatro sitios experimentales en Colombia (Planta de Tratamiento de Aguas Residuales (PTAR) El-Salitre, PTAR San Fernando, Estación Elevadora de Gibraltar y un Humedal Construido/Tanque de Almacenamiento) y un sitio en Austria (Graz-West R05 Catchment outlet). El proceso propuesto completo consta de etapas a ser aplicadas a las series de tiempo de absorbancia UV-Vis y son: (i) entradas, series de tiempo de absorbancia UV-Vis,(ii) pre-procesamiento de las series de tiempo, análisis de outliers, completar los valores ausentes y reducción de la dimensionalidad,y (iii) procedimientos de pronóstico y evaluación de los resultados. La metodología propuesta fue aplicada a la series de tiempo con diferentes características (absorbancia), esta consiste del enventaneo Winsorising como paso para la remoción de outliers y la aplicación de la transformada discreta de Fourier (DFT) para reemplazar valores ausentes. Los nuevos valores reemplazando o los outliers o los valores ausentes presentan la misma o al menos la misma forma de la serie de tiempo original, permitiendo una visión macro en la coherencia de la serie de tiempo. La reducción de la dimensionalidad en las series de tiempo de absorbancia multivariadas permite obtener menor número de variables a ser procesadas: el análisis por componentes principales (PCA) como transformación lineal captura más del 97% de la variabilidad en cada serie de tiempo (en un rango de una a seis, dependiendo del comportamiento de la series de tiempo absorbancia) y el proceso de Clustering (k-means) combinado con cadenas de Markov. Los procedimientos de pronóstico basados en señales periódicas como la DFT, Chebyshev, Legendre y Regresión Polinomial fueron aplicados y estos pueden capturar el comportamiento dinámico de las series de tiempo. Algunas técnicas de aprendizaje de máquina fueron probadas y fue posible capturar el comportamiento de las series de tiempo en la etapa de calibración, los valores de pronóstico pueden seguir el comportamiento general comparado con los valores observados (excepto ANFIS, GA y Filtro de Kalman). Por lo tanto, ANN y SVM tiene buen rendimiento de pronóstico para la primer parte del horizonte de pronóstico (2 horas). La evaluación de cada metodología de pronóstico fue realizada utilizando cuatro indicadores estadísticos tales como porcentaje absoluto de error (APE), incertidumbre extendida (EU), conjunto de valores dentro del intervalo de confianza (CI) y suma de valores de incertidumbre extendida más el conjunto de valores dentro del intervalo de confianza. El rendimiento de los indicadores provee información acerca de los resultados de pronóstico multivariado con el fin de estimar y evaluar los tiempos de pronóstico para cierta metodología de pronóstico y determinar cuál metodología de pronóstico es mejor adaptada a diferentes rangos de longitudes de onda (espectros de absorbancia) para cada serie de tiempo de absorbancia UV-Vis en cada sitio de estudio. Los resultados en la comparación de las diferentes metodologías de pronóstico, resaltan que no es posible obtener la mejor metodología de pronóstico, porque todas las metodologías de pronóstico propuestas podrían generar un amplio número de valores que permitirán complementar cada una con las otras para diferentes pasos de tiempo de pronóstico y en diferentes rangos del espectro (UV y/o Vis). Por lo tanto, es propuesto un sistema híbrido que es basado en siete metodologías de pronóstico. Así, los valores de los espectros de absorbancia pronosticados fueron transformados a los correspondientes indicadores de calidad de agua (WQI) para utilización en la práctica. Los resultados de pronóstico multivariado presentan valores bajos de APE comparados con los resultados de pronóstico univariado utilizando directamente los valores WQI observados. Estos resultados, probablemente, son obtenidos porque el pronóstico multivariado incluye la correlación presente en todo el rango de los espectros de absorbancia (se captura de forma completa o al menos gran parte de la variabilidad de las series de tiempo),una longitud de onda interfiere con otra u otras longitudes de onda. Finalmente, los resultados obtenidos para el humedal construido/tanque de almacenamiento presentan que es posible obtener apreciables resultados de pronóstico en términos de tiempos de detección para eventos de lluvia. Adicionalmente, la inclusión de variables como escorrentía (nivel de agua para este caso) mejora substancialmente los resultados de pronóstico de la calidad del agua.The monitoring of pollutants in urban sewer systems is generally conducted by sampling campaigns, and the resulting samples must be transported, stored and analyzed in laboratory. However, the developments in optics and electronics have enabled the merge of them into the UV-Vis Spectrometry. UV-Vis probes have the purpose of determining the dynamics of loads of organic materials (i.e. Chemical Oxygen Demand (COD) and Biochemical Oxygen Demand (BOD5)), nitrates, nitrites and Total Suspended Solids (TSS). In addition to the methods used for the calibration of the probes and the analysis of the time series of UV-Vis absorbance spectra, it is necessary to develop forecasting methods in order to use the online control monitoring in real time. The information from the collected data can also be used for decision making purposes and for real-time control applications. Forecasting is important for decision-making processes. Therefore, the objective of this research work was to develop either a forecasting method or forecasting methods applied to UV-Vis spectrometry time series data for online water quality monitoring in operating urban sewer systems. Five UV-Vis Absorbance time series collected at different on-line measurement sites were used, for a total of 5705 UV-Vis absorbance spectra data: four sites in Colombia (El-Salitre Wastewater Treatment Plant-WWTP, San Fernando WWTP, Pumping Station (PS) sewage called Gibraltar and constructed-wetland/reservoir-tank (CWRT)) and one site in Austria (Graz-West R05 Catchment outlet). The complete process proposed to be applied to UV-Vis absorbance time series has several stages and these are: (i) inputs, the UV-Vis absorbance time series,(ii) the time series pre-processing, outliers analysis, complete missing values and time series dimensionality reduction,and (iii) forecasting procedures and evaluation of results. The methodology proposed was applied to the time series with different characteristics (absorbance), this consists of Winsorising as a step in outlier removal and the application of the Discrete Fourier Transform (DFT) to complete the missing values. The new values replaced either outliers or missing values present the same, or almost the same, shape as the original time series, granted the macro vision of the time series coherence. Dimensionality reduction of multivariate absorbance time series allows to have less variables to be processed: PCA linear transformation captures more than 97% of variability for each time series (PC ranging from one to six, depending on absorbance time series behavior), and Clustering process (k-means) combined with Markov Chains. Forecasting procedures based on periodic signals as DFT, Chebyshev, Legendre and Polynomial Regression were applied and they can capture the dynamic behaviour of the time series. Several Machine Learning technics were tested and it was possible to capture the behaviour of the time series at calibration stage, the forecasting obtained valúes can follow the general behaviour compared with observed valúes (with exception of ANFIS, GA and Kalman Filter). Therefore, ANN and SVM have good forecasting performances for first part of forecasting horizon (2 hours). The evaluation of each forecasting methodology was done using four statistic indicators as Absolute Percentage Error (APE), Extended Uncertainty (EU), Set of observed values within Confidence Interval (CI) and sum of EU and Set of observed values within CI. The performance indicators provided valuable information about multivariate forecasting results to estimate and evaluate the forecasting time for a given forecasting methodology and determine which forecasting methodology is best suited for different wavelength ranges (absorbance spectra) at each study site s UV-Vis absorbance time series. Results from different comparison of several forecasting methodologies, highlight that there is not possibility to have a best forecasting methodology among the proposed ones, because all of them could provide a wide forecasting values that would complemented each other for different forecasting time steps and spectra range (UV and/or Vis). Therefore, it is proposed a hybrid system that is based on seven forecasting methodologies. Thus, the forecasted absorbance spectra were transformed to Water Quality Indicators (WQI) for practical uses. The multivariate forecasting results show lower APE values compared to the univariate forecasting results (APE values) using the observed WQI. These results, probably, were obtained because multivariate forecasting includes the correlation presented at whole absorbance spectra range (captures complete or at least great part of time series variability),one wavelength interferes with another and/or other wavelengths. Finally, the results obtained for a constructed-wetland/reservoir-tank system show that it is possible to obtain valuable forecasting results in terms of time detection for some rainfall events. In addition, the inclusion of runoff variables (water level in this case) improves the water quality forecasting results.Doctor en IngenieríaDoctorad

Computational Intelligence Techniques for Control and Optimization of Wastewater Treatment Plants

The development of novel, practice-oriented and reliable instrumentation and control strategies for wastewater treatment plants in order to improve energy efficiency, while guaranteeing process stability and maintenance of high cleaning capacity, has become a priority for WWTP operators due to increasing treatment costs. To achieve these ambitious and even contradictory objectives, this thesis investigates a combination of online measurement systems, computational intelligence and machine learning methods as well as dynamic simulation models. Introducing the state-of-the-art in the fields of WWTP operation, process monitoring and control, three novel computational intelligence enabled instrumentation, control and automation (ICA) methods are developed and presented. Furthermore, their potential for practical implementation is assessed. The methods are, on the one hand, the automated calibration of a simulation model for the Rospe WWTP that provides a basis for the development and evaluation of the subsequent methods, and on the other hand, the development of soft sensors for the WWTP inflow which estimate the crucial process variables COD and NH4-N, and the estimation of WWTP operating states using Self- Organising Maps (SOM) that are used to determine the optimal control parameters for each state. These collectively, provide the basis for achieving comprehensive WWTP optimization. Results show that energy consumption and cleaning capacity can be improved by more than 50%

Multi-tier framework for the inferential measurement and data-driven modeling

A framework for the inferential measurement and data-driven modeling has been proposed and assessed in several real-world application domains. The architecture of the framework has been structured in multiple tiers to facilitate extensibility and the integration of new components. Each of the proposed four tiers has been assessed in an uncoupled way to verify their suitability. The first tier, dealing with exploratory data analysis, has been assessed with the characterization of the chemical space related to the biodegradation of organic chemicals. This analysis has established relationships between physicochemical variables and biodegradation rates that have been used for model development. At the preprocessing level, a novel method for feature selection based on dissimilarity measures between Self-Organizing maps (SOM) has been developed and assessed. The proposed method selected more features than others published in literature but leads to models with improved predictive power. Single and multiple data imputation techniques based on the SOM have also been used to recover missing data in a Waste Water Treatment Plant benchmark. A new dynamic method to adjust the centers and widths of in Radial basis Function networks has been proposed to predict water quality. The proposed method outperformed other neural networks. The proposed modeling components have also been assessed in the development of prediction and classification models for biodegradation rates in different media. The results obtained proved the suitability of this approach to develop data-driven models when the complex dynamics of the process prevents the formulation of mechanistic models. The use of rule generation algorithms and Bayesian dependency models has been preliminary screened to provide the framework with interpretation capabilities. Preliminary results obtained from the classification of Modes of Toxic Action (MOA) indicate that this could be a promising approach to use MOAs as proxy indicators of human health effects of chemicals.Finally, the complete framework has been applied to three different modeling scenarios. A virtual sensor system, capable of inferring product quality indices from primary process variables has been developed and assessed. The system was integrated with the control system in a real chemical plant outperforming multi-linear correlation models usually adopted by chemical manufacturers. A model to predict carcinogenicity from molecular structure for a set of aromatic compounds has been developed and tested. Results obtained after the application of the SOM-dissimilarity feature selection method yielded better results than models published in the literature. Finally, the framework has been used to facilitate a new approach for environmental modeling and risk management within geographical information systems (GIS). The SOM has been successfully used to characterize exposure scenarios and to provide estimations of missing data through geographic interpolation. The combination of SOM and Gaussian Mixture models facilitated the formulation of a new probabilistic risk assessment approach.Aquesta tesi proposa i avalua en diverses aplicacions reals, un marc general de treball per al desenvolupament de sistemes de mesurament inferencial i de modelat basats en dades. L'arquitectura d'aquest marc de treball s'organitza en diverses capes que faciliten la seva extensibilitat així com la integració de nous components. Cadascun dels quatre nivells en que s'estructura la proposta de marc de treball ha estat avaluat de forma independent per a verificar la seva funcionalitat. El primer que nivell s'ocupa de l'anàlisi exploratòria de dades ha esta avaluat a partir de la caracterització de l'espai químic corresponent a la biodegradació de certs compostos orgànics. Fruit d'aquest anàlisi s'han establert relacions entre diverses variables físico-químiques que han estat emprades posteriorment per al desenvolupament de models de biodegradació. A nivell del preprocés de les dades s'ha desenvolupat i avaluat una nova metodologia per a la selecció de variables basada en l'ús del Mapes Autoorganitzats (SOM). Tot i que el mètode proposat selecciona, en general, un major nombre de variables que altres mètodes proposats a la literatura, els models resultants mostren una millor capacitat predictiva. S'han avaluat també tot un conjunt de tècniques d'imputació de dades basades en el SOM amb un conjunt de dades estàndard corresponent als paràmetres d'operació d'una planta de tractament d'aigües residuals. Es proposa i avalua en un problema de predicció de qualitat en aigua un nou model dinàmic per a ajustar el centre i la dispersió en xarxes de funcions de base radial. El mètode proposat millora els resultats obtinguts amb altres arquitectures neuronals. Els components de modelat proposat s'han aplicat també al desenvolupament de models predictius i de classificació de les velocitats de biodegradació de compostos orgànics en diferents medis. Els resultats obtinguts demostren la viabilitat d'aquesta aproximació per a desenvolupar models basats en dades en aquells casos en els que la complexitat de dinàmica del procés impedeix formular models mecanicistes. S'ha dut a terme un estudi preliminar de l'ús de algorismes de generació de regles i de grafs de dependència bayesiana per a introduir una nova capa que faciliti la interpretació dels models. Els resultats preliminars obtinguts a partir de la classificació dels Modes d'acció Tòxica (MOA) apunten a que l'ús dels MOA com a indicadors intermediaris dels efectes dels compostos químics en la salut és una aproximació factible.Finalment, el marc de treball proposat s'ha aplicat en tres escenaris de modelat diferents. En primer lloc, s'ha desenvolupat i avaluat un sensor virtual capaç d'inferir índexs de qualitat a partir de variables primàries de procés. El sensor resultant ha estat implementat en una planta química real millorant els resultats de les correlacions multilineals emprades habitualment. S'ha desenvolupat i avaluat un model per a predir els efectes carcinògens d'un grup de compostos aromàtics a partir de la seva estructura molecular. Els resultats obtinguts desprès d'aplicar el mètode de selecció de variables basat en el SOM milloren els resultats prèviament publicats. Aquest marc de treball s'ha usat també per a proporcionar una nova aproximació al modelat ambiental i l'anàlisi de risc amb sistemes d'informació geogràfica (GIS). S'ha usat el SOM per a caracteritzar escenaris d'exposició i per a desenvolupar un nou mètode d'interpolació geogràfica. La combinació del SOM amb els models de mescla de gaussianes dona una nova formulació al problema de l'anàlisi de risc des d'un punt de vista probabilístic

Rethinking construction cost overruns: an artificial neural network approach to construction cost estimation

The main concern of a construction client is to procure a facility that is able to meet its functional requirements, of the required quality, and delivered within an acceptable budget and timeframe. The cost aspect of these key performance indicators usually ranks highest. In spite of the importance of cost estimation, it is undeniably neither simple nor straightforward because of the lack of information in the early stages of the project. Construction projects therefore have routinely overrun their estimates. Cost overrun has been attributed to a number of sources including technical error in design, managerial incompetence, risk and uncertainty, suspicions of foul play and even corruption. Furthermore, even though it is accepted that factors such as tendering method, location of project, procurement method or size of project have an effect on likely final cost of a project, it is difficult to establish their measured financial impact. Estimators thus have to rely largely on experience and intuition when preparing initial estimates, often neglecting most of these factors in the final cost build-up. The decision-to-build for most projects is therefore largely based on unrealistic estimates that would inevitably be exceeded. The main aim of this research is to re-examine the sources of cost overrun on construction projects and to develop final cost estimation models that could help in reaching more reliable final cost estimates at the tendering stage of the project. The research identified two predominant schools of thought on the sources of overruns – referred to here as the PsychoStrategists and Evolution Theorists. Another finding was that there is no unanimity on the reference point from which cost performance could be assessed, leading to a large disparity in the size of overruns reported. Another misunderstanding relates to the term “cost overrun” itself. The experimental part of the research, conducted in collaboration with two industry partners, used a combination of non-parametric bootstrapping and ensemble modelling with artificial neural networks to develop final project cost models based on about 1,600 water infrastructure projects. 92% of the validation predictions were within ±10% of the actual final cost of the project. The models will be particularly useful at the pre-contract stage as they will provide a benchmark for evaluating submitted tenders and also allow the quick generation of various alternative solutions for a construction project using what-if scenarios. The original contribution of the study is a fresh thinking of construction “cost overruns”, now proposed to be more appropriately known as “cost growth” based on a synthesises of the two schools of thought into a conceptual model. The second contribution is the development of novel models of construction cost estimation utilising artificial neural networks coupled with bootstrapping and ensemble modelling

Applications of machine learning to water resources management: A review of present status and future opportunities

Data availability: No data was used for the research described in the article.The corrected proof will be replaced by version of record in due course.Copyright © 2024 The Authors. Water is the most valuable natural resource on earth that plays a critical role in the socio-economic development of humans worldwide. Water is used for various purposes, including, but not limited to, drinking, recreation, irrigation, and hydropower production. The expected population growth at a global scale, coupled with the predicted climate change-induced impacts, warrants the need for proactive and effective management of water resources. Over the recent decades, machine learning tools have been widely applied to various water resources management-related fields and have often shown promising results. Despite the publication of several review articles on machine learning applications in water-related fields, this review paper presents for the first time a comprehensive review of machine learning techniques applied to water resources management, focusing on the most recent achievements. The study examines the potential for advanced machine learning techniques to improve decision support systems in the various sectors within the realm of water resources management, which includes groundwater management, streamflow forecasting, water distribution systems, water quality and wastewater treatment, water demand and consumption, hydropower and marine energy, water drainage systems, and flood management and defence. This study provides an overview of the state-of-the-art machine learning approaches to the water industry and how they can be used to ensure water supply sustainability, quality, and flood and drought mitigation. This review covers the most recent related studies to provide the most recent snapshot of machine learning applications in the water industry. Overall, LSTM networks have been proven to exhibit reliable performance, often outperforming ANN models, traditional machine learning models, and established physics-based models. Hybrid ML techniques have exhibited great forecasting accuracy across all water-related fields, often showing superior computational power over traditional ANNs architectures. In addition to purely data-driven models, physical-based hybrid models have also been developed to improve prediction performance. These efforts further demonstrate that Machine learning can be a powerful practical tool for water resources management. It provides insights, predictions, and optimisation capabilities to help enhance sustainable water use and management and improve socio-economic development, healthy ecosystems and human existence.EPSRC project reference 2339403 to S. Sayed and A. Ahmed

Smart Urban Water Networks

This book presents the paper form of the Special Issue (SI) on Smart Urban Water Networks. The number and topics of the papers in the SI confirm the growing interest of operators and researchers for the new paradigm of smart networks, as part of the more general smart city. The SI showed that digital information and communication technology (ICT), with the implementation of smart meters and other digital devices, can significantly improve the modelling and the management of urban water networks, contributing to a radical transformation of the traditional paradigm of water utilities. The paper collection in this SI includes different crucial topics such as the reliability, resilience, and performance of water networks, innovative demand management, and the novel challenge of real-time control and operation, along with their implications for cyber-security. The SI collected fourteen papers that provide a wide perspective of solutions, trends, and challenges in the contest of smart urban water networks. Some solutions have already been implemented in pilot sites (i.e., for water network partitioning, cyber-security, and water demand disaggregation and forecasting), while further investigations are required for other methods, e.g., the data-driven approaches for real time control. In all cases, a new deal between academia, industry, and governments must be embraced to start the new era of smart urban water systems

The application of the artificial neural network model for river water quality classification with emphasis on the impact of land use activities: a case study from several catchments in Malaysia

Several methods of river water quality assessment techniques have been introduced. Among the most commonly used are the water quality index system and classification scheme. These two systems are designed to simplify the huge amount of water quality data down to its simplest form, while retaining the essential meaning of the information. They offer the means for measuring the effectiveness of pollution abatement strategies by comparing the status of water quality both temporally and spatially. In this way, it is useful for management purposes, especially in determining priorities for resource allocation and planning of new development areas. The water quality index system and the classification schemes currently available, however have some limitations in their structural design. They often exhibit inherent loss of information, are complex and may involve subjective judgement in their interpretation. However, because of the critical issues on water pollution and the scarcity of water resources, these systems are being applied despite of these limitations. The current situation is that, different countries are applying different models of water quality assessment system. Based on the limitations of the existing assessment systems, it is appropriate to explore other approaches that can be more flexible, robust to noisy data, and adaptable to new form of environmental data, in order to provide direct and prompt results for classifying of river water quality. One avenue for research is that based on Artificial Neural Network (ANN). Artificial Neural Network comprises of several techniques. One of this technique that is widely being used is the Back-Error Propagation (BEP). BEP of ANN was used in this research in conjunction with the Interim National Water Quality Standard (INWQS) data for Malaysia. The findings of the study shows that the classification results based on the evaluation of the water quality variables were good when compared with the results obtained from other water quality classification models, which include: the Department of Environment Water Quality Index (DOE-WQI), the Harkins'-WQI, Mahalanobis Distance Classifier, Maximum Likelihood Distance Classifier and the Decision Tree Classifier. The accuracy for BEP of ANN was found to be 86.9% and correlated well with all of these five models. The highest correlation was, with the Mahalanobis Distance Classifier and the DOE-WQI. The analysis on sensitivity shows that the BEP of ANN was sensitive to Dissolved Oxygen, a condition similar to the DOE-WQI model. Comparisons were made with two types of BEP of ANN architecture, a simple network with less number of hidden nodes and a relatively complex network with more hidden nodes. It can be concluded from the analysis that a small and simple network performed well with large samples and with test data that are widely distributed than the complex network with more hidden nodes. Using the same model, different approaches were used in evaluating the classification of water quality were applied, such as the used of the land use variables and hydrological features (LUVHF) to replace the water quality data. Using these variables, the performance of the BEP of ANN in classification of water quality was low (24% and 31%). However, its performance can be improved, if more samples with wider range of LUVHF were used. Throughout this study, the BEP of ANN model has shown some remarkable achievements. In view of these, several knowledge contributions have been made. The first contribution is the flexibility of the system approach and operationally simple to perform. Secondly, it provides a practical approach in classification of river water quality, such that through a single network computation of a sample, the results are presented promptly as the probability value and the class grade value. The third contribution is that the water quality can also be classified based on the land use variables and hydrological features, without dependence on water quality data. This approach is suitable for remote areas, where accessibility is relatively difficult

Application of data mining techniques to predict the performance of matured Vertical Flow Constructed Wetlands Systems treating urban wastewater

The rapid urbanisation and industrialisation, due to technological advancement, led to severe environmental pollution. The environmental pollution in the last few decades resulted in an adverse impact on the environment causing massive accumulation of wastewater. Wastewater is one of the closest sources of environmental problems, at the same time water scarcity is becoming alarming due to its high demand as the global population is increasing. Hence, the application for managing available water resources becomes crucial. The ever-increasing demand for water brings the need for wastewater treatment as an alternative source of water. Constructed Wetlands (CW) have gained broader research attention due to their environmental and safety benefits for wastewater treatment. In this study, over three years of monitoring performance data from 03rd December 2014 to 28th March 2018 (thirty-nine months) of the vertical flow vertical wetlands system, receiving and treating domestic wastewater, were collected and utilised to assess and investigate the treatment performance efficiency of the Vertical Flow Constructed Wetland Systems (VFCWs) for removing pollutants from wastewater. Different laboratory-scale vertical-flow constructed wetlands filters filled with gravel and planted with common reed were built to remove removal from wastewater. The overall evaluation of the system treatment performance was calculated using percentage removal efficiency. The results were recorded it was observed that all vertical flow constructed wetland filters had recorded high removal performance for the water quality parameters, irrespective of filter set-up and operation. The system was discovered to be very useful in pollutants removal (water quality parameters) with significant efficiency. However, the high cost of analysis laboratory tests, time-consuming parameters couple with uncertainties associated with an analysis of water quality variables, lead to the development of two data mining technique models Multiple Linear Regressions (MLR) and Multilayer Perceptron (MLP). To predict the wastewater treatment performance of CW by predicting selected output water quality parameters these include Chemical Oxygen Demand (COD), Biological Oxygen Demand (BOD), orthophosphate phosphorous (PO4-P), ammonium nitrogen (NH4-N) and suspended solids (SS) with respect to other known input parameters that will provide comfortable, reliable and cost-effective methods. Correlation analysis was conducted to select the most highly correlated input parameters to be used for the model development (prediction of output parameter). The monitoring dataset of all the parameters used was divided into training dataset to build prediction models (MLR and MLP) and testing dataset to validate the models constructed. In this current work, 70% of the whole data was used as a training dataset while the remaining 30% of the data set was used as a testing dataset. The prediction models built were evaluated and compared using two model evaluation criteria: graphical model evaluation (scatter plot and hydrograph) and numerical model error evaluation criteria using five model evaluation criteria, these include: Root Mean Square Error (RMSE), regression coefficient (r), Relative Absolute Error (RAE), mean absolute error (MAE) and root relative squared error (RRSE). The results obtained indicated that the predicted values of output parameters were in good agreement and relationship with their respective measured parameters. Thus, this showed that the two models built yielded satisfactory predictions and both models had performed reasonably well in predicting output variables concentrations accurately given the value of input dependent variable. Furthermore, the comparison between the model's outcomes showed that MLP model prediction performance was discovered to be better than the MLR model in a majority of water quality parameters. Both models built could be effectively used as a tool for predicting removal of water quality parameters efficiency of vertical flow constructed wetlands treating domestic wastewater and in predicting constructed wetland performance in wastewater treatment process in term of pollutants removal. The results demonstrated the potentiality of vertical flow constructed wetlands to treat domestic wastewater and remove pollutants for future reuse