Water demand forecasting using machine learning on weather and smart metering data

Water scarcity is a global threat due to lifestyle and climate changes, pollution of water resources, as well as a rapidly growing population. The UK water industry’s regulators demand plans from water companies to sustainably manage their water resources, reduce per capita consumption and leakage, and create projections for climate change scenarios. This work addresses critical problems of water demand by expanding the understanding of water use and developing improved forecasting methods. As part of this effort, the influence of the weather is thoroughly investigated, using a disaggregated, big-data statistical analysis. Results show that the weather effect on water consumption is overall limited, non-linear, and variable over time and households. Next, a short-term demand forecasting model is developed, based on Random Forests, that predicts household consumption using several socio-economic, customer and temporal characteristics. This model is of significant value due to its accuracy as well as accompanying methodology that allows the interpretation of results. In order to further improve the forecasting accuracy achieved using Random Forests, a new modelling technique is developed. The new method that uses model stacking and bias correction, outperforms most other forecasting models, especially when past consumption data are not available, as well as for peak consumption days. Finally, a water demand forecasting model based on Gradient Boosting Machines is trained at different levels of spatial aggregation, for different input configurations. Results show that the spatial scale has a strong influence on the best model predictors and the maximum forecasting accuracy that can be achieved. The methodology developed here can be used as a guide for researchers, water utilities and network operators to identify the methods, data and models to produce accurate water demand forecasts, based on the characteristics and limitations of the problem

Event Management and Event Response Planning for Smart Water Networks

The water industry in the UK and worldwide has a pressing need to better manage interruptions to water supply caused by various failure events, such as pipe bursts, equipment failures or water treatment work shutdowns. One way of doing this is by making use of the increasingly available real-time sensor data collected in water distribution systems, as well as by using hydraulic models in real-time. Currently, real-time sensor data and real-time hydraulic modelling are not used much in a water utility’s control room, especially when it comes to identifying a suitable strategy to respond to failure events in near real-time. This PhD project aims to develop, test/validate and demonstrate a new response methodology to support decisions made by control room operators when dealing with various failure events in a water distribution system. An integral part of this work is to develop an interactive decision-support tool, which will guide/support operators in identifying an effective response solution in near real-time (i.e. usually required up to 1 hour after the event detection/localisation if unforeseen events in the field are not considered). The tool will be used in this thesis to test and validate the response methodology. The proposed response methodology considers: (i) structured yet flexible approach supporting and guiding the operator throughout the entire response process to water network failure events, whilst allowing the operator to have a final say; (ii) novel interaction with the operator in near real-time via the proposed tool (e.g. allowing operators to propose different ‘what-if’ scenarios without being hydraulic experts); (iii) provision of automatically generated advices (e.g. near optimal response solutions via a novel heuristic algorithm and assessed end-impacts); and (iv) improved impact assessment. An integral part of the response methodology is the development of a novel method to identify near optimal response to failures in water distribution networks. The response problem is formulated as a two-objective optimisation problem with objectives being the minimisation of failure impacts and related operational costs. The heuristic-based method is developed and used to solve this problem. For the first objective (i.e. impact assessment), a new impact assessment method is developed, using realistic impact indicators that cover different aspects of the event - which are consistently calculated for every proposed response solution (to facilitate easy comparison between different response solutions). The response methodology was tested, validated and demonstrated on a semi-real case study. The implementation of the response methodology via the tool enabled operators to identify a response solution better (i.e. with lower end-impact and cost) than the solution based on the current response practice of utilities. The results obtained from this case study, demonstrate that the response methodology works well and that it has a potential to improve water utilities’ current practice. The heuristic optimisation method that is integral part of the response methodology was validated and demonstrated on two semi-real case studies. Based on the results obtained it can be concluded that the heuristic-based method works well (i.e. it is reliable and robust) and is able to identify near optimal response solutions in a computationally fast manner. This, in turn, enables this method to be used in near real-time in real-life situations.United Utilities plcUnited Utilities pl

Multilevel Delayed Acceptance MCMC with Applications to Hydrogeological Inverse Problems

Quantifying the uncertainty of model predictions is a critical task for engineering decision support systems. This is a particularly challenging effort in the context of statistical inverse problems, where the model parameters are unknown or poorly constrained, and where the data is often scarce. Many such problems emerge in the fields of hydrology and hydro--environmental engineering in general, and in hydrogeology in particular. While methods for rigorously quantifying the uncertainty of such problems exist, they are often prohibitively computationally expensive, particularly when the forward model is high--dimensional and expensive to evaluate. In this thesis, I present a Metropolis--Hastings algorithm, namely the Multilevel Delayed Acceptance (MLDA) algorithm, which exploits a hierarchy of forward models of increasing computational cost to significantly reduce the total cost of quantifying the uncertainty of high--dimensional, expensive forward models. The algorithm is shown to be in detailed balance with the posterior distribution of parameters, and the computational gains of the algorithm is demonstrated on multiple examples. Additionally, I present an approach for exploiting a deep neural network as an ultra--fast model approximation in an MLDA model hierarchy. This method is demonstrated in the context of both 2D and 3D groundwater flow modelling. Finally, I present a novel approach to adaptive optimal design of groundwater surveying, in which MLDA is employed to construct the posterior Monte Carlo estimates. This method utilises the posterior uncertainty of the primary problem in conjunction with the expected solution to an adjoint problem to sequentially determine the optimal location of the next datapoint.Engineering and Physical Sciences Research Council (EPSRC)Alan Turing InstituteEngineering and Physical Sciences Research Council (EPSRC

Automated analysis of sewer CCTV surveys

Sewers across the globe must be regularly inspected to ensure their smooth running and effective maintenance. Furthermore, surveys are often performed reactively, often diagnosing suspected faults within a network. Almost all surveys in the UK and abroad are performed using closed circuit television (CCTV) cameras where pipes are too small for manual inspection. As such vast quantities of footage are recorded by surveying teams on a daily basis. This footage is currently analysed manually, requiring a trained engineer to watch through its entirety, annotating potential faults. This thesis examines methods of improving this labelling process, implementing various machine learning and image processing techniques to automate this procedure. The thesis presents two distinct methodologies: the first for the detection of faults, using only raw CCTV footage, whilst the second identifies the type of a detected fault according to the Manual of Sewer Condition Classification. The fault detection methodology identifies the presence of a fault within a CCTV image. The methodology calculates a GIST feature descriptor for each video frame, before utilising a Random Forest classifier, to predict the presence of a fault. The basic methodology was further refined with the inclusion of smoothing, to eliminate isolated inconstancies, and stacking to intuitively combine the results of multiple machine learning classifiers. The final methodology achieved a detection accuracy of 86% on unseen real-life data from the UK. The fault classification methodology identifies the fault type in images, where faults have been previously detected using the above technique. The tool again calculates a frame’s GIST descriptor before applying multiple Random Forest classifiers in a ‘1 vs all’ architecture to predict the type of a given fault. This architecture allowed for comparative classifications and later enabled the identification of multiple faults within a single frame. The methodology achieved a peak accuracy of 74% when classifying faults well represented by the dataset (at least 100 examples). Furthermore, when including multi-label functionality, the tool achieved an accuracy of 67% across all fault types. Both methods have been developed to be holistic and practical, utilising only industry standard CCTV footage and generalising well across all types of sewer system (size, shape and material). Furthermore, as both methodologies rely on the same feature descriptor, they integrate well to form a methodology that could be applied in real time. As such the thesis also explores the practical implications of creating a detection support tool capable of integration with current working practices. In combination both methodologies and their additions present a unique contribution to the field of automated sewer surveying. Achieving competitive accuracies with a streamlined methodology, the technology shows promise for future application in industry, greatly increasing the speed, accuracy and consistency of CCTV sewer surveys