Identifying Sampling Interval for Event Detection in Water Distribution Networks

It is a generally adopted policy, albeit unofficially, to sample flow and pressure data at a 15-min interval for water distribution system hydraulic measurements. Further, for flow, this is usually averaged, whereas pressure is instantaneous. This paper sets out the findings of studies into the potential benefits of a higher sampling rate and averaging for flow and pressure measurements in water distribution systems. A data set comprising sampling at 5 s (in the case of pressure), 1 min, 5 min, and 15 min, both instantaneous and averaged, for a set of flow and pressure sensors deployed within two DMAs has been used. Engineered events conducted by opening fire hydrants/wash outs were used to form a controlled baseline detection comparison with known event start times. A data analysis system using support vector regression (SVR) was used to analyze the flow and pressure time series data from the deployed sensors and hence, detect these abnormal events. Results are analyzed over different sensors and events. The overall trend in the results is that a faster sampling rate leads to earlier event detection. However, it is concluded that a sampling interval of 1 or 5 min does not significantly improve detection to the point at which it is worth the added increase in power, communications, and data management requirements with current technologies. It was discovered that averaging pressure data can result in more rapid detection when compared with using the same instantaneous sampling rate. Averaging of pressure data is also likely to provide better regulatory compliance and provide improved data for EPS hydraulic modelling. This improvement can be achieved without any additional overheads on communications by a simple firmware alteration and hence, is potentially a very low cost upgrade with significant gains

Online modelling of water distribution systems: a UK case study

Hydraulic simulation models of water distribution networks are routinely used for operational investigations and network design purposes. However, their full potential is often never realised because, in the majority of cases, they have been calibrated with data collected manually from the field during a single historic time period and, as such, reflect the network operational conditions that were prevalent at that time, and they are then applied as part of a reactive, desktop investigation. In order to use a hydraulic model to assist proactive distribution network management its element asset information must be up to date and it should be able to access current network information to drive simulations. Historically this advance has been restricted by the high cost of collecting and transferring the necessary field measurements. However, recent innovation and cost reductions associated with data transfer is resulting in collection of data from increasing numbers of sensors in water supply systems, and automatic transfer of the data to point of use. This means engineers potentially have access to a constant stream of current network data that enables a new era of "on-line" modelling that can be used to continually assess standards of service compliance for pressure and reduce the impact of network events, such as mains bursts, on customers. A case study is presented here that shows how an online modelling system can give timely warning of changes from normal network operation, providing capacity to minimise customer impact

Water quality event detection and customer complaint clustering analysis in distribution systems

Safe, clean drinking water is a foundation of society and water quality monitoring can contribute to ensuring this. A case study application of the CANARY software to historic data from a UK drinking water distribution system is described. Sensitivity studies explored appropriate choice of algorithmic parameter settings for a baseline site, performance was evaluated with artificial events and the system then transferred to all sites. Results are presented for analysis of nine water quality sensors measuring six parameters and deployed in three connected district meter areas (DMAs), fed from a single water source (service reservoir), for a 1 year period and evaluated using comprehensive water utility records with 86% of event clusters successfully correlated to causes (spatially limited to DMA level). False negatives, defined by temporal clusters of water quality complaints in the pilot area not corresponding to detections, were only approximately 25%. It was demonstrated that the software could be configured and applied retrospectively (with potential for future near real time application) to detect various water quality event types (with a wider remit than contamination alone) for further interpretation

Knowledge management for more sustainable water systems

The management and sharing of complex data, information and knowledge is a fundamental and growing concern in the Water and other Industries for a variety of reasons. For example, risks and uncertainties associated with climate, and other changes require knowledge to prepare for a range of future scenarios and potential extreme events. Formal ways in which knowledge can be established and managed can help deliver efficiencies on acquisition, structuring and filtering to provide only the essential aspects of the knowledge really needed. Ontologies are a key technology for this knowledge management. The construction of ontologies is a considerable overhead on any knowledge management programme. Hence current computer science research is investigating generating ontologies automatically from documents using text mining and natural language techniques. As an example of this, results from application of the Text2Onto tool to stakeholder documents for a project on sustainable water cycle management in new developments are presented. It is concluded that by adopting ontological representations sooner, rather than later in an analytical process, decision makers will be able to make better use of highly knowledgeable systems containing automated services to ensure that sustainability considerations are included

Self-Organizing Maps For Knowledge Discovery From Corporate Databases To Develop Risk Based Prioritization For Stagnation 

Stagnation or low turnover of water within water distribution systems may result in water quality issues, even for relatively short durations of stagnation / low turnover if other factors such as deteriorated aging pipe infrastructure are present. As leakage management strategies, including the creation of smaller pressure management zones, are implemented increasingly more dead ends are being created within networks and hence potentially there is an increasing risk to water quality due to stagnation / low turnover. This paper presents results of applying data driven tools to the large corporate databases maintained by UK water companies. These databases include multiple information sources such as asset data, regulatory water quality sampling, customer complaints etc. A range of techniques exist for exploring the interrelationships between various types of variables, with a number of studies successfully using Artificial Neural Networks (ANNs) to probe complex data sets. Self Organising Maps (SOMs), are a class of unsupervised ANN that perform dimensionality reduction of the feature space to yield topologically ordered maps, have been used successfully for similar problems to that posed here. Notably for this application, SOM are trained without classes attached in an unsupervised fashion. Training combines competitive learning (learning the position of a data cloud) and co-operative learning (self-organising of neighbourhoods). Specifically, in this application SOMs performed multidimensional data analysis of a case study area (covering a town for an eight year period). The visual output of the SOM analysis provides a rapid and intuitive means of examining covariance between variables and exploring hypotheses for increased understanding. For example, water age (time from system entry, from hydraulic modelling) in combination with high pipe specific residence time and old cast iron pipe were found to be strong explanatory variables. This derived understanding could ultimately be captured in a tool providing risk based prioritisation scores

Predicting combined sewer overflows chamber depth using artificial neural networks with rainfall radar data

Combined sewer overflows (CSOs) represent a common feature in combined urban drainage systems and are used to discharge excess water to the environment during heavy storms. To better understand the performance of CSOs, the UK water industry has installed a large number of monitoring systems that provide data for these assets. This paper presents research into the prediction of the hydraulic performance of CSOs using artificial neural networks (ANN) as an alternative to hydraulic models. Previous work has explored using an ANN model for the prediction of chamber depth using time series for depth and rain gauge data. Rainfall intensity data that can be provided by rainfall radar devices can be used to improve on this approach. Results are presented using real data from a CSO for a catchment in the North of England, UK. An ANN model trained with the pseudo-inverse rule was shown to be capable of providing prediction of CSO depth with less than 5% error for predictions more than one hour ahead for unseen data. Such predictive approaches are important to the future management of combined sewer systems

Ensemble decision tree models using RUSBoost for estimating risk of iron failure in drinking water distribution systems

Safe, trusted drinking water is fundamental to society. Discolouration is a key aesthetic indicator visible to customers. Investigations to understand discolouration and iron failures in water supply systems require assessment of large quantities of disparate, inconsistent, multidimensional data from multiple corporate systems. A comprehensive data matrix was assembled for a seven year period across the whole of a UK water company (serving three million people). From this a novel data driven tool for assessment of iron risk was developed based on a yearly update and ranking procedure, for a subset of the best quality data. To avoid a ‘black box’ output, and provide an element of explanatory (human readable) interpretation, classification decision trees were utilised. Due to the very limited number of iron failures, results from many weak learners were melded into one high-quality ensemble predictor using the RUSBoost algorithm which is designed for class imbalance. Results, exploring simplicity vs predictive power, indicate enough discrimination between variable relationships in the matrix to produce ensemble decision tree classification models with good accuracy for iron failure estimation at District Management Area (DMA) scale. Two model variants were explored: ‘Nowcast’ (situation at end of calendar year) and ‘Futurecast’ (predict end of next year situation from this year’s data). The Nowcast 2014 model achieved 100% True Positive Rate (TPR) and 95.3% True Negative Rate (TNR), with 3.3% of DMAs classified High Risk for un-sampled instances. The Futurecast 2014 achieved 60.5% TPR and 75.9% TNR, with 25.7% of DMAs classified High Risk for un-sampled instances. The output can be used to focus preventive measures to improve iron compliance

Multivariate data mining for estimating the rate of discolouration material accumulation in drinking water distribution systems

Particulate material accumulates over time as cohesive layers on internal pipeline surfaces in water distribution systems (WDS). When mobilised, this material can cause discolouration. This paper explores factors expected to be involved in this accumulation process. Two complementary machine learning methodologies are applied to significant amounts of real world field data from both a qualitative and a quantitative perspective. First, Kohonen self-organising maps were used for integrative and interpretative multivariate data mining of potential factors affecting accumulation. Second, evolutionary polynomial regression (EPR), a hybrid data-driven technique, was applied that combines genetic algorithms with numerical regression for developing easily interpretable mathematical model expressions. EPR was used to explore producing novel simple expressions to highlight important accumulation factors. Three case studies are presented: UK national and two Dutch local studies. The results highlight bulk water iron concentration, pipe material and looped network areas as key descriptive parameters for the UK study. At the local level, a significantly increased third data set allowed K-fold cross validation. The mean cross validation coefficient of determination was 0.945 for training data and 0.930 for testing data for an equation utilising amount of material mobilised and soil temperature for estimating daily regeneration rate. The approach shows promise for developing transferable expressions usable for pro-active WDS management

Ensemble decision tree models using RUSBoost for estimating risk of iron failure in drinking water distribution systems

Safe, trusted drinking water is fundamental to society. Discolouration is a key aesthetic indicator visible to customers. Investigations to understand discolouration and iron failures in water supply systems require assessment of large quantities of disparate, inconsistent, multidimensional data from multiple corporate systems. A comprehensive data matrix was assembled for a seven year period across the whole of a UK water company (serving three million people). From this a novel data driven tool for assessment of iron risk was developed based on a yearly update and ranking procedure, for a subset of the best quality data. To avoid a ‘black box’ output, and provide an element of explanatory (human readable) interpretation, classification decision trees were utilised. Due to the very limited number of iron failures, results from many weak learners were melded into one high-quality ensemble predictor using the RUSBoost algorithm which is designed for class imbalance. Results, exploring simplicity vs predictive power, indicate enough discrimination between variable relationships in the matrix to produce ensemble decision tree classification models with good accuracy for iron failure estimation at District Management Area (DMA) scale. Two model variants were explored: ‘Nowcast’ (situation at end of calendar year) and ‘Futurecast’ (predict end of next year situation from this year’s data). The Nowcast 2014 model achieved 100% True Positive Rate (TPR) and 95.3% True Negative Rate (TNR), with 3.3% of DMAs classified High Risk for un-sampled instances. The Futurecast 2014 achieved 60.5% TPR and 75.9% TNR, with 25.7% of DMAs classified High Risk for un-sampled instances. The output can be used to focus preventive measures to improve iron compliance