Hybrid SOM+k-Means Clustering to Improve Planning, Operation and Management in Water Distribution Systems

[EN] With the advance of new technologies and emergence of the concept of the smart city, there has been a dramatic increase in available information. Water distribution systems (WDSs) in which databases can be updated every few minutes are no exception. Suitable techniques to evaluate available information and produce optimized responses are necessary for planning, operation, and management. This can help identify critical characteristics, such as leakage patterns, pipes to be replaced, and other features. This paper presents a clustering method based on self-organizing maps coupled with k-means algorithms to achieve groups that can be easily labeled and used for WDS decision-making. Three case-studies are presented, namely a classification of Brazilian cities in terms of their water utilities; district metered area creation to improve pressure control; and transient pressure signal analysis to identify burst pipes. In the three cases, this hybrid technique produces excellent results. © 2018 Elsevier Ltd. All rights reserved.This work is partially supported by Capes and CNPq, Brazilian research agencies. The use of English was revised by John Rawlins.Brentan, BM.; Meirelles, G.; Luvizotto, E.; Izquierdo Sebastián, J. (2018). Hybrid SOM+k-Means Clustering to Improve Planning, Operation and Management in Water Distribution Systems. Environmental Modelling & Software. 106:77-88. https://doi.org/10.1016/j.envsoft.2018.02.013S778810

Gerenciamento ótimo das pressões em redes de abastecimento de água através da criação de distritos de medição com base na aprendizagem de máquinas

Integrated management of water supply systems with efficient use of natural resources requires optimization of operational performances. Dividing the water supply networks into small units, so-called district metered areas (DMAs), is a strategy that allows the development of specific operational rules, responsible for improving the network performance. In this context, clustering methods congregate neighboring nodes in groups according to similar features, such as elevation or distance to the water source. Taking into account hydraulic, operational and mathematical criteria to determine the configuration of DMAs, this work presents the k-means model and a hybrid model, that combines a self-organizing map (SOM) with the k-means algorithm, as clustering methods, comparing four mathematical criteria to determine the number of DMAs, namely Silhouette, GAP, Calinski-Harabasz and Davies Bouldin. The influence of three clustering topological criteria is evaluated: the water demand, node elevation and pipe length, in order to determine the optimal number of clusters. Furthermore, to identify the best DMA configuration, the particle swarm optimization (PSO) method was applied to determine the number, cost, pressure setting of Pressure Reducing Valves and location of DMA entrances.24A gestão integrada dos sistemas de abastecimento de água com o uso eficiente dos recursos requer a otimização das operações. O agrupamento das redes de abastecimento de água em pequenas unidades, chamadas de distritos de medição (DMAs), é uma estratégia que permite o desenvolvimento de regras operacionais específicas, responsáveis por melhorar o desempenho da rede. Neste contexto, os métodos de classificação agrupam os nós vizinhos de acordo com características semelhantes, como elevação ou distância à fonte de água. Utilizando os critérios topológicos, operacionais e matemáticos para determinar a configuração dos DMAs, o trabalho apresenta um modelo k-means e um modelo híbrido, que combina um mapa auto-organizado (SOM) com o algoritmo k-means, como métodos de agrupamento. Comparou-se quatro critérios matemáticos, Silhouette, GAP, CalinskiHarabasz e Davies-Bouldin e analisou-se a influência de três critérios topológicos variáveis, a demanda de água, a elevação dos nós e o comprimento do tubo, para determinar o número ótimo de agrupamentos. Ademais, com o intuito de identificar a melhor configuração de DMAs, o método de otimização de enxame de partículas (PSO) foi aplicado para determinar o número, o custo, as pressões e a localização das entradas do DMA

Smart Water: Short-Term Forecasting Application in Water Utilities

The unyielding interconnection between water and energy has made demand forecasting a necessity for water utilities. Electricity prices driven by the time of use has impelled water utilities towards short-term water demand forecasting. The progressive new Smart Water Grid platform has helped water utilities in utilizing their Water Distribution Networks. This two-way platform has provided developers and decision makers with robust models that rely on consumer feedback. Among these models is the water demand forecasting models. Multitudinous demand forecasting methods have been developed but none have utilized model implementation practicality. Utilities differ in size, capacity, and interest. While small size utilities focus on model simplicity, larger utilities prioritize model accuracy. This work focuses on a water utility located in Essex County, Ontario, Canada. This study presents three papers that focus on investigation and evaluation of short-term water demand forecasting techniques. The first paper compares water usage between two crops (tomatoes and bell peppers) in an effort to evaluate a crop to crop forecast technique that relies on one crops watering data in order to produce forecasts for another crop, The second paper examines the effect of model type, input type, and data size on model performance and computational load. The third paper proposes a new methodology where model performance is not sacrificed for model simplification

Pattern Recognition and Clustering of Transient Pressure Signals for Burst Location

[EN] A large volume of the water produced for public supply is lost in the systems between sources and consumers. An important-in many cases the greatest-fraction of these losses are physical losses, mainly related to leaks and bursts in pipes and in consumer connections. Fast detection and location of bursts plays an important role in the design of operation strategies for water loss control, since this helps reduce the volume lost from the instant the event occurs until its effective repair (run time). The transient pressure signals caused by bursts contain important information about their location and magnitude, and stamp on any of these events a specific "hydraulic signature". The present work proposes and evaluates three methods to disaggregate transient signals, which are used afterwards to train artificial neural networks (ANNs) to identify burst locations and calculate the leaked flow. In addition, a clustering process is also used to group similar signals, and then train specific ANNs for each group, thus improving both the computational efficiency and the location accuracy. The proposed methods are applied to two real distribution networks, and the results show good accuracy in burst location and characterization.Manzi, D.; Brentan, BM.; Meirelles, G.; Izquierdo Sebastián, J.; Luvizotto Jr., E. (2019). Pattern Recognition and Clustering of Transient Pressure Signals for Burst Location. Water. 11(11):1-13. https://doi.org/10.3390/w11112279S1131111Creaco, E., & Walski, T. (2017). Economic Analysis of Pressure Control for Leakage and Pipe Burst Reduction. Journal of Water Resources Planning and Management, 143(12), 04017074. doi:10.1061/(asce)wr.1943-5452.0000846Campisano, A., Creaco, E., & Modica, C. (2010). RTC of Valves for Leakage Reduction in Water Supply Networks. Journal of Water Resources Planning and Management, 136(1), 138-141. doi:10.1061/(asce)0733-9496(2010)136:1(138)Campisano, A., Modica, C., Reitano, S., Ugarelli, R., & Bagherian, S. (2016). Field-Oriented Methodology for Real-Time Pressure Control to Reduce Leakage in Water Distribution Networks. Journal of Water Resources Planning and Management, 142(12), 04016057. doi:10.1061/(asce)wr.1943-5452.0000697Vítkovský, J. P., Simpson, A. R., & Lambert, M. F. (2000). Leak Detection and Calibration Using Transients and Genetic Algorithms. Journal of Water Resources Planning and Management, 126(4), 262-265. doi:10.1061/(asce)0733-9496(2000)126:4(262)Pérez, R., Puig, V., Pascual, J., Quevedo, J., Landeros, E., & Peralta, A. (2011). Methodology for leakage isolation using pressure sensitivity analysis in water distribution networks. Control Engineering Practice, 19(10), 1157-1167. doi:10.1016/j.conengprac.2011.06.004Jung, D., & Kim, J. (2017). Robust Meter Network for Water Distribution Pipe Burst Detection. Water, 9(11), 820. doi:10.3390/w9110820Colombo, A. F., Lee, P., & Karney, B. W. (2009). A selective literature review of transient-based leak detection methods. Journal of Hydro-environment Research, 2(4), 212-227. doi:10.1016/j.jher.2009.02.003Choi, D., Kim, S.-W., Choi, M.-A., & Geem, Z. (2016). Adaptive Kalman Filter Based on Adjustable Sampling Interval in Burst Detection for Water Distribution System. Water, 8(4), 142. doi:10.3390/w8040142Christodoulou, S. E., Kourti, E., & Agathokleous, A. (2016). Waterloss Detection in Water Distribution Networks using Wavelet Change-Point Detection. Water Resources Management, 31(3), 979-994. doi:10.1007/s11269-016-1558-5Guo, X., Yang, K., & Guo, Y. (2012). Leak detection in pipelines by exclusively frequency domain method. Science China Technological Sciences, 55(3), 743-752. doi:10.1007/s11431-011-4707-3Holloway, M. B., & Hanif Chaudhry, M. (1985). Stability and accuracy of waterhammer analysis. Advances in Water Resources, 8(3), 121-128. doi:10.1016/0309-1708(85)90052-1Sanz, G., Pérez, R., Kapelan, Z., & Savic, D. (2016). Leak Detection and Localization through Demand Components Calibration. Journal of Water Resources Planning and Management, 142(2), 04015057. doi:10.1061/(asce)wr.1943-5452.0000592Zhang, Q., Wu, Z. Y., Zhao, M., Qi, J., Huang, Y., & Zhao, H. (2016). Leakage Zone Identification in Large-Scale Water Distribution Systems Using Multiclass Support Vector Machines. Journal of Water Resources Planning and Management, 142(11), 04016042. doi:10.1061/(asce)wr.1943-5452.0000661Mounce, S. R., & Machell, J. (2006). Burst detection using hydraulic data from water distribution systems with artificial neural networks. Urban Water Journal, 3(1), 21-31. doi:10.1080/15730620600578538Covas, D., Ramos, H., & de Almeida, A. B. (2005). Standing Wave Difference Method for Leak Detection in Pipeline Systems. Journal of Hydraulic Engineering, 131(12), 1106-1116. doi:10.1061/(asce)0733-9429(2005)131:12(1106)Liggett, J. A., & Chen, L. (1994). Inverse Transient Analysis in Pipe Networks. Journal of Hydraulic Engineering, 120(8), 934-955. doi:10.1061/(asce)0733-9429(1994)120:8(934)Caputo, A. C., & Pelagagge, P. M. (2002). An inverse approach for piping networks monitoring. Journal of Loss Prevention in the Process Industries, 15(6), 497-505. doi:10.1016/s0950-4230(02)00036-0Van Zyl, J. E. (2014). Theoretical Modeling of Pressure and Leakage in Water Distribution Systems. Procedia Engineering, 89, 273-277. doi:10.1016/j.proeng.2014.11.187Izquierdo, J., & Iglesias, P. . (2004). Mathematical modelling of hydraulic transients in complex systems. Mathematical and Computer Modelling, 39(4-5), 529-540. doi:10.1016/s0895-7177(04)90524-9Lin, J., Keogh, E., Wei, L., & Lonardi, S. (2007). Experiencing SAX: a novel symbolic representation of time series. Data Mining and Knowledge Discovery, 15(2), 107-144. doi:10.1007/s10618-007-0064-zNavarrete-López, C., Herrera, M., Brentan, B., Luvizotto, E., & Izquierdo, J. (2019). Enhanced Water Demand Analysis via Symbolic Approximation within an Epidemiology-Based Forecasting Framework. Water, 11(2), 246. doi:10.3390/w11020246Meirelles, G., Manzi, D., Brentan, B., Goulart, T., & Luvizotto, E. (2017). Calibration Model for Water Distribution Network Using Pressures Estimated by Artificial Neural Networks. Water Resources Management, 31(13), 4339-4351. doi:10.1007/s11269-017-1750-2Adamowski, J., & Chan, H. F. (2011). A wavelet neural network conjunction model for groundwater level forecasting. Journal of Hydrology, 407(1-4), 28-40. doi:10.1016/j.jhydrol.2011.06.013Brentan, B., Meirelles, G., Luvizotto, E., & Izquierdo, J. (2018). Hybrid SOM+ k -Means clustering to improve planning, operation and management in water distribution systems. Environmental Modelling & Software, 106, 77-88. doi:10.1016/j.envsoft.2018.02.013Calinski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics - Theory and Methods, 3(1), 1-27. doi:10.1080/0361092740882710

Pattern recognition and clustering of transient pressure signals for burst location

A large volume of the water produced for public supply is lost in the systems between sources and consumers. An important-in many cases the greatest-fraction of these losses are physical losses, mainly related to leaks and bursts in pipes and in consumer connections. Fast detection and location of bursts plays an important role in the design of operation strategies for water loss control, since this helps reduce the volume lost from the instant the event occurs until its effective repair (run time). The transient pressure signals caused by bursts contain important information about their location and magnitude, and stamp on any of these events a specific "hydraulic signature". The present work proposes and evaluates three methods to disaggregate transient signals, which are used afterwards to train artificial neural networks (ANNs) to identify burst locations and calculate the leaked flow. In addition, a clustering process is also used to group similar signals, and then train specific ANNs for each group, thus improving both the computational efficiency and the location accuracy. The proposed methods are applied to two real distribution networks, and the results show good accuracy in burst location and characterization111

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science

Unsupervised learning based coordinated multi-task allocation for unmanned surface vehicles

In recent decades, unmanned surface vehicles (USVs) are attracting increasing attention due to their underlying capability in autonomously undertaking complex maritime tasks in constrained environments. However, the autonomy level of USVs is still limited, especially when being deployed to conduct multiple tasks simultaneously. This paper, therefore, aims to improve USVs autonomy level by investigating and developing an effective and efficient task management algorithm for multi-USV systems. To better deal with challenging requirements such as allocating vast tasks in cluttered environments, the task management has been de-composed into two submissions, i.e., task allocation and task execution. More specifically, unsupervised learning strategies have been used with an improved K-means algorithm proposed to first assign different tasks for a multi-USV system then a self-organising map (SOM) been implemented to deal with the task execution problem based upon the assigned tasks for each USV. Differing to other work, the communication problem that is crucial for USVs in a constrained environment has been specifically resolved by designing a new competition strategy for K-means algorithm. Key factors that will influence the communication capability in practical applications have been taken into account. A holistic task management architecture has been designed by integrating both the task allocation and task execution algorithms, and a number of simulations in both simulated and practical maritime environments have been carried out to validate the effectiveness of the proposed algorithms

Demand Response in Smart Grids

The Special Issue “Demand Response in Smart Grids” includes 11 papers on a variety of topics. The success of this Special Issue demonstrates the relevance of demand response programs and events in the operation of power and energy systems at both the distribution level and at the wide power system level. This reprint addresses the design, implementation, and operation of demand response programs, with focus on methods and techniques to achieve an optimized operation as well as on the electricity consumer

Advanced and novel modeling techniques for simulation, optimization and monitoring chemical engineering tasks with refinery and petrochemical unit applications

Engineers predict, optimize, and monitor processes to improve safety and profitability. Models automate these tasks and determine precise solutions. This research studies and applies advanced and novel modeling techniques to automate and aid engineering decision-making. Advancements in computational ability have improved modeling software’s ability to mimic industrial problems. Simulations are increasingly used to explore new operating regimes and design new processes. In this work, we present a methodology for creating structured mathematical models, useful tips to simplify models, and a novel repair method to improve convergence by populating quality initial conditions for the simulation’s solver. A crude oil refinery application is presented including simulation, simplification tips, and the repair strategy implementation. A crude oil scheduling problem is also presented which can be integrated with production unit models. Recently, stochastic global optimization (SGO) has shown to have success of finding global optima to complex nonlinear processes. When performing SGO on simulations, model convergence can become an issue. The computational load can be decreased by 1) simplifying the model and 2) finding a synergy between the model solver repair strategy and optimization routine by using the initial conditions formulated as points to perturb the neighborhood being searched. Here, a simplifying technique to merging the crude oil scheduling problem and the vertically integrated online refinery production optimization is demonstrated. To optimize the refinery production a stochastic global optimization technique is employed. Process monitoring has been vastly enhanced through a data-driven modeling technique Principle Component Analysis. As opposed to first-principle models, which make assumptions about the structure of the model describing the process, data-driven techniques make no assumptions about the underlying relationships. Data-driven techniques search for a projection that displays data into a space easier to analyze. Feature extraction techniques, commonly dimensionality reduction techniques, have been explored fervidly to better capture nonlinear relationships. These techniques can extend data-driven modeling’s process-monitoring use to nonlinear processes. Here, we employ a novel nonlinear process-monitoring scheme, which utilizes Self-Organizing Maps. The novel techniques and implementation methodology are applied and implemented to a publically studied Tennessee Eastman Process and an industrial polymerization unit