Use of an artificial neural network to capture the domain knowledge of a conventional hydraulic simulation model

"The definitive peer-reviewed and edited version of this article is published in Journal of Hydroinformatics , vol.. 9, n. 1[ (15-27) 2007. DOI: 10.2166/hydro.2006.01 and is available at www.iwapublishing.com.”[EN] As part of the POWADIMA research project, this paper describes the technique used to predict the consequences of different control settings on the performance of the water-distribution network, in the context of real-time, near-optimal control. Since the use of a complex hydraulic simulation model is somewhat impractical for real-time operations as a result of the computational burden it imposes, the approach adopted has been to capture its domain knowledge in a far more efficient form by means of an artificial neural network (ANN). The way this is achieved is to run the hydraulic simulation model off-line, with a large number of different combinations of initial tank-storage levels, demands, pump and valve settings, to predict future tank-storage water levels, hydrostatic pressures and flow rates at critical points throughout the network. These input/output data sets are used to train an ANN, which is then verified using testing sets. Thereafter, the ANN is employed in preference to the hydraulic simulation model within the optimization process. For experimental purposes, this technique was initially applied to a small, hypothetical water-distribution network, using EPANET as the hydraulic simulation package. The application to two real networks is described in subsequent papers of this series.The POWADIMA research project was funded by the European Commission under its Vth Framework thematic programme on Energy, Environment and Sustainable Development (Contract Number EVK1-CT-2000-00084). The authors would like to take this opportunity to thank the Commission and project officers for their support throughout the duration of the contract.Rao, Z.; Alvarruiz Bermejo, F. (2007). Use of an artificial neural network to capture the domain knowledge of a conventional hydraulic simulation model. Journal of Hydroinformatics. 9(1):15-24. doi:10.2166/hydro.2006.014S15249

Reducción del Tiempo de Simulación de Redes de Distribución de Agua, mediante el Método de Mallas y la Computación de Altas Prestaciones

[EN] Computer simulation of water distribution networks by means of mathematical models is nowadays an indispensable tool for the design and exploitation of those networks. Simulation is used not only for the design of new supply systems, or modifications and extensions of existing systems, but also for the normal operation tasks carried out in any network. Two main types of simulation can be differentiated: hydraulic simulation, by means of which the pressures and flows registered in the network are computed, and water quality simulation, the objective of which is to obtain information about chemical substance concentrations. The need for simulation comes often in the context of a wider problem of optimization or reliability analysis, which requires performing a large number of simulations, thus resulting in a process with considerable computational complexity. This fact, added to the growing size and level of detail of network models, as a consequence of the automatic incorporation of data coming from Geographical Information Systems, means that the performance of the simulation solver has a great impact in the overall computing time. In this context, this thesis considers and explores different strategies to improve the performance of water distribution network simulation. The first strategy consists of making some contributions to the hydraulic simulation method known as Looped Newton-Raphson (or more simply the loop method), which is based on the consideration of flow corrections associated to a set of independent loops within the network. Even though the method known as Global Gradient Algorithm (GGA) is more widely used and accepted, the loop method has the potential to be faster, owing to the smaller size of the underlying linear systems. In this thesis some contributions are presented to improve the performance of the loop method for hydraulic simulation. Firstly, efficient algorithms are developed for the selection of a suitable set of independent loops, leading to a highly sparse linear system. Secondly, methods are developed for efficient modeling of hydraulic valves, and especially pressure reducing/sustaining valves. The second strategy explored is the introduction of high performance computing in the hydraulic simulation using distributed memory platforms. In particular, the code of Epanet, a widely accepted water distribution network simulation software, is taken as the starting point for the introduction of parallel simulation algorithms, using the Message Passing Interface (MPI) tool for inter-process communications. As a result of this work, firstly a parallel algorithm is presented for the simulation of flows and pressures by means of the GGA method, making use of multifrontal algorithms for the parallel solution of the underlying linear systems. Secondly, a parallel algorithm for water quality simulation by means of the Discrete Volume Element Method (DVEM) is described, based on partitioning the network by means of multilevel recursive bisection algorithms. Thirdly, a parallel method is presented for leakage minimization by finding the optimal pressure settings for a set of pressure-reducing valves. In distributed memory platforms the overhead due to communication and synchronization can be excessively high, counterbalancing the gain derived from the division of the computation among the processors. This effect is less pronounced in shared memory platforms such as multicore systems, which have gained popularity over the last years. This fact motivates the third strategy explored in this thesis, which is the development of parallel algorithms for simulation of flows and pressures using multicore systems. OpenMP is the tool used for the parallelization, both of the method GGA as implemented in Epanet software and of the loop method with the contributions on it that have been made in the context of this thesis.[ES] La simulación por computador de las redes de distribución de agua potable, mediante el uso de modelos matemáticos, es hoy en día una herramienta indispensable para el diseño y la explotación de dichas redes. La simulación se utiliza tanto en el diseño de nuevos abastecimientos y en ampliaciones o modificaciones de abastecimientos existentes, como en las tareas de operación normales de cualquier red. Se puede diferenciar entre dos tipos de simulación: la simulación hidráulica, que permite obtener las presiones y caudales que se registran en la red, y la simulación de la calidad del agua, cuyo objetivo es obtener información sobre concentraciones de sustancias químicas. A menudo la necesidad de simulación surge dentro de un problema más amplio de optimización o de análisis de fiabilidad, que requiere llevar a cabo un gran número de simulaciones, con lo que el proceso completo resulta de una complejidad computacional considerable. Esto, añadido al hecho de que el tamaño y nivel de detalle de los modelos de redes crece constantemente, como consecuencia de la incorporación automática de datos contenidos en Sistemas de Información Geográfica, hace que las prestaciones del solver de simulación tengan un gran impacto en el tiempo total de cálculo necesario. En este contexto, esta tesis considera y explora distintas vías para mejorar las prestaciones de la simulación de redes de distribución de agua. La primera de estas vías consiste en realizar algunas aportaciones al método de simulación hidráulica conocido como método de Newton-Raphson de mallas, el cual se basa en la consideración de caudales correctores asociados a un conjunto de mallas independientes definidas sobre la red. Aunque el método conocido como Algoritmo del Gradiente Global (GGA) goza de mayor aceptación, el método de mallas tiene el potencial de ser más rápido, debido al menor tamaño de los sistemas lineales subyacentes. Esta tesis presenta aportaciones para mejorar las prestaciones del método de mallas de simulación hidráulica. En primer lugar, se desarrollan algoritmos eficientes para la selección de un conjunto de mallas adecuado, que conduzca a un sistema altamente disperso. En segundo lugar se desarrollan métodos para la modelización eficiente de válvulas, y especialmente válvulas reductoras/sostenedoras de presión. La segunda vía explorada es la introducción de la computación de altas prestaciones en la simulación hidráulica usando plataformas de memoria distribuida. En particular, se parte del código de Epanet, un software de simulación de redes de amplia aceptación, y se introducen en él algoritmos paralelos de simulación, usando la herramienta Message Passing Interface (MPI) para la comunicación entre procesos. Como resultado de ello, se presenta en primer lugar un algoritmo paralelo para la simulación de caudales y presiones por medio del método GGA, haciendo uso de algoritmos multifrontales para la resolución paralela de los sistemas lineales subyacentes. En segundo lugar, se describe un algoritmo paralelo para la simulación de la calidad del agua mediante el Método de Elementos Discretos de Volumen (DVEM), particionando la red por medio de algoritmos de bisección recursiva multinivel. En tercer lugar, se presenta un método paralelo para la minimización de fugas mediante la determinación de las consignas óptimas de una serie de válvulas reductoras de presión. Finalmente, la tercera vía explorada es el desarrollo de algoritmos paralelos sobre memoria compartida para la simulación de presiones y caudales. Se considera con ello un tipo de plataformas que han ganado popularidad en los últimos años. Se utiliza la herramienta OpenMP para la paralelización, tanto de Epanet y de su implementación del método GGA, como del método de mallas, con las aportaciones al mismo que se han realizado en el contexto de esta tesis.[CA] La simulació per computador de les xarxes de distribució d'aigua potable, per mitjà de l'ús de models matemàtics, es hui en dia una ferramenta indispensable per al disseny i l'explotació d'abastiments d'aigua. La simulació s'utilitza tant per al disseny de nous abastiments o ampliacions i modificacions d'abastiments existents, com per a les tasques d'operació normals en qualsevol xarxa. Es pot diferenciar entre dos tipus de simulació: la simulació hidràulica, que permet obtindre les pressions i cabals que es produeixen en la xarxa, i la simulació de la qualitat de l'aigua, l'objectiu de la qual és obtindre informació sobre concentracions de substàncies químiques. Sovint la necessitat de simulació sorgeix dins d'un problema més ampli d'optimització o d'anàlisi de fiabilitat, que requereix dur a terme un gran nombre de simulacions, amb la qual cosa el procés complet resulta d'una complexitat computacional considerable. Això, afegit al fet de que la grandària i nivell de detall del models de xarxes creix constantment, com a conseqüència de la incorporació automàtica de dades contingudes en Sistemes d'Informació Geogràfica, fa que les prestacions del solver de simulació tinguen un gran impacte en el temps total de càlcul necessari. En este context, esta tesi considera i explora diferents vies per a millorar les prestacions de la simulació de xarxes de distribució d'aigua. La primera d'estes vies consisteix en realitzar algunes contribucions al mètode de simulació hidràulica conegut com mètode de Newton-Raphson de malles (o simplement mètode de malles), el qual es basa en la consideració de cabals correctors associats a un conjunt de malles independents definides en la xarxa. Encara que el mètode conegut com Algorisme del Gradient Global (GGA) gaudeix de major acceptació, el mètode de malles té el potencial de ser més ràpid, degut a la menor grandària dels sistemes lineals subjacents. En esta tesi es presenten contribucions per a millorar les prestacions del mètode de malles de simulació hidràulica. En concret, en primer lloc es desenvolupen algorismes eficients per a la selecció d'un conjunt de malles adequat, que conduïsca a un sistema lineal altament dispers. En segon lloc es desenvolupen mètodes per a la modelització eficient de vàlvules, i especialment vàlvules reductores/sostenidores de pressió. La segona via explorada és la introducció de la computació d'altes prestacions en la simulació hidràulica utilitzant plataformes de memòria distribuïda. En concret, es parteix del codi d'Epanet, un programari de simulació de xarxes de distribució d'aigua d'amplia acceptació, i s'hi introdueixen algorismes paral·lels de simulació, utilitzant la ferramenta Message Passing Interface (MPI) per a la comunicació entre processos. Com a resultat d'este treball, es presenta en primer lloc un algorisme paral·lel per a la simulació de cabals i pressions per mitjà del mètode GGA, fent ús d'algorismes multifrontals per a la resolució en paral·lel dels sistemes lineals subjacents. En segon lloc, es descriu un algorisme paral·lel per a la simulació de la qualitat d'aigua amb el Mètode d'Elements Discrets de Volum (DVEM), particionant la xarxa per mitjà d'algoritmes de bisecció recursiva multinivell. En tercer lloc es presenta un mètode paral·lel per a la minimització de fugues mitjançant la determinació de les consignes òptimes d'una sèrie de vàlvules reductores de pressió. Finalment, la tercera via explorada és el desenvolupament d'algorismes paral·lels sobre memòria compartida per a la simulació de pressions i cabals. Es considera amb això un tipus de plataformes que han guanyat popularitat en els últims anys. S'utilitza la ferramenta OpenMP per a la paral·lelització, tant del programari Epanet i de la seua implementació del mètode GGA, com del mètode de malles, amb les contribucions al mateix que s'han realitzat en el context d'esta tesi.Alvarruiz Bermejo, F. (2016). Reducción del Tiempo de Simulación de Redes de Distribución de Agua, mediante el Método de Mallas y la Computación de Altas Prestaciones [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/61764TESI

Improving the performance of water distribution systems’ simulation on multicore systems

The final publication is available at Springer via http://dx.doi.org/10.1007/s11227-015-1607-5Hydraulic solvers for the simulation of flows and pressures in water distribution systems (WDS) are used extensively, and their computational performance is key when considering optimization problems. This paper presents an approach to speedup the hydraulic solver using OpenMP with two efficient methods for WDS simulation. The paper identifies the different tasks carried out in the simulation, showing their contribution to the execution time, and selecting the target tasks for parallelization. After describing the algorithms for the selected tasks, parallel OpenMP versions are derived, with emphasis on the task of linear system update. Results are presented for four different large WDS models, showing considerable reduction in computing timeThis work has been partially supported by Ministerio de Economia y Competitividad from Spain, under the project TEC2012-38142-C04-01, and by project PROMETEO FASE II 2014/003 of Generalitat Valenciana.Alvarruiz Bermejo, F.; Martínez Alzamora, F.; Vidal Maciá, AM. (2016). Improving the performance of water distribution systems’ simulation on multicore systems. Journal of Supercomputing. 1-13. https://doi.org/10.1007/s11227-015-1607-5S113Abraham E, Stoianov I (2015) Efficient preconditioned iterative methods for hydraulic simulation of large scale water distribution networks. Proc Eng 119:623–632Abraham E, Stoianov I (2015) Sparse null space algorithms for hydraulic analysis of large-scale water supply networks. J Hydraul Eng. doi: 10.1061/(ASCE)HY.1943-7900.0001089Alonso JM, Alvarruiz F, Guerrero D et al (2000) Parallel computing in water network analysis and leakage minimization. J Water Resour Plan Manag 126(4):251–260Alvarruiz F, Martínez-Alzamora F, Vidal AM (2015) Efficient simulation of water distribution systems using openmp. In: 15th International conference computational and mathematical methods in computational mathematics, science and engineering, pp 125–129Alvarruiz F, Martínez-Alzamora F, Vidal AM (2015) Improving the efficiency of the loop method for the simulation of water distribution systems. J Water Resour Plan Manag 141(10):04015019Burger G, Sitzenfrei R, Kleidorfer M, Rauch W (2015) Quest for a new solver for EPANET 2. J Water Resour Plan Manag. doi: 10.1061/(ASCE)WR.1943-5452.0000596Creaco E, Franchini M (2014) Comparison of Newton–Raphson global and loop algorithms for water distribution network resolution. J Hydraul Eng 140(3):313–321Creaco E, Franchini M (2015) The identification of loops in water distribution networks. Proc Eng 119:506–515 Computing and Control for the Water Industry (CCWI2015) Sharing the best practice in water managementCrous PA, van Zyl JE, Roodt Y (2012) The potential of graphical processing units to solve hydraulic network equations. J Hydroinf 14:603–612Elhay S, Simpson A, Deuerlein J, Alexander B, Schilders W (2014) Reformulated co-tree flows method competitive with the global gradient algorithm for solving water distribution system equations. J Water Resour Plan Manag 140(12):04014040Epp R, Fowler AG (1970) Efficient code for steady-state flows in networks. J Hydraul Div 96(1):43–56Guidolin M, Burovskiy P, Kapelan Z, Savić D (2010) Cwsnet: an object-oriented toolkit for water distribution system simulations. In: Proceedings of 12th water distribution system analysis symposium, ASCE, Reston, VAGuidolin M, Kapelan Z, Savic D (2013) Using high performance techniques to accelerate demand-driven hydraulic solvers. J Hydroinf 15(1):38–54Guidolin M, Kapelan Z, Savic D, Giustolisi O (2010) High performance hydraulic simulations with epanet on graphics processing units. In: Proceedings of 9th international conference on hydroinformaticsOstfeld A, Uber J, Salomons E et al (2008) The battle of the water sensor networks (BWSN): a design challenge for engineers and algorithms. J Water Resour Plan Manag 134(6):556–568Rossman AL (2000) Epanet 2 users manual. Water Supply and Water Resources Division, US Environment Protection AgencyTodini E, Pilati S (1988) Computer applications in water supply: vol. 1—systems analysis and simulation. In: Coulbeck B, Orr CH (eds) A gradient algorithm for the analysis of pipe networks. Research Studies Press Ltd, Letchworth, Hertfordshire, UK, pp 1–2

Efficient Modeling of Active Control Valves in Water Distribution Systems Using the Loop Method

[EN] This paper presents a novel approach to model pressure- and flow-regulating devices in the context of the Newton-Raphson loop method for water distribution network simulation. The proposed approach uses a symmetric matrix for the underlying linear systems, which enables simpler implementation and faster solution, while producing iterations very close to the global gradient algorithm of EPANET. The structure of the matrix is kept unchanged regardless of the operational status of the valves. The paper presents results that validate its formulation, accuracy, and speed in various case studies.Alvarruiz Bermejo, F.; Martínez Alzamora, F.; Vidal Maciá, AM. (2018). Efficient Modeling of Active Control Valves in Water Distribution Systems Using the Loop Method. Journal of Water Resources Planning and Management. 144(10):1-9. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000982S191441

Improving the efficiency of the loop method for the simulation of water distribution networks

Efficiency of hydraulic solvers for the simulation of flows and pressures in water distribution systems (WDSs) is very important, especially in the context of optimization and risk analysis problems, where the hydraulic simulation has to be repeated many times. Among the methods used for hydraulic solvers, the most prominent nowadays is the global gradient algorithm (GGA), based on a hybrid node-loop formulation. Previously, another method based just on loop flow equations was proposed, which presents the advantage that it leads to a system matrix that is in most cases much smaller than in the GGA method, but has also some disadvantages, mainly a less sparse system matrix and the fact that introducing some types of valves requires the redefinition of the set of network loops initially defined. The contribution of this paper is to present solutions for overcoming the mentioned disadvantages of the method based on loop flow equations. In particular, efficient procedures are shown for selecting the network loops so as to achieve a highly sparse matrix and methods are presented to incorporate check valves and automatic control valves while avoiding the need to redefine the loops initially selected. (C) 2015 American Society of Civil Engineers.This work has been partially supported by "Ministerio de Economia y Competitividad" from Spain, under the project TEC2012-38142-C04-01 and by PROMETEO FASE II 2014/003 project of Generalitat Valenciana.Alvarruiz Bermejo, F.; Martínez Alzamora, F.; Vidal Maciá, AM. (2015). Improving the efficiency of the loop method for the simulation of water distribution networks. Journal of Water Resources Planning and Management. 141(10):1-10. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000539S1101411

Efficient Simulation of Water Distribution Systems using OpenMP

[EN] Hydraulic solvers for the simulation of flows and pressures in water distribution systems (WDS) are used extensively, and their computational performance is key when considering optimization problems. This paper presents an approach to speedup the hydrualic solver using OpenMP. The procedure is based on a parallel algorithm for the generation of the linear system of equations. Preliminary results show the validity of the approach.The authors would like to thank the Spanish “Ministerio de Economía y Competitividad” fort he project “Migrable Elastic Virtual Clusters on Hybrid Cloud Infrastructures” (TIN2013-44390-R).Alvarruiz Bermejo, F.; Martínez Alzamora, F.; Vidal Maciá, AM. (2015). Efficient Simulation of Water Distribution Systems using OpenMP. CMMSE. http://hdl.handle.net/10251/75303

Accurate Approximation of the Matrix Hyperbolic Cosine Using Bernoulli Polynomials

[EN] This paper presents three different alternatives to evaluate the matrix hyperbolic cosine using Bernoulli matrix polynomials, comparing them from the point of view of accuracy and computational complexity. The first two alternatives are derived from two different Bernoulli series expansions of the matrix hyperbolic cosine, while the third one is based on the approximation of the matrix exponential by means of Bernoulli matrix polynomials. We carry out an analysis of the absolute and relative forward errors incurred in the approximations, deriving corresponding suitable values for the matrix polynomial degree and the scaling factor to be used. Finally, we use a comprehensive matrix testbed to perform a thorough comparison of the alternative approximations, also taking into account other current state-of-the-art approaches. The most accurate and efficient options are identified as results.This research was supported by the Vicerrectorado de Investigacion de la Universitat Politecnica de Valencia (PAID-11-21).Alonso Abalos, JM.; Ibáñez González, JJ.; Defez Candel, E.; Alvarruiz Bermejo, F. (2023). Accurate Approximation of the Matrix Hyperbolic Cosine Using Bernoulli Polynomials. Mathematics. 11(3):1-22. https://doi.org/10.3390/math1103052012211

Parallel Computing in Water Network Analysis and Leakage Minimization

[EN] In this paper a parallel computing based software demonstrator for the simulation and leakage minimization of water networks is presented. This demonstrator, based on the EPANET package, tackles three different types of problems making use of parallel computing. First, the solution of the hydraulic problem is treated by means of the gradient method. The key point in the parallelization of the method is the solution of the underlying linear systems, which is carried out by means of a multifrontal Choleski method. Second, the water quality simulation problem is approached by using the discrete volume element method. The application of parallel computing is based on dividing the water network in several parts using the multilevel recursive bisection graph partitioning algorithm. Finally, the problem of leakage minimization using pressure reducing valves is approached. This results in the formulation of an optimization problem for each time step, which is solved by means of sequential quadratic programming. Because these subproblems are independent of each other, they can be solved in parallel.The writers wish to acknowledge the financial support provided by the ESPRIT program of the European Commission (HIPERWATER, ESPRIT project 24003), by the CICYT TIC96-1062-C03-01 project, and also by research staff training grants from the Spanish government and the autonomous government of the Comunidad Valenciana in Spain.Alonso Ábalos, JM.; Alvarruiz Bermejo, F.; Guerrero López, D.; Hernández García, V.; Ruiz Martínez, PA.; Vidal Maciá, AM.; Martínez Alzamora, F.... (2000). Parallel Computing in Water Network Analysis and Leakage Minimization. Journal of Water Resources Planning and Management. 126(4):251-260. https://doi.org/10.1061/(ASCE)0733-9496(2000)126:4(251)S251260126

Improvements to SLEPc in releases 3.14-3.18

[EN] This short article describes the main newfeatures added to SLEPc, the Scalable Library for Eigenvalue Problem Computations, in the past two and a half years, corresponding to five release versions. The main novelty is the extension of the SVD module with new problem types, such as the generalized SVD or the hyperbolic SVD. Additionally, many improvements have been incorporated in different parts of the library, including contour integral eigensolvers, preconditioning, and GPU support.Jose E. Roman; Alvarruiz Bermejo, F.; Campos, C.; Dalcin, L.; Jolivet, P.; Lamas Daviña, A. (2023). Improvements to SLEPc in releases 3.14-3.18. ACM Transactions on Mathematical Software. 49(3). https://doi.org/10.1145/360337349

Ejercicios de programación paralela con OpenMP y MPI

Este libro recopila un conjunto de ejemplos de programación paralela, presentados de forma didáctica a modo de ejercicios resueltos.Las herramientas utilizadas son OpenMP y MPI,dos de las más utilizadas en la actualidad para la implementación de algoritmos paralelos. Los algoritmos paralelos se emplean para aprovechar al máximo la capacidad de los computadores paralelos actuales.OpenMP es un sistema de programación basado en directivas de compilador apropiado para computadores de memoria compartida, tales como los procesadores multinúcleo.Por su parte, MPI se utiliza para la programación de computadores paralelos de memoria distribuida, ejemplo de los cuales son los clusters: un conjunto de ordenadores conectados mediante una red que se pueden utilizar simultáneamente para resolver problemas de gran coste computacional. El libro se dirige a estudiantes universitarios o profesionales con interés en la programación paralela. A diferencia de otros libros, la orientación es muy práctica, mostrándose casos que se pueden encontrar habitualmente. Para poder seguir el libro es necesario tener conocimientos de programación en lenguaje CRomán Moltó, JE.; Alonso Ábalos, JM.; Blanquer Espert, I.; Guerrero López, D.; Ibáñez González, JJ.; Ramos Peinado, E.; Alvarruiz Bermejo, F. (2018). Ejercicios de programación paralela con OpenMP y MPI. Editorial Universitat Politècnica de València. http://hdl.handle.net/10251/113119EDITORIA