Water pressure optimisation for leakage management using deep reinforcement learning

In this thesis, we introduce a novel approach to pressure management using deep reinforcement learning (DRL) algorithms. Exploiting DRL algorithms to optimise pressure management in water distribution networks (WDNs) provides a more computationally efficient and resilient method to reduce background and burst leakage. Using DRL to manage pressure has proven as a valuable method to reduce leakage and carbon emissions in two case studies based on a real and benchmark water network. A cohort of eight DRL algorithms of varying natures are implemented on a benchmark test network and real network model of varying sizes to prove their scalability. An investigation on their ability to reduce both background and burst leakage is conducted to highlight their abilities with regards to different leak sizes. The application of deep reinforcement learning algorithms to control leakage in WDNs builds on from two extensive reviews of leakage management and DRL applications in the urban water systems. Collating this literature pinpoints the novelty in applying deep reinforcement learning algorithms to control pressure in WDNs and provides context to the thesis. To develop DRL algorithms fit for WDN operations, a novel python-based environment is created that can communicate the hydraulic capabilities of EPANET to the DRL agent. This involved multiple design choices including action space and observation space selection as well as formulating a reward function suitable for the multiple objectives relating to leakage reduction. Regarding background leakage, the best performing DRL algorithm resulted in 65.2% reduction in leakage in the benchmark network. However, the investigation on the real water network provided by Northumbrian Water Living has proved the strong dependency between valve locations and pressure management hence resulting in a negligible background leakage reduction. The ability of the DRL algorithms to deal with uncertainty through randomised burst nodes was investigated in the second case study. DRL policies demonstrated resilience in comparison to the standard optimisation algorithms used (differential evolution, particle swarm optimisation, and nelder mead). The best performing DRL algorithm predicted a 58.46% leakage reduction and 5650kg of reduced CO2 emissions in the benchmark water network. On the other hand, the best DRL performance optimised the real water network by reducing the leakage by 5.79% and carbon emissions by 1999kg of CO2

Proceedings of the 2nd Conference on Production Systems and Logistics (CPSL 2021)

Proceedings of the CPSL 202