Optimal control towards sustainable wastewater treatment plants based on multi-agent reinforcement learning

Wastewater treatment plants are designed to eliminate pollutants and alleviate environmental pollution. However, the construction and operation of WWTPs consume resources, emit greenhouse gases (GHGs) and produce residual sludge, thus require further optimization. WWTPs are complex to control and optimize because of high nonlinearity and variation. This study used a novel technique, multi-agent deep reinforcement learning, to simultaneously optimize dissolved oxygen and chemical dosage in a WWTP. The reward function was specially designed from life cycle perspective to achieve sustainable optimization. Five scenarios were considered: baseline, three different effluent quality and cost-oriented scenarios. The result shows that optimization based on LCA has lower environmental impacts compared to baseline scenario, as cost, energy consumption and greenhouse gas emissions reduce to 0.890 CNY/m3-ww, 0.530 kWh/m3-ww, 2.491 kg CO2-eq/m3-ww respectively. The cost-oriented control strategy exhibits comparable overall performance to the LCA driven strategy since it sacrifices environmental bene ts but has lower cost as 0.873 CNY/m3-ww. It is worth mentioning that the retrofitting of WWTPs based on resources should be implemented with the consideration of impact transfer. Specifically, LCA SW scenario decreases 10 kg PO4-eq in eutrophication potential compared to the baseline within 10 days, while significantly increases other indicators. The major contributors of each indicator are identified for future study and improvement. Last, the author discussed that novel dynamic control strategies required advanced sensors or a large amount of data, so the selection of control strategies should also consider economic and ecological conditions

A contribution to support decision making in energy/water sypply chain optimisation

The seeking of process sustainability forces enterprises to change their operations. Additionally, the industrial globalization implies a very dynamic market that, among other issues, promotes the enterprises competition. Therefore, the efficient control and use of their Key Performance Indicators, including profitability, cost reduction, demand satisfaction and environmental impact associated to the development of new products, is a significant challenge. All the above indicators can be efficiently controlled through the Supply Chain Management. Thus, companies work towards the optimization of their individual operations under competitive environments taking advantage of the flexibility provided by the virtually inexistent world market restrictions. This is achieved by the coordination of the resource flows, across all the entities and echelons belonging to the system network. Nevertheless, such coordination is significantly complicated if considering the presence of uncertainty and even more if seeking for a win-win outcome. The purpose of this thesis is extending the current decision making strategies to expedite these tasks in industrial processes. Such a contribution is based on the development of efficient mathematical models that allows coordinating large amount of information synchronizing the production and distribution tasks in terms of economic, environmental and social criteria. This thesis starts presents an overview of the requirements of sustainable production processes, describing and analyzing the current methods and tools used and identifying the most relevant open issues. All the above is always within the framework of Process System Engineering literature. The second part of this thesis is focused in stressing the current Multi-Objective solution strategies. During this part, first explores how the profitability of the Supply Chain can be enhanced by considering simultaneously multiple objectives under demand uncertainties. Particularly, solution frameworks have been proposed in which different multi-criteria decision making strategies have been combined with stochastic approaches. Furthermore, additional performance indicators (including financial and operational ones) have been included in the same solution framework to evaluate its capabilities. This framework was also applied to decentralized supply chains problems in order to explore its capabilities to produce solution that improves the performances of each one of the SC entities simultaneously. Consequently, a new generalized mathematical formulation which integrates many performance indicators in the production process within a supply chain is efficiently solved. Afterwards, the third part of the thesis extends the proposed solution framework to address the uncertainty management. Particularly, the consideration of different types and sources of uncertainty (e.g. external and internal ones) where considered, through the implementation of preventive approaches. This part also explores the use of solution strategies that efficiently selects the number of scenarios that represent the uncertainty conditions. Finally, the importance and effect of each uncertainty source over the process performance is detailed analyzed through the use of surrogate models that promote the sensitivity analysis of those uncertainties. The third part of this thesis is focused on the integration of the above multi-objective and uncertainty approaches for the optimization of a sustainable Supply Chain. Besides the integration of different solution approaches, this part also considers the integration of hierarchical decision levels, by the exploitation of mathematical models that assess the consequences of considering simultaneously design and planning decisions under centralized and decentralized Supply Chains. Finally, the last part of this thesis provides the final conclusions and further work to be developed.La globalización industrial genera un ambiente dinámico en los mercados que, entre otras cosas, promueve la competencia entre corporaciones. Por lo tanto, el uso eficiente de las los indicadores de rendimiento, incluyendo rentabilidad, satisfacción de la demanda y en general el impacto ambiental, representa un area de oportunidad importante. El control de estos indicadores tiene un efecto positivo si se combinan con la gestión de cadena de suministro. Por lo tanto, las compañías buscan definir sus operaciones para permanecer activas dentro de un ambiente competitivo, tomando en cuenta las restricciones en el mercado mundial. Lo anterior puede ser logrado mediante la coordinación de los flujos de recursos a través de todas las entidades y escalones pertenecientes a la red del sistema. Sin embargo, dicha coordinación se complica significativamente si se quiere considerar la presencia de incertidumbre, y aún más, si se busca exclusivamente un ganar-ganar. El propósito de esta tesis es extender el alcance de las estrategias de toma de decisiones con el fin de facilitar estas tareas dentro de procesos industriales. Estas contribuciones se basan en el desarrollo de modelos matemáticos eficientes que permitan coordinar grandes cantidades de información sincronizando las tareas de producción y distribución en términos económicos, ambientales y sociales. Esta tesis inicia presentando una visión global de los requerimientos de un proceso de producción sostenible, describiendo y analizando los métodos y herramientas actuales así como identificando las áreas de oportunidad más relevantes dentro del marco de ingeniería de procesos La segunda parte se enfoca en enfatizar las capacidades de las estrategias de solución multi-objetivo, durante la cual, se explora el mejoramiento de la rentabilidad de la cadena de suministro considerando múltiples objetivos bajo incertidumbres en la demanda. Particularmente, diferentes marcos de solución han sido propuestos en los que varias estrategias de toma de decisión multi-criterio han sido combinadas con aproximaciones estocásticas. Por otra parte, indicadores de rendimiento (incluyendo financiero y operacional) han sido incluidos en el mismo marco de solución para evaluar sus capacidades. Este marco fue aplicado también a problemas de cadenas de suministro descentralizados con el fin de explorar sus capacidades de producir soluciones que mejoran simultáneamente el rendimiento para cada uno de las entidades dentro de la cadena de suministro. Consecuentemente, una nueva formulación que integra varios indicadores de rendimiento en los procesos de producción fue propuesta y validada. La tercera parte de la tesis extiende el marco de solución propuesto para abordar el manejo de incertidumbres. Particularmente, la consideración de diferentes tipos y fuentes de incertidumbre (p.ej. externos e internos) fueron considerados, mediante la implementación de aproximaciones preventivas. Esta parte también explora el uso de estrategias de solución que elige eficientemente el número de escenarios necesario que representan las condiciones inciertas. Finalmente, la importancia y efecto de cada una de las fuentes de incertidumbre sobre el rendimiento del proceso es analizado en detalle mediante el uso de meta modelos que promueven el análisis de sensibilidad de dichas incertidumbres. La tercera parte de esta tesis se enfoca en la integración de las metodologías de multi-objetivo e incertidumbre anteriormente expuestas para la optimización de cadenas de suministro sostenibles. Además de la integración de diferentes métodos. Esta parte también considera la integración de diferentes niveles jerárquicos de decisión, mediante el aprovechamiento de modelos matemáticos que evalúan lasconsecuencias de considerar simultáneamente las decisiones de diseño y planeación de una cadena de suministro centralizada y descentralizada. La parte final de la tesis detalla las conclusiones y el trabajo a futuro necesario sobre esta línea de investigaciónPostprint (published version

On the modeling and real-time control of urban drainage systems: A survey

Trabajo presentado a la 11th International Conference on Hydroinformatics celebrada en New York (US) del 17 al 21 de agosto de 2014.Drainage networks are complex systems composed by several processes including recollection, transport, storing, treatment, and releasing the water to a receiving environment. The way Urban Drainage Systems (UDS) manage wastewater is through the convenient handling of active elements such as gates (redirection and/or retention), storing tanks, and pumping stations, when needed. Therefore, modeling and control of UDS basically consists in knowing and representing the (dynamical) behavior of these elements and managing them properly in order to achieve a given set of control objectives, such as minimization of flooding in streets or maximization of treated wastewater in the system. Given the large number of elements composing an UDS and the interaction between them, management and control strategies may depend on highly complex system models, which implies the explicit difficulty for designing real-time control (RTC) strategies. This paper makes a review of the models used to describe, simulate, and control UDS, proposes a revision of the techniques and strategies commonly used for the control UDS, and finally compares several control strategies based on a case study.This work has been partially supported by project N°548-2012 “Drenaje Urbano y Cambio Climático: Hacia los Sistemas de Alcantarillado del Futuro.”, Mexichem Colombia S.A, the scholarships of Colciencias N°567-2012 and 647-2013, and the EU Project EFFINET (FP7-ICT-2011-8-31855) and the DGR of Generalitat de Catalunya (SAC group Ref. 2009/SGR/1491).Peer Reviewe

Integrated design and control of chemical processes : part I : revision and clasification

[EN] This work presents a comprehensive classification of the different methods and procedures for integrated synthesis, design and control of chemical processes, based on a wide revision of recent literature. This classification fundamentally differentiates between “projecting methods”, where controllability is monitored during the process design to predict the trade-offs between design and control, and the “integrated-optimization methods” which solve the process design and the control-systems design at once within an optimization framework. The latter are revised categorizing them according to the methods to evaluate controllability and other related properties, the scope of the design problem, the treatment of uncertainties and perturbations, and finally, the type the optimization problem formulation and the methods for its resolution.[ES] Este trabajo presenta una clasificación integral de los diferentes métodos y procedimientos para la síntesis integrada, diseño y control de procesos químicos. Esta clasificación distingue fundamentalmente entre los "métodos de proyección", donde se controla la controlabilidad durante el diseño del proceso para predecir los compromisos entre diseño y control, y los "métodos de optimización integrada" que resuelven el diseño del proceso y el diseño de los sistemas de control a la vez dentro de un marco de optimización. Estos últimos se revisan clasificándolos según los métodos para evaluar la controlabilidad y otras propiedades relacionadas, el alcance del problema de diseño, el tratamiento de las incertidumbres y las perturbaciones y, finalmente, el tipo de la formulación del problema de optimización y los métodos para su resolución

On The Modeling And Real-Time Control Of Urban Drainage Systems: A Survey

Drainage network are complex systems composed by several processes including recollection, transport, storing, wastewater and/or rain treatment, and return of the water to a receiving environment. Urban drainage systems (UDS) involve most of these processes inside cities and can be either separate or combined systems, depending on how wastewater and rainwater are managed. The way UDS manage the wastewater is through the convenient handling of active elements such as gates (redirection and/or retention), storing tanks and pumping stations, when needed. Therefore, the modeling and control of UDS basically consists in knowing and representing the (dynamical) behavior of those elements and manage them properly in order to achieve a given set of control objectives, such as minimization of flooding in streets or maximization of treated wastewater in the system. Given the large number of elements composing a UDS and the interaction between them, management and control strategies may depend on highly complex system models, what implies the explicit difficulty for designing real-time control strategies. This paper makes a review on the huge world of models used to describe, simulate, and control UDS. Moreover, a revision of the techniques and strategies commonly used for the control of these systems is also presented and discussed. Mechanisms that ensure the correct operation of the UDS under presence of failures or communication flaws in the system are considered as well

On the modeling and real-time control of urban drainage systems: A survey

Drainage networks are complex systems composed by several processes including recollection, transport, storing, treatment, and releasing the water to a receiving environment. The way Urban Drainage Systems (UDS) manage wastewater is through the convenient handling of active elements such as gates (redirection and/or retention), storing tanks, and pumping stations, when needed. Therefore, modeling and control of UDS basically consists in knowing and representing the (dynamical) behavior of these elements and managing them properly in order to achieve a given set of control objectives, such as minimization of flooding in streets or maximization of treated wastewater in the system. Given the large number of elements composing an UDS and the interaction between them, management and control strategies may depend on highly complex system models, which implies the explicit difficulty for designing real-time control (RTC) strategies. This paper makes a review of the models used to describe, simulate, and control UDS, proposes a revision of the techniques and strategies commonly used for the control UDS, and finally compares several control strategies based on a case study.Peer ReviewedPostprint (author’s final draft

Multi-objective optimal control of small-size wastewater treatment plants

In this work, a multi-objective dynamic optimization of the operating strategy of a small-size wastewater treatment plant is carried out. In-situ incineration of the excess sludge produced for electricity production is investigated in order to reduce the operating costs. The trade-offs between the treatment quality and the operating costs are characterized. Compared to the literature, emphasis is put on a more rigorous formulation of the problem and an accurate modeling of the underlying phenomena so as to get physically relevant solutions. Thus, from a mathematical perspective, the problem is formulated so that the solution is less sensitive to the – arbitrarily chosen – plant initial conditions. Modeling of physical phenomena e.g. the detrimental effect of the concentration of suspended solids in the mixed liquor, on oxygen transfer rate, has been included in the model. Several constraints are added to the problem so as to maintain the optimal solutions within the limits of validity of the mathematical model. The results provided a clear picture about the trade-offs between the treatment quality and the exploitation costs. Sludge incineration was shown to be of a high energetic profit, but it does not allow the plant to be electrically autonomou

Supply chain management for the process industry

This thesis investigates some important problems in the supply chain management (SCM) for the process industry to fill the gap in the literature work, covering production planning and scheduling, production, distribution planning under uncertainty, multiobjective supply chain optimisation and water resources management in the water supply chain planning. To solve these problems, models and solution approaches are developed using mathematical programming, especially mixed-integer linear programming (MILP), techniques. First, the medium-term planning of continuous multiproduct plants with sequence-dependent changeovers is addressed. An MILP model is developed using Travelling Salesman Problem (TSP) classic formulation. A rolling horizon approach is also proposed for large instances. Compared with several literature models, the proposed models and approaches show significant computational advantage. Then, the short-term scheduling of batch multiproduct plants is considered. TSP-based formulation is adapted to model the sequence-dependent changeovers between product groups. An edible-oil deodoriser case study is investigated. Later, the proposed TSP-based formulation is incorporated into the supply chain planning with sequence-dependent changeovers and demand elasticity of price. Model predictive control (MPC) is applied to the production, distribution and inventory planning of supply chains under demand uncertainty. A multiobjective optimisation problem for the production, distribution and capacity planning of a global supply chain of agrochemicals is also addressed, considering cost, responsiveness and customer service level as objectives simultaneously. Both ε- constraint method and lexicographic minimax method are used to find the Pareto-optimal solutions Finally, the integrated water resources management in the water supply chain management is addressed, considering desalinated water, wastewater and reclaimed water, simultaneously. The optimal production, distribution and storage systems are determined by the proposed MILP model. Real cases of two Greek islands are studied