Optimal Control of District Cooling Energy Plant with Reinforcement Learning and MPC

We consider the problem of optimal control of district cooling energy plants (DCEPs) consisting of multiple chillers, a cooling tower, and a thermal energy storage (TES), in the presence of time-varying electricity price. A straightforward application of model predictive control (MPC) requires solving a challenging mixed-integer nonlinear program (MINLP) because of the on/off of chillers and the complexity of the DCEP model. Reinforcement learning (RL) is an attractive alternative since its real-time control computation is much simpler. But designing an RL controller is challenging due to myriad design choices and computationally intensive training. In this paper, we propose an RL controller and an MPC controller for minimizing the electricity cost of a DCEP, and compare them via simulations. The two controllers are designed to be comparable in terms of objective and information requirements. The RL controller uses a novel Q-learning algorithm that is based on least-squares policy iteration. We describe the design choices for the RL controller, including the choice of state space and basis functions, that are found to be effective. The proposed MPC controller does not need a mixed integer solver for implementation, but only a nonlinear program (NLP) solver. A rule-based baseline controller is also proposed to aid in comparison. Simulation results show that the proposed RL and MPC controllers achieve similar savings over the baseline controller, about 17%.Comment: 18 pages, 12 figures. arXiv admin note: text overlap with arXiv:2203.0750

Supervisory model predictive control of building integrated renewable and low carbon energy systems

To reduce fossil fuel consumption and carbon emission in the building sector, renewable and low carbon energy technologies are integrated in building energy systems to supply all or part of the building energy demand. In this research, an optimal supervisory controller is designed to optimize the operational cost and the CO2 emission of the integrated energy systems. For this purpose, the building energy system is defined and its boundary, components (subsystems), inputs and outputs are identified. Then a mathematical model of the components is obtained. For mathematical modelling of the energy system, a unified modelling method is used. With this method, many different building energy systems can be modelled uniformly. Two approaches are used; multi-period optimization and hybrid model predictive control. In both approaches the optimization problem is deterministic, so that at each time step the energy consumption of the building, and the available renewable energy are perfectly predicted for the prediction horizon. The controller is simulated in three different applications. In the first application the controller is used for a system consisting of a micro-combined heat and power system with an auxiliary boiler and a hot water storage tank. In this application the controller reduces the operational cost and CO2 emission by 7.31 percent and 5.19 percent respectively, with respect to the heat led operation. In the second application the controller is used to control a farm electrification system consisting of PV panels, a diesel generator and a battery bank. In this application the operational cost with respect to the common load following strategy is reduced by 3.8 percent. In the third application the controller is used to control a hybrid off-grid power system consisting of PV panels, a battery bank, an electrolyzer, a hydrogen storage tank and a fuel cell. In this application the controller maximizes the total stored energies in the battery bank and the hydrogen storage tank

Chilled Water Storage Feasibility with District Cooling Chiller in Tropical Environment

The difficulties of efficiently operating a chiller cooling system are manifest in the high-energy consumption under partial-cooling loads. The performance of a chiller cooling system declines when operating away from the optimal design conditions, which is typically 75% of chiller capacity. One pathway has been found to overcome this problem using multiple smaller chillers within the same chiller plant, accompanied by a smart control system that is designed and constructed based on the cooling demand profile. Thermal energy storage integration with chiller cooling system is proposed to shave the cooling peak demand. This can be achieved by storing chilled water during the lower electricity-tariff period by the thermal energy storage system, which will then be discharged during the higher tariff-rate, thus, aiming for sustainable operating cost. The present paper studies the feasibility of sensible thermal energy storage to be integrated with two chillers, of 30-ton capacity each, under hot-and-humid climates. A computational model validated with experimental results is developed for three chiller cooling system case scenarios. The smart control scenario, as well as the thermal energy storage scenario results, showed great potential for energy and electricity cost saving. In addition, the carbon dioxide emissions reduction is calculated based on the amount of energy saving

Trade-off between optimal design and operation in district cooling networks

Especially in densely populated areas, district cooling represents an opportunity to reduce energy consumption and emissions. Nevertheless, this technology is characterised by large capital costs which impede its diffusion. As a consequence, optimization tools can significantly help to unleash their potential. In this paper, a methodology is proposed to combinedly optimize the design and operation of a district cooling system based on a Mixed Integer Quadratic Programming. The model is compared to the design only optimization, based on a properly tailored heuristic approach. The models, when applied to a case study characterized by seasonal demand, provide similar solutions, which differ by 0.5 % in terms of objective value for a standard scenario. The simultaneous design and operation optimization does not provide sensible savings with respect to optimizing solely the design. A sensitivity analysis is performed to prove the robustness of the results. The results showed that the simulta- neous operation and design optimization would be limited to 1 % of total costs in the case of seasonal cooling demand. On the other hand, if the cooling demand persists throughout the year, as in tropical climates, the combined optimization provides significant benefits, since these savings reach 4.7 % of total costs

Closed-Loop Scheduling for Cost Minimization in HVAC Central Plants

In this paper, we examine closed-loop operation of an HVAC central plant to demonstrate that closed-loop receding-horizon scheduling provides robustness to inaccurate forecasts, and that economic performance is not seriously impaired by shortened prediction horizons or inaccurate forecasts when feedback is employed. Using a general mixed-integer linear programming formulation for the scheduling problem, we show that optimization can be performed in real time. Furthermore, we demonstrate that closed-loop operation with a moderate prediction horizon is not significantly worse than a long-horizon implementation in the nominal case, and that closed-loop operation can correct for inaccurate long-term forecasts without significant cost increase. In addition, we show that terminal constraints can be employed to ensure recursive feasibility. The end result is that forecasts of demand need not be extremely accurate over long times, indicating that closed-loop scheduling can be implemented in new or existing central plants

Dynamic optimization of energy systems with thermal energy storage

textThermal energy storage (TES), the storage of heat or cooling, is a cost-effective energy storage technology that can greatly enhance the performance of the energy systems with which it interacts. TES acts as a buffer between transient supply and demand of energy. In solar thermal systems, TES enables the power output of the plant to be effectively regulated, despite fluctuating solar irradiance. In district energy systems, TES can be used to shift loads, allowing the system to avoid or take advantage of peak energy prices. The benefit of TES, however, can be significantly enhanced by dynamically optimizing the complete energy system. The ability of TES to shift loads gives the system newfound degrees of freedom which can be exploited to yield optimal performance. In the hybrid solar thermal/fossil fuel system explored in this work, the use of TES enables the system to extract nearly 50% more solar energy when the system is optimized. This requires relaxing some constraints, such as fixed temperature and power control, and dynamically optimizing the over a one-day time horizon. In a district cooling system, TES can help equipment to run more efficiently, by shifting cooling loads, not only between chillers, but temporally, allowing the system to take advantage of the most efficient times for running this equipment. This work also highlights the use of TES in a district energy system, where heat, cooling and electrical power are generated from central locations. Shifting the cooling load frees up electrical generation capacity, which is used to sell power to the grid at peak prices. The combination of optimization, TES, and participation in the electricity market yields a 16% cost savings. The problems encountered in this work require modeling a diverse range of systems including the TES, the solar power plant, boilers, gas and steam turbines, heat recovery equipment, chillers, and pumps. These problems also require novel solution methods that are efficient and effective at obtaining workable solutions. A simultaneous solution method is used for optimizing the solar power plant, while a static/dynamic decoupling method is used for the district energy system.Chemical Engineerin