Learning Agent for a Heat-Pump Thermostat With a Set-Back Strategy Using Model-Free Reinforcement Learning

The conventional control paradigm for a heat pump with a less efficient auxiliary heating element is to keep its temperature set point constant during the day. This constant temperature set point ensures that the heat pump operates in its more efficient heat-pump mode and minimizes the risk of activating the less efficient auxiliary heating element. As an alternative to a constant set-point strategy, this paper proposes a learning agent for a thermostat with a set-back strategy. This set-back strategy relaxes the set-point temperature during convenient moments, e.g. when the occupants are not at home. Finding an optimal set-back strategy requires solving a sequential decision-making process under uncertainty, which presents two challenges. A first challenge is that for most residential buildings a description of the thermal characteristics of the building is unavailable and challenging to obtain. A second challenge is that the relevant information on the state, i.e. the building envelope, cannot be measured by the learning agent. In order to overcome these two challenges, our paper proposes an auto-encoder coupled with a batch reinforcement learning technique. The proposed approach is validated for two building types with different thermal characteristics for heating in the winter and cooling in the summer. The simulation results indicate that the proposed learning agent can reduce the energy consumption by 4-9% during 100 winter days and by 9-11% during 80 summer days compared to the conventional constant set-point strategyComment: Submitted to Energies - MDPI.co

Performance Assessment of Ancient Wind Catchers - an Experimental and Analytical Study

AbstractWind catchers – “Baud-Geers” in Persian– are the main component of the traditional buildings in the hot regions of Iran. A Baud-Geer is a tower linked to a building that uses wind to provide natural ventilation and passive cooling. This passive renewable strategy offers the opportunity to improve the ambient comfort conditions in buildings whilst reducing the energy consumption of air-conditioning systems. In this research the natural ventilation performance of a typical wind tower in a hot dry central region of Iran -Yazd city- is studied. The tower is equipped with wind, temperature, air-velocity and solar sensors to acquire a climatic database. Using the measured data, the theoretical values of the ventilation rates are estimated and analysed to assess the performance of the wind tower. Additionally the data collected from the on-site measurements will assist in the validation of a CFD computer model. Finally the findings from this field study will lead to a discussion on the potential of Baud-Geers in achieving thermal comfort. This can contribute to energy savings for cooling and to the reuse and reappraisal of wind towers in Iran

Optimum Blood Pressure in Patients With Shock After Acute Myocardial Infarction and Cardiac Arrest

BACKGROUND In patients with shock after acute myocardial infarction (AMI), the optimal level of pharmacologic support is unknown. Whereas higher doses may increase myocardial oxygen consumption and induce arrhythmias, diastolic hypotension may reduce coronary perfusion and increase infarct size. OBJECTIVES This study aimed to determine the optimal mean arterial pressure (MAP) in patients with AMI and shock after cardiac arrest. METHODS This study used patient-level pooled analysis of post-cardiac arrest patients with shock after AMI randomized in the Neuroprotect (Neuroprotective Goal Directed Hemodynamic Optimization in Post-cardiac Arrest Patients; NCT02541591) and COMACARE (Carbon Dioxide, Oxygen and Mean Arterial Pressure After Cardiac Arrest and Resuscitation; NCT02698917) trials who were randomized to MAP 65 mm Hg or MAP 80/85 to 100 mm Hg targets during the first 36 h after admission. The primary endpoint was the area under the 72-h high-sensitivity troponin-T curve. RESULTS Of 235 patients originally randomized, 120 patients had AMI with shock. Patients assigned to the higher MAP target (n = 58) received higher doses of norepinephrine (p = 0.004) and dobutamine (p = 0.01) and reached higher MAPs (86 +/- 9 mm Hg vs. 72 +/- 10 mm Hg, p <0.001). Whereas admission hemodynamics and angiographic findings were all well-balanced and revascularization was performed equally effective, the area under the 72-h high-sensitivity troponin-T curve was lower in patients assigned to the higher MAP target (median: 1.14 mu g.72 h/l [interquartile range: 0.35 to 2.31 mu g.72 h/l] vs. median: 1.56 mu g.72 h/l [interquartile range: 0.61 to 4.72 mu g. 72 h/l]; p = 0.04). Additional pharmacologic support did not increase the risk of a new cardiac arrest (p = 0.88) or atrial fibrillation (p = 0.94). Survival with good neurologic outcome at 180 days was not different between both groups (64% vs. 53%, odds ratio: 1.55; 95% confidence interval: 0.74 to 3.22). CONCLUSIONS In post-cardiac arrest patients with shock after AMI, targeting MAP between 80/85 and 100 mm Hg with additional use of inotropes and vasopressors was associated with smaller myocardial injury. (C) 2020 by the American College of Cardiology Foundation.Peer reviewe

A Flexible Stochastic Optimization Method for Wind Power Balancing With PHEVs

This paper proposes a flexible optimization method, based on state of the art algorithms, for the smart control of plug-in hybrid electric vehicles (PHEVs) to balance wind power production. The problem is approached from the perspective of a balance responsible party (BRP) with a large share of wind power in its portfolio. The BRP uses controllable PHEVs to minimize the imbalance of its portfolio resulting from wind power forecast errors. A Markov Decision Process (MDP) formulation in combination with dynamic programming is used to solve the multistage stochastic problem. The main difficulty for applying MDPs to this problem is to efficiently include time interdependence of the wind power forecast error. In the presented approach, the probability distribution and time interdependence of the forecast error are represented by a scenario tree. Because of the MDP formulation, the algorithm is adaptable to deal with different transition models and constraints. This feature enables to use the algorithm in a dynamic environment such as the future smart grid. To demonstrate this, a generic charging model for PHEVs is used in the BRP wind balancing case. The flexibility of the algorithm is shown by investigating the solution for different degrees of complexity in the charging model.status: publishe

Unlocking the full sustainability potential of school buildings by reconciling building properties with educational and societal needs

This study explores how school buildings can be exploited more efficiently in the future, since, at present, they remain unused for a substantial amount of time. One possibility to tackle this inefficiency, is to involve the local community more closely in usage of its school building. First, a theoretical analysis was carried out to increase the fundamental understanding of the underlying dynamics related to opening school infrastructure to the local community. Second, focus group discussions were organized to research whether involving the local community in the school building was compatible with educational needs. The first highlighted that more extensive building usage could lead to positive social, environmental, educational and economic benefits. In the second, educational experts stressed that they wanted to adopt more innovative and flexible forms of teaching in the future, such as team teaching. Technical directors expressed concerns on safety issues if the local community is to be more closely involved. In the final step, all findings were translated into their technical consequences. From this analysis, it could be concluded that a school building with a high degree of short-term flexibility was the preferred option to reconcile societal and educational needs

Unlocking the Full Sustainability Potential of School Buildings by Reconciling Building Properties with Educational and Societal Needs

This study explores how school buildings can be exploited more efficiently in the future, since, at present, they remain unused for a substantial amount of time. One possibility to tackle this inefficiency, is to involve the local community more closely in usage of its school building. First, a theoretical analysis was carried out to increase the fundamental understanding of the underlying dynamics related to opening school infrastructure to the local community. Second, focus group discussions were organized to research whether involving the local community in the school building was compatible with educational needs. The first highlighted that more extensive building usage could lead to positive social, environmental, educational and economic benefits. In the second, educational experts stressed that they wanted to adopt more innovative and flexible forms of teaching in the future, such as team teaching. Technical directors expressed concerns on safety issues if the local community is to be more closely involved. In the final step, all findings were translated into their technical consequences. From this analysis, it could be concluded that a school building with a high degree of short-term flexibility was the preferred option to reconcile societal and educational needs

Sequential Decision-Making Strategy for a Demand Response Aggregator in a Two-Settlement Electricity Market

This paper proposes a novel sequential decision-making strategy for a demand response aggregator that participates in the day-ahead market and reacts to imbalance prices. Finding such a participation strategy requires solving a multistage optimization problem under uncertainty that entails both an open-loop (day-ahead market) and a nested closed-loop (imbalance system) problem. Driven by the possibility of using data-driven models and reinforcement learning techniques, we formulate the problem as a Markov Decision Process (MDP). Standard MDP-based methods, however, often suffer from the curse of dimensionality. To address this challenge, we use techniques from approximate dynamic programming. Our proposed method applies a cross-entropy method with a simulation-based approximate policy iteration algorithm nested inside. The crossentropy method is compared with a separated planning method, that optimizes the day-ahead and real-time decisions separately. Both planning methods are evaluated for an aggregator with a fleet of electric vehicles using data from the Belgian electricity market.status: publishe