Understanding Reinforcement Learning Control in Cyber-Physical Energy Systems

The possibility of modeling a renewable energy system as a Cyber-Physical Energy System (CPES) offers new possibilities in terms of control. More precisely, this document discusses the applicability of Reinforcement Learning (RL) techniques to CPES. By considering a benchmark algorithm, we focus on conceptual and implementation details and on how such details relate to the problem of interest. In this case, we simulate how a RL model can optimize the energy storage control in order to reduce energy costs. The work also discusses the issues that arise in RL models and the possible approaches to these difficulties. Specifically, we propose investigating a better exploitation of the memory mechanism

Modeling of Renewable Energy Communities: the RECoupled approach

The increase in energy production and consumption from Renewable Energy Sources (RES) is becoming strategic to reduce CO2 emissions and to contrast climate-change related issues. In this view, EU promoted the creation of Renewable Energy Communities (RECs) to foster the sharing of local RES production among end users. Even if technological aspects of these energy systems are not critical, their complexity in management and planning is presently arising due to intermittent RES generation and end users demands. Storage solutions can contribute to RES exploitation and to the flexibility of RECs, but introducing further complexity in the system. In this context, an adequate model of the physical and management layouts of the REC becomes crucial to perform energy, economic and environmental analyses. Consequently, in this paper a modelling approach of interconnected multi-energy systems named RECoupled is proposed to simulate such RECs in the Italian context, taking into account the corresponding rules and peculiarities

A MILP Approach for Demand Management in Renewable Energy Communities with Residential End-Users

Nowadays, the energy sharing of RES production within Renewable Energy Communities (REC) is promoting the diffusion of a more decentralized energy system, where dispersed renewable generation can be locally self-consumed by REC members. The maximization of self-consumption through the matching between generation and demand is thus fundamental to ensure higher economic and environmental benefits for residential end-users joining REC configurations. However residential electricity demand and the corresponding load profiles are generally influenced by end-users’ behaviour. In fact, even if most of the household appliances can be assumed as fixed loads, the usage of some appliances depends basically on the residents’ habits. The engagement of customers in changing their energy consumption patterns is then challenging to promote flexibility in electricity demand to further increase the benefits of adopting and joining renewable energy communities. In this view, a MILP approach is proposed to model end-users’ flexibility for investigating how the changing in consumption habits can potentially improve the energy sharing by maximizing the match between RES production and demand. User’s discomfort is evaluated consequently as the distance between the desired or usual consumption pattern and the optimized one. An Italian multifamily residential building case study, where end-users adopt a collective self-consumption scheme, is considered to highlight energy and economic results assuming different level of end-users’ flexibility. Finally, a comparison between the maximization of energy sharing and the minimization of discomfort rate is pointed out through weighted sum method to identify solutions with different relevance of the end-users’ flexibility

Metabolic responses to high pCO2 conditions at a CO2 vent site in juveniles of a marine isopod species assemblage

We are starting to understand the relationship between metabolic rate responses and species' ability to respond to exposure to high pCO2. However, most of our knowledge has come from investigations of single species. The examination of metabolic responses of closely related species with differing distributions around natural elevated CO2 areas may be useful to inform our understanding of their adaptive significance. Furthermore, little is known about the physiological responses of marine invertebrate juveniles to high pCO2, despite the fact they are known to be sensitive to other stressors, often acting as bottlenecks for future species success. We conducted an in situ transplant experiment using juveniles of isopods found living inside and around a high pCO2 vent (Ischia, Italy): the CO2 'tolerant' Dynamene bifida and 'sensitive' Cymodoce truncata and Dynamene torelliae. This allowed us to test for any generality of the hypothesis that pCO2 sensitive marine invertebrates may be those that experience trade-offs between energy metabolism and cellular homoeostasis under high pCO2 conditions. Both sensitive species were able to maintain their energy metabolism under high pCO2 conditions, but in C. truncata this may occur at the expense of [carbonic anhydrase], confirming our hypothesis. By comparison, the tolerant D. bifida appeared metabolically well adapted to high pCO2, being able to upregulate ATP production without recourse to anaerobiosis. These isopods are important keystone species; however, given they differ in their metabolic responses to future pCO2, shifts in the structure of the marine ecosystems they inhabit may be expected under future ocean acidification conditions

A data-driven approach to predict hourly load profiles from time-of-use electricity bills

The design of renewable-based and collective energy systems requires consumption data with fine temporal and spatial resolution. Despite the increasing deployment of smart meters, obtaining such data directly from measurements can still be challenging, particularly when attempting to gather historical data over a reasonable period for many end users. As a result, there is a need for models to simulate or predict these consumption data (e.g., hourly load profiles). Typically, these models rely on numerous specific and detailed observations, such as load type, household size for residential customers, or operating hours for commercial ones. However, gathering this level of detail becomes increasingly difficult as the number and diversity of end users increase. Therefore, this paper proposes a data-driven approach to predict hourly load profiles of heterogeneous end users using only their monthly time-of-use electricity bills as inputs. We create a training set using a limited number of hourly measurements from diverse categories of end users and, differently from other approaches aimed at classifying the end users, we develop a regression model to map monthly electricity bills to typical load profiles. Experimental results using one year of data from various end-user categories demonstrate an average normalized mean absolute error of approximately 26% for instantaneous consumption and less than 4% for time-of-use values. Comparative analysis with standard load profiles and a two-step data-driven approach based on classification reveals that our proposed method outperforms the others in terms of prediction accuracy and statistical metrics

Data-Driven Constraint Handling in Multi-Objective Inductor Design

This paper analyses the multi-objective design of an inductor for a DC-DC buck converter. The core volume and total losses are the two competing objectives, which should be minimised while satisfying the design constraints on the required differential inductance profile and the maximum overheating. The multi-objective optimisation problem is solved by means of a population-based metaheuristic algorithm based on Artificial Immune Systems (AIS). Despite its effectiveness in finding the Pareto front, the algorithm requires the evaluation of many candidate solutions before converging. In the case of the inductor design problem, the evaluation of a configuration is time-consuming. In fact, a non-linear iterative technique (fixed point) is needed to obtain the differential inductance profile of the configuration, as it may operate in conditions of partial saturation. However, many configurations evaluated during an optimisation do not comply with the design constraint, resulting in expensive and unnecessary calculations. Therefore, this paper proposes the adoption of a data-driven surrogate model in a pre-selection phase of the optimisation. The adopted model should classify newly generated configurations as compliant or not with the design constraint. Configurations classified as unfeasible are disregarded, thus avoiding the computational burden of their complete evaluation. Interesting results have been obtained, both in terms of avoided configuration evaluations and the quality of the Pareto front found by the optimisation procedure

A look to the future acidified ocean through the eyes of the alien and invasive alga Caulerpa cylindracea (Chlorophyta, Ulvophyceae)

Underwater CO2 vents represent natural laboratories where the responses of marine organisms to ocean acidification can be tested. In a such context, we investigated the changes in the physiology, anatomy, and ultrastructure of the non-indigenous algal species Caulerpa cylindracea growing along a natural pH/CO2 gradient, by conducting a reciprocal transplant experiment between two populations from an acidified vs a non-acidified site. Stress effects in transplants from current to lowered pH conditions resulted in a decrease in the number of active chloroplasts together with an increased number of dilatations between thylakoid membranes and a higher amount of plastoglobules. These changes were consistent with a decrease in the chlorophyll content and in photosynthetic efficiency, matched by an increase in carotenoid content and non-photochemical yields. On the opposite side, transplants from low to current pH showed a recovery to original conditions. Unexpectedly, no significant difference was recorded between wild populations living at current and lowered pH. These results suggest an ongoing acclimation process to lowered pH in the C. cylindracea populations growing in the vent area. This confirms the high plasticity of this invasive species, able to cope not only with different light and temperature conditions but even with a new acidified scenario

Decarbonizing residential energy consumption under the Italian collective self-consumption regulation

The recent Italian regulatory framework is promoting Collective Self-Consumption to play a key role in the energy transition. In fact, this new scheme allows sharing and exchange of electricity produced from Renewable Energy Sources (RES) among different end-users living in the same multi-family building block. In this context, this paper aims to perform an energy, economic and environmental assessment of creating a RES-based energy community formed by end-users located in the same residential building, taking into account the currently available incentive schemes. In particular, two different progressive scenarios have been analyzed: the first one includes only a photovoltaic system to supply the aggregated electricity demand of the apartments, while a heat pump is further integrated in the second scenario for supplying and electrifying/decarbonizing also the space heating demand of the building. Available temperature and solar irradiance datasets at national level were then used to spread the analysis to the whole country, at different latitudes. Although differences exist at regional levels for the proposed scenarios, the results highlight how the RES-based Collective Self-Consumption scheme is economically profitable for Italian residential end-users with cost savings up to 32% and environmentally sustainable with carbon emissions reduction up to 60%