Chance-Constrained Control with Lexicographic Deep Reinforcement Learning

This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes, in which the controller seeks to ensure that the probability of satisfying the constraint is above a given threshold. Standard DeepRL approaches require i) the constraints to be included as additional weighted terms in the cost function, in a multi-objective fashion, and ii) the tuning of the introduced weights during the training phase of the Deep Neural Network (DNN) according to the probability thresholds. The proposed approach, instead, requires to separately train one constraint-free DNN and one DNN associated to each constraint and then, at each time-step, to select which DNN to use depending on the system observed state. The presented solution does not require any hyper-parameter tuning besides the standard DNN ones, even if the probability thresholds changes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations

Bellman's principle of optimality and deep reinforcement learning for time-varying tasks

This paper presents the first framework (up to the authors' knowledge) to address time-varying objectives in finite-horizon Deep Reinforcement Learning (DeepRL), based on a switching control solution developed on the ground of Bellman's principle of optimality. By augmenting the state space of the system with information on its visit time, the DeepRL agent is able to solve problems in which its task dynamically changes within the same episode. To address the scalability problems caused by the state space augmentation, we propose a procedure to partition the episode length to define separate sub-problems that are then solved by specialised DeepRL agents. Contrary to standard solutions, with the proposed approach the DeepRL agents correctly estimate the value function at each time-step and are hence able to solve time-varying tasks. Numerical simulations validate the approach in a classic RL environment

Controlled optimal black start procedures in smart grids for service restoration in presence of electrical storage systems

This paper presents an optimisation problem to determine the optimal reclosure order of remotely operable switches deployed in a smart grid consisting in a distribution network equipped with one or more Energy Storage Systems (ESS). The proposed solution integrates nonlinear real and reactive power flow equations, by reconducting them to a set of conic constraints, together with several network operator requirements, such as network radiality and ampacity limits. A numerical simulation validates the approach and concludes the work

Attack-Surface Metrics, OSSTMM and Common Criteria Based Approach to “Composable Security” in Complex Systems

In recent studies on Complex Systems and Systems-of-Systems theory, a huge effort has been put to cope with behavioral problems, i.e. the possibility of controlling a desired overall or end-to-end behavior by acting on the individual elements that constitute the system itself. This problem is particularly important in the “SMART” environments, where the huge number of devices, their significant computational capabilities as well as their tight interconnection produce a complex architecture for which it is difficult to predict (and control) a desired behavior; furthermore, if the scenario is allowed to dynamically evolve through the modification of both topology and subsystems composition, then the control problem becomes a real challenge. In this perspective, the purpose of this paper is to cope with a specific class of control problems in complex systems, the “composability of security functionalities”, recently introduced by the European Funded research through the pSHIELD and nSHIELD projects (ARTEMIS-JU programme). In a nutshell, the objective of this research is to define a control framework that, given a target security level for a specific application scenario, is able to i) discover the system elements, ii) quantify the security level of each element as well as its contribution to the security of the overall system, and iii) compute the control action to be applied on such elements to reach the security target. The main innovations proposed by the authors are: i) the definition of a comprehensive methodology to quantify the security of a generic system independently from the technology and the environment and ii) the integration of the derived metrics into a closed-loop scheme that allows real-time control of the system. The solution described in this work moves from the proof-of-concepts performed in the early phase of the pSHIELD research and enrich es it through an innovative metric with a sound foundation, able to potentially cope with any kind of pplication scenarios (railways, automotive, manufacturing, ...)

Joint Model Predictive Control of Electric and Heating Resources in a Smart Building

The new challenge in power systems design and operation is to organize and control smart micro grids supplying aggregation of users and special loads as electric vehicles charging stations. The presence of renewable and storage can help the optimal operation only if a good control manages all the elements of the grid. New models of green buildings and energy communities are proposed. For a real application they need an appropriate and advanced power system equipped with a building automation control system. This article presents an economic model predictive control approach to the problem of managing the electric and heating resources in a smart building in a coordinated way, for the purpose of achieving in real time nearly zero energy consumption and automated participation to demand response programs. The proposed control, leveraging a mixed integer quadratic programming problem, allows to meet manifold thermal and electric users' requirements and react to inbound demand response signals, while still guaranteeing stable operation of the building's electric and thermal storage equipment. The simulation results, performed for a real case study in Italy, highlight the peculiarities of the proposed approach in the joint handling of electric and thermal building flexibility

“The heart in a bag”: The lived experience of patient-caregiver dyads with left ventricular assist device during cardiac rehabilitation

Objective: The Left Ventricular Assist Device (LVAD) has increasingly become a primary therapeutic option for longer-waiting heart transplant lists. Although survival rates are growing, the device requires complex home care. Therefore, the presence of a caregiver trained in the LVAD management is important for the success of the therapy. The LVAD leads both patients and their caregivers to experience new challenges and adapt to new lifestyle changes and limitations – but their subjective beliefs before home management remained little explored. Design: This study identified, using a phenomenological hermeneutic approach, the main components of the LVAD experience of six patient-caregiver dyads interviewed during cardiac rehabilitation. Results: We identified 4 master themes: Being between life and death, Being human with a heart of steel, Sharing is caring (and a burden), and Being small and passive. Conclusion: The knowledge from this study can be used as a guide for healthcare providers in counseling LVAD recipients and their caregivers

The relationship between emotional intelligence, obesity and eating disorder in children and adolescents: A systematic mapping review

Eating and weight disorders often develop early in life and cause a long-standing significant health burden. Given the documented role of emotional intelligence (EI) in shaping the body image and predicting the onset of eating disorders, knowledge of the mechanisms involved in EI among youth is fundamental to designing specific interventions for screening and prevention of obesity and eating disorders (EDs). The present systematic mapping review was aimed to explore and quantify the nature and distribution of existing research investigating the impact of EI on EDs in young people. A systematic search for relevant articles was conducted using PubMed, Scopus, PsycINFO and Web of Science databases. The Appraisal tool for Cross-Sectional Studies (AXIS) was used to assess the included studies’ methodological quality. The included studies’ results were mapped based on stratification by age groups (children, preadolescents, and adolescents), population (clinical vs. non-clinical) and disordered eating outcomes. Nine studies were included, supporting the association between EI and body image dissatisfaction, ED risk and bulimic symptomatology, but not with anorexic symptoms. Research on children and clinical populations was scant. Further studies are needed to deepen the role of EI in the genesis and maintenance of EDs

Magnetic carbon nanotubes: a new tool for shepherding mesenchymal stem cells by magnetic fields

We investigated the interaction between magnetic carbon nanotubes (CNTs) and mesenchymal stem cells (MSCs), and their ability to guide these intravenously injected cells in living rats by using an external magnetic field. MATERIALS & METHODS: Multiwalled CNTs were used to treat MSCs derived from rat bone marrow. Cytotoxicity induced by nanotubes was studied using the WST-1 proliferation and Hoechest 33258 apoptosis assays. The effects of nanotubes on MSCs were evaluated by monitoring the effects on cellular growth rates, immunophenotyping and differentiation, and on the arrangement of cytoskeletal actin. MSCs loaded with nanotubes were injected in vivo in the portal vein of rats driving their localization in the liver by magnetic field. An histological analysis was performed on the liver, lungs and kidneys of all animals. RESULTS: CNTs did not affect cell viability and their ability to differentiate in osteocytes and adipocytes. Both the CNTs and the magnetic field did not alter the cell growth rate, phenotype and cytoskeletal conformation. CNTs, when exposed to magnetic fields, are able to shepherd MSCs towards the magnetic source in vitro. Moreover, the application of a magnetic field alters the biodistribution of CNT-labelled MSCs after intravenous injection into rats, increasing the accumulation of cells into the target organ (liver). CONCLUSION: Multiwalled CNTs hold the potential for use as nanodevices to improve therapeutic protocols for transplantation and homing of stem cells in vivo. This could pave the way for the development of new strategies for the manipulation/guidance of MSCs in regenerative medicine and cell transplantation

Investigation of interactions between poly-l-lysine-coated boron nitride nanotubes and C2C12 cells: up-take, cytocompatibility, and differentiation

Boron nitride nanotubes (BNNTs) have generated considerable interest within the scientific community by virtue of their unique physical properties, which can be exploited in the biomedical field. In the present in vitro study, we investigated the interactions of poly-l-lysine-coated BNNTs with C2C12 cells, as a model of muscle cells, in terms of cytocompatibility and BNNT internalization. The latter was performed using both confocal and transmission electron microscopy. Finally, we investigated myoblast differentiation in the presence of BNNTs, evaluating the protein synthesis of differentiating cells, myotube formation, and expression of some constitutive myoblastic markers, such as MyoD and Cx43, by reverse transcription – polymerase chain reaction and Western blot analysis. We demonstrated that BNNTs are highly internalized by C2C12 cells, with neither adversely affecting C2C12 myoblast viability nor significantly interfering with myotube formation