Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management

The utilization of large-scale distributed renewable energy promotes the development of the multi-microgrid (MMG), which raises the need of developing an effective energy management method to minimize economic costs and keep self energy-sufficiency. The multi-agent deep reinforcement learning (MADRL) has been widely used for the energy management problem because of its real-time scheduling ability. However, its training requires massive energy operation data of microgrids (MGs), while gathering these data from different MGs would threaten their privacy and data security. Therefore, this paper tackles this practical yet challenging issue by proposing a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning (FL) mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data. In addition, a decentralized MMG model is built, and the energy of each participated MG is managed by an agent, which aims to minimize economic costs and keep self energy-sufficiency according to the physics-informed reward. At first, MGs individually execute the self-training based on local energy operation data to train their local agent models. Then, these local models are periodically uploaded to a server and their parameters are aggregated to build a global agent, which will be broadcasted to MGs and replace their local agents. In this way, the experience of each MG agent can be shared and the energy operation data is not explicitly transmitted, thus protecting the privacy and ensuring data security. Finally, experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system, and the comparisons are carried out to verify the effectiveness of introducing the FL mechanism and the outperformance of our proposed F-MADRL.Comment: Accepted by IEEE Transactions on Neural Networks and Learning System

STAMP alters the growth of transformed and ovarian cancer cells

<p>Abstract</p> <p>Background</p> <p>Steroid receptors play major roles in the development, differentiation, and homeostasis of normal and malignant tissue. STAMP is a novel coregulator that not only enhances the ability of p160 coactivator family members TIF2 and SRC-1 to increase gene induction by many of the classical steroid receptors but also modulates the potency (or EC₅₀) of agonists and the partial agonist activity of antisteroids. These modulatory activities of STAMP are not limited to gene induction but are also observed for receptor-mediated gene repression. However, a physiological role for STAMP remains unclear.</p> <p>Methods</p> <p>The growth rate of HEK293 cells stably transfected with STAMP plasmid and overexpressing STAMP protein is found to be decreased. We therefore asked whether different STAMP levels might also contribute to the abnormal growth rates of cancer cells. Panels of different stage human cancers were screened for altered levels of STAMP mRNA. Those cancers with the greatest apparent changes in STAMP mRNA were pursued in cultured cancer cell lines.</p> <p>Results</p> <p>Higher levels of STAMP are shown to have the physiologically relevant function of reducing the growth of HEK293 cells but, unexpectedly, in a steroid-independent manner. STAMP expression was examined in eight human cancer panels. More extensive studies of ovarian cancers suggested the presence of higher levels of STAMP mRNA. Lowering STAMP mRNA levels with siRNAs alters the proliferation of several ovarian cancer tissue culture lines in a cell line-specific manner. This cell line-specific effect of STAMP is not unique and is also seen for the conventional effects of STAMP on glucocorticoid receptor-regulated gene transactivation.</p> <p>Conclusions</p> <p>This study indicates that a physiological function of STAMP in several settings is to modify cell growth rates in a manner that can be independent of steroid hormones. Studies with eleven tissue culture cell lines of ovarian cancer revealed a cell line-dependent effect of reduced STAMP mRNA on cell growth rates. This cell-line dependency is also seen for STAMP effects on glucocorticoid receptor-mediated transactivation. These preliminary findings suggest that further studies of STAMP in ovarian cancer may yield insight into ovarian cancer proliferation and may be useful in the development of biomarker panels.</p

Layered hybrid phase Li2NaV2(PO4)3/carbon dot nanocomposite cathodes for Li+/Na+ mixed-ion batteries

Hybrid phase Li2NaV2(PO4)3 (H-LNVP) is one of the most promising cathode materials for Li+/Na+ mixed-ion batteries.</p

Impact of the aqueous corrosion induced alteration layer on mechanical properties of pharmaceutical glasses.

It is known that network modifying ions (such as alkali or alkaline earth ions) make glasses susceptible to aqueous corrosion, resulting in the alteration of their surface layers. However, the effect of the altered layers on the mechanical properties of glasses has not been well understood. In this work we study this effect using the pharmaceutical boroaluminosilicate (BAS) glasses as objects by performing nano- and macroscale mechanical tests. The results show that extending the corrosion time increases the thickness of the alteration layer of the BAS glass. The water-related species in the alteration layer lowers the nanohardness, the reduced modulus, the nanowear resistance and Vickers hardness. The corrosion-induced “silica-like” structure in alteration layer benefits the densification of the subsurface caused by nanoindentation and nanowear, and thereby enhances the fracture toughness of the BAS glass. The correlation between the water content in the alteration layer and the mechanical properties has been revealed. This work is instrumental in the design of the next generation of pharmaceutical glasses with higher toughness

Probabilistic Charging Power Forecast of EVCS: Reinforcement Learning Assisted Deep Learning Approach

The electric vehicle (EV) and electric vehicle charging station (EVCS) have been widely deployed with the development of large-scale transportation electrifications. However, since charging behaviors of EVs show large uncertainties, the forecasting of EVCS charging power is non-trivial. This paper tackles this issue by proposing a reinforcement learning assisted deep learning framework for the probabilistic EVCS charging power forecasting to capture its uncertainties. Since the EVCS charging power data are not standard time-series data like electricity load, they are first converted to the time-series format. On this basis, one of the most popular deep learning models, the long short-term memory (LSTM) is used and trained to obtain the point forecast of EVCS charging power. To further capture the forecast uncertainty, a Markov decision process (MDP) is employed to model the change of LSTM cell states, which is solved by our proposed adaptive exploration proximal policy optimization (AePPO) algorithm based on reinforcement learning. Finally, experiments are carried out on the real EVCSs charging data from Caltech, and Jet Propulsion Laboratory, USA, respectively. The results and comparative analysis verify the effectiveness and outperformance of our proposed framework.Comment: Accepted by IEEE Transactions on Intelligent Vehicle