Continuous Multiagent Control using Collective Behavior Entropy for Large-Scale Home Energy Management

With the increasing popularity of electric vehicles, distributed energy generation and storage facilities in smart grid systems, an efficient Demand-Side Management (DSM) is urgent for energy savings and peak loads reduction. Traditional DSM works focusing on optimizing the energy activities for a single household can not scale up to large-scale home energy management problems. Multi-agent Deep Reinforcement Learning (MA-DRL) shows a potential way to solve the problem of scalability, where modern homes interact together to reduce energy consumers consumption while striking a balance between energy cost and peak loads reduction. However, it is difficult to solve such an environment with the non-stationarity, and existing MA-DRL approaches cannot effectively give incentives for expected group behavior. In this paper, we propose a collective MA-DRL algorithm with continuous action space to provide fine-grained control on a large scale microgrid. To mitigate the non-stationarity of the microgrid environment, a novel predictive model is proposed to measure the collective market behavior. Besides, a collective behavior entropy is introduced to reduce the high peak loads incurred by the collective behaviors of all householders in the smart grid. Empirical results show that our approach significantly outperforms the state-of-the-art methods regarding power cost reduction and daily peak loads optimization

Applications of New Technologies and New Methods in ZHENG Differentiation

With the hope to provide an effective approach for personalized diagnosis and treatment clinically, Traditional Chinese Medicine (TCM) is being paid increasing attention as a complementary and alternative medicine. It performs treatment based on ZHENG (TCM syndrome) differentiation, which could be identified as clinical special phenotypes by symptoms and signs of patients. However, it caused skepticism and criticism because ZHENG classification only depends on observation, knowledge, and clinical experience of TCM practitioners, which is lack of objectivity and repeatability. Scientists have done fruitful researches for its objectivity and standardization. Compared with traditional four diagnostic methods (looking, listening and smelling, asking, and touching), in this paper, the applications of new technologies and new methods on the ZHENG differentiation were systemically reviewed, including acquisition, analysis, and integration of clinical data or information. Furthermore, the characteristics and application range of these technologies and methods were summarized. It will provide reference for further researches

Metabonomic Evaluation of ZHENG Differentiation and Treatment by Fuzhenghuayu Tablet in Hepatitis-B-Caused Cirrhosis

In Traditional Chinese Medicine (TCM), treatment based on ZHENG (also called TCM syndrome and pattern) differentiation has been applied for about 3 thousand years, while there are some difficulties to communicate with western medicine. In the present work, metabonomic methods were utilized to differentiate ZHENG types and evaluate the therapeutic efficiency of Fuzhenghuayu (FZHY) tablet in hepatitis-B-caused cirrhosis (HBC). Urine samples of 12 healthy volunteers (control group, CG) and 31 HBC patients (HBCG) were analyzed by gas chromatography mass spectrometry (GC/MS) and multivariate statistical analysis. The significantly changed metabolites between CG and HBCG were selected by PLS-DA loading plot analysis. Moreover, 4 ZHENGs were differentiated mutually, suggesting that there was urine metabolic material basis in ZHENG differentiation. The efficiency of FZHY tablet on subjects with spleen deficiency with dampness encumbrance syndrome (SDDES) and liver-kidney yin deficiency syndrome (LKYDS) was better than that of other syndromes. The efficiency of FZHY treatment based on ZHENG differentiation indicated that accurately ZHENG differentiating could guide the appropriate TCM treatment in HBC

1,2,3,4-Tetra­hydro­phenazine 5,10-dioxide

The complete mol­ecule of the title compound, C12H12N2O2, lies on two crystallographic symmetry elements: a twofold axis and a mirror plane. In the mol­ecular structure, the quinoxaline ring and two methyl­ene substituents lie on the mirror plane while the other two methyl­ene groups are disordered about the plane. The crystal packing is stabilized by weak inter­molecular π–π stacking inter­actions with centroid–centroid distances of 3.6803 (7) Å

OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments

In reinforcement learning, the optimism in the face of uncertainty (OFU) is a mainstream principle for directing exploration towards less explored areas, characterized by higher uncertainty. However, in the presence of environmental stochasticity (noise), purely optimistic exploration may lead to excessive probing of high-noise areas, consequently impeding exploration efficiency. Hence, in exploring noisy environments, while optimism-driven exploration serves as a foundation, prudent attention to alleviating unnecessary over-exploration in high-noise areas becomes beneficial. In this work, we propose Optimistic Value Distribution Explorer (OVD-Explorer) to achieve a noise-aware optimistic exploration for continuous control. OVD-Explorer proposes a new measurement of the policy's exploration ability considering noise in optimistic perspectives, and leverages gradient ascent to drive exploration. Practically, OVD-Explorer can be easily integrated with continuous control RL algorithms. Extensive evaluations on the MuJoCo and GridChaos tasks demonstrate the superiority of OVD-Explorer in achieving noise-aware optimistic exploration.Comment: Accepted by AAAI 2024, with appendi