NA2\text{A}^\text{2}Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning

Value decomposition is widely used in cooperative multi-agent reinforcement learning, however, its implicit credit assignment mechanism is not yet fully understood due to black-box networks. In this work, we study an interpretable value decomposition framework via the family of generalized additive models. We present a novel method, named Neural Attention Additive Q-learning~(N$\text{A}^\text{2}$Q), providing inherent intelligibility of collaboration behavior. N$\text{A}^\text{2}$Q can explicitly factorize the optimal joint policy induced by enriching shape functions to model all possible coalitions of agents into individual policies. Moreover, we construct identity semantics to promote estimating credits together with the global state and individual value functions, where local semantic masks help us diagnose whether each agent captures relevant-task information. Extensive experiments show that N$\text{A}^\text{2}$Q consistently achieves superior performance compared to different state-of-the-art methods on all challenging tasks, while yielding human-like interpretability

MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees

While achieving tremendous success in various fields, existing multi-agent reinforcement learning (MARL) with a black-box neural network architecture makes decisions in an opaque manner that hinders humans from understanding the learned knowledge and how input observations influence decisions. Instead, existing interpretable approaches, such as traditional linear models and decision trees, usually suffer from weak expressivity and low accuracy. To address this apparent dichotomy between performance and interpretability, our solution, MIXing Recurrent soft decision Trees (MIXRTs), is a novel interpretable architecture that can represent explicit decision processes via the root-to-leaf path and reflect each agent's contribution to the team. Specifically, we construct a novel soft decision tree to address partial observability by leveraging the advances in recurrent neural networks, and demonstrate which features influence the decision-making process through the tree-based model. Then, based on the value decomposition framework, we linearly assign credit to each agent by explicitly mixing individual action values to estimate the joint action value using only local observations, providing new insights into how agents cooperate to accomplish the task. Theoretical analysis shows that MIXRTs guarantees the structural constraint on additivity and monotonicity in the factorization of joint action values. Evaluations on the challenging Spread and StarCraft II tasks show that MIXRTs achieves competitive performance compared to widely investigated methods and delivers more straightforward explanations of the decision processes. We explore a promising path toward developing learning algorithms with both high performance and interpretability, potentially shedding light on new interpretable paradigms for MARL

BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization

Evolutionary reinforcement learning (ERL) algorithms recently raise attention in tackling complex reinforcement learning (RL) problems due to high parallelism, while they are prone to insufficient exploration or model collapse without carefully tuning hyperparameters (aka meta-parameters). In the paper, we propose a general meta ERL framework via bilevel optimization (BiERL) to jointly update hyperparameters in parallel to training the ERL model within a single agent, which relieves the need for prior domain knowledge or costly optimization procedure before model deployment. We design an elegant meta-level architecture that embeds the inner-level's evolving experience into an informative population representation and introduce a simple and feasible evaluation of the meta-level fitness function to facilitate learning efficiency. We perform extensive experiments in MuJoCo and Box2D tasks to verify that as a general framework, BiERL outperforms various baselines and consistently improves the learning performance for a diversity of ERL algorithms.Comment: Published as a conference paper at European Conference on Artificial Intelligence (ECAI) 202

Vitamin D Signaling through Induction of Paneth Cell Defensins Maintains Gut Microbiota and Improves Metabolic Disorders and Hepatic Steatosis in Animal Models.

Metabolic syndrome (MetS), characterized as obesity, insulin resistance, and non-alcoholic fatty liver diseases (NAFLD), is associated with vitamin D insufficiency/deficiency in epidemiological studies, while the underlying mechanism is poorly addressed. On the other hand, disorder of gut microbiota, namely dysbiosis, is known to cause MetS and NAFLD. It is also known that systemic inflammation blocks insulin signaling pathways, leading to insulin resistance and glucose intolerance, which are the driving force for hepatic steatosis. Vitamin D receptor (VDR) is highly expressed in the ileum of the small intestine, which prompted us to test a hypothesis that vitamin D signaling may determine the enterotype of gut microbiota through regulating the intestinal interface. Here, we demonstrate that high-fat-diet feeding (HFD) is necessary but not sufficient, while additional vitamin D deficiency (VDD) as a second hit is needed, to induce robust insulin resistance and fatty liver. Under the two hits (HFD+VDD), the Paneth cell-specific alpha-defensins including α-defensin 5 (DEFA5), MMP7 which activates the pro-defensins, as well as tight junction genes, and MUC2 are all suppressed in the ileum, resulting in mucosal collapse, increased gut permeability, dysbiosis, endotoxemia, systemic inflammation which underlie insulin resistance and hepatic steatosis. Moreover, under the vitamin D deficient high fat feeding (HFD+VDD), Helicobacter hepaticus, a known murine hepatic-pathogen, is substantially amplified in the ileum, while Akkermansia muciniphila, a beneficial symbiotic, is diminished. Likewise, the VD receptor (VDR) knockout mice exhibit similar phenotypes, showing down regulation of alpha-defensins and MMP7 in the ileum, increased Helicobacter hepaticus and suppressed Akkermansia muciniphila. Remarkably, oral administration of DEFA5 restored eubiosys, showing suppression of Helicobacter hepaticus and increase of Akkermansia muciniphila in association with resolving metabolic disorders and fatty liver in the HFD+VDD mice. An in vitro analysis showed that DEFA5 peptide could directly suppress Helicobacter hepaticus. Thus, the results of this study reveal critical roles of a vitamin D/VDR axis in optimal expression of defensins and tight junction genes in support of intestinal integrity and eubiosis to suppress NAFLD and metabolic disorders

Pygo2 expands mammary progenitor cells by facilitating histone H3 K4 methylation

Recent studies have unequivocally identified multipotent stem/progenitor cells in mammary glands, offering a tractable model system to unravel genetic and epigenetic regulation of epithelial stem/progenitor cell development and homeostasis. In this study, we show that Pygo2, a member of an evolutionarily conserved family of plant homeo domain–containing proteins, is expressed in embryonic and postnatal mammary progenitor cells. Pygo2 deficiency, which is achieved by complete or epithelia-specific gene ablation in mice, results in defective mammary morphogenesis and regeneration accompanied by severely compromised expansive self-renewal of epithelial progenitor cells. Pygo2 converges with Wnt/β-catenin signaling on progenitor cell regulation and cell cycle gene expression, and loss of epithelial Pygo2 completely rescues β-catenin–induced mammary outgrowth. We further describe a novel molecular function of Pygo2 that is required for mammary progenitor cell expansion, which is to facilitate K4 trimethylation of histone H3, both globally and at Wnt/β-catenin target loci, via direct binding to K4-methyl histone H3 and recruiting histone H3 K4 methyltransferase complexes

Multiport Energy Management System Design for a 150 kW Range-Extended Towing Vessel

This paper proposes a multiport energy management system (EMS) and its rule-based expert control strategy for a 150 kW range-extended towing vessel (RETV). The system integrates a diesel generator system, a permanent magnet synchronous motor, a lithium battery, and supercapacitors. To verify its feasibility and effectiveness, the proposed multiport EMS was modelled and tested through MATLAB/Simulink. Simulation results demonstrate that the designed multiport EMS works efficiently under the five typical operating conditions of the 150 kW RETV. In addition, two case studies were conducted and compared to investigate the impact of the battery’s initial state of charge (SoC) on the system’s energy efficiency. It was found that an overall 85% energy efficiency can be achieved for the RETV when the initial SoC is either 75% or 15%. The battery consistently operates within the optimal SoC range of 20% to 80%, and the supercapacitors effectively meet the instantaneous high-power demand

An Increase in Consuming Adequately Iodized Salt May Not Be Enough to Rectify Iodine Deficiency in Pregnancy in an Iodine-Sufficient Area of China

Universal salt iodization (USI) has been implemented for two decades in China. It is crucial to periodically monitor iodine status in the most vulnerable population, such as pregnant women. A cross-sectional study was carried out in an evidence-proved iodine-sufficient province to evaluate iodine intake in pregnancy. According to the WHO/UNICEF/ICCIDD recommendation criteria of adequate iodine intake in pregnancy (150–249 µg/L), the median urinary iodine concentration (UIC) of the total 8159 recruited pregnant women was 147.5 µg/L, which indicated pregnant women had iodine deficiency at the province level. Overall, 51.0% of the total study participants had iodine deficiency with a UIC < 150 µg/L and only 32.9% of them had adequate iodine. Participants living in coastal areas had iodine deficiency with a median UIC of 130.1 µg/L, while those in inland areas had marginally adequate iodine intake with a median UIC of 158.1 µg/L (p < 0.001). Among the total study participants, 450 pregnant women consuming non-iodized salt had mild-moderate iodine deficiency with a median UIC of 99.6 µg/L; 7363 pregnant women consuming adequately iodized salt had a lightly statistically higher median UIC of 151.9 µg/L, compared with the recommended adequate level by the WHO/UNICEF/ICCIDD (p < 0.001). Consuming adequately iodized salt seemed to lightly increase the median UIC level, but it may not be enough to correct iodine nutrition status to an optimum level as recommended by the WHO/UNICEF/ICCIDD. We therefore suggest that, besides strengthening USI policy, additional interventive measure may be needed to improve iodine intake in pregnancy

Supramolecular Fluorescent Polymers Containing α‑Cyanostilbene-Based Stereoisomers: <i>Z</i>/<i>E</i>‑Isomerization Induced Multiple Reversible Switching

Photoresponsive materials play an important role in smart sensors and readable optical devices. The α-cyanostilbene moiety gathers <i>Z</i>/<i>E</i>-isomerization, mesogenic, and aggregation-induced enhanced emission (AIEE) properties together and can be considered as a multifunctional building block bearing luminogen, AIEgen, mesogen, and chromphore characteristics. Here, we report isomerization induced a series of notable reversible switching based on α-cyanostilbene containing hydrogen-bonded supramolecular polymers. <i>Z</i>- and <i>E</i>-stereoisomers of dendritic α-cyanostilbene derivatives (<i>Z</i>-CNBP and <i>E</i>-CNBP) were hydrogen-bonded with poly­(4-vinylpyridine) (P4VP), giving rise to P4VP­(<i>Z</i>-CNBP)_<i>x</i> and P4VP­(<i>E</i>-CNBP)_<i>x</i>. P4VP­(<i>Z</i>-CNBP)_<i>x</i> exhibit a hexagonal columnar (Φ_h) phase at 0.4 ≤ <i>x</i> ≤ 1.0, while the lamellar phase could be formed for P4VP­(<i>E</i>-CNBP)_<i>x</i> at 0.3 ≤ <i>x</i> ≤ 0.7. P4VP­(<i>Z</i>-CNBP)_0.8 shows photothermal <i>E</i>/<i>Z</i>-isomerization induced reversible switching, including phase structures, surface topographies, birefringence, and fluorescence properties, whereas P4VP­(<i>E</i>-CNBP)_0.5 exhibits enhanced fluorescence emission upon UV irradiation

Effects of the cap layer on the properties of AlN barrier HEMT grown on 6-inch Si(111) substrate

A series of AlN/GaN heterostructures were grown on 150 mm Si substrates by metal organic chemical vapor deposition (MOCVD). Different cap layer structures, including gallium nitride (GaN) and silicon nitride (SiN _x ), were used to passivate the heterostructure surface. A 3.5 nm thick SiN _x cap is able to maintain the two dimensional electron gas (2DEG) stability in a long period. An AlN/GaN heterostructure with a 4.5 nm thick AlN barrier exhibits the best 2DEG properties, in terms of sheet resistance, carrier mobility and stability. The carrier mobility of the 2DEG can be enhanced by a combination of SiNx and GaN cap layers to over 1400 cm ^2 /Vs