38 research outputs found
NQ: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning
Value decomposition is widely used in cooperative multi-agent reinforcement
learning, however, its implicit credit assignment mechanism is not yet fully
understood due to black-box networks. In this work, we study an interpretable
value decomposition framework via the family of generalized additive models. We
present a novel method, named Neural Attention Additive
Q-learning~(NQ), providing inherent intelligibility of
collaboration behavior. NQ can explicitly factorize the
optimal joint policy induced by enriching shape functions to model all possible
coalitions of agents into individual policies. Moreover, we construct identity
semantics to promote estimating credits together with the global state and
individual value functions, where local semantic masks help us diagnose whether
each agent captures relevant-task information. Extensive experiments show that
NQ consistently achieves superior performance compared to
different state-of-the-art methods on all challenging tasks, while yielding
human-like interpretability
MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees
While achieving tremendous success in various fields, existing multi-agent
reinforcement learning (MARL) with a black-box neural network architecture
makes decisions in an opaque manner that hinders humans from understanding the
learned knowledge and how input observations influence decisions. Instead,
existing interpretable approaches, such as traditional linear models and
decision trees, usually suffer from weak expressivity and low accuracy. To
address this apparent dichotomy between performance and interpretability, our
solution, MIXing Recurrent soft decision Trees (MIXRTs), is a novel
interpretable architecture that can represent explicit decision processes via
the root-to-leaf path and reflect each agent's contribution to the team.
Specifically, we construct a novel soft decision tree to address partial
observability by leveraging the advances in recurrent neural networks, and
demonstrate which features influence the decision-making process through the
tree-based model. Then, based on the value decomposition framework, we linearly
assign credit to each agent by explicitly mixing individual action values to
estimate the joint action value using only local observations, providing new
insights into how agents cooperate to accomplish the task. Theoretical analysis
shows that MIXRTs guarantees the structural constraint on additivity and
monotonicity in the factorization of joint action values. Evaluations on the
challenging Spread and StarCraft II tasks show that MIXRTs achieves competitive
performance compared to widely investigated methods and delivers more
straightforward explanations of the decision processes. We explore a promising
path toward developing learning algorithms with both high performance and
interpretability, potentially shedding light on new interpretable paradigms for
MARL
BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization
Evolutionary reinforcement learning (ERL) algorithms recently raise attention
in tackling complex reinforcement learning (RL) problems due to high
parallelism, while they are prone to insufficient exploration or model collapse
without carefully tuning hyperparameters (aka meta-parameters). In the paper,
we propose a general meta ERL framework via bilevel optimization (BiERL) to
jointly update hyperparameters in parallel to training the ERL model within a
single agent, which relieves the need for prior domain knowledge or costly
optimization procedure before model deployment. We design an elegant meta-level
architecture that embeds the inner-level's evolving experience into an
informative population representation and introduce a simple and feasible
evaluation of the meta-level fitness function to facilitate learning
efficiency. We perform extensive experiments in MuJoCo and Box2D tasks to
verify that as a general framework, BiERL outperforms various baselines and
consistently improves the learning performance for a diversity of ERL
algorithms.Comment: Published as a conference paper at European Conference on Artificial
Intelligence (ECAI) 202
Vitamin D Signaling through Induction of Paneth Cell Defensins Maintains Gut Microbiota and Improves Metabolic Disorders and Hepatic Steatosis in Animal Models.
Metabolic syndrome (MetS), characterized as obesity, insulin resistance, and non-alcoholic fatty liver diseases (NAFLD), is associated with vitamin D insufficiency/deficiency in epidemiological studies, while the underlying mechanism is poorly addressed. On the other hand, disorder of gut microbiota, namely dysbiosis, is known to cause MetS and NAFLD. It is also known that systemic inflammation blocks insulin signaling pathways, leading to insulin resistance and glucose intolerance, which are the driving force for hepatic steatosis. Vitamin D receptor (VDR) is highly expressed in the ileum of the small intestine, which prompted us to test a hypothesis that vitamin D signaling may determine the enterotype of gut microbiota through regulating the intestinal interface. Here, we demonstrate that high-fat-diet feeding (HFD) is necessary but not sufficient, while additional vitamin D deficiency (VDD) as a second hit is needed, to induce robust insulin resistance and fatty liver. Under the two hits (HFD+VDD), the Paneth cell-specific alpha-defensins including α-defensin 5 (DEFA5), MMP7 which activates the pro-defensins, as well as tight junction genes, and MUC2 are all suppressed in the ileum, resulting in mucosal collapse, increased gut permeability, dysbiosis, endotoxemia, systemic inflammation which underlie insulin resistance and hepatic steatosis. Moreover, under the vitamin D deficient high fat feeding (HFD+VDD), Helicobacter hepaticus, a known murine hepatic-pathogen, is substantially amplified in the ileum, while Akkermansia muciniphila, a beneficial symbiotic, is diminished. Likewise, the VD receptor (VDR) knockout mice exhibit similar phenotypes, showing down regulation of alpha-defensins and MMP7 in the ileum, increased Helicobacter hepaticus and suppressed Akkermansia muciniphila. Remarkably, oral administration of DEFA5 restored eubiosys, showing suppression of Helicobacter hepaticus and increase of Akkermansia muciniphila in association with resolving metabolic disorders and fatty liver in the HFD+VDD mice. An in vitro analysis showed that DEFA5 peptide could directly suppress Helicobacter hepaticus. Thus, the results of this study reveal critical roles of a vitamin D/VDR axis in optimal expression of defensins and tight junction genes in support of intestinal integrity and eubiosis to suppress NAFLD and metabolic disorders
Pygo2 expands mammary progenitor cells by facilitating histone H3 K4 methylation
Recent studies have unequivocally identified multipotent stem/progenitor cells in mammary glands, offering a tractable model system to unravel genetic and epigenetic regulation of epithelial stem/progenitor cell development and homeostasis. In this study, we show that Pygo2, a member of an evolutionarily conserved family of plant homeo domain–containing proteins, is expressed in embryonic and postnatal mammary progenitor cells. Pygo2 deficiency, which is achieved by complete or epithelia-specific gene ablation in mice, results in defective mammary morphogenesis and regeneration accompanied by severely compromised expansive self-renewal of epithelial progenitor cells. Pygo2 converges with Wnt/β-catenin signaling on progenitor cell regulation and cell cycle gene expression, and loss of epithelial Pygo2 completely rescues β-catenin–induced mammary outgrowth. We further describe a novel molecular function of Pygo2 that is required for mammary progenitor cell expansion, which is to facilitate K4 trimethylation of histone H3, both globally and at Wnt/β-catenin target loci, via direct binding to K4-methyl histone H3 and recruiting histone H3 K4 methyltransferase complexes
Multiport Energy Management System Design for a 150 kW Range-Extended Towing Vessel
This paper proposes a multiport energy management system (EMS) and its rule-based expert control strategy for a 150 kW range-extended towing vessel (RETV). The system integrates a diesel generator system, a permanent magnet synchronous motor, a lithium battery, and supercapacitors. To verify its feasibility and effectiveness, the proposed multiport EMS was modelled and tested through MATLAB/Simulink. Simulation results demonstrate that the designed multiport EMS works efficiently under the five typical operating conditions of the 150 kW RETV. In addition, two case studies were conducted and compared to investigate the impact of the battery’s initial state of charge (SoC) on the system’s energy efficiency. It was found that an overall 85% energy efficiency can be achieved for the RETV when the initial SoC is either 75% or 15%. The battery consistently operates within the optimal SoC range of 20% to 80%, and the supercapacitors effectively meet the instantaneous high-power demand
An Increase in Consuming Adequately Iodized Salt May Not Be Enough to Rectify Iodine Deficiency in Pregnancy in an Iodine-Sufficient Area of China
Universal salt iodization (USI) has been implemented for two decades in China. It is crucial to periodically monitor iodine status in the most vulnerable population, such as pregnant women. A cross-sectional study was carried out in an evidence-proved iodine-sufficient province to evaluate iodine intake in pregnancy. According to the WHO/UNICEF/ICCIDD recommendation criteria of adequate iodine intake in pregnancy (150–249 µg/L), the median urinary iodine concentration (UIC) of the total 8159 recruited pregnant women was 147.5 µg/L, which indicated pregnant women had iodine deficiency at the province level. Overall, 51.0% of the total study participants had iodine deficiency with a UIC < 150 µg/L and only 32.9% of them had adequate iodine. Participants living in coastal areas had iodine deficiency with a median UIC of 130.1 µg/L, while those in inland areas had marginally adequate iodine intake with a median UIC of 158.1 µg/L (p < 0.001). Among the total study participants, 450 pregnant women consuming non-iodized salt had mild-moderate iodine deficiency with a median UIC of 99.6 µg/L; 7363 pregnant women consuming adequately iodized salt had a lightly statistically higher median UIC of 151.9 µg/L, compared with the recommended adequate level by the WHO/UNICEF/ICCIDD (p < 0.001). Consuming adequately iodized salt seemed to lightly increase the median UIC level, but it may not be enough to correct iodine nutrition status to an optimum level as recommended by the WHO/UNICEF/ICCIDD. We therefore suggest that, besides strengthening USI policy, additional interventive measure may be needed to improve iodine intake in pregnancy
Supramolecular Fluorescent Polymers Containing α‑Cyanostilbene-Based Stereoisomers: <i>Z</i>/<i>E</i>‑Isomerization Induced Multiple Reversible Switching
Photoresponsive materials play an
important role in smart sensors
and readable optical devices. The α-cyanostilbene moiety gathers <i>Z</i>/<i>E</i>-isomerization, mesogenic, and aggregation-induced
enhanced emission (AIEE) properties together and can be considered
as a multifunctional building block bearing luminogen, AIEgen, mesogen,
and chromphore characteristics. Here, we report isomerization induced
a series of notable reversible switching based on α-cyanostilbene
containing hydrogen-bonded supramolecular polymers. <i>Z</i>- and <i>E</i>-stereoisomers of dendritic α-cyanostilbene
derivatives (<i>Z</i>-CNBP and <i>E</i>-CNBP)
were hydrogen-bonded with polyÂ(4-vinylpyridine) (P4VP), giving rise
to P4VPÂ(<i>Z</i>-CNBP)<sub><i>x</i></sub> and
P4VPÂ(<i>E</i>-CNBP)<sub><i>x</i></sub>. P4VPÂ(<i>Z</i>-CNBP)<sub><i>x</i></sub> exhibit a hexagonal
columnar (Φ<sub>h</sub>) phase at 0.4 ≤ <i>x</i> ≤ 1.0, while the lamellar phase could be formed for P4VPÂ(<i>E</i>-CNBP)<sub><i>x</i></sub> at 0.3 ≤ <i>x</i> ≤ 0.7. P4VPÂ(<i>Z</i>-CNBP)<sub>0.8</sub> shows photothermal <i>E</i>/<i>Z</i>-isomerization
induced reversible switching, including phase structures, surface
topographies, birefringence, and fluorescence properties, whereas
P4VPÂ(<i>E</i>-CNBP)<sub>0.5</sub> exhibits enhanced fluorescence
emission upon UV irradiation
Effects of the cap layer on the properties of AlN barrier HEMT grown on 6-inch Si(111) substrate
A series of AlN/GaN heterostructures were grown on 150 mm Si substrates by metal organic chemical vapor deposition (MOCVD). Different cap layer structures, including gallium nitride (GaN) and silicon nitride (SiN _x ), were used to passivate the heterostructure surface. A 3.5 nm thick SiN _x cap is able to maintain the two dimensional electron gas (2DEG) stability in a long period. An AlN/GaN heterostructure with a 4.5 nm thick AlN barrier exhibits the best 2DEG properties, in terms of sheet resistance, carrier mobility and stability. The carrier mobility of the 2DEG can be enhanced by a combination of SiNx and GaN cap layers to over 1400 cm ^2 /Vs