Robustifying a Policy in Multi-Agent RL with Diverse Cooperative Behaviors and Adversarial Style Sampling for Assistive Tasks

Autonomous assistance of people with motor impairments is one of the most promising applications of autonomous robotic systems. Recent studies have reported encouraging results using deep reinforcement learning (RL) in the healthcare domain. Previous studies showed that assistive tasks can be formulated as multi-agent RL, wherein there are two agents: a caregiver and a care-receiver. However, policies trained in multi-agent RL are often sensitive to the policies of other agents. In such a case, a trained caregiver's policy may not work for different care-receivers. To alleviate this issue, we propose a framework that learns a robust caregiver's policy by training it for diverse care-receiver responses. In our framework, diverse care-receiver responses are autonomously learned through trials and errors. In addition, to robustify the care-giver's policy, we propose a strategy for sampling a care-receiver's response in an adversarial manner during the training. We evaluated the proposed method using tasks in an Assistive Gym. We demonstrate that policies trained with a popular deep RL method are vulnerable to changes in policies of other agents and that the proposed framework improves the robustness against such changes.Comment: 7 pages, accepted for ICRA 202

シシンケイ サイセイ リョウホウ ノ ミライ

The optic nerve is a part of the central nervous system （CNS） and convey visual signals from the retina along their axons to the brain. Axonal damage can be induced by trauma, ischemia or in glaucoma, the most common cause of blindness in Japan. Like other CNS axons, the optic nerve has a very limited regenerative capacity. However, recent advances in research have revealed that combinational treatments, for example, overcoming the inhibitory environment of the glial scar and activating the intrinsic growth program, yield robust optic nerve regeneration. In addition, we revealed that overexpression of dedicator of cytokinesis 3 （Dock3）, one of the atypical Rho-guanine nucleotide exchange factors （Rho-GEFs）, plays important roles in promoting optic nerve regeneration. In response to the brain-derived neurotrophic factor （BDNF）, Dock3 activates multiple pathways that stimulate both actin polymerization and microtubule assembly, which are processes involved in neuroregeneration. Furthermore, Dock3 prevents glaucomatous retinal degeneration by suppressing both glutamate neurotoxicity and oxidative stress, suggesting that Dock3 signaling is a potential therapeutic target for both optic nerve regeneration and retinal neuroprotection. Based on our current knowledge, a combinatory approach including stimulation of Dock3 signalling may be effective for the treatment of complex diseases such as glaucoma, and this type of strategy may be available for future regeneration therapy using induced pluripotent stem （iPS） cells

Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning

In deep reinforcement learning, estimating the value function to evaluate the quality of states and actions is essential. The value function is often trained using the least squares method, which implicitly assumes a Gaussian error distribution. However, a recent study suggested that the error distribution for training the value function is often skewed because of the properties of the Bellman operator, and violates the implicit assumption of normal error distribution in the least squares method. To address this, we proposed a method called Symmetric Q-learning, in which the synthetic noise generated from a zero-mean distribution is added to the target values to generate a Gaussian error distribution. We evaluated the proposed method on continuous control benchmark tasks in MuJoCo. It improved the sample efficiency of a state-of-the-art reinforcement learning method by reducing the skewness of the error distribution.Comment: Accepted at AAAI 2024: The 38th Annual AAAI Conference on Artificial Intelligence (Main Tech Track

<ORIGINAL REPORT>IMMUNOHISTOCHEMICAL STUDIES ON LACTATE DEHYDROGENASE SUBUNITS IN LUNG CANCER CELLS

この論文は国立情報学研究所の学術雑誌公開支援事業により電子化されました。Immunohistochemical distribution of lactate dehydrogenase (LDH) subunits in human lung cancer cells were studied using fluorescent antibody technique. Both in cytological and histological specimen, specific fluorescence of LDH-H and LDH-M were seen in the cytoplasm of cancer cells, and in most cases no remarkable difference could be demonstrated between the distribution of H and M subunit. But in some cases specific fluorescence of M-subunit was stronger than that of H-subunit. The fluorescence of M-subunit was demonstrated as fine granules diffusely in the cytoplasm. On the other hand the fluorescence of H-subunit was rather localized and demonstrated as rather coarse granules. And this observation was discussed

Rutile-type Ge_xSn_{1−x}O_2 alloy layers lattice-matched to TiO_2 substrates for device applications

TiO₂に格子整合した高品質ルチル型GeₓSn₁−ₓO₂デバイスの動作実証 --高耐圧パワーデバイスへの応用--. 京都大学プレスリリース. 2024-01-19.We report the characterization and application of mist-CVD-grown rutile-structured Ge_xSn_{1−x}O_2 (x = ∼0.53) films lattice-matched to isostructural TiO_2(001) substrates. The grown surface was flat throughout the growth owing to the lattice-matching epitaxy. Additionally, the film was single-crystalline without misoriented domains and TEM-detectable threading dislocations due to the coherent heterointerface. Using the Ge_{0.49}Sn_{0.51}O_2 film with a carrier density of 7.8 × 10^{18} cm^{−3} and a mobility of 24 cm^2V^{−1}s^{−1}, lateral Schottky barrier diodes were fabricated with Pt anodes and Ti/Au cathodes. The diodes exhibited rectifying properties with a rectification ratio of 8.2 × 10^4 at ±5 V, showing the potential of Ge_xSn_{1−x}O_2 as a practical semiconductor

Delayed Onset of Experimental Autoimmune Encephalomyelitis in Olig1 Deficient Mice

BACKGROUND: Olig1 is a basic helix-loop-helix (bHLH) transcription factor that is essential for oligodendrogenesis and efficient remyelination. However, its role in neurodegenerative disorders has not been well-elucidated. METHODOLOGY/PRINCIPAL FINDINGS: Here we investigated the effects of Olig1 deficiency on experimental autoimmune encephalomyelitis (EAE), an animal model of multiple sclerosis (MS). We show that the mean disease onset of myelin oligodendrocyte glycoprotein (MOG)-induced EAE in Olig1(-/-) mice is significantly slower than wide-type (WT) mice (19.8 ± 2.2 in Olig1(-/-) mice and 9.5 ± 0.3 days in WT mice). In addition, 10% of Olig1(-/-) mice did not develop EAE by the end of the observation periods (60 days). The severity of EAE, the extent of demyelination, and the activation of microglial cells and astrocytes in spinal cords, were significantly milder in Olig1(-/-) mice compared with WT mice in the early stage. Moreover, the visual function, as assessed by the second-kernel of multifocal electroretinograms, was better preserved, and the number of degenerating axons in the optic nerve was significantly reduced in Olig1(-/-) mice. Interestingly, Olig1 deficiency had no effect on T cell response capability, however, it reduced the expression of myelin proteins such as MOG, myelin basic protein (MBP) and myelin-associated glycoprotein (MAG). The expression of Olig2 remained unchanged in the optic nerve and brain, and it was reduced in the spinal cord of Olig1(-/-) mice. CONCLUSIONS/SIGNIFICANCE: Our results suggest that the Olig1 signaling pathways may be involved in the incidence rate and the severity of neurological symptoms in MS