Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization

Deep reinforcement learning with domain randomization learns a control policy in various simulations with randomized physical and sensor model parameters to become transferable to the real world in a zero-shot setting. However, a huge number of samples are often required to learn an effective policy when the range of randomized parameters is extensive due to the instability of policy updates. To alleviate this problem, we propose a sample-efficient method named cyclic policy distillation (CPD). CPD divides the range of randomized parameters into several small sub-domains and assigns a local policy to each one. Then local policies are learned while cyclically transitioning to sub-domains. CPD accelerates learning through knowledge transfer based on expected performance improvements. Finally, all of the learned local policies are distilled into a global policy for sim-to-real transfers. CPD's effectiveness and sample efficiency are demonstrated through simulations with four tasks (Pendulum from OpenAIGym and Pusher, Swimmer, and HalfCheetah from Mujoco), and a real-robot, ball-dispersal task. We published code and videos from our experiments at https://github.com/yuki-kadokawa/cyclic-policy-distillation

Comparing the Osteogenic Potential and Bone Regeneration Capacities of Dedifferentiated Fat Cells and Adipose-Derived Stem Cells In Vitro and In Vivo: Application of DFAT Cells Isolated by a Mesh Method

Background: We investigated and compared the osteogenic potential and bone regeneration capacities of dedifferentiated fat cells (DFAT cells) and adipose-derived stem cells (ASCs). Method: We isolated DFAT cells and ASCs from GFP mice. DFAT cells were established by a new culture method using a mesh culture instead of a ceiling culture. The isolated DFAT cells and ASCs were incubated in osteogenic medium, then alizarin red staining, alkaline phosphatase (ALP) assays, and RT-PCR (for RUNX2, osteopontin, DLX5, osterix, and osteocalcin) were performed to evaluate the osteoblastic differentiation ability of both cell types in vitro. In vivo, the DFAT cells and ASCs were incubated in osteogenic medium for four weeks and seeded on collagen composite scaffolds, then implanted subcutaneously into the backs of mice. We then performed hematoxylin and eosin staining and immunostaining for GFP and osteocalcin. Results: The alizarin red-stained areas in DFAT cells showed weak calcification ability at two weeks, but high calcification ability at three weeks, similar to ASCs. The ALP levels of ASCs increased earlier than in DFAT cells and showed a significant difference (p Conclusion: DFAT cells are easily isolated from a small amount of adipose tissue and are readily expanded with high purity; thus, DFAT cells are applicable to many tissue-engineering strategies and cell-based therapies

子宮内膜側に発症する (Subtypel) 子宮腺筋症は黄体ホルモン療法による多量性器出血の危険因子である

We aimed to retrospectively analyze the risk factors of a continuous dienogest (DNG) therapy for serious unpredictable bleeding in patients with symptomatic adenomyosis. This is a retrospective study based on data extracted from medical records of 84 women treated with 2 mg of DNG orally each day between 2008 and 2017. 47 subjects were excluded from the original analyses due to an inadequate subcategorization into subtype I and subtype II and a lack of hemoglobin levels. The influence of various independent variables on serious unpredictable bleeding was assessed. Of the 37 eligible patients who received the continuous DNG therapy, 14 patients experienced serious unpredictable bleeding. Univariate analysis revealed that the serious bleeding group had subtype I adenomyosis (P = 0.027). There was no correlation between age, parity, minimum hemoglobin level before treatment, previous endometrial curettage, and duration of DNG administration, or uterine or adenomyosis size and the serious bleeding. A DNG-related serious unpredictable bleeding is associated with the structural type of adenomyosis (subtype I) in patients with symptomatic adenomyosis.博士（医学）・甲第800号・令和3年9月29日© The Author(s) 2019 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Self-Organized Synchronous Calcium Transients in a Cultured Human Neural Network Derived from Cerebral Organoids

The cerebrum is a major center for brain function, and its activity is derived from the assembly of activated cells in neural networks. It is currently difficult to study complex human cerebral neuronal network activity. Here, using cerebral organoids, we report self-organized and complex human neural network activities that include synchronized and non-synchronized patterns. Self-organized neuronal network formation was observed following a dissociation culture of human embryonic stem cell-derived cerebral organoids. The spontaneous individual and synchronized activity of the network was measured via calcium imaging, and subsequent analysis enabled the examination of detailed cell activity patterns, providing simultaneous raster plots, cluster analyses, and cell distribution data. Finally, we demonstrated the feasibility of our system to assess drug-inducible dynamic changes of the network activity. The comprehensive functional analysis of human neuronal networks using this system may offer a powerful tool to access human brain function

Experimental model for the irradiation-mediated abscopal effect and factors influencing this effect

Radiotherapy (RT) is the primary treatment for cancer. Ionizing radiation from RT induces tumor damage at the irradiated site, and, although clinically infrequent, may cause regression of tumors distant from the irradiated site-a phenomenon known as the abscopal effect. Recently, the abscopal effect has been related to prolongation of overall survival time in cancer patients, though the factors that influence the abscopal effect are not well understood. The aim of this study is to clarify the factors influencing on abscopal effect. Here, we established a mouse model in which we induced the abscopal effect. We injected MC38 (mouse colon adenocarcinoma) cells subcutaneously into C57BL/6 mice at two sites. Only one tumor was irradiated and the sizes of both tumors were measured over time. The non-irradiated-site tumor showed regression, demonstrating the abscopal effect. This effect was enhanced by an increase in the irradiated-tumor volume and by administration of anti-PD1 antibody. When the abscopal effect was induced by a combination of RT and anti-PD1 antibody, it was also influenced by radiation dose and irradiated-tumor volume. These phenomena were also verified in other cell line, B16F10 cells (mouse melanoma cells). These findings provide further evidence of the mechanism for, and factors that influence, the abscopal effect in RT