Reinforcement Learning Guided by Double Replay Memory

Experience replay memory in reinforcement learning enables agents to remember and reuse past experiences. Most of the reinforcement models are subject to single experience replay memory to operate agents. In this article, we propose a framework that accommodates doubly used experience replay memory, exploiting both important transitions and new transitions simultaneously. In numerical studies, the deep Q-networks (DQN) equipped with double experience replay memory are examined under various scenarios. A self-driving car requires an automated agent to figure out when to adequately change lanes on the real-time basis. To this end, we apply our proposed agent to the simulation of urban mobility (SUMO) experiments. Besides, we also verify its applicability to reinforcement learning whose action space is discrete (e.g., computer game environments). Taken all together, we conclude that the proposed framework outperforms priorly known reinforcement learning models in the virtue of double experience replay memory

A Study of Skin Reflectance Using Kubelka-Munk Model

빛은 피부에서 개개인마다 상이한 반사, 흡수 및 산란 등의 광학적 거동을 보여준다. 특히, 피부에서 빛의반사는 분광반사율이라는 물리량 측정을 통해 개인의 피부 밝기지표로 널리 활용되어 왔다. 따라서 피부에서빛의 반사 거동 연구는 개인 맞춤형 화장품, 특히 메이크업 제품 개발에 있어 좀 더 효율적인 처방 개선에 기여하거나 새로운 평가법에 활용될 수 있다. 본 연구에서는 Kubelka-Munk 모델을 이용하여 개인의 피부 특성에 따른 빛의 분광반사율 거동에 대해 다각적인 분석을 수행하였다. 또한, 상기 모델과 기존에 알려진 문헌 정보를이용하여 개인의 피부 분광반사율에 영향을 줄 수 있는 피부 두께 및 헤모글로빈 등의 다양한 파라미터들에 대한기여도 분석을 수행하였다. 이를 통해, 일반적인 여성의 피부에서의 분광반사율 이론치를 계산하였고, 분광반사율 실측을 통해 이론치와 실측치 간의 유사성을 확인하였다. Kubelka-Munk 모델을 이용한 피부 분광반사율연구는 향후 새로운 개인 맞춤형 메이크업 화장품 개발에 유용하게 이용될 수 있을 것으로 사료된다. Light shows various optical behaviors such as reflection, absorption, and scattering on skin for individuals. In particular, reflection of light from the skin has been widely used as the brightness index of the skin of individuals through the measurement of the physical quantity of spectral reflectance. Therefore, the study of light behavior on skin would be useful for the preparation of new evaluation method in the development stage of make-up products. In this study, multi-dimensional analysis for spectral reflectance behavior of light on individual skin was performed using Kubelka-Munk model. Also, we analyzed the contribution of skin parameters such as skin thickness and hemoglobin, which could affect the spectral reflectance, using above model and literature information. Base on this, we calculated the theoretical reflectance of normal women for visual light, which showed good agreement with the measured reflectance. Our study of light propagation in skin based on Kubelka-Munk model provides useful insight for the development of personalized cosmetic in the near future.22Nkc

Three-dimensional heart extracellular matrix enhances chemically induced direct cardiac reprogramming

© 2022 The Authors.Direct cardiac reprogramming has emerged as a promising therapeutic approach for cardiac regeneration. Full chemical reprogramming with small molecules to generate cardiomyocytes may be more amenable than genetic reprogramming for clinical applications as it avoids safety concerns associated with genetic manipulations. However, challenges remain regarding low conversion efficiency and incomplete cardiomyocyte maturation. Furthermore, the therapeutic potential of chemically induced cardiomyocytes (CiCMs) has not been investigated. Here, we report that a three-dimensional microenvironment reconstituted with decellularized heart extracellular matrix can enhance chemical reprogramming and cardiac maturation of fibroblasts to cardiomyocytes. The resultant CiCMs exhibit elevated cardiac marker expression, sarcomeric organization, and improved electrophysiological features and drug responses. We investigated the therapeutic potential of CiCMs reprogrammed in three-dimensional heart extracellular matrix in a rat model of myocardial infarction. Our platform can facilitate the use of CiCMs for regenerative medicine, disease modeling, and drug screening.11Nsciescopu