SHIRO: Soft Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) algorithms have been demonstrated to perform well on high-dimensional decision making and robotic control tasks. However, because they solely optimize for rewards, the agent tends to search the same space redundantly. This problem reduces the speed of learning and achieved reward. In this work, we present an Off-Policy HRL algorithm that maximizes entropy for efficient exploration. The algorithm learns a temporally abstracted low-level policy and is able to explore broadly through the addition of entropy to the high-level. The novelty of this work is the theoretical motivation of adding entropy to the RL objective in the HRL setting. We empirically show that the entropy can be added to both levels if the Kullback-Leibler (KL) divergence between consecutive updates of the low-level policy is sufficiently small. We performed an ablative study to analyze the effects of entropy on hierarchy, in which adding entropy to high-level emerged as the most desirable configuration. Furthermore, a higher temperature in the low-level leads to Q-value overestimation and increases the stochasticity of the environment that the high-level operates on, making learning more challenging. Our method, SHIRO, surpasses state-of-the-art performance on a range of simulated robotic control benchmark tasks and requires minimal tuning

組織再編の法理と立法 -利害関係者の保護と救済-

早大学位記番号:新7889早稲田大

Analysis of Kinang Jingkion Material as Road Subbase Layer Using Soil Cement Method

This research aims to analyze the California Bearing Ratio (CBR) and Unconfined Compressive Strength Test (UCS) values as road foundation layers using local materials, namely Kinang Jingkion. The soil and Kinang Jingkion materials for the study were taken from the Apalapsili quarry in Yalimo Regency. Additionally, the additive or soil stabilizer used was matos obtained from the testing of soil characteristics, including sieve analysis or gradation, Standard Proctor Test for light density, and liquid limit test. After testing, it was found that the soil from the Apalapsili quarry is granular soil and can be categorized as A-1-a according to SNI 03-6797-2002. Subsequent testing involved CBR and UCS testing on Mix Designs made of soil and cement with concentrations of 4%, 6%, and 8%. Next, additional Mix Designs were created with compositions of Kinang Jingkion, soil, cement at 4%, 6%, and 8%. Further Mix Designs included compositions of Kinang Jingkion, soil, matos at 2%, and cement at 4%, 6%, and 8%. From the UCS testing results, Mix Designs with soil and cement concentrations of 4% and 6% did not meet the compressive strength criteria, with compressive strength values of 9.26 Kg/Cm² and 17.4 Kg/Cm² respectively. According to SNI 6887-2012, the compressive strength value after 7 days of immersion should be within 25-35 Kg/Cm², while other Mix Designs met the criteria of SNI 6887-2012. CBR testing also utilized the same Mix Designs, and the highest CBR result was obtained from the Mix Design with compositions of Kinang Jingkion, soil, cement at 8%, and matos at 2%, with a CBR value of 112%. According to the 2018 Revised General Specifications, a CBR value of 112% indicates a Class A Foundation Layer

Trimerization and genotype-phenotype correlation of COL4A5 mutants in Alport syndrome

INTRODUCTION: Alport syndrome is a hereditary glomerulonephritis that results from the disruption of collagen α345(IV) heterotrimerization caused by mutation in METHODS: We selected 9 α5(IV) missense mutants with typical glycine substitutions that clinically differed in disease progression. To quantify the trimerization of each mutant, split nanoluciferase-fused α3/α5 mutants and α4 were transfected into the cells, and intracellular and secreted heterotrimer were detected by luminescence using an assay that we developed previously. RESULTS: Trimer formation and secretion patterns tended to be similar to the wild type in most of the mutations that did not show proteinuria at a young age. On the other hand, trimer secretion was significantly reduced in all the mutations that showed proteinuria and early onset of renal failure. One of these mutants has low ability of intracellular trimer formation, and the others had the defect of low-level secretion. In addition, the mutant that is assumed to be nonpathogenic has similar trimer formation and secretion pattern as wild-type α5(IV). CONCLUSION: The result of cell-based α345(IV) heterotrimer formation assay was largely correlated with clinical genotype-phenotype. These trimerization assessments provide additional phenotypic considerations and may help to distinguish between pathogenic and nonpathogenic mutations

iPSC-derived type IV collagen α5-expressing kidney organoids model Alport syndrome

ヒトiPS細胞から作製した腎オルガノイドを用いたアルポート症候群病態モデルの開発. 京都大学プレスリリース. 2023-09-28.iPSC-derived kidney organoids to model a lifelong renal disease. 京都大学プレスリリース. 2023-10/17.Alport syndrome (AS) is a hereditary glomerulonephritis caused by COL4A3, COL4A4 or COL4A5 gene mutations and characterized by abnormalities of glomerular basement membranes (GBMs). Due to a lack of curative treatments, the condition proceeds to end-stage renal disease even in adolescents. Hampering drug discovery is the absence of effective in vitro methods for testing the restoration of normal GBMs. Here, we aimed to develop kidney organoid models from AS patient iPSCs for this purpose. We established iPSC-derived collagen α5(IV)-expressing kidney organoids and confirmed that kidney organoids from COL4A5 mutation-corrected iPSCs restore collagen α5(IV) protein expression. Importantly, our model recapitulates the differences in collagen composition between iPSC-derived kidney organoids from mild and severe AS cases. Furthermore, we demonstrate that a chemical chaperone, 4-phenyl butyric acid, has the potential to correct GBM abnormalities in kidney organoids showing mild AS phenotypes. This iPSC-derived kidney organoid model will contribute to drug discovery for AS

Diseño e implementación de un sistema RFID para seguimiento de personas dependientes en el hogar

Este Trabajo Final de Grado se basa en el estudio, diseño, montaje y evaluación de un sistema RFID (Radio Frequency IDentification) para el posicionamiento de personas dependientes dentro de edificios, a fin de proporcionar un servicio a sus habitantes. El primer paso es entender lo que son los sistemas RFID y estudiar casos de uso existentes. Después, se estudia el funcionamiento de los lectores RFID disponibles y las etiquetas (Tags) a rastrear; seguidamente se aprende a configurarlos y se diseña e implementa un programa que pueda establecer una comunicación con los equipos y gestionar los cálculos necesarios lo suficientemente rápido como para realizar un rastreo en tiempo real. Una vez conseguido esto, se busca el método de cálculo de distancias y de coordenadas lo más exacto posible y se corrigen los errores alcanzando la precisión necesaria. Para ello, se realizan continuamente pruebas de rastreo y posicionamiento tanto estáticos como en movimiento.In this final project a RFID (Radio Frequency IDentification) system is studied, designed, implemented and evaluated. The system aims to track and position dependent people in buildings, in order to provide different services to its inhabitants. Firstly, it is needed to understand what RFID systems are, and study real-life use cases. Later, the operation and configuration of the available RFID readers and tags is studied. Once this is accomplished, a computer program is designed and implemented. This computer program must communicate with the RFID equipment and perform the calculations fast enough so that it is possible to track items in real time. Afterwards, errors are corrected and the best formulas for calculating distances and coordinates are found, in order to facilitate the required results and precision. Throughout the process, the system is continuously tested with static and moving tracking and positioning.Catalá Adrama, L. (2015). Diseño e implementación de un sistema RFID para seguimiento de personas dependientes en el hogar. Universitat Politècnica de València. http://hdl.handle.net/10251/55170TFG

Timing of therapeutic interventions against infection-triggered encephalopathy syndrome: a scoping review of the pediatric literature

Our goal was to conduct a scoping review of the literature on the treatment of infection-triggered encephalopathy syndrome/acute encephalopathy in children, focusing on treatment targets and treatment initiation timing. We performed literature searches using PubMed for articles reporting treatments of infection-triggered encephalopathy syndrome/acute encephalopathy. We included articles describing specific treatments for acute encephalopathy with control groups. For the purpose of searching new therapies only experimentally tried in the case series, we also included case series studies without control groups in this review, if the studies contained at least two cases with clear treatment goals. Therapies were classified based on their mechanisms of action into brain protection therapy, immunotherapy, and other therapies. We operationally categorized the timing of treatment initiation as T1 (6–12 h), T2 (12–24 h), T3 (24–48 h), and T4 (>48 h) after the onset of seizures and/or impaired consciousness. Thirty articles were included in this review; no randomized control study was found. Eleven retrospective/historical cohort studies and five case–control studies included control groups with or without specific therapies or outcomes. The targeted conditions and treatment timing varied widely across studies. However, the following three points were suggested to be effective in multiple studies: (1) Careful seizure management and targeted temperature management within 12 h (T1) of onset of febrile seizure/prolonged impaired consciousness without multiple organ failure may reduce the development of acute encephalopathy with biphasic seizures and late reduced diffusion; (2) immunotherapy using corticosteroids, tocilizumab, or plasma exchange within 24 h (T1–T2) of onset of acute necrotizing encephalopathy may reduce sequelae; and (3) anakinra therapy and ketogenic diet demonstrate little evidence of neurologic sequelae reduction, but may reduce seizure frequency and allow for weaning from barbiturates, even when administered weeks (T4) after onset in children with febrile infection-related epilepsy syndrome. Although available studies have no solid evidence in the treatment of infection-triggered encephalopathy syndrome/acute encephalopathy, this scoping review lays the groundwork for future prospective clinical trials

Molecular Characterization of Haemaphysalis Species and a Molecular Genetic Key for the Identification of Haemaphysalis of North America

Haemaphysalis longicornis (Acari: Ixodidae), the Asian longhorned tick, is native to East Asia, but has become established in Australia and New Zealand, and more recently in the United States. In North America, there are other native Haemaphysalis species that share similar morphological characteristics and can be difficult to identify if the specimen is damaged. The goal of this study was to develop a cost-effective and rapid molecular diagnostic assay to differentiate between exotic and native Haemaphysalis species to aid in ongoing surveillance of H. longicornis within the United States and help prevent misidentification. We demonstrated that restriction fragment length polymorphisms (RFLPs) targeting the 16S ribosomal RNA and the cytochrome c oxidase subunit I (COI) can be used to differentiate H. longicornis from the other Haemaphysalis species found in North America. Furthermore, we show that this RFLP assay can be applied to Haemaphysalis species endemic to other regions of the world for the rapid identification of damaged specimens. The work presented in this study can serve as the foundation for region specific PCR-RFLP keys for Haemaphysalis and other tick species and can be further applied to other morphometrically challenging taxa