Multi-Agent Deep Reinforcement Learning for Walkers

This project was motivated by seeking an AI method towards Artificial General Intelligence (AGI), that is, more similar to learning behavior of human-beings. As of today, Deep Reinforcement Learning (DRL) is the most closer to the AGI compared to other machine learning methods. To better understand the DRL, we compares and contrasts to other related methods: Deep Learning, Dynamic Programming and Game Theory. We apply one of state-of-art DRL algorithms, called Proximal Policy Op- timization (PPO) to the robot walkers locomotion, as a simple yet challenging environment, inherently continuous and high-dimensional state/action space. The end goal of this project is to train the agent by finding the optimal sequential actions (policy/strategy) of multi-walkers leading them to move forward as far as possible to maximize the accumulated reward (performance). This goal can be accomplished by finding the tuned hyperparameters of the PPO algorithm by monitoring the performances for the multi-agent DRL (MADRL) settings. At the end, we can draw three conclusions from our findings based on the various MADRL experiments: 1) Unlike DL with explicit target labels, DRL needs larger minibatch size for better estimate of values from various gradients. There- fore, a minibatch size and its pool size (experience replay buffer) are critical hyperparameters in PPO algorithm. 2) For the homogeneous multi-agent envi- ronments, there is a mutual transferability between single-agent and multi-agent environments to be able to reuse the tuned hyperparameters. 3) For the homo- geneous multi-agent environments with a well tuned hyperparameter set, the parameter sharing is a better strategy for the MADRL in terms of performance and efficiency with reduced parameters and less memory. To conclude, reward-driven, sequential and evaluative learning, the DRL, would be closer to AGI if multiple DRL agents learn to collaborate to capture the true signal from the shared environment. This work provides one instance of implicit cooperative learning of MADRL

Structure of the chlorobenzene–argon dimer: Microwave spectrum and ab initio analysis

The rotational spectra of the 35Cl35Cl and 37Cl37Cl isotopes of the chlorobenzene–argon van der Waals dimer have been assigned using Fourier transform microwave spectroscopy techniques. Rotational constants and chlorine nuclear quadrupole coupling constants were determined which confirm that the complex has CsCs symmetry. The argon is over the aromatic ring, shifted from a position above the geometrical ring center towards the substituted carbon atom, and at a distance of about 3.68 Å from it. This distance is 0.1–0.2 Å shorter than the similar distance in the benzene–argon and fluorobenzene–argon complexes. Experimental results are confirmed and explained with the help of second-order Møller–Plesset perturbation calculations using a VDZP+diffVDZP+diff basis set. The complex binding energy of the chlorobenzene–argon complex is 1.28 kcal/mol (fluorobenzene–argon, 1.17; benzene–argon, 1.12 kcal/mol) reflecting an increase in stability caused by larger dispersion interactions when replacing one benzene H atom by F or by Cl. The structure and stability of Ar⋅C6H5–XAr⋅C6H5–X complexes are explained in terms of a balance between stabilizing dispersion and destabilizing exchange repulsion interactions between the monomers. © 2000 American Institute of Physics.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/70251/2/JCPSA6-113-20-9051-1.pd

Spatiotemporally controlled drug delivery via photothermally driven conformational change of self-integrated plasmonic hybrid nanogels

Background Spatiotemporal regulation is one of the major considerations for developing a controlled and targeted drug delivery system to treat diseases efficiently. Light-responsive plasmonic nanostructures take advantage due to their tunable optical and photothermal properties by changing size, shape, and spatial arrangement. Results In this study, self-integrated plasmonic hybrid nanogels (PHNs) are developed for spatiotemporally controllable drug delivery through light-driven conformational change and photothermally-boosted endosomal escape. PHNs are easily synthesized through the simultaneous integration of gold nanoparticles (GNPs), thermo-responsive poly (N-isopropyl acrylamide), and linker molecules during polymerization. Wave-optic simulations reveal that the size of the PHNs and the density of the integrated GNPs are crucial factors in modulating photothermal conversion. Several linkers with varying molecular weights are inserted for the optimal PHNs, and the alginate-linked PHN (A-PHN) achieves more than twofold enhanced heat conversion compared with others. Since light-mediated conformational changes occur transiently, drug delivery is achieved in a spatiotemporally controlled manner. Furthermore, light-induced heat generation from cellular internalized A-PHNs enables pinpoint cytosolic delivery through the endosomal rupture. Finally, the deeper penetration for the enhanced delivery efficiency by A-PHNs is validated using multicellular spheroid. Conclusion This study offers a strategy for synthesizing light-responsive nanocarriers and an in-depth understanding of light-modulated site-specific drug delivery.This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (Ministry of Science, ICT and Future Planning, No. 2023R1A2C2003947) for S.L., D.K., H.K., and I.C. This work was supported by the Samsung Research Funding and Incubation Center for Future Technology (SRFC-IT1802-03) for S.L., J.Y., H.K., and I.C. This work was supported by Basic Science Research Program through the NRF funded by the MSIT (No. 2021R1F1A1062856) and was also supported by Regional Innovation Strategy (RIS) through the NRF funded by the Ministry of Education (MOE) (2021RIS-004) for S.K. and J.S

Significant THz absorption in CH3NH2 molecular defect-incorporated organic-inorganic hybrid perovskite thin film

The valid strong THz absorption at 1.58 THz was probed in the organic-inorganic hybrid perovskite thin film, CH3NH3PbI3, fabricated by sequential vacuum evaporation method. In usual solution-based methods such as 2-step solution and antisolvent, we observed the relatively weak two main absorption peaks at 0.95 and 1.87 THz. The measured absorption spectrum is analyzed by density-functional theory calculations. The modes at 0.95 and 1.87 THz are assigned to the Pb-I vibrations of the inorganic components in the tetragonal phase. By contrast, the origin of the 1.58 THz absorption is due to the structural deformation of Pb-I bonding at the grain boundary incorporated with a CH3NH2 molecular defect

Acetylation changes tau interactome to degrade tau in Alzheimer’s disease animal and organoid models

© 2019 The Authors. Aging Cell published by the Anatomical Society and John Wiley & Sons Ltd.Alzheimer's disease (AD) is an age-related neurodegenerative disease. The most common pathological hallmarks are amyloid plaques and neurofibrillary tangles in the brain. In the brains of patients with AD, pathological tau is abnormally accumulated causing neuronal loss, synaptic dysfunction, and cognitive decline. We found a histone deacetylase 6 (HDAC6) inhibitor, CKD-504, changed the tau interactome dramatically to degrade pathological tau not only in AD animal model (ADLPAPT) brains containing both amyloid plaques and neurofibrillary tangles but also in AD patient-derived brain organoids. Acetylated tau recruited chaperone proteins such as Hsp40, Hsp70, and Hsp110, and this complex bound to novel tau E3 ligases including UBE2O and RNF14. This complex degraded pathological tau through proteasomal pathway. We also identified the responsible acetylation sites on tau. These dramatic tau-interactome changes may result in tau degradation, leading to the recovery of synaptic pathology and cognitive decline in the ADLPAPT mice11Nsciescopu

A logical network-based drug-screening platform for Alzheimer’s disease representing pathological features of human brain organoids

Developing effective drugs for Alzheimer’s disease (AD), the most common cause of dementia, has been difficult because of complicated pathogenesis. Here, we report an efficient, network-based drug-screening platform developed by integrating mathematical modeling and the pathological features of AD with human iPSC-derived cerebral organoids (iCOs), including CRISPR-Cas9-edited isogenic lines. We use 1300 organoids from 11 participants to build a high-content screening (HCS) system and test blood–brain barrier-permeable FDA-approved drugs. Our study provides a strategy for precision medicine through the convergence of mathematical modeling and a miniature pathological brain model using iCOs. © 2021, The Author(s).1

Chemically treated plasma Aβ is a potential blood-based biomarker for screening cerebral amyloid deposition

Background Plasma β-amyloid (Aβ) is a potential candidate for an Alzheimers disease (AD) biomarker because blood is an easily accessible bio-fluid, which can be collected routinely, and Aβ is one of the major hallmarks of AD pathogenesis in the brain. However, the association between plasma Aβ levels and AD diagnosis is still unclear due to the instability and inaccurate measurements of plasma Aβ levels in the blood of patients with AD. If a consistent value of plasma Aβ from the blood can be obtained, this might help determine whether plasma Aβ is a potential biomarker for AD diagnosis. Methods We predicted the brain amyloid deposit by measuring the plasma Aβ levels. This cross-sectional study included 353 participants (215 cognitively normal, 79 with mild cognitive impairment, and 59 with AD dementia) who underwent Pittsburgh-compound B positron emission tomography (PiB-PET) scans. We treated a mixture of protease inhibitors and phosphatase inhibitors (MPP) and detected plasma Aβ42 and Aβ40 (MPP-Aβ42 and MPP-Aβ40) in a stable manner using xMAP technology. Results MPP-Aβ40 and MPP-Aβ42/40 (MPP-Aβs) were significantly different between subjects with positive amyloid deposition (PiB+) and those with negative amyloid deposition (PiB–) (P < 0.0001). Furthermore, MPP-Aβ40 (P < 0.0001, r = 0.23) and MPP-Aβ42/40 ratio (P < 0.0001, r = –0.23) showed significant correlation with global PiB deposition (standardized uptake value ratio). In addition, our integrated multivariable (MPP-Aβ42/40, gender, age, and apolipoprotein E genotypes) logistic regression model proposes a new standard for the prediction of cerebral amyloid deposition. Conclusions MPP-Aβ might be one of the potential blood biomarkers for the prediction of PiB-PET positivity in the brain

Genetic associations of in vivo pathology influence Alzheimers disease susceptibility

Introduction Although the heritability of sporadic Alzheimers disease (AD) is estimated to be 60–80%, addressing the genetic contribution to AD risk still remains elusive. More specifically, it remains unclear whether genetic variants are able to affect neurodegenerative brain features that can be addressed by in vivo imaging techniques. Methods Targeted sequencing analysis of the coding and UTR regions of 132 AD susceptibility genes was performed. Neuroimaging data using 11C-Pittsburgh Compound B positron emission tomography (PET), 18F-fluorodeoxyglucose PET, and MRI that are available from the KBASE (Korean Brain Aging Study for Early Diagnosis and Prediction of Alzheimers disease) cohort were acquired. A total of 557 participants consisted of 336 cognitively normal (CN) adults, 137 mild cognitive impairment (MCI), and 84 AD dementia (ADD) groups. Results We called 5391 high-quality single nucleotide variants (SNVs) on AD susceptibility genes and selected significant associations between variants and five in vivo AD pathologies: (1) amyloid β (Aβ) deposition, (2) AD-signature region cerebral glucose metabolism (AD-Cm), (3) posterior cingulate cortex (PCC) cerebral glucose metabolism (PCC-Cm), (4) AD-signature region cortical thickness (AD-Ct), and (5) hippocampal volume (Hv). The association analysis for common variants (allele frequency (AF) > 0.05) yielded several novel loci associated with Aβ deposition (PIWIL1-rs10848087), AD-Cm (NME8-rs2722372 and PSEN2-rs75733498), AD-Ct (PSEN1-rs7523) and, Hv (CASS4-rs3746625). Meanwhile, in a gene-based analysis for rare variants (AF < 0.05), cases carrying rare variants in LPL, FERMT2, NFAT5, DSG2, and ITPR1 displayed associations with the neuroimaging features. Exploratory voxel-based brain morphometry between the variant carriers and non-carriers was performed subsequently. Finally, we document a strong association of previously reported APOE variants with the in vivo AD pathologies and demonstrate that the variants exert a causal effect on AD susceptibility via neuroimaging features. Conclusions This study provides novel associations of genetic factors to Aβ accumulation and AD-related neurodegeneration to influence AD susceptibility.The study was supported by grants from the National Research Foundation of Korea (2014M3C7A1046049 and 2018M3C9A5064708 for Choi M and 2014M3C7A1046042 for Lee DY) and grants from the Ministry of Health and Welfare of Korea (HI18C0630 for Mook-Jung IH and Lee DY, and HI19C0149 for Lee DY)

Induction of Neuronal Death by Microglial AGE-Albumin: Implications for Alzheimer’s Disease

Advanced glycation end products (AGEs) have long been considered as potent molecules promoting neuronal cell death and contributing to neurodegenerative disorders such as Alzheimer’s disease (AD). In this study, we demonstrate that AGE-albumin, the most abundant AGE product in human AD brains, is synthesized in activated microglial cells and secreted into the extracellular space. The rate of AGE-albumin synthesis in human microglial cells is markedly increased by amyloid-β exposure and oxidative stress. Exogenous AGE-albumin upregulates the receptor protein for AGE (RAGE) and augments calcium influx, leading to apoptosis of human primary neurons. In animal experiments, soluble RAGE (sRAGE), pyridoxamine or ALT-711 prevented Aβ-induced neuronal death in rat brains. Collectively, these results provide evidence for a new mechanism by which microglial cells promote death of neuronal cells through synthesis and secretion of AGE-albumin, thereby likely contributing to neurodegenerative diseases such as AD