Spatial reasoning and planning for deep embodied agents

Humans can perform complex tasks with long-term objectives by planning, reasoning, and forecasting outcomes of actions. For embodied agents (e.g. robots) to achieve similar capabilities, they must gain knowledge of the environment transferable to novel scenarios with a limited budget of additional trial and error. Learning-based approaches, such as deep reinforcement learning, can discover and take advantage of inherent regularities and characteristics of the application domain from data, and continuously improve their performances, however at a cost of large amounts of training data. This thesis explores the development of data-driven techniques for spatial reasoning and planning tasks, focusing on enhancing learning efficiency, interpretability, and transferability across novel scenarios. Four key contributions are made. Firstly, CALVIN, a differential planner that learns interpretable models of the world for long-term planning. It successfully navigated partially observable 3D environments, such as mazes and indoor rooms, by learning the rewards (goals and obstacles) and state transitions (robot dynamics) from expert demonstrations. Secondly, SOAP, a reinforcement learning algorithm that discovers macro-actions (options) unsupervised for long-horizon tasks. Options segment a task into subtasks and enable consistent execution of the subtask. SOAP showed robust performances on history-conditional corridor tasks as well as classical benchmarks such as Atari. Thirdly, LangProp, a code optimisation framework using Large Language Models to solve embodied agent problems that require reasoning by treating code as learnable policies. The framework successfully generated interpretable code with comparable or superior performance to human-written experts in the CARLA autonomous driving benchmark. Finally, Voggite, an embodied agent with a vision-to-action transformer backend that solves complex tasks in Minecraft. It achieved third place in the MineRL BASALT Competition by identifying action triggers to segment tasks into multiple stages. These advancements provide new avenues for applications of learning-based methods in complex spatial reasoning and planning challenges

Bacterial nitrate assimilation: gene distribution and regulation

In the context of the global nitrogen cycle, the importance of inorganic nitrate for the nutrition and growth of marine and freshwater autotrophic phytoplankton has long been recognized. In contrast, the utilization of nitrate by heterotrophic bacteria has historically received less attention because the primary role of these organisms has classically been considered to be the decomposition and mineralization of dissolved and particulate organic nitrogen. In the pre-genome sequence era, it was known that some, but not all, heterotrophic bacteria were capable of growth on nitrate as a sole nitrogen source. However, examination of currently available prokaryotic genome sequences suggests that assimilatory nitrate reductase (Nas) systems are widespread phylogenetically in bacterial and archaeal heterotrophs. Until now, regulation of nitrate assimilation has been mainly studied in cyanobacteria. In contrast, in heterotrophic bacterial strains, the study of nitrate assimilation regulation has been limited to Rhodobacter capsulatus, Klebsiella oxytoca, Azotobacter vinelandii and Bacillus subtilis. In Gram-negative bacteria, the nas genes are subjected to dual control: ammonia repression by the general nitrogen regulatory (Ntr) system and specific nitrate or nitrite induction. The Ntr system is widely distributed in bacteria, whereas the nitrate/nitrite-specific control is variable depending on the organism

Activation of LXR Receptors and Inhibition of TRAP1 Causes Synthetic Lethality in Solid Tumors

Cholesterol is a pivotal factor for cancer cells to entertain their relentless growth. In this case, we provide a novel strategy to inhibit tumor growth by simultaneous activation of liver-X-receptors and interference with Tumor Necrosis Factor Receptor-associated Protein 1 (TRAP1). Informed by a transcriptomic and subsequent gene set enrichment analysis, we demonstrate that inhibition of TRAP1 results in suppression of the cholesterol synthesis pathway in stem-like and established glioblastoma (GBM) cells by destabilizing the transcription factor SREBP2. Notably, TRAP1 inhibition induced cell death, which was rescued by cholesterol and mevalonate. Activation of liver X receptor (LXR) by a clinically validated LXR agonist, LXR623, along with the TRAP1 inhibitor, gamitrinib (GTPP), results in synergistic reduction of tumor growth and cell death induction in a broad range of solid tumors, which is rescued by exogenous cholesterol. The LXR agonist and TRAP1 inhibitor mediated cell death is regulated at the level of Bcl-2 family proteins with an elevation of pro-apoptotic Noxa. Silencing of Noxa and its effector BAK attenuates cell death mediated by the combination treatment of LXR agonists and TRAP1 inhibition. Combined inhibition of TRAP1 and LXR agonists elicits a synergistic activation of the integrated stress response with an increase in activating transcription factor 4 (ATF4) driven by protein kinase RNA-like endoplasmic reticulum kinase (PERK). Silencing of ATF4 attenuates the increase of Noxa by using the combination treatment. Lastly, we demonstrate in patient-derived xenografts that the combination treatment of LXR623 and gamitrinib reduces tumor growth more potent than each compound. Taken together, these results suggest that TRAP1 inhibition and simultaneous activation of LXR might be a potent novel treatment strategy for solid malignancies

Weak quasistatic magnetism in the frustrated Kondo lattice Pr_2Ir_2O_7

Muon spin relaxation experiments have been performed in the pyrochlore iridate Pr_2Ir_2O_7 for temperatures in the range 0.025-250 K. Kubo-Toyabe relaxation functions are observed up to > 200 K, indicating static magnetism over this temperature range. The T -> 0 static muon spin relaxation rate Delta(0) ~ 8 mus^-1 implies a weak quasistatic moment (~0.1 mu_B). The temperature dependence of Delta is highly non-mean-field-like, decreasing smoothly by orders of magnitude but remaining nonzero below ~150 K. The data rule out ordering of the full Pr^3+ CEF ground-state moment (3.0 mu_B) down to 0.025 K. The weak static magnetism is most likely due to hyperfine-enhanced ^141Pr nuclear magnetism. The dynamic relaxation rate lambda increases markedly below ~20 K, probably due to slowing down of spin fluctuations in the spin-liquid state. At low temperatures lambda is strong and temperature-independent, indicative of a high density of low-lying spin excitations as is common in frustrated antiferromagnets.Comment: 4 pages, 3 figures, to appear in Proc. 11th Int. Conf. on Muon Spin Rotation, Relaxation and Resonance (muSR2008

Dual Inhibition of Bcl-2/Bcl-xL and XPO1 is synthetically lethal in glioblastoma model systems

XPO1 has recently emerged as a viable treatment target for solid malignancies, including glioblastoma (GBM), the most common primary malignant brain tumor in adults. However, given that tumors become commonly resistant to single treatments, the identification of combination therapies is critical. Therefore, we tested the hypothesis that inhibition of anti-apoptotic Bcl-2 family members and XPO1 are synthetically lethal. To this purpose, two clinically validated drug compounds, the BH3-mimetic, ABT263, and the XPO1 inhibitor, Selinexor, were used in preclinical GBM model systems. Our results show that inhibition of XPO1 reduces cellular viability in glioblastoma cell cultures. Moreover, addition of ABT263 significantly enhances the efficacy of XPO1 inhibition on the reduction of cellular viability, which occurs in a synergistic manner. While selinexor inhibits the proliferation of glioblastoma cells, the combination treatment of ABT263 and selinexor results in substantial induction of cell death, which is accompanied by activation of effector- initiator caspases and cleavage of PARP. Mechanistically we find that XPO1 inhibition results in down-regulation of anti-apoptotic Mcl-1 and attenuates ABT263 driven Mcl-1 up-regulation. Consistently, siRNA mediated silencing of Mcl-1 sensitizes for ABT263 mediated cell death and partially for the combination treatment. By using a human patient-derived xenograft model of glioblastoma in mice, we demonstrate that the combination treatment of ABT263 and Selinexor reduces tumor growth significantly more than each compound alone. Collectively, these results suggest that inhibition of XPO1 and Bcl-2/Bcl-xL might be a potential strategy for the treatment of malignant glial tumors

Interleukin-10 Promoter Polymorphism is Associated with the Predisposition to the Development of IgA Nephropathy and Focal Segmental Glomerulosclerosis in Korea

The roles of interleukin-10 (IL-10) have been emphasized in several models of glomerulonephritis (GN). Three biallelic polymorphisms within the IL-10 promoter region, at positions -1,082, -819, and -592 from the transcription initiation site, were shown to affect the level of IL-10 production. To investigate the effect of IL-10 promoter polymorphisms on the predisposition to development of GN in Korea, IL-10 promoter polymorphisms were assayed by polymerase chain reaction followed by restriction fragment length polymorphism in 108 patients with IgA nephropathy (IgAN), 49 focal segmental glomerulosclerosis (FSGS), and 100 healthy controls. In comparison with the control, the frequency of -1,082*G alleles were lower in IgAN and the frequencies of -592*C and -819*C were lower in FSGS, respectively. As for the haplotype, GCC haplotype was less frequent among IgAN than the control and ATA haplotype was more frequent among FSGS than the control (p<0.05). The frequency of intermediate producer genotypes (GCC/ACC and GCC/ATA) were lower among IgAN or FSGS than the control. Our findings suggested that IL-10 promoter polymorphism predisposed to the development of IgAN and FSGS in Korean patients