Reparameterized Policy Learning for Multimodal Trajectory Optimization

We investigate the challenge of parametrizing policies for reinforcement learning (RL) in high-dimensional continuous action spaces. Our objective is to develop a multimodal policy that overcomes limitations inherent in the commonly-used Gaussian parameterization. To achieve this, we propose a principled framework that models the continuous RL policy as a generative model of optimal trajectories. By conditioning the policy on a latent variable, we derive a novel variational bound as the optimization objective, which promotes exploration of the environment. We then present a practical model-based RL method, called Reparameterized Policy Gradient (RPG), which leverages the multimodal policy parameterization and learned world model to achieve strong exploration capabilities and high data efficiency. Empirical results demonstrate that our method can help agents evade local optima in tasks with dense rewards and solve challenging sparse-reward environments by incorporating an object-centric intrinsic reward. Our method consistently outperforms previous approaches across a range of tasks. Code and supplementary materials are available on the project page https://haosulab.github.io/RPG

Deductive Verification of Chain-of-Thought Reasoning

Large Language Models (LLMs) significantly benefit from Chain-of-Thought (CoT) prompting in performing various reasoning tasks. While CoT allows models to produce more comprehensive reasoning processes, its emphasis on intermediate reasoning steps can inadvertently introduce hallucinations and accumulated errors, thereby limiting models' ability to solve complex reasoning tasks. Inspired by how humans engage in careful and meticulous deductive logical reasoning processes to solve tasks, we seek to enable language models to perform explicit and rigorous deductive reasoning, and also ensure the trustworthiness of their reasoning process through self-verification. However, directly verifying the validity of an entire deductive reasoning process is challenging, even with advanced models like ChatGPT. In light of this, we propose to decompose a reasoning verification process into a series of step-by-step subprocesses, each only receiving their necessary context and premises. To facilitate this procedure, we propose Natural Program, a natural language-based deductive reasoning format. Our approach enables models to generate precise reasoning steps where subsequent steps are more rigorously grounded on prior steps. It also empowers language models to carry out reasoning self-verification in a step-by-step manner. By integrating this verification process into each deductive reasoning stage, we significantly enhance the rigor and trustfulness of generated reasoning steps. Along this process, we also improve the answer correctness on complex reasoning tasks. Code will be released at https://github.com/lz1oceani/verify_cot

Comprehensive genomic analysis of Oesophageal Squamous Cell Carcinoma reveals clinical relevance

Abstract Oesophageal carcinoma is the fourth leading cause of cancer-related death in China, and more than 90% of these tumours are oesophageal squamous cell carcinoma (ESCC). Although several ESCC genomic sequencing studies have identified mutated somatic genes, the number of samples in each study was relatively small, and the molecular basis of ESCC has not been fully elucidated. Here, we performed an integrated analysis of 490 tumours by combining the genomic data from 7 previous ESCC projects. We identified 18 significantly mutated genes (SMGs). PTEN, DCDC1 and CUL3 were first reported as SMGs in ESCC. Notably, the AJUBA mutations and mutational signature4 were significantly correlated with a poorer survival in patients with ESCC. Hierarchical clustering analysis of the copy number alteration (CNA) of cancer gene census (CGC) genes in ESCC patients revealed three subtypes, and subtype3 exhibited more CNAs and marked for worse prognosis compared with subtype2. Moreover, database annotation suggested that two significantly differential CNA genes (PIK3CA and FBXW7) between subtype3 and subtype2 may serve as therapeutic drug targets. This study has extended our knowledge of the genetic basis of ESCC and shed some light into the clinical relevance, which would help improve the therapy and prognosis of ESCC patients

Whole exome sequencing of insulinoma reveals recurrent T372R mutations in YY1

Functional pancreatic neuroendocrine tumours (PNETs) are mainly represented by insulinoma, which secrete insulin independent of glucose and cause hypoglycaemia. The major genetic alterations in sporadic insulinomas are still unknown. Here we identify recurrent somatic T372R mutations in YY1 by whole exome sequencing of 10 sporadic insulinomas. Further screening in 103 additional insulinomas reveals this hotspot mutation in 30% (34/113) of all tumours. T372R mutation alters the expression of YY1 target genes in insulinomas. Clinically, the T372R mutation is associated with the later onset of tumours. Genotyping of YY1, a target of mTOR inhibitors, may contribute to medical treatment of insulinomas. Our findings highlight the importance of YY1 in pancreatic β-cells and may provide therapeutic targets for PNETs

Rapid Turnover of 2-LTR HIV-1 DNA during Early Stage of Highly Active Antiretroviral Therapy

BACKGROUND: Despite prolonged treatment with highly active antiretroviral therapy (HAART), the infectious HIV-1 continues to replicate and resides latently in the resting memory CD4+ T lymphocytes, which blocks the eradication of HIV-1. The viral persistence of HIV-1 is mainly caused by its proviral DNA being either linear nonintegrated, circular nonintegrated, or integrated. Previous reports have largely focused on the dynamics of HIV-1 DNA from the samples collected with relatively long time intervals during the process of disease and HAART treatment, which may have missed the intricate changes during the intervals in early treatment. METHODOLOGY/PRINCIPAL FINDINGS: In this study, we investigated the dynamics of HIV-1 DNA in patients during the early phase of HARRT treatment. Using optimized real time PCR, we observed significant changes in 2-LTR during the first 12-week of treatment, while total and integrated HIV-1 DNA remained stable. The doubling time and half-life of 2-LTR were not correlated with the baseline and the rate of changes in plasma viral load and various CD4+ T-cell populations. Longitudinal analyses on 2-LTR sequences and plasma lipopolysaccharide (LPS) levels did not reveal any significant changes in the same treatment period. CONCLUSIONS/SIGNIFICANCE: Our study revealed the rapid changes in 2-LTR concentration in a relatively large number of patients during the early HAART treatment. The rapid changes indicate the rapid infusion and clearance of cells bearing 2-LTR in the peripheral blood. Those changes are not expected to be caused by the blocking of viral integration, as our study did not include the integrase inhibitor raltegravir. Our study helps better understand the dynamics of HIV-DNA and its potential role as a biomarker for the diseases and for the treatment efficacy of HAART

Genomic Analyses Reveal Mutational Signatures and Frequently Altered Genes in Esophageal Squamous Cell Carcinoma

Esophageal squamous cell carcinoma (ESCC) is one of the most common cancers worldwide and the fourth most lethal cancer in China. However, although genomic studies have identified some mutations associated with ESCC, we know little of the mutational processes responsible. To identify genome-wide mutational signatures, we performed either whole-genome sequencing (WGS) or whole-exome sequencing (WES) on 104 ESCC individuals and combined our data with those of 88 previously reported samples. An APOBEC-mediated mutational signature in 47% of 192 tumors suggests that APOBEC-catalyzed deamination provides a source of DNA damage in ESCC. Moreover, PIK3CA hotspot mutations (c.1624G>A [p.Glu542Lys] and c.1633G>A [p.Glu545Lys]) were enriched in APOBEC-signature tumors, and no smoking-associated signature was observed in ESCC. In the samples analyzed by WGS, we identified focal (<100 kb) amplifications of CBX4 and CBX8. In our combined cohort, we identified frequent inactivating mutations in AJUBA, ZNF750, and PTCH1 and the chromatin-remodeling genes CREBBP and BAP1, in addition to known mutations. Functional analyses suggest roles for several genes (CBX4, CBX8, AJUBA, and ZNF750) in ESCC. Notably, high activity of hedgehog signaling and the PI3K pathway in approximately 60% of 104 ESCC tumors indicates that therapies targeting these pathways might be particularly promising strategies for ESCC. Collectively, our data provide comprehensive insights into the mutational signatures of ESCC and identify markers for early diagnosis and potential therapeutic targets

Combining Load and Motor Encoders to Compensate Nonlinear Disturbances for High Precision Tracking Control of Gear-Driven Gimbal

High-performance position control can be improved by the compensation of disturbances for a gear-driven control system. This paper presents a mode-free disturbance observer (DOB) based on sensor-fusion to reduce some errors related disturbances for a gear-driven gimbal. This DOB uses the rate deviation to detect disturbances for implementation of a high-gain compensator. In comparison with the angular position signal the rate deviation between load and motor can exhibits the disturbances exiting in the gear-driven gimbal quickly. Due to high bandwidth of the motor rate closed loop, the inverse model of the plant is not necessary to implement DOB. Besides, this DOB requires neither complex modeling of plant nor the use of additive sensors. Without rate sensors providing angular rate, the rate deviation is easily detected by encoders mounted on the side of motor and load, respectively. Extensive experiments are provided to demonstrate the benefits of the proposed algorithm

Integrated Transcriptome and Metabolome Analysis Reveals Phenylpropanoid Biosynthesis and Phytohormone Signaling Contribute to “<i>Candidatus</i> Liberibacter asiaticus” Accumulation in Citrus Fruit Piths (Fluffy Albedo)

“Candidatus Liberibacter asiaticus” (CLas) is a phloem-restricted α-proteobacterium that is associated with citrus huanglongbing (HLB), which is the most destructive disease that affects all varieties of citrus. Although midrib is usually used as a material for CLas detection, we recently found that the bacterium was enriched in fruits, especially in the fruit pith. However, no study has revealed the molecular basis of these two parts in responding to CLas infection. Therefore, we performed transcriptome and UHPLC–MS-based targeted and untargeted metabolomics analyses in order to organize the essential genes and metabolites that are involved. Transcriptome and metabolome characterized 4834 differentially expressed genes (DEGs) and 383 differentially accumulated metabolites (DAMs) between the two materials, wherein 179 DEGs and 44 DAMs were affected by HLB in both of the tissues, involving the pathways of phenylpropanoid biosynthesis, phytohormone signaling transduction, starch and sucrose metabolism, and photosynthesis. Notably, we discovered that the gene expression that is related to beta-glucosidase and endoglucanase was up-regulated in fruits. In addition, defense-related gene expression and metabolite accumulation were significantly down-regulated in infected fruits. Taken together, the decreased amount of jasmonic acid, coupled with the reduced accumulation of phenylpropanoid and the increased proliferation of indole-3-acetic acid, salicylic acid, and abscisic acid, compared to leaf midribs, may contribute largely to the enrichment of CLas in fruit piths, resulting in disorders of photosynthesis and starch and sucrose metabolism

Table_2_Gout and risk of dementia, Alzheimer's disease or vascular dementia: a meta-epidemiology study.DOC

ObjectivesThe association between gout and dementia, Alzheimer's disease (AD), or vascular dementia (VD) is not fully understood. The aim of this meta-analysis was to evaluate the risk of all-cause dementia, AD, and VD in gout patients with or without medication.MethodsData sources were PubMed, Embase, the Cochrane Library, and reference lists of included studies. This meta-analysis included cohort studies assessing whether the risk of all-cause dementia, AD, and VD was associated with gout. The risk of bias was assessed using the Newcastle-Ottawa Quality Assessment Scale (NOS). The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system was used to access the overall certainty of evidence. Risk ratios (RR) with 95% confidence intervals (CI) were pooled using a random-effects model, and publication bias was assessed with funnel plots and Egger's test.ResultsA total of six cohort studies involving 2,349,605 individuals were included in this meta-analysis, which were published between 2015 and 2022. The pooling analysis shows that the risk of all-cause dementia was decreased in gout patients [RR = 0.67, 95% CI (0.51, 0.89), I2 = 99%, P = 0.005, very low quality], especially in gout patients with medication [RR = 0.50, 95% CI (0.31, 0.79), I2 = 93%, P = 0.003, low quality]. The risk of AD [RR = 0.70, 95% CI (0.63, 0.79), I2 = 57.2%, P = 0.000, very low quality] and VD [RR = 0.68, 95% CI (0.49, 0.95), I2 = 91.2%, P = 0.025, very low quality] was also decreased in gout patients. Despite the large heterogeneity, the sensitivity analysis indicated that the results were robust, and there was little evidence of publication bias.ConclusionThe risk of all-cause dementia, AD, and VD is decreased in gout patients, but the quality of evidence is generally low. More studies are still needed to validate and explore the mechanisms of this association.Systematic review registrationhttps://www.crd.york.ac.uk/prospero/#recordDetails, identifier: CRD42022353312.</p