Guided Cooperation in Hierarchical Reinforcement Learning via Model-based Rollout

Goal-conditioned hierarchical reinforcement learning (HRL) presents a promising approach for enabling effective exploration in complex long-horizon reinforcement learning (RL) tasks via temporal abstraction. Yet, most goal-conditioned HRL algorithms focused on the subgoal discovery, regardless of inter-level coupling. In essence, for hierarchical systems, the increased inter-level communication and coordination can induce more stable and robust policy improvement. Here, we present a goal-conditioned HRL framework with Guided Cooperation via Model-based Rollout (GCMR), which estimates forward dynamics to promote inter-level cooperation. The GCMR alleviates the state-transition error within off-policy correction through a model-based rollout, further improving the sample efficiency. Meanwhile, to avoid being disrupted by these corrected but possibly unseen or faraway goals, lower-level Q-function gradients are constrained using a gradient penalty with a model-inferred upper bound, leading to a more stable behavioral policy. Besides, we propose a one-step rollout-based planning to further facilitate inter-level cooperation, where the higher-level Q-function is used to guide the lower-level policy by estimating the value of future states so that global task information is transmitted downwards to avoid local pitfalls. Experimental results demonstrate that incorporating the proposed GCMR framework with ACLG, a disentangled variant of HIGL, yields more stable and robust policy improvement than baselines and substantially outperforms previous state-of-the-art (SOTA) HRL algorithms in both hard-exploration problems and robotic control

Sorting with Robots: where to drop off the parcel?

This paper presents a method for assigning destinations to drop off points in robotic sorting systems, taking into account robot congestion

Multiscale Superpixel Structured Difference Graph Convolutional Network for VL Representation

Within the multimodal field, the key to integrating vision and language lies in establishing a good alignment strategy. Recently, benefiting from the success of self-supervised learning, significant progress has been made in multimodal semantic representation based on pre-trained models for vision and language. However, there is still room for improvement in visual semantic representation. The lack of spatial semantic coherence and vulnerability to noise makes it challenging for current pixel or patch-based methods to accurately extract complex scene boundaries. To this end, this paper develops superpixel as a comprehensive compact representation of learnable image data, which effectively reduces the number of visual primitives for subsequent processing by clustering perceptually similar pixels. To mine more precise topological relations, we propose a Multiscale Difference Graph Convolutional Network (MDGCN). It parses the entire image as a fine-to-coarse hierarchical structure of constituent visual patterns, and captures multiscale features by progressively merging adjacent superpixels as graph nodes. Moreover, we predict the differences between adjacent nodes through the graph structure, facilitating key information aggregation of graph nodes to reason actual semantic relations. Afterward, we design a multi-level fusion rule in a bottom-up manner to avoid understanding deviation by learning complementary spatial information at different regional scales. Our proposed method can be well applied to multiple downstream task learning. Extensive experiments demonstrate that our method is competitive with other state-of-the-art methods in visual reasoning. Our code will be released upon publication

In situ stress distribution law of fault zone in tunnel site area based on the inversion method with optimized fitting conditions

Tunnel construction in high geo-stress strata faces the risk of extreme natural disasters such as large squeezing deformation and rockburst. Therefore, it is of great significance to adopt a high-precision inversion method to investigate the distribution law of in situ stress in the tunnel site area. In this paper, the in situ stress inversion research was carried out based on a plateau tunnel with a buried depth of more than 1000 m. The idea of improving the inversion accuracy by unifying displacement constraints was proposed by aiming at the defects of the traditional method on the boundary conditions. Furthermore, the impact of the constant term in the regression model on the fitting accuracy was discussed. According to the inversion method with optimized fitting conditions, the in situ stress distribution characteristics in the tunnel site area were obtained, and the variation law of the in situ stress near the fault zone was discussed. The results showed that after unifying displacement constraints, the comprehensive inversion accuracy comprehensive indicator reflecting the inversion accuracy decreased from 15.291 to 12.895, indicating that the inversion error was effectively controlled. Whether the constant term should be retained had a random effect on the inversion accuracy, so it was recommended that this issue be independently verified when fitting the data. When approaching the inner side of the fault from the outer side, the in situ stress first increased slightly and then decreased significantly. Moreover, the wider the fault impact zone and the farther the fault distribution distance, the more significant the amplitude of stress change, e.g., the maximum amplitude of stress change reached 9.0 MPa. In addition, the in situ stress orientation near the fault can be significantly deflected. And the wider the fault impact zone, the more pronounced the deflection

Utterance Augmentation for Speaker Recognition

The speaker recognition problem is to automatically recognize a person from their voice. The training of a speaker recognition model typically requires a very large training corpus, e.g., multiple voice samples from a very large number of individuals. In the diverse domains of application of speaker recognition, it is often impractical to obtain a training corpus of the requisite size. This disclosure describes techniques that augment utterances, e.g., by cutting, splitting, shuffling, etc., such that the need for collections of raw voice samples from individuals is substantially reduced. In effect, the original model works better on the augmented utterances on the target domain

Evaluation of the efficacy and safety of intravenous remdesivir in adult patients with severe COVID-19: study protocol for a phase 3 randomized, double-blind, placebo-controlled, multicentre trial.

BACKGROUND: Coronavirus disease 2019 (COVID-19), caused by a novel corinavirus (later named SARS-CoV-2 virus), was fistly reported in Wuhan, Hubei Province, China towards the end of 2019. Large-scale spread within China and internationally led the World Health Organization to declare a Public Health Emergency of International Concern on 30th January 2020. The clinical manifestations of COVID-19 virus infection include asymptomatic infection, mild upper respiratory symptoms, severe viral pneumonia with respiratory failure, and even death. There are no antivirals of proven clinical efficacy in coronavirus infections. Remdesivir (GS-5734), a nucleoside analogue, has inhibitory effects on animal and human highly pathogenic coronaviruses, including MERS-CoV and SARS-CoV, in in vitro and in vivo experiments. It is also inhibitory against the COVID-19 virus in vitro. The aim of this study is to assess the efficacy and safety of remdesivir in adult patients with severe COVID-19. METHODS: The protocol is prepared in accordance with the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidelines. This is a phase 3, randomized, double-blind, placebo-controlled, multicentre trial. Adults (≥ 18 years) with laboratory-confirmed COVID-19 virus infection, severe pneumonia signs or symptoms, and radiologically confirmed severe pneumonia are randomly assigned in a 2:1 ratio to intravenously administered remdesivir or placebo for 10 days. The primary endpoint is time to clinical improvement (censored at day 28), defined as the time (in days) from randomization of study treatment (remdesivir or placebo) until a decline of two categories on a six-category ordinal scale of clinical status (1 = discharged; 6 = death) or live discharge from hospital. One interim analysis for efficacy and futility will be conducted once half of the total number of events required has been observed. DISCUSSION: This is the first randomized, placebo-controlled trial in COVID-19. Enrolment began in sites in Wuhan, Hubei Province, China on 6th February 2020. TRIAL REGISTRATION: ClinicalTrials.gov: NCT04257656. Registered on 6 February 2020

Prospective evaluation of a rapid clinical metagenomics test for bacterial pneumonia

Background: The diagnosis of bacterial pathogens in lower respiratory tract infections (LRI) using conventional culture methods remains challenging and time-consuming.  Objectives: To evaluate the clinical performance of a rapid nanopore-sequencing based metagenomics test for diagnosis of bacterial pathogens in common LRIs through a large-scale prospective study.  Methods: We enrolled 292 hospitalized patients suspected to have LRIs between November 2018 and June 2019 in a single-center, prospective cohort study. Rapid clinical metagenomics test was performed on-site, and the results were compared with those of routine microbiology tests.  Results: 171 bronchoalveolar lavage fluid (BAL) and 121 sputum samples were collected from patients with six kinds of LRIs. The turnaround time (from sample registration to result) for the rapid metagenomics test was 6.4 ± 1.4 hours, compared to 94.8 ± 34.9 hours for routine culture. Compared with culture and real-time PCR validation tests, rapid metagenomics achieved 96.6% sensitivity and 88.0% specificity and identified pathogens in 63 out of 161 (39.1%) culture-negative samples. Correlation between enriched anaerobes and lung abscess was observed by Gene Set Enrichment Analysis. Moreover, 38 anaerobic species failed in culture was identified by metagenomics sequencing. The hypothetical impact of metagenomics test proposed antibiotic de-escalation in 34 patients compared to 1 using routine culture.  Conclusions: Rapid clinical metagenomics test improved pathogen detection yield in the diagnosis of LRI. Empirical antimicrobial therapy could be de-escalated if rapid metagenomics test results were hypothetically applied to clinical management

Bronchogenic Carcinoma after Lung Transplantation: A Case Report and Literature Review

Background and objective Lung transplantation is an efficient therapeutic option for patients with end-stage pulmonary diseases, but less is known about lung cancer after lung transplantation. The aim of this study is to improve the awareness, diagnosis and treatment of bronchogenic carcinoma after lung transplantation with a case report and related literatures. Methods We reported a 65-year-old male with idiopathic pulmonary fibrosis (IPF) who underwent right lung transplantation under extracorporeal membrane oxygenation (ECMO) support in May 2007 in our hospital. The patient recovered smoothly and discharged from the hospital 46 days after the procedure with regular follow-up. Immunosuppression therapy was triple drug maintenance regimen including tacrolimus (Tac), mycophenolate mofetil (MMF) and steroids. Results Small cell lung cancer in the left lung with multiple osseous metastases was found 13 months after the lung transplantation. Symptoms were relieved a bit by administering chemotherapeutics (etoposide and cisplatin) for 4 cycles. However, the patient was succumbed to his illness within 11 months after the diagnosis of lung cancer. Conclusion Lung cancer after lung transplantation has been suggested as one of causes of late mortality with the risk factors such as chronic obstructive pulmonary disease (COPD), IPF, cigarette smoking history and immunosuppression etc. Early diagnosis and treatment are very important to improve the prognosis

Lung Transplantation for Lung Carcinoma: A Case Report and Literature Review

Background and objective Lung carcinoma is a relative contraindication for lung transplantation (LTx). The improvement of this procedure may successfully treat novel end-stage pulmonary diseases. In the present study, we reported one case of bilateral LTx (BLT) in a young man with bilateral lung carcinoma and reviewed related literature to investigate patient selection and curative effect. Methods A 42-year-old male patient underwent BLT on October 21, 2010 in our hospital. Preoperative chest CT and PET-CT showed bilateral multiple nodules and mass without positive lymph nodes or extrapulmonary metastasis. Bronchioloalveolar carcinoma was identified histologically (T4N0M0, stage IIIb). Results Routine therapies were performed postoperatively. The patient was discharged on post-operative day 66. There was no clinical or radiologic evidence of the recurrence of lung carcinoma in his latest follow-up of 6 months postoperatively. Conclusion LTx may be proposed as an efficient therapeutic option in selected patients of lung carcinoma