Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection

Human-Object Interaction (HOI) detection is a challenging computer vision task that requires visual models to address the complex interactive relationship between humans and objects and predict HOI triplets. Despite the challenges posed by the numerous interaction combinations, they also offer opportunities for multimodal learning of visual texts. In this paper, we present a systematic and unified framework (RmLR) that enhances HOI detection by incorporating structured text knowledge. Firstly, we qualitatively and quantitatively analyze the loss of interaction information in the two-stage HOI detector and propose a re-mining strategy to generate more comprehensive visual representation.Secondly, we design more fine-grained sentence- and word-level alignment and knowledge transfer strategies to effectively address the many-to-many matching problem between multiple interactions and multiple texts.These strategies alleviate the matching confusion problem that arises when multiple interactions occur simultaneously, thereby improving the effectiveness of the alignment process. Finally, HOI reasoning by visual features augmented with textual knowledge substantially improves the understanding of interactions. Experimental results illustrate the effectiveness of our approach, where state-of-the-art performance is achieved on public benchmarks. We further analyze the effects of different components of our approach to provide insights into its efficacy.Comment: ICCV202

The Generalization Gap in Offline Reinforcement Learning

Despite recent progress in offline learning, these methods are still trained and tested on the same environment. In this paper, we compare the generalization abilities of widely used online and offline learning methods such as online reinforcement learning (RL), offline RL, sequence modeling, and behavioral cloning. Our experiments show that offline learning algorithms perform worse on new environments than online learning ones. We also introduce the first benchmark for evaluating generalization in offline learning, collecting datasets of varying sizes and skill-levels from Procgen (2D video games) and WebShop (e-commerce websites). The datasets contain trajectories for a limited number of game levels or natural language instructions and at test time, the agent has to generalize to new levels or instructions. Our experiments reveal that existing offline learning algorithms struggle to match the performance of online RL on both train and test environments. Behavioral cloning is a strong baseline, outperforming state-of-the-art offline RL and sequence modeling approaches when trained on data from multiple environments and tested on new ones. Finally, we find that increasing the diversity of the data, rather than its size, improves performance on new environments for all offline learning algorithms. Our study demonstrates the limited generalization of current offline learning algorithms highlighting the need for more research in this area.Comment: Published as a conference paper at ICLR 2024; First two authors contributed equall

The Investigation of Plume-Regolith Interaction and Dust Dispersal during Chang’E-5 Descent Stage

The plume-surface interaction that occurs as a result of a variable-thrust engine exhaust plume impinging on soil during landings is critical for future lunar mission design. Unique lunar environmental properties, such as low gravity, high vacuum, and the regolith layer, make this study complex and challenging. In this paper, we build a reliable simulation model, with constraints based on landing photos, to characterize the erosion properties induced by a low-thrust engine plume. We focus on the low-thrust plume-surface erosion process and erosion properties during the Chang’E-5 mission, aiming to determine the erosion difference between high- and low-thrust conditions; this is a major concern, as the erosion process for a low-thrust lunar mission is rarely studied. First, to identify the entire erosion process and its relative effect on the flat lunar surface, a one-to-one rocket nozzle simulation model is built; ground experimental results are utilized to verify the simulated inlet parameters of the vacuum plume flow field. Following that, plume flow is considered using the finite volume method, and the Roberts erosion model, based on excess shear stress, is adopted to describe plume-surface interaction properties. Finally, a Lagrangian framework using the discrete phase model is selected to investigate the dynamic properties of lunar dust particles. Results show that erosion depth, total ejected mass, and the maximum particle incline angle during the Chang’E-5 landing period are approximately 0.2 cm, 335.95 kg, and 4.16°, respectively. These results are not only useful for the Chang’E-5 lunar sample analysis, but also for future lunar mission design

Are medical record front page data suitable for risk adjustment in hospital performance measurement? Development and validation of a risk model of in-hospital mortality after acute myocardial infarction

Objectives To develop a model of in-hospital mortality using medical record front page (MRFP) data and assess its validity in case-mix standardisation by comparison with a model developed using the complete medical record data.Design A nationally representative retrospective study.Setting Representative hospitals in China, covering 161 hospitals in modelling cohort and 156 hospitals in validation cohort.Participants Representative patients admitted for acute myocardial infarction. 8370 patients in modelling cohort and 9704 patients in validation cohort.Primary outcome measures In-hospital mortality, which was defined explicitly as death that occurred during hospitalisation, and the hospital-level risk standardised mortality rate (RSMR).Results A total of 14 variables were included in the model predicting in-hospital mortality based on MRFP data, with the area under receiver operating characteristic curve of 0.78 among modelling cohort and 0.79 among validation cohort. The median of absolute difference between the hospital RSMR predicted by hierarchical generalised linear models established based on MRFP data and complete medical record data, which was built as ‘reference model’, was 0.08% (10th and 90th percentiles: −1.8% and 1.6%). In the regression model comparing the RSMR between two models, the slope and intercept of the regression equation is 0.90 and 0.007 in modelling cohort, while 0.85 and 0.010 in validation cohort, which indicated that the evaluation capability from two models were very similar.Conclusions The models based on MRFP data showed good discrimination and calibration capability, as well as similar risk prediction effect in comparison with the model based on complete medical record data, which proved that MRFP data could be suitable for risk adjustment in hospital performance measurement