20 research outputs found
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Video Foundation Models (VFMs) have received limited exploration due to high
computational costs and data scarcity. Previous VFMs rely on Image Foundation
Models (IFMs), which face challenges in transferring to the video domain.
Although VideoMAE has trained a robust ViT from limited data, its low-level
reconstruction poses convergence difficulties and conflicts with high-level
cross-modal alignment. This paper proposes a training-efficient method for
temporal-sensitive VFMs that integrates the benefits of existing methods. To
increase data efficiency, we mask out most of the low-semantics video tokens,
but selectively align the unmasked tokens with IFM, which serves as the
UnMasked Teacher (UMT). By providing semantic guidance, our method enables
faster convergence and multimodal friendliness. With a progressive pre-training
framework, our model can handle various tasks including scene-related,
temporal-related, and complex video-language understanding. Using only public
sources for pre-training in 6 days on 32 A100 GPUs, our scratch-built ViT-L/16
achieves state-of-the-art performances on various video tasks. The code and
models will be released at https://github.com/OpenGVLab/unmasked_teacher.Comment: 16 pages, 5 figures, 28 table
Harvest Video Foundation Models via Efficient Post-Pretraining
Building video-language foundation models is costly and difficult due to the
redundant nature of video data and the lack of high-quality video-language
datasets. In this paper, we propose an efficient framework to harvest video
foundation models from image ones. Our method is intuitively simple by randomly
dropping input video patches and masking out input text during the
post-pretraining procedure. The patch dropping boosts the training efficiency
significantly and text masking enforces the learning of cross-modal fusion. We
conduct extensive experiments to validate the effectiveness of our method on a
wide range of video-language downstream tasks including various zero-shot
tasks, video question answering, and video-text retrieval. Despite its
simplicity, our method achieves state-of-the-art performances, which are
comparable to some heavily pretrained video foundation models. Our method is
extremely efficient and can be trained in less than one day on 8 GPUs,
requiring only WebVid-10M as pretraining data. We hope our method can serve as
a simple yet strong counterpart for prevalent video foundation models, provide
useful insights when building them, and make large pretrained models more
accessible and sustainable. This is part of the InternVideo project
\url{https://github.com/OpenGVLab/InternVideo}
Mendelian randomization supports genetic liability to hospitalization for COVID-19 as a risk factor of pre-eclampsia
BackgroundPre-eclampsia and eclampsia are among the major threats to pregnant women and fetuses, but they can be mitigated by prevention and early screening. Existing observational research presents conflicting evidence regarding the causal effects of coronavirus disease 2019 (COVID-19) on pre-eclampsia risk. Through Mendelian randomization (MR), this study aims to investigate the causal effect of three COVID-19 severity phenotypes on the risk of pre-eclampsia and eclampsia to provide more rigorous evidence.MethodsTwo-sample MR was utilized to examine causal effects. Summary-level data from genome-wide association studies (GWAS) of individuals of European ancestry were acquired from the GWAS catalog and FinnGen databases. Single-nucleotide polymorphisms associated with COVID-19 traits at p < 5 × −8 were obtained and pruned for linkage disequilibrium to generate instrumental variables for COVID-19. Inverse variance weighted estimates were used as the primary MR results, with weighted median and MR-Egger as auxiliary analyses. The robustness of the MR findings was also evaluated through sensitivity analyses. Bonferroni correction was applied to primary results, with a p < 0.0083 considered significant evidence and a p within 0.083–0.05 considered suggestive evidence.ResultsCritical ill COVID-19 [defined as hospitalization for COVID-19 with either a death outcome or respiratory support, OR (95% CI): 1.17 (1.03–1.33), p = 0.020] and hospitalized COVID-19 [defined as hospitalization for COVID-19, OR (95% CI): 1.10 (1.01–1.19), p = 0.026] demonstrated suggestive causal effects on pre-eclampsia, while general severe acute respiratory syndrome coronavirus 2 infection did not exhibit a significant causal effect on pre-eclampsia. None of the three COVID-19 severity phenotypes exhibited a significant causal effect on eclampsia.ConclusionsOur investigation demonstrates a suggestive causal effect of genetic susceptibility to critical ill COVID-19 and hospitalized COVID-19 on pre-eclampsia. The COVID-19 severity exhibited a suggestive positive dose–response relationship with the risk of pre-eclampsia. Augmented attention should be paid to pregnant women hospitalized for COVID-19, especially those needing respiratory support
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation
This paper introduces InternVid, a large-scale video-centric multimodal
dataset that enables learning powerful and transferable video-text
representations for multimodal understanding and generation. The InternVid
dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M
video clips accompanied by detailed descriptions of total 4.1B words. Our core
contribution is to develop a scalable approach to autonomously build a
high-quality video-text dataset with large language models (LLM), thereby
showcasing its efficacy in learning video-language representation at scale.
Specifically, we utilize a multi-scale approach to generate video-related
descriptions. Furthermore, we introduce ViCLIP, a video-text representation
learning model based on ViT-L. Learned on InternVid via contrastive learning,
this model demonstrates leading zero-shot action recognition and competitive
video retrieval performance. Beyond basic video understanding tasks like
recognition and retrieval, our dataset and model have broad applications. They
are particularly beneficial for generating interleaved video-text data for
learning a video-centric dialogue system, advancing video-to-text and
text-to-video generation research. These proposed resources provide a tool for
researchers and practitioners interested in multimodal video understanding and
generation.Comment: Data and Code:
https://github.com/OpenGVLab/InternVideo/tree/main/Data/InternVi
Expert Consensus on Microtransplant for Acute Myeloid Leukemia in Elderly Patients -Report From the International Microtransplant Interest Group
Recent studies have shown that microtransplant (MST) could improve outcome of patients with elderly acute myeloid leukemia (EAML). To further standardize the MST therapy and improve outcomes in EAML patients, based on analysis of the literature on MST, especially MST with EAML from January 1st, 2011 to November 30th, 2022, the International Microtransplant Interest Group provides recommendations and considerations for MST in the treatment of EAML. Four major issues related to MST for treating EAML were addressed: therapeutic principle of MST (1), candidates for MST (2), induction chemotherapy regimens (3), and post-remission therapy based on MST (4). Others included donor screening, infusion of donor cells, laboratory examinations, and complications of treatment
A Novel Address-Matching Framework Based on Region Proposal
Geocoding is a fundamental component of geographic information science that plays a crucial role in various geographical studies and applications involving text data. Current mainstream geocoding methods fall into two categories: geodesic-grid prediction and address matching. However, the geodesic-grid-prediction method’s localization accuracy is hindered by the density of grid partitioning, struggling to strike a balance between prediction accuracy and grid density. Address-matching methods mainly focus on the semantics of query text. However, they tend to ignore keyword information that can be used to distinguish candidates and introduce potential interference, which reduces matching accuracy. Inspired by the human map-usage process, we propose a two-stage address-matching approach that integrates geodesic-grid prediction and text-matching models. Initially, a multi-level text-classification model is used to generate a retrieval region proposal for an input query text. Subsequently, we search for the most relevant point of interest (POI) within the region-proposal area using a semantics-based text-retrieval model. We evaluated the proposed method using POI data from the Beijing Chaoyang District. The experimental results indicate that the proposed method provides high address-matching accuracy, increasing Recall@1 by 0.55 to 1.56 percentage points and MRR@5 by 0.54 to 1.68 percentage points
Detection of Genetic Mutations by Next-Generation Sequencing for Predicting Prognosis of Extensive-Stage Small-Cell Lung Cancer
Some studies have revealed that specific genetic mutations could be associated with chemotherapy response or even survival in small-cell lung cancer (SCLC). Our retrospective study aimed to identify the correlation between genetic mutations and progression-free survival (PFS) in extensive-stage SCLC after first-line chemotherapy. A total of 75 patients with extensive-stage SCLC confirmed by histopathology from February 2018 to February 2019 were retrospectively analyzed. The biopsy specimens of all patients were analyzed by Next-Generation Sequencing (NGS). All patients received first-line chemotherapy and follow-up at Shanghai Chest Hospital. Eleven genes were mutated in, at least, 10% of the 75 patients, including TP53 (96%), RB1 (77%), SMAD4 (32%), NOTCH1 (21%), PTEN (16%), FGFR1 (16%), KDR (15%), PIK3CA (15%), ROS1 (15%), BRCA2 (13%), and ERBB4 (10%). The median number of mutated genes among all patients was 5. Patients with more than 5 mutated genes (PFS = 6.7 months, P=0.004), mutant TP53 (PFS = 5.0 months, P=0.011), and mutant BRCA2 (PFS = 6.7 months, P=0.046) had better PFS after first-line chemotherapy than other patients. Multivariate Cox regression analysis showed that patients who achieved a PR (HR 3.729, 95% CI 2.038–6.822), had more than 5 mutated genes (HR 1.929, 95% CI 1.096–3.396), had BRCA2 mutations (HR 4.581, 95% CI 1.721–12.195), and had no liver metastasis (HR 0.415, 95% CI 0.181–0.951) showed improvements in PFS after first-line chemotherapy. In conclusion, the number of mutated genes and BRCA2 mutation status in extensive-stage SCLC were significantly related to PFS after first-line chemotherapy
VideoChat: Chat-Centric Video Understanding
In this study, we initiate an exploration into video understanding by
introducing VideoChat, an end-to-end chat-centric video understanding system.
It integrates video foundation models and large language models via a learnable
neural interface, excelling in spatiotemporal reasoning, event localization,
and causal relationship inference. To instructively tune this system, we
propose a video-centric instruction dataset, composed of thousands of videos
matched with detailed descriptions and conversations. This dataset emphasizes
spatiotemporal reasoning and causal relationships, providing a valuable asset
for training chat-centric video understanding systems. Preliminary qualitative
experiments reveal our system's potential across a broad spectrum of video
applications and set the standard for future research. Access our code and data
at https://github.com/OpenGVLab/Ask-AnythingComment: Technical repor
Table3_Mendelian randomization supports genetic liability to hospitalization for COVID-19 as a risk factor of pre-eclampsia.xlsx
BackgroundPre-eclampsia and eclampsia are among the major threats to pregnant women and fetuses, but they can be mitigated by prevention and early screening. Existing observational research presents conflicting evidence regarding the causal effects of coronavirus disease 2019 (COVID-19) on pre-eclampsia risk. Through Mendelian randomization (MR), this study aims to investigate the causal effect of three COVID-19 severity phenotypes on the risk of pre-eclampsia and eclampsia to provide more rigorous evidence.MethodsTwo-sample MR was utilized to examine causal effects. Summary-level data from genome-wide association studies (GWAS) of individuals of European ancestry were acquired from the GWAS catalog and FinnGen databases. Single-nucleotide polymorphisms associated with COVID-19 traits at p −8 were obtained and pruned for linkage disequilibrium to generate instrumental variables for COVID-19. Inverse variance weighted estimates were used as the primary MR results, with weighted median and MR-Egger as auxiliary analyses. The robustness of the MR findings was also evaluated through sensitivity analyses. Bonferroni correction was applied to primary results, with a p ResultsCritical ill COVID-19 [defined as hospitalization for COVID-19 with either a death outcome or respiratory support, OR (95% CI): 1.17 (1.03–1.33), p = 0.020] and hospitalized COVID-19 [defined as hospitalization for COVID-19, OR (95% CI): 1.10 (1.01–1.19), p = 0.026] demonstrated suggestive causal effects on pre-eclampsia, while general severe acute respiratory syndrome coronavirus 2 infection did not exhibit a significant causal effect on pre-eclampsia. None of the three COVID-19 severity phenotypes exhibited a significant causal effect on eclampsia.ConclusionsOur investigation demonstrates a suggestive causal effect of genetic susceptibility to critical ill COVID-19 and hospitalized COVID-19 on pre-eclampsia. The COVID-19 severity exhibited a suggestive positive dose–response relationship with the risk of pre-eclampsia. Augmented attention should be paid to pregnant women hospitalized for COVID-19, especially those needing respiratory support.</p
Table2_Mendelian randomization supports genetic liability to hospitalization for COVID-19 as a risk factor of pre-eclampsia.xlsx
BackgroundPre-eclampsia and eclampsia are among the major threats to pregnant women and fetuses, but they can be mitigated by prevention and early screening. Existing observational research presents conflicting evidence regarding the causal effects of coronavirus disease 2019 (COVID-19) on pre-eclampsia risk. Through Mendelian randomization (MR), this study aims to investigate the causal effect of three COVID-19 severity phenotypes on the risk of pre-eclampsia and eclampsia to provide more rigorous evidence.MethodsTwo-sample MR was utilized to examine causal effects. Summary-level data from genome-wide association studies (GWAS) of individuals of European ancestry were acquired from the GWAS catalog and FinnGen databases. Single-nucleotide polymorphisms associated with COVID-19 traits at p −8 were obtained and pruned for linkage disequilibrium to generate instrumental variables for COVID-19. Inverse variance weighted estimates were used as the primary MR results, with weighted median and MR-Egger as auxiliary analyses. The robustness of the MR findings was also evaluated through sensitivity analyses. Bonferroni correction was applied to primary results, with a p ResultsCritical ill COVID-19 [defined as hospitalization for COVID-19 with either a death outcome or respiratory support, OR (95% CI): 1.17 (1.03–1.33), p = 0.020] and hospitalized COVID-19 [defined as hospitalization for COVID-19, OR (95% CI): 1.10 (1.01–1.19), p = 0.026] demonstrated suggestive causal effects on pre-eclampsia, while general severe acute respiratory syndrome coronavirus 2 infection did not exhibit a significant causal effect on pre-eclampsia. None of the three COVID-19 severity phenotypes exhibited a significant causal effect on eclampsia.ConclusionsOur investigation demonstrates a suggestive causal effect of genetic susceptibility to critical ill COVID-19 and hospitalized COVID-19 on pre-eclampsia. The COVID-19 severity exhibited a suggestive positive dose–response relationship with the risk of pre-eclampsia. Augmented attention should be paid to pregnant women hospitalized for COVID-19, especially those needing respiratory support.</p