LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

In this work, we present a novel method to tackle the token generation challenge in Vision Language Models (VLMs) for video and image understanding, called LLaMA-VID. Current VLMs, while proficient in tasks like image captioning and visual question answering, face computational burdens when processing long videos due to the excessive visual tokens. LLaMA-VID addresses this issue by representing each frame with two distinct tokens, namely context token and content token. The context token encodes the overall image context based on user input, whereas the content token encapsulates visual cues in each frame. This dual-token strategy significantly reduces the overload of long videos while preserving critical information. Generally, LLaMA-VID empowers existing frameworks to support hour-long videos and pushes their upper limit with an extra context token. It is proved to surpass previous methods on most of video- or image-based benchmarks. Code is available https://github.com/dvlab-research/LLaMA-VID}{https://github.com/dvlab-research/LLaMA-VIDComment: Code is available at https://github.com/dvlab-research/LLaMA-VI

GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding

Self-supervised 3D representation learning aims to learn effective representations from large-scale unlabeled point clouds. Most existing approaches adopt point discrimination as the pretext task, which assigns matched points in two distinct views as positive pairs and unmatched points as negative pairs. However, this approach often results in semantically identical points having dissimilar representations, leading to a high number of false negatives and introducing a "semantic conflict" problem. To address this issue, we propose GroupContrast, a novel approach that combines segment grouping and semantic-aware contrastive learning. Segment grouping partitions points into semantically meaningful regions, which enhances semantic coherence and provides semantic guidance for the subsequent contrastive representation learning. Semantic-aware contrastive learning augments the semantic information extracted from segment grouping and helps to alleviate the issue of "semantic conflict". We conducted extensive experiments on multiple 3D scene understanding tasks. The results demonstrate that GroupContrast learns semantically meaningful representations and achieves promising transfer learning performance.Comment: CVPR 202

Hierarchical Dense Correlation Distillation for Few-Shot Segmentation-Extended Abstract

Few-shot semantic segmentation (FSS) aims to form class-agnostic models segmenting unseen classes with only a handful of annotations. Previous methods limited to the semantic feature and prototype representation suffer from coarse segmentation granularity and train-set overfitting. In this work, we design Hierarchically Decoupled Matching Network (HDMNet) mining pixel-level support correlation based on the transformer architecture. The self-attention modules are used to assist in establishing hierarchical dense features, as a means to accomplish the cascade matching between query and support features. Moreover, we propose a matching module to reduce train-set overfitting and introduce correlation distillation leveraging semantic correspondence from coarse resolution to boost fine-grained segmentation. Our method performs decently in experiments. We achieve 50.0% mIoU on COCO dataset one-shot setting and 56.0% on five-shot segmentation, respectively. The code will be available on the project website. We hope our work can benefit broader industrial applications where novel classes with limited annotations are required to be decently identified.Comment: Accepted to CVPR 2023 VISION Workshop, Oral. The extended abstract of Hierarchical Dense Correlation Distillation for Few-Shot Segmentation. arXiv admin note: substantial text overlap with arXiv:2303.1465

Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models

In this work, we introduce Mini-Gemini, a simple and effective framework enhancing multi-modality Vision Language Models (VLMs). Despite the advancements in VLMs facilitating basic visual dialog and reasoning, a performance gap persists compared to advanced models like GPT-4 and Gemini. We try to narrow the gap by mining the potential of VLMs for better performance and any-to-any workflow from three aspects, i.e., high-resolution visual tokens, high-quality data, and VLM-guided generation. To enhance visual tokens, we propose to utilize an additional visual encoder for high-resolution refinement without increasing the visual token count. We further construct a high-quality dataset that promotes precise image comprehension and reasoning-based generation, expanding the operational scope of current VLMs. In general, Mini-Gemini further mines the potential of VLMs and empowers current frameworks with image understanding, reasoning, and generation simultaneously. Mini-Gemini supports a series of dense and MoE Large Language Models (LLMs) from 2B to 34B. It is demonstrated to achieve leading performance in several zero-shot benchmarks and even surpasses the developed private models. Code and models are available at https://github.com/dvlab-research/MiniGemini.Comment: Code and models are available at https://github.com/dvlab-research/MiniGemin

Mediterranean Diet and Mortality in People with Cardiovascular Disease: A Meta-Analysis of Prospective Cohort Studies

The association of the Mediterranean diet (MD) with mortality among people with a history of cardiovascular disease (CVD) has not been systematically examined. Hereby, our objective was to investigate the association of MD with all-cause and cardiovascular mortality in people with a history of CVD. We searched five electronic databases including Embase, PubMed, Scopus, Web of Science, and Cochrane Central Register of Controlled Trials to screen eligible studies published before 31 August 2020. A random-effect model was used to examine the association of a 2-unit increment in MD score with the risk of all-cause and cardiovascular mortality. We conducted sensitivity and subgroup analyses and examined potential publication bias by Egger’s and Begg’s tests. Seven cohort studies (eight datasets) with a total of 37,879 participants who had a history of CVD were eligible for the main analysis. The pooled hazard ratios were 0.85 (95% CIs: 0.78–0.93; n = 8) for all-cause mortality and 0.91 (95% CIs; 0.82–1.01; n = 4) for cardiovascular mortality for each 2-unit increment in a score of adherence to MD. Subgroup analyses for all-cause mortality showed that the association appeared relatively stronger in Mediterranean areas (HR = 0.76 [0.69–0.83]) than non-Mediterranean areas (HR = 0.95 [0.93–0.98]) and in studies with a shorter duration (HR = 0.75 [0.66–0.84] for <7 years vs. HR = 0.94 [0.91–0.98] for ≥7 years). No evidence of publication bias was observed. The present meta-analysis of prospective cohort studies provided evidence that adherence to MD improved survival in people with a history of CVD

Unraveling ferroptosis in osteogenic lineages: implications for dysregulated bone remodeling during periodontitis progression

Abstract Periodontitis is a highly prevalent disease characterized by inflammation and destruction of tooth-supporting tissues that leads to tooth loss in extreme situations. Elucidating the underlying mechanisms of periodontitis pathogenesis and progression will establish the groundwork for developing effective treatment strategies. Recently, evidence concerning the role of ferroptosis in periodontitis progression has emerged. Osteogenic lineage cells are key regulators of bone remodeling. Osteogenic cell death, as observed in experimental periodontitis models, disrupts the balance between bone resorption and bone formation. However, whether the osteogenic lineage undergoes ferroptosis during periodontitis and the corresponding effect on periodontitis progression remain elusive. Here, we investigated cell-specific ferroptosis within the alveolar bone in a murine periodontitis model. Through immunofluorescence double staining and immunohistochemistry, we identified ferroptotic osteocytes and osteoblasts in inflammatory alveolar bone. Next, in vivo administration of erastin or liproxstatin-1 was conducted to either induce or inhibit ferroptosis, respectively. Severe bone resorption and inflammation, accompanied by increased osteoclast formation and impaired osteogenic potential were detected following ferroptosis activation. Subsequently, we carried out in vitro experiments on osteocytes and further verified that ferroptosis enhanced the osteocytic expression of RANKL and IL-6. These findings suggest that ferroptosis occurring within the osteogenic lineage acts as a catalyst in the progression of periodontitis by stimulating osteoclastogenesis through the secretion of inflammatory cytokines and inhibiting osteoblastic function, providing insights into ferroptosis-induced alterations in microenvironment-based intercellular communication. Ferroptosis is a promising target for controlling inflammation and preventing bone resorption in periodontitis

Research trends and hotspots of circular RNA in cardiovascular disease: A bibliometric analysis

From a global perspective, cardiovascular diseases (CVDs), the leading factor accounting for population mortality, and circRNAs, RNA molecules with stable closed-loop structures, have been proven to be closely related. The latent clinical value and the potential role of circRNAs in CVDs have been attracting increasing, active research interest, but bibliometric studies in this field are still lacking. Thus, in this study, we conducted a bibliometric analysis by using software such as VOSviewer, CiteSpace, Microsoft Excel, and the R package to determine the current research progress and hotspots and ultimately provide an overview of the development trends and future frontiers in this field. In our study, based on our search strategy, a total of 1206 publications published before July 31, 2023 were accessed from the WOSCC database. According to our findings, there is a notable increasing trend in global publications in the field of circRNA in CVDs. China was found to be the dominant country in terms of publication number, but a lack of high-quality articles was a significant fault. A cluster analysis on the co-cited references indicated that dilated cardiomyopathy, AMI, and cardiac hypertrophy are the greatest objects of concern. In contrast, a keywords analysis indicated that high importance has been ascribed to MI, abdominal aortic aneurysm, cell proliferation, and coronary artery diseases

Patients with Asian-type DEL can safely be transfused using RhD-positive blood

Red blood cells (RBCs) of the Asian-type DEL phenotype express few RhD proteins and are typed as serologic RhD-negative (D-) in routine testing. RhD-positive (D+) RBC transfusion for Asian-type DEL patients has been proposed but has not been generally adopted due to a lack of direct evidence regarding its safety and underlying mechanism. We performed a single-arm multicenter clinical trial to document the outcome of D+ RBC transfusion in Asian-type DEL patients; none of the recipients (0/42; 95% confidence interval, 0%-8.40%) developed alloanti-D after a median follow-up of 226 days. We conducted a large retrospective study to detect alloanti-D immunization in 4,045 serologic D- pregnant women throughout China; alloanti-D was found only in true D- individuals (2.63%, 79/3,009), but not in those with Asian-type DEL (0/1,032). We further retrospectively examined 127 serologic D- pregnant women who had developed alloanti-D and found none with Asian-type DEL (0/127). Finally, we analyzed RHD transcripts from Asian-type DEL erythroblasts and examined antigen epitopes expressed by various RHD transcripts in vitro, finding a low abundance of full-length RHD transcripts (0.18% of the total) expressing RhD antigens carrying the entire repertoire of epitopes, which could explain the immune tolerance against D+ RBCs. Our results provide multiple lines of evidence that individuals with Asian-type DEL cannot produce alloanti-D when exposed to D+ RBCs following transfusion or pregnancy. Therefore, we recommend considering D+ RBC transfusion and discontinuing anti-D prophylaxis in Asian-type DEL patients, including pregnant women. This clinical trial is registered at www.clinicaltrials.gov as NCT03727230