8 research outputs found
Contrastive Vision-Language Alignment Makes Efficient Instruction Learner
We study the task of extending the large language model (LLM) into a
vision-language instruction-following model. This task is crucial but
challenging since the LLM is trained on text modality only, making it hard to
effectively digest the visual modality. To address this, existing methods
typically train a visual adapter to align the representation between a
pre-trained vision transformer (ViT) and the LLM by a generative image
captioning loss. However, we find that the generative objective can only
produce weak alignment for vision and language, making the aligned
vision-language model very hungry for the instruction fine-tuning data. In this
paper, we propose CG-VLM that applies both Contrastive and Generative alignment
objectives to effectively align the representation of ViT and LLM. Different
from image level and sentence level alignment in common contrastive learning
settings, CG-VLM aligns the image-patch level features and text-token level
embeddings, which, however, is very hard to achieve as no explicit grounding
patch-token relation provided in standard image captioning datasets. To address
this issue, we propose to maximize the averaged similarity between pooled
image-patch features and text-token embeddings. Extensive experiments
demonstrate that the proposed CG-VLM produces strong vision-language alignment
and is an efficient instruction learner. For example, using only 10%
instruction tuning data, we reach 95% performance of state-of-the-art method
LLaVA [29] on the zero-shot ScienceQA-Image benchmark.Comment: 17 pages, 10 pages for main paper, 7 pages for supplementar
CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation
We study the task of weakly-supervised point cloud semantic segmentation with
sparse annotations (e.g., less than 0.1% points are labeled), aiming to reduce
the expensive cost of dense annotations. Unfortunately, with extremely sparse
annotated points, it is very difficult to extract both contextual and object
information for scene understanding such as semantic segmentation. Motivated by
masked modeling (e.g., MAE) in image and video representation learning, we seek
to endow the power of masked modeling to learn contextual information from
sparsely-annotated points. However, directly applying MAE to 3D point clouds
with sparse annotations may fail to work. First, it is nontrivial to
effectively mask out the informative visual context from 3D point clouds.
Second, how to fully exploit the sparse annotations for context modeling
remains an open question. In this paper, we propose a simple yet effective
Contextual Point Cloud Modeling (CPCM) method that consists of two parts: a
region-wise masking (RegionMask) strategy and a contextual masked training
(CMT) method. Specifically, RegionMask masks the point cloud continuously in
geometric space to construct a meaningful masked prediction task for subsequent
context learning. CMT disentangles the learning of supervised segmentation and
unsupervised masked context prediction for effectively learning the very
limited labeled points and mass unlabeled points, respectively. Extensive
experiments on the widely-tested ScanNet V2 and S3DIS benchmarks demonstrate
the superiority of CPCM over the state-of-the-art.Comment: Accepted by ICCV 202
A Study on the Comparison and Enhancement of Health Literacy of College Students in Guangdong Province in 2020 and 2022
In order to compare the health literacy level of college students in Guangdong province in 2020 and 2022, so as to provide a scientific basis for targeted health literacy intervention and policy formulation for college students in Guangdong province, surveys were respectively conducted in 2020 and 2022. The data collation and analysis were performed using SPSS 19.0 statistical software. The χ2 test was used to compare different health literacy, and logistic regression was performed to analyse the factors influencing health literacy. The results show that the general health literacy level of college students in Guangdong province in 2022 is 46.5%, 6.3% higher than 40.2% in 2020, which difference is statistically significant. The three dimensions and six aspects of health literacy all have improved. The results of both years showed that health skills, basic medical literacy and health information literacy were at a low level. According to logistic regression analysis, the health literacy level of senior students is higher than that of junior students,and those who have taken health related courses have higher health literacy level. The most desirable type of health knowledge is prevention and treatment of infectious diseases, and the new media access is becoming more popular among students to gain health knowledge. In conclusion, Guangdong college students’ health literacy is relatively high, but still needs to be improved, especially in health skills, basic medical care and health information literacy. The government, colleges and universities should work together to improve college students’ health literacy
Design and Simulation Analysis of Docking Interface of Linked In-Orbit Replacement Module
On-orbit service for spacecraft relies heavily on on-orbit docking with the orbital replacement unit docking interface. Foreign research on the docking interface of the orbit replaceable unit has been in-depth, while the domestic work is still limited. Currently, most design on the docking interface relies on the axial feed of the manipulator, which may result in insufficient docking interface mating force under specific conditions. In view of the above problems, it requires a linear plug-in locking interface for the docking of the orbital replaceable unit, and the design scheme of the tapered rod guide and linkage locking parts needs to be determined. Optimization of the linkage locking mechanism is completed by a finite element simulation. The effect of clearance of the taper rod, effective locking points and friction coefficient have been analyzed by means of dynamics modelling during the docking and locking processes. The research also verified the design rationality for the orbital replaceable unit linkage. A processing path and verification for the prototype have been made as well. This work introduces the idea of self-plugging during the orbital docking process. It lays a foundation for the prototype development and control strategy of the orbital replaceable unit
Synthesis and Biological Evaluation of PEGylated MWO4 Nanoparticles as Sonodynamic AID Inhibitors in Treating Diffuse Large B-Cell Lymphoma
Sonodynamic therapy (SDT) triggered by ultrasound (US) has attracted increasing attention owing to its ability to overcome critical limitations, including low tissue-penetration depth and phototoxicity in photodynamic therapy (PDT). Biogenic metal oxide nanoparticles (NPs) have been used as anti-cancer drugs due to their biocompatibility properties with most biological systems. Here, sonosensitizer MWO4-PEG NPs (M = Fe Mn Co Ni) were synthesized as inhibitors to activation-induced cytidine deaminase (AID), thus neutralizing the extensive carcinogenesis of AID in diffuse large B-cell lymphoma (DLBCL). The physiological properties of these nanomaterials were examined using transmission electron microscopy (TEM). The inhibition of NPs to AID was primarily identified by the affinity interaction prediction between reactive oxygen species (ROS) and AID through molecular dynamics and molecular docking technology. The cell apoptosis and ROS generation in US-triggered NPs treated DLBCL cells (with high levels of AID) were also detected to indicate the sonosensitivity and toxicity of MWO4-PEG NPs to DLBCL cells. The anti-lymphoma studies using DLBCL and AID-deficient DLBCL cell lines indicated a concentration-dependent profile. The synthesized MWO4-PEG NPs in this study manifested good sonodynamic inhibitory effects to AID and well treatment for AID-positive hematopoietic cancers