11 research outputs found
D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation
Temporal sentence grounding (TSG) aims to locate a specific moment from an
untrimmed video with a given natural language query. Recently, weakly
supervised methods still have a large performance gap compared to fully
supervised ones, while the latter requires laborious timestamp annotations. In
this study, we aim to reduce the annotation cost yet keep competitive
performance for TSG task compared to fully supervised ones. To achieve this
goal, we investigate a recently proposed glance-supervised temporal sentence
grounding task, which requires only single frame annotation (referred to as
glance annotation) for each query. Under this setup, we propose a Dynamic
Gaussian prior based Grounding framework with Glance annotation (D3G), which
consists of a Semantic Alignment Group Contrastive Learning module (SA-GCL) and
a Dynamic Gaussian prior Adjustment module (DGA). Specifically, SA-GCL samples
reliable positive moments from a 2D temporal map via jointly leveraging
Gaussian prior and semantic consistency, which contributes to aligning the
positive sentence-moment pairs in the joint embedding space. Moreover, to
alleviate the annotation bias resulting from glance annotation and model
complex queries consisting of multiple events, we propose the DGA module, which
adjusts the distribution dynamically to approximate the ground truth of target
moments. Extensive experiments on three challenging benchmarks verify the
effectiveness of the proposed D3G. It outperforms the state-of-the-art weakly
supervised methods by a large margin and narrows the performance gap compared
to fully supervised methods. Code is available at
https://github.com/solicucu/D3G.Comment: ICCV202
Line-Monitoring, Hyperspectral Fluorescence Setup for Simultaneous Multi-Analyte Biosensing
Conventional fluorescence scanners utilize multiple filters to distinguish different fluorescent labels, and problems arise because of this filter-based mechanism. In this work we propose a line-monitoring, hyperspectral fluorescence technique which is designed and optimized for applications in multi-channel microfluidic systems. In contrast to the filter-based mechanism, which only records fluorescent intensities, the hyperspectral technique records the full spectrum for every point on the sample plane. Multivariate data exploitation is then applied to spectra analysis to determine ratios of different fluorescent labels and eliminate unwanted artifacts. This sensor is designed to monitor multiple fluidic channels simultaneously, providing the potential for multi-analyte biosensing. The detection sensitivity is approximately 0.81 fluors/μm2, and this sensor is proved to act with a good homogeneity. Finally, a model experiment of detecting short oligonucleotides has demonstrated the biomedical application of this hyperspectral fluorescence biosensor
Analysis on spatial-temporal variation characteristics of climate in Qinling-Huaihe demarcation zone since 1961
The emerging global environmental crisis, intensified by global warming, necessitates a deeper understanding of climate variations, especially in critical geographical divisions like the Qinling-Huaihe demarcation zone, which acts as a natural division between northern and southern mainland China. This study investigates the spatial–temporal differentiation of climatic elements within the Qinling-Huaihe demarcation zone from 1961 to 2018, employing data accumulated from 144 meteorological stations. Utilizing methodologies including the univariate linear regression, the Mann-Kendall test, and the Kriging interpolation, this investigation unveils the spatial–temporal variation characteristics of precipitation, relative humidity, and temperature. Temporally, the climatic alterations in the zone are predominantly marked by warming trends, significantly pronounced since the 1990s, with the elevation in minimum temperature being the most notable. Both precipitation and relative humidity exhibit a fluctuating ascending trend. Spatial analysis reveals a west-to-east increase in precipitation, while relative humidity presents lower values in the central region, escalating towards the eastern and western edges. The spatial distribution patterns for average, maximum, and minimum temperatures align closely, showcasing higher temperatures in the central region, extending to the Huaihe River area, with the Qinling Mountains exhibiting the lowest temperatures, particularly noticeable north of the Qinling Mountains
Multideep Feature Fusion Algorithm for Clothing Style Recognition
In order to improve recognition accuracy of clothing style and fully exploit the advantages of deep learning in extracting deep semantic features from global to local features of clothing images, this paper utilizes the target detection technology and deep residual network (ResNet) to extract comprehensive clothing features, which aims at focusing on clothing itself in the process of feature extraction procedure. Based on that, we propose a multideep feature fusion algorithm for clothing image style recognition. First, we use the improved target detection model to extract the global area, main part, and part areas of clothing, which constitute the image, so as to weaken the influence of the background and other interference factors. Then, the three parts were inputted, respectively, to improve ResNet for feature extraction, which has been trained beforehand. The ResNet model is improved by optimizing the convolution layer in the residual block and adjusting the order of the batch-normalized layer and the activation layer. Finally, the multicategory fusion features were obtained by combining the overall features of the clothing image from the global area, the main part, to the part areas. The experimental results show that the proposed algorithm eliminates the influence of interference factors, makes the recognition process focus on clothing itself, greatly improves the accuracy of the clothing style recognition, and is better than the traditional deep residual network-based methods
Open-Vocabulary Multi-Label Classification via Multi-modal Knowledge Transfer
Real-world recognition system often encounters a plenty of unseen labels in
practice. To identify such unseen labels, multi-label zero-shot learning
(ML-ZSL) focuses on transferring knowledge by a pre-trained textual label
embedding (e.g., GloVe). However, such methods only exploit singlemodal
knowledge from a language model, while ignoring the rich semantic information
inherent in image-text pairs. Instead, recently developed open-vocabulary (OV)
based methods succeed in exploiting such information of image-text pairs in
object detection, and achieve impressive performance. Inspired by the success
of OV-based methods, we propose a novel open-vocabulary framework, named
multimodal knowledge transfer (MKT), for multi-label classification.
Specifically, our method exploits multi-modal knowledge of image-text pairs
based on a vision and language pretraining (VLP) model. To facilitate
transferring the imagetext matching ability of VLP model, knowledge
distillation is used to guarantee the consistency of image and label
embeddings, along with prompt tuning to further update the label embeddings. To
further recognize multiple objects, a simple but effective two-stream module is
developed to capture both local and global features. Extensive experimental
results show that our method significantly outperforms state-of-theart methods
on public benchmark datasets. Code will be available at
https://github.com/seanhe97/MKT.Comment: 13 pages, 10 figure
Research on Adaptability Evaluation Method of Polymer by Nuclear Magnetic Resonance Technology
In order to study the matching relationship between polymer(HPAM) molecular weight and reservoir permeability, in this paper, the injection performance of polymers with different molecular weights in rock cores with different permeability is studied. Using nuclear magnetic resonance technology combined with conventional core displacement equipment, the change law of the displacement process was analyzed from three aspects of nuclear magnetic resonance T2 spectrum, core layering, and imaging. Finally, the fluidity of the polymer solution in the core was analyzed by injection pressure control features. The experimental results show that the polymer solution with a molecular weight of 25 million has the best retention effect in the core flooding experiment and can stay in the dominant channel of the core for a long time to control the water flooding mobility. In rocks with a permeability of 500, 1000, and 2000 mD, subsequent water flooding can expand the swept volume by about 25% compared with polymer flooding. This method can effectively establish the adaptability matching relationship between the polymer molecular weight and the reservoir permeability