41 research outputs found
Knowledge Prompting for Few-shot Action Recognition
Few-shot action recognition in videos is challenging for its lack of
supervision and difficulty in generalizing to unseen actions. To address this
task, we propose a simple yet effective method, called knowledge prompting,
which leverages commonsense knowledge of actions from external resources to
prompt a powerful pre-trained vision-language model for few-shot
classification. We first collect large-scale language descriptions of actions,
defined as text proposals, to build an action knowledge base. The collection of
text proposals is done by filling in handcraft sentence templates with external
action-related corpus or by extracting action-related phrases from captions of
Web instruction videos.Then we feed these text proposals into the pre-trained
vision-language model along with video frames to generate matching scores of
the proposals to each frame, and the scores can be treated as action semantics
with strong generalization. Finally, we design a lightweight temporal modeling
network to capture the temporal evolution of action semantics for
classification.Extensive experiments on six benchmark datasets demonstrate that
our method generally achieves the state-of-the-art performance while reducing
the training overhead to 0.001 of existing methods
Teaching What You Should Teach: A Data-Based Distillation Method
In real teaching scenarios, an excellent teacher always teaches what he (or
she) is good at but the student is not. This gives the student the best
assistance in making up for his (or her) weaknesses and becoming a good one
overall. Enlightened by this, we introduce the "Teaching what you Should Teach"
strategy into a knowledge distillation framework, and propose a data-based
distillation method named "TST" that searches for desirable augmented samples
to assist in distilling more efficiently and rationally. To be specific, we
design a neural network-based data augmentation module with priori bias, which
assists in finding what meets the teacher's strengths but the student's
weaknesses, by learning magnitudes and probabilities to generate suitable data
samples. By training the data augmentation module and the generalized
distillation paradigm in turn, a student model is learned with excellent
generalization ability. To verify the effectiveness of our method, we conducted
extensive comparative experiments on object recognition, detection, and
segmentation tasks. The results on the CIFAR-10, ImageNet-1k, MS-COCO, and
Cityscapes datasets demonstrate that our method achieves state-of-the-art
performance on almost all teacher-student pairs. Furthermore, we conduct
visualization studies to explore what magnitudes and probabilities are needed
for the distillation process.Comment: 13 pages, 4 figure
Bootstrap Generalization Ability from Loss Landscape Perspective
Domain generalization aims to learn a model that can generalize well on the
unseen test dataset, i.e., out-of-distribution data, which has different
distribution from the training dataset. To address domain generalization in
computer vision, we introduce the loss landscape theory into this field.
Specifically, we bootstrap the generalization ability of the deep learning
model from the loss landscape perspective in four aspects, including backbone,
regularization, training paradigm, and learning rate. We verify the proposed
theory on the NICO++, PACS, and VLCS datasets by doing extensive ablation
studies as well as visualizations. In addition, we apply this theory in the
ECCV 2022 NICO Challenge1 and achieve the 3rd place without using any domain
invariant methods.Comment: 18 pages, 4 figure
Adiponectin-Mediated Promotion of CD44 Suppresses Diabetic Vascular Inflammatory Effects
While adiponectin (APN) was known to significantly abolish the diabetic endothelial inflammatory response, the specific mechanisms have yet to be elucidated. Aortic vascular tissues from mice fed normal and high-fat diets (HFD) were analyzed by transcriptome analysis. GO functional annotation showed that APN inhibited vascular endothelial inflammation in an APPL1-dependent manner. We confirmed that activation of the Wnt/β-catenin signaling plays a key role in APN-mediated anti-inflammation. Mechanistically, APN promoted APPL1/reptin complex formation and β-catenin nuclear translocation. Simultaneously, we identified APN promoted the expression of CD44 by activating TCF/LEF in an APPL1-mediated manner. Clinically, the serum levels of APN and CD44 were decreased in diabetes; the levels of these two proteins were positively correlated. Functionally, treatment with CD44 C-terminal polypeptides protected diabetes-induced vascular endothelial inflammation in vivo. Collectively, we provided a roadmap for APN-inhibited vascular inflammatory effects and CD44 might represent potential targets against the diabetic endothelial inflammatory effect
Response of Soil Respiration to Soil Temperature and Moisture in a 50-Year-Old Oriental Arborvitae Plantation in China
China possesses large areas of plantation forests which take up great quantities of carbon. However, studies on soil respiration in these plantation forests are rather scarce and their soil carbon flux remains an uncertainty. In this study, we used an automatic chamber system to measure soil surface flux of a 50-year-old mature plantation of Platycladus orientalis at Jiufeng Mountain, Beijing, China. Mean daily soil respiration rates (Rs) ranged from 0.09 to 4.87 µmol CO2 m−2s−1, with the highest values observed in August and the lowest in the winter months. A logistic model gave the best fit to the relationship between hourly Rs and soil temperature (Ts), explaining 82% of the variation in Rs over the annual cycle. The annual total of soil respiration estimated from the logistic model was 645±5 g C m−2 year−1. The performance of the logistic model was poorest during periods of high soil temperature or low soil volumetric water content (VWC), which limits the model's ability to predict the seasonal dynamics of Rs. The logistic model will potentially overestimate Rs at high Ts and low VWC. Seasonally, Rs increased significantly and linearly with increasing VWC in May and July, in which VWC was low. In the months from August to November, inclusive, in which VWC was not limiting, Rs showed a positively exponential relationship with Ts. The seasonal sensitivity of soil respiration to Ts (Q10) ranged from 0.76 in May to 4.38 in October. It was suggested that soil temperature was the main determinant of soil respiration when soil water was not limiting
Anticipating Future Relations via Graph Growing for Action Prediction
Predicting actions from partially observed videos is challenging as the partial videos containing incomplete action executions have insufficient discriminative information for classification. Recent progress has been made through enriching the features of the observed video part or generating the features for the unobserved video part, but without explicitly modeling the fine-grained evolution of visual object relations over both space and time. In this paper, we investigate how the interaction and correlation between visual objects evolve and propose a graph growing method to anticipate future object relations from limited video observations for reliable action prediction. There are two tasks in our method. First, we work with spatial-temporal graph neural networks to reason object relations in the observed video part. Then, we synthesize the spatial-temporal relation representation for the unobserved video part via graph node generation and aggregation. These two tasks are jointly learned to enable the anticipated future relation representation informative to action prediction. Experimental results on two action video datasets demonstrate the effectiveness of our method