Knowledge Prompting for Few-shot Action Recognition

Few-shot action recognition in videos is challenging for its lack of supervision and difficulty in generalizing to unseen actions. To address this task, we propose a simple yet effective method, called knowledge prompting, which leverages commonsense knowledge of actions from external resources to prompt a powerful pre-trained vision-language model for few-shot classification. We first collect large-scale language descriptions of actions, defined as text proposals, to build an action knowledge base. The collection of text proposals is done by filling in handcraft sentence templates with external action-related corpus or by extracting action-related phrases from captions of Web instruction videos.Then we feed these text proposals into the pre-trained vision-language model along with video frames to generate matching scores of the proposals to each frame, and the scores can be treated as action semantics with strong generalization. Finally, we design a lightweight temporal modeling network to capture the temporal evolution of action semantics for classification.Extensive experiments on six benchmark datasets demonstrate that our method generally achieves the state-of-the-art performance while reducing the training overhead to 0.001 of existing methods

Teaching What You Should Teach: A Data-Based Distillation Method

In real teaching scenarios, an excellent teacher always teaches what he (or she) is good at but the student is not. This gives the student the best assistance in making up for his (or her) weaknesses and becoming a good one overall. Enlightened by this, we introduce the "Teaching what you Should Teach" strategy into a knowledge distillation framework, and propose a data-based distillation method named "TST" that searches for desirable augmented samples to assist in distilling more efficiently and rationally. To be specific, we design a neural network-based data augmentation module with priori bias, which assists in finding what meets the teacher's strengths but the student's weaknesses, by learning magnitudes and probabilities to generate suitable data samples. By training the data augmentation module and the generalized distillation paradigm in turn, a student model is learned with excellent generalization ability. To verify the effectiveness of our method, we conducted extensive comparative experiments on object recognition, detection, and segmentation tasks. The results on the CIFAR-10, ImageNet-1k, MS-COCO, and Cityscapes datasets demonstrate that our method achieves state-of-the-art performance on almost all teacher-student pairs. Furthermore, we conduct visualization studies to explore what magnitudes and probabilities are needed for the distillation process.Comment: 13 pages, 4 figure

Bootstrap Generalization Ability from Loss Landscape Perspective

Domain generalization aims to learn a model that can generalize well on the unseen test dataset, i.e., out-of-distribution data, which has different distribution from the training dataset. To address domain generalization in computer vision, we introduce the loss landscape theory into this field. Specifically, we bootstrap the generalization ability of the deep learning model from the loss landscape perspective in four aspects, including backbone, regularization, training paradigm, and learning rate. We verify the proposed theory on the NICO++, PACS, and VLCS datasets by doing extensive ablation studies as well as visualizations. In addition, we apply this theory in the ECCV 2022 NICO Challenge1 and achieve the 3rd place without using any domain invariant methods.Comment: 18 pages, 4 figure

Adiponectin-Mediated Promotion of CD44 Suppresses Diabetic Vascular Inflammatory Effects

While adiponectin (APN) was known to significantly abolish the diabetic endothelial inflammatory response, the specific mechanisms have yet to be elucidated. Aortic vascular tissues from mice fed normal and high-fat diets (HFD) were analyzed by transcriptome analysis. GO functional annotation showed that APN inhibited vascular endothelial inflammation in an APPL1-dependent manner. We confirmed that activation of the Wnt/β-catenin signaling plays a key role in APN-mediated anti-inflammation. Mechanistically, APN promoted APPL1/reptin complex formation and β-catenin nuclear translocation. Simultaneously, we identified APN promoted the expression of CD44 by activating TCF/LEF in an APPL1-mediated manner. Clinically, the serum levels of APN and CD44 were decreased in diabetes; the levels of these two proteins were positively correlated. Functionally, treatment with CD44 C-terminal polypeptides protected diabetes-induced vascular endothelial inflammation in vivo. Collectively, we provided a roadmap for APN-inhibited vascular inflammatory effects and CD44 might represent potential targets against the diabetic endothelial inflammatory effect

Response of Soil Respiration to Soil Temperature and Moisture in a 50-Year-Old Oriental Arborvitae Plantation in China

China possesses large areas of plantation forests which take up great quantities of carbon. However, studies on soil respiration in these plantation forests are rather scarce and their soil carbon flux remains an uncertainty. In this study, we used an automatic chamber system to measure soil surface flux of a 50-year-old mature plantation of Platycladus orientalis at Jiufeng Mountain, Beijing, China. Mean daily soil respiration rates (Rs) ranged from 0.09 to 4.87 µmol CO2 m−2s−1, with the highest values observed in August and the lowest in the winter months. A logistic model gave the best fit to the relationship between hourly Rs and soil temperature (Ts), explaining 82% of the variation in Rs over the annual cycle. The annual total of soil respiration estimated from the logistic model was 645±5 g C m−2 year−1. The performance of the logistic model was poorest during periods of high soil temperature or low soil volumetric water content (VWC), which limits the model's ability to predict the seasonal dynamics of Rs. The logistic model will potentially overestimate Rs at high Ts and low VWC. Seasonally, Rs increased significantly and linearly with increasing VWC in May and July, in which VWC was low. In the months from August to November, inclusive, in which VWC was not limiting, Rs showed a positively exponential relationship with Ts. The seasonal sensitivity of soil respiration to Ts (Q10) ranged from 0.76 in May to 4.38 in October. It was suggested that soil temperature was the main determinant of soil respiration when soil water was not limiting

Anticipating Future Relations via Graph Growing for Action Prediction

Predicting actions from partially observed videos is challenging as the partial videos containing incomplete action executions have insufficient discriminative information for classification. Recent progress has been made through enriching the features of the observed video part or generating the features for the unobserved video part, but without explicitly modeling the fine-grained evolution of visual object relations over both space and time. In this paper, we investigate how the interaction and correlation between visual objects evolve and propose a graph growing method to anticipate future object relations from limited video observations for reliable action prediction. There are two tasks in our method. First, we work with spatial-temporal graph neural networks to reason object relations in the observed video part. Then, we synthesize the spatial-temporal relation representation for the unobserved video part via graph node generation and aggregation. These two tasks are jointly learned to enable the anticipated future relation representation informative to action prediction. Experimental results on two action video datasets demonstrate the effectiveness of our method