64 research outputs found
Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks
It is desirable to train convolutional networks (CNNs) to run more
efficiently during inference. In many cases however, the computational budget
that the system has for inference cannot be known beforehand during training,
or the inference budget is dependent on the changing real-time resource
availability. Thus, it is inadequate to train just inference-efficient CNNs,
whose inference costs are not adjustable and cannot adapt to varied inference
budgets. We propose a novel approach for cost-adjustable inference in CNNs -
Stochastic Downsampling Point (SDPoint). During training, SDPoint applies
feature map downsampling to a random point in the layer hierarchy, with a
random downsampling ratio. The different stochastic downsampling configurations
known as SDPoint instances (of the same model) have computational costs
different from each other, while being trained to minimize the same prediction
loss. Sharing network parameters across different instances provides
significant regularization boost. During inference, one may handpick a SDPoint
instance that best fits the inference budget. The effectiveness of SDPoint, as
both a cost-adjustable inference approach and a regularizer, is validated
through extensive experiments on image classification
Continual Semantic Segmentation with Automatic Memory Sample Selection
Continual Semantic Segmentation (CSS) extends static semantic segmentation by
incrementally introducing new classes for training. To alleviate the
catastrophic forgetting issue in CSS, a memory buffer that stores a small
number of samples from the previous classes is constructed for replay. However,
existing methods select the memory samples either randomly or based on a
single-factor-driven handcrafted strategy, which has no guarantee to be
optimal. In this work, we propose a novel memory sample selection mechanism
that selects informative samples for effective replay in a fully automatic way
by considering comprehensive factors including sample diversity and class
performance. Our mechanism regards the selection operation as a decision-making
process and learns an optimal selection policy that directly maximizes the
validation performance on a reward set. To facilitate the selection decision,
we design a novel state representation and a dual-stage action space. Our
extensive experiments on Pascal-VOC 2012 and ADE 20K datasets demonstrate the
effectiveness of our approach with state-of-the-art (SOTA) performance
achieved, outperforming the second-place one by 12.54% for the 6stage setting
on Pascal-VOC 2012.Comment: Accepted to CVPR202
Effective Action Recognition with Embedded Key Point Shifts
Temporal feature extraction is an essential technique in video-based action
recognition. Key points have been utilized in skeleton-based action recognition
methods but they require costly key point annotation. In this paper, we propose
a novel temporal feature extraction module, named Key Point Shifts Embedding
Module (), to adaptively extract channel-wise key point shifts across
video frames without key point annotation for temporal feature extraction. Key
points are adaptively extracted as feature points with maximum feature values
at split regions, while key point shifts are the spatial displacements of
corresponding key points. The key point shifts are encoded as the overall
temporal features via linear embedding layers in a multi-set manner. Our method
achieves competitive performance through embedding key point shifts with
trivial computational cost, achieving the state-of-the-art performance of
82.05% on Mini-Kinetics and competitive performance on UCF101,
Something-Something-v1, and HMDB51 datasets.Comment: 35 pages, 10 figure
Towards Balanced Active Learning for Multimodal Classification
Training multimodal networks requires a vast amount of data due to their
larger parameter space compared to unimodal networks. Active learning is a
widely used technique for reducing data annotation costs by selecting only
those samples that could contribute to improving model performance. However,
current active learning strategies are mostly designed for unimodal tasks, and
when applied to multimodal data, they often result in biased sample selection
from the dominant modality. This unfairness hinders balanced multimodal
learning, which is crucial for achieving optimal performance. To address this
issue, we propose three guidelines for designing a more balanced multimodal
active learning strategy. Following these guidelines, a novel approach is
proposed to achieve more fair data selection by modulating the gradient
embedding with the dominance degree among modalities. Our studies demonstrate
that the proposed method achieves more balanced multimodal learning by avoiding
greedy sample selection from the dominant modality. Our approach outperforms
existing active learning strategies on a variety of multimodal classification
tasks. Overall, our work highlights the importance of balancing sample
selection in multimodal active learning and provides a practical solution for
achieving more balanced active learning for multimodal classification.Comment: 12 pages, accepted by ACMMM 202
Research Progress on the Bioactivity and Mechanisms of Jujube Polysaccharides
Jujube has a history of being used as both food and herbal medicine in China for thousands of years. Its nutritional value has long been recognized, as it contains various health-promoting bioactive components. Jujube polysaccharides are one of the major bioactive constituents. Extensive studies have shown that jujube polysaccharides exert multiple biological activities, such as antioxidant, anti-inflammatory, anti-tumor, immune regulatory, anti-fatigue, liver-protective, blood sugar-lowering, and blood lipid-lowering effects, through different or synergistic mechanisms. This article provides an overview of the physicochemical properties and structure-activity relationship of jujube polysaccharides, and summarizes the research progress on the bioactivity and mechanisms of jujube polysaccharides, both domestically and internationally in recent years. Based on these findings, the therapeutic potential of jujube polysaccharides for various diseases is discussed, aiming to provide references for further research and the development of related medicinal treatments involving jujube polysaccharides
- …