32 research outputs found
Discrete denoising of heterogenous two-dimensional data
We consider discrete denoising of two-dimensional data with characteristics
that may be varying abruptly between regions.
Using a quadtree decomposition technique and space-filling curves, we extend
the recently developed S-DUDE (Shifting Discrete Universal DEnoiser), which was
tailored to one-dimensional data, to the two-dimensional case. Our scheme
competes with a genie that has access, in addition to the noisy data, also to
the underlying noiseless data, and can employ different two-dimensional
sliding window denoisers along distinct regions obtained by a quadtree
decomposition with leaves, in a way that minimizes the overall loss. We
show that, regardless of what the underlying noiseless data may be, the
two-dimensional S-DUDE performs essentially as well as this genie, provided
that the number of distinct regions satisfies , where is the total
size of the data. The resulting algorithm complexity is still linear in both
and , as in the one-dimensional case. Our experimental results show that
the two-dimensional S-DUDE can be effective when the characteristics of the
underlying clean image vary across different regions in the data.Comment: 16 pages, submitted to IEEE Transactions on Information Theor
Visual Concept Reasoning Networks
A split-transform-merge strategy has been broadly used as an architectural
constraint in convolutional neural networks for visual recognition tasks. It
approximates sparsely connected networks by explicitly defining multiple
branches to simultaneously learn representations with different visual concepts
or properties. Dependencies or interactions between these representations are
typically defined by dense and local operations, however, without any
adaptiveness or high-level reasoning. In this work, we propose to exploit this
strategy and combine it with our Visual Concept Reasoning Networks (VCRNet) to
enable reasoning between high-level visual concepts. We associate each branch
with a visual concept and derive a compact concept state by selecting a few
local descriptors through an attention module. These concept states are then
updated by graph-based interaction and used to adaptively modulate the local
descriptors. We describe our proposed model by
split-transform-attend-interact-modulate-merge stages, which are implemented by
opting for a highly modularized architecture. Extensive experiments on visual
recognition tasks such as image classification, semantic segmentation, object
detection, scene recognition, and action recognition show that our proposed
model, VCRNet, consistently improves the performance by increasing the number
of parameters by less than 1%.Comment: Preprin
Regularization and Kernelization of the Maximin Correlation Approach
Robust classification becomes challenging when each class consists of
multiple subclasses. Examples include multi-font optical character recognition
and automated protein function prediction. In correlation-based
nearest-neighbor classification, the maximin correlation approach (MCA)
provides the worst-case optimal solution by minimizing the maximum
misclassification risk through an iterative procedure. Despite the optimality,
the original MCA has drawbacks that have limited its wide applicability in
practice. That is, the MCA tends to be sensitive to outliers, cannot
effectively handle nonlinearities in datasets, and suffers from having high
computational complexity. To address these limitations, we propose an improved
solution, named regularized maximin correlation approach (R-MCA). We first
reformulate MCA as a quadratically constrained linear programming (QCLP)
problem, incorporate regularization by introducing slack variables in the
primal problem of the QCLP, and derive the corresponding Lagrangian dual. The
dual formulation enables us to apply the kernel trick to R-MCA so that it can
better handle nonlinearities. Our experimental results demonstrate that the
regularization and kernelization make the proposed R-MCA more robust and
accurate for various classification tasks than the original MCA. Furthermore,
when the data size or dimensionality grows, R-MCA runs substantially faster by
solving either the primal or dual (whichever has a smaller variable dimension)
of the QCLP.Comment: Submitted to IEEE Acces
Complementary Domain Adaptation and Generalization for Unsupervised Continual Domain Shift Learning
Continual domain shift poses a significant challenge in real-world
applications, particularly in situations where labeled data is not available
for new domains. The challenge of acquiring knowledge in this problem setting
is referred to as unsupervised continual domain shift learning. Existing
methods for domain adaptation and generalization have limitations in addressing
this issue, as they focus either on adapting to a specific domain or
generalizing to unseen domains, but not both. In this paper, we propose
Complementary Domain Adaptation and Generalization (CoDAG), a simple yet
effective learning framework that combines domain adaptation and generalization
in a complementary manner to achieve three major goals of unsupervised
continual domain shift learning: adapting to a current domain, generalizing to
unseen domains, and preventing forgetting of previously seen domains. Our
approach is model-agnostic, meaning that it is compatible with any existing
domain adaptation and generalization algorithms. We evaluate CoDAG on several
benchmark datasets and demonstrate that our model outperforms state-of-the-art
models in all datasets and evaluation metrics, highlighting its effectiveness
and robustness in handling unsupervised continual domain shift learning
Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation
Sequential recommenders have made great strides in capturing a user's
preferences. Nevertheless, the cold-start recommendation remains a fundamental
challenge as they typically involve limited user-item interactions for
personalization. Recently, gradient-based meta-learning approaches have emerged
in the sequential recommendation field due to their fast adaptation and
easy-to-integrate abilities. The meta-learning algorithms formulate the
cold-start recommendation as a few-shot learning problem, where each user is
represented as a task to be adapted. While meta-learning algorithms generally
assume that task-wise samples are evenly distributed over classes or values,
user-item interactions in real-world applications do not conform to such a
distribution (e.g., watching favorite videos multiple times, leaving only
positive ratings without any negative ones). Consequently, imbalanced user
feedback, which accounts for the majority of task training data, may dominate
the user adaptation process and prevent meta-learning algorithms from learning
meaningful meta-knowledge for personalized recommendations. To alleviate this
limitation, we propose a novel sequential recommendation framework based on
gradient-based meta-learning that captures the imbalanced rating distribution
of each user and computes adaptive loss for user-specific learning. Our work is
the first to tackle the impact of imbalanced ratings in cold-start sequential
recommendation scenarios. Through extensive experiments conducted on real-world
datasets, we demonstrate the effectiveness of our framework.Comment: Accepted by CIKM 202