23 research outputs found
Two generalizations of Kohonen clustering
The relationship between the sequential hard c-means (SHCM), learning vector quantization (LVQ), and fuzzy c-means (FCM) clustering algorithms is discussed. LVQ and SHCM suffer from several major problems. For example, they depend heavily on initialization. If the initial values of the cluster centers are outside the convex hull of the input data, such algorithms, even if they terminate, may not produce meaningful results in terms of prototypes for cluster representation. This is due in part to the fact that they update only the winning prototype for every input vector. The impact and interaction of these two families with Kohonen's self-organizing feature mapping (SOFM), which is not a clustering method, but which often leads ideas to clustering algorithms is discussed. Then two generalizations of LVQ that are explicitly designed as clustering algorithms are presented; these algorithms are referred to as generalized LVQ = GLVQ; and fuzzy LVQ = FLVQ. Learning rules are derived to optimize an objective function whose goal is to produce 'good clusters'. GLVQ/FLVQ (may) update every node in the clustering net for each input vector. Neither GLVQ nor FLVQ depends upon a choice for the update neighborhood or learning rate distribution - these are taken care of automatically. Segmentation of a gray tone image is used as a typical application of these algorithms to illustrate the performance of GLVQ/FLVQ
Robust Class-Conditional Distribution Alignment for Partial Domain Adaptation
Unwanted samples from private source categories in the learning objective of
a partial domain adaptation setup can lead to negative transfer and reduce
classification performance. Existing methods, such as re-weighting or
aggregating target predictions, are vulnerable to this issue, especially during
initial training stages, and do not adequately address class-level feature
alignment. Our proposed approach seeks to overcome these limitations by delving
deeper than just the first-order moments to derive distinct and compact
categorical distributions. We employ objectives that optimize the intra and
inter-class distributions in a domain-invariant fashion and design a robust
pseudo-labeling for efficient target supervision. Our approach incorporates a
complement entropy objective module to reduce classification uncertainty and
flatten incorrect category predictions. The experimental findings and ablation
analysis of the proposed modules demonstrate the superior performance of our
proposed model compared to benchmarks
Curriculum Guided Domain Adaptation in the Dark
Addressing the rising concerns of privacy and security, domain adaptation in
the dark aims to adapt a black-box source trained model to an unlabeled target
domain without access to any source data or source model parameters. The need
for domain adaptation of black-box predictors becomes even more pronounced to
protect intellectual property as deep learning based solutions are becoming
increasingly commercialized. Current methods distill noisy predictions on the
target data obtained from the source model to the target model, and/or separate
clean/noisy target samples before adapting using traditional noisy label
learning algorithms. However, these methods do not utilize the easy-to-hard
learning nature of the clean/noisy data splits. Also, none of the existing
methods are end-to-end, and require a separate fine-tuning stage and an initial
warmup stage. In this work, we present Curriculum Adaptation for Black-Box
(CABB) which provides a curriculum guided adaptation approach to gradually
train the target model, first on target data with high confidence (clean)
labels, and later on target data with noisy labels. CABB utilizes
Jensen-Shannon divergence as a better criterion for clean-noisy sample
separation, compared to the traditional criterion of cross entropy loss. Our
method utilizes co-training of a dual-branch network to suppress error
accumulation resulting from confirmation bias. The proposed approach is
end-to-end trainable and does not require any extra finetuning stage, unlike
existing methods. Empirical results on standard domain adaptation datasets show
that CABB outperforms existing state-of-the-art black-box DA models and is
comparable to white-box domain adaptation models
Towards interpretable-by-design deep learning algorithms
The proposed framework named IDEAL (Interpretable-by-design DEep learning
ALgorithms) recasts the standard supervised classification problem into a
function of similarity to a set of prototypes derived from the training data,
while taking advantage of existing latent spaces of large neural networks
forming so-called Foundation Models (FM). This addresses the issue of
explainability (stage B) while retaining the benefits from the tremendous
achievements offered by DL models (e.g., visual transformers, ViT) pre-trained
on huge data sets such as IG-3.6B + ImageNet-1K or LVD-142M (stage A). We show
that one can turn such DL models into conceptually simpler,
explainable-through-prototypes ones.
The key findings can be summarized as follows: (1) the proposed models are
interpretable through prototypes, mitigating the issue of confounded
interpretations, (2) the proposed IDEAL framework circumvents the issue of
catastrophic forgetting allowing efficient class-incremental learning, and (3)
the proposed IDEAL approach demonstrates that ViT architectures narrow the gap
between finetuned and non-finetuned models allowing for transfer learning in a
fraction of time \textbf{without} finetuning of the feature space on a target
dataset with iterative supervised methods