1,305 research outputs found
A review of domain adaptation without target labels
Domain adaptation has become a prominent problem setting in machine learning
and related fields. This review asks the question: how can a classifier learn
from a source domain and generalize to a target domain? We present a
categorization of approaches, divided into, what we refer to as, sample-based,
feature-based and inference-based methods. Sample-based methods focus on
weighting individual observations during training based on their importance to
the target domain. Feature-based methods revolve around on mapping, projecting
and representing features such that a source classifier performs well on the
target domain and inference-based methods incorporate adaptation into the
parameter estimation procedure, for instance through constraints on the
optimization procedure. Additionally, we review a number of conditions that
allow for formulating bounds on the cross-domain generalization error. Our
categorization highlights recurring ideas and raises questions important to
further research.Comment: 20 pages, 5 figure
Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification
Within the scope of natural language processing, the domain of multi-label
text classification is uniquely challenging due to its expansive and uneven
label distribution. The complexity deepens due to the demand for an extensive
set of annotated data for training an advanced deep learning model, especially
in specialized fields where the labeling task can be labor-intensive and often
requires domain-specific knowledge. Addressing these challenges, our study
introduces a novel deep active learning strategy, capitalizing on the Beta
family of proper scoring rules within the Expected Loss Reduction framework. It
computes the expected increase in scores using the Beta Scoring Rules, which
are then transformed into sample vector representations. These vector
representations guide the diverse selection of informative samples, directly
linking this process to the model's expected proper score. Comprehensive
evaluations across both synthetic and real datasets reveal our method's
capability to often outperform established acquisition techniques in
multi-label text classification, presenting encouraging outcomes across various
architectural and dataset scenarios.Comment: 7 pages AAAI 202
mlplasmids : a user-friendly tool to predict plasmid- and chromosome-derived sequences for single species
Assembly of bacterial short-read whole-genome sequencing data frequently results in hundreds of contigs for which the origin, plasmid or chromosome, is unclear. Complete genomes resolved by long-read sequencing can be used to generate and label short-read contigs. These were used to train several popular machine learning methods to classify the origin of contigs from Enterococcus faecium, Klebsiella pneumoniae and Escherichia colt using pentamer frequencies. We selected support-vector machine (SVM) models as the best classifier for all three bacterial species (Fl-score E. faecium=0.92, F1-score K. pneumoniae=0.90, F1-score E. coli=0.76), which outperformed other existing plasmid prediction tools using a benchmarking set of isolates. We demonstrated the scalability of our models by accurately predicting the plasmidome of a large collection of 1644 E. faecium isolates and illustrate its applicability by predicting the location of antibiotic-resistance genes in all three species. The SVM classifiers are publicly available as an R package and graphical-user interface called 'mlplasmids'. We anticipate that this tool may significantly facilitate research on the dissemination of plasmids encoding antibiotic resistance and/or contributing to host adaptation.Peer reviewe
Does Confidence Calibration Help Conformal Prediction?
Conformal prediction, as an emerging uncertainty qualification technique,
constructs prediction sets that are guaranteed to contain the true label with
high probability. Previous works usually employ temperature scaling to
calibrate the classifier, assuming that confidence calibration can benefit
conformal prediction. In this work, we first show that post-hoc calibration
methods surprisingly lead to larger prediction sets with improved calibration,
while over-confidence with small temperatures benefits the conformal prediction
performance instead. Theoretically, we prove that high confidence reduces the
probability of appending a new class in the prediction set. Inspired by the
analysis, we propose a novel method,
(ConfTS), which rectifies the objective through the gap between the threshold
and the non-conformity score of the ground-truth label. In this way, the new
objective of ConfTS will optimize the temperature value toward an optimal set
that satisfies the . Experiments demonstrate that
our method can effectively improve widely-used conformal prediction methods
The Multimodal Tutor: Adaptive Feedback from Multimodal Experiences
This doctoral thesis describes the journey of ideation, prototyping and empirical testing of the Multimodal Tutor, a system designed for providing digital feedback that supports psychomotor skills acquisition using learning and multimodal data capturing. The feedback is given in real-time with machine-driven assessment of the learner's task execution. The predictions are tailored by supervised machine learning models trained with human annotated samples. The main contributions of this thesis are: a literature survey on multimodal data for learning, a conceptual model (the Multimodal Learning Analytics Model), a technological framework (the Multimodal Pipeline), a data annotation tool (the Visual Inspection Tool) and a case study in Cardiopulmonary Resuscitation training (CPR Tutor). The CPR Tutor generates real-time, adaptive feedback using kinematic and myographic data and neural networks
XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensics
With the rise of complex cyber devices Cyber Forensics (CF) is facing many
new challenges. For example, there are dozens of systems running on
smartphones, each with more than millions of downloadable applications. Sifting
through this large amount of data and making sense requires new techniques,
such as from the field of Artificial Intelligence (AI). To apply these
techniques successfully in CF, we need to justify and explain the results to
the stakeholders of CF, such as forensic analysts and members of the court, for
them to make an informed decision. If we want to apply AI successfully in CF,
there is a need to develop trust in AI systems. Some other factors in accepting
the use of AI in CF are to make AI authentic, interpretable, understandable,
and interactive. This way, AI systems will be more acceptable to the public and
ensure alignment with legal standards. An explainable AI (XAI) system can play
this role in CF, and we call such a system XAI-CF. XAI-CF is indispensable and
is still in its infancy. In this paper, we explore and make a case for the
significance and advantages of XAI-CF. We strongly emphasize the need to build
a successful and practical XAI-CF system and discuss some of the main
requirements and prerequisites of such a system. We present a formal definition
of the terms CF and XAI-CF and a comprehensive literature review of previous
works that apply and utilize XAI to build and increase trust in CF. We discuss
some challenges facing XAI-CF. We also provide some concrete solutions to these
challenges. We identify key insights and future research directions for
building XAI applications for CF. This paper is an effort to explore and
familiarize the readers with the role of XAI applications in CF, and we believe
that our work provides a promising basis for future researchers interested in
XAI-CF
- …