978 research outputs found
A Generic Approach for Reproducible Model Distillation
Model distillation has been a popular method for producing interpretable
machine learning. It uses an interpretable "student" model to mimic the
predictions made by the black box "teacher" model. However, when the student
model is sensitive to the variability of the data sets used for training even
when keeping the teacher fixed, the corresponded interpretation is not
reliable. Existing strategies stabilize model distillation by checking whether
a large enough corpus of pseudo-data is generated to reliably reproduce student
models, but methods to do so have so far been developed for a specific student
model. In this paper, we develop a generic approach for stable model
distillation based on central limit theorem for the average loss. We start with
a collection of candidate student models and search for candidates that
reasonably agree with the teacher. Then we construct a multiple testing
framework to select a corpus size such that the consistent student model would
be selected under different pseudo samples. We demonstrate the application of
our proposed approach on three commonly used intelligible models: decision
trees, falling rule lists and symbolic regression. Finally, we conduct
simulation experiments on Mammographic Mass and Breast Cancer datasets and
illustrate the testing procedure throughout a theoretical analysis with Markov
process. The code is publicly available at
https://github.com/yunzhe-zhou/GenericDistillation.Comment: 31 pages, 8 figure
On the Intersection of Explainable and Reliable AI for physical fatigue prediction
In the era of Industry 4.0, the use of Artificial Intelligence (AI) is widespread in occupational settings. Since dealing with human safety, explainability and trustworthiness of AI are even more important than achieving high accuracy. eXplainable AI (XAI) is investigated in this paper to detect physical fatigue during manual material handling task simulation. Besides comparing global rule-based XAI models (LLM and DT) to black-box models (NN, SVM, XGBoost) in terms of performance, we also compare global models with local ones (LIME over XGBoost). Surprisingly, global and local approaches achieve similar conclusions, in terms of feature importance. Moreover, an expansion from local rules to global rules is designed for Anchors, by posing an appropriate optimization method (Anchors coverage is enlarged from an original low value, 11%, up to 43%). As far as trustworthiness is concerned, rule sensitivity analysis drives the identification of optimized regions in the feature space, where physical fatigue is predicted with zero statistical error. The discovery of such ânon-fatigue regionsâ helps certifying the organizational and clinical decision making
A survey of face recognition techniques under occlusion
The limited capacity to recognize faces under occlusions is a long-standing
problem that presents a unique challenge for face recognition systems and even
for humans. The problem regarding occlusion is less covered by research when
compared to other challenges such as pose variation, different expressions,
etc. Nevertheless, occluded face recognition is imperative to exploit the full
potential of face recognition for real-world applications. In this paper, we
restrict the scope to occluded face recognition. First, we explore what the
occlusion problem is and what inherent difficulties can arise. As a part of
this review, we introduce face detection under occlusion, a preliminary step in
face recognition. Second, we present how existing face recognition methods cope
with the occlusion problem and classify them into three categories, which are
1) occlusion robust feature extraction approaches, 2) occlusion aware face
recognition approaches, and 3) occlusion recovery based face recognition
approaches. Furthermore, we analyze the motivations, innovations, pros and
cons, and the performance of representative approaches for comparison. Finally,
future challenges and method trends of occluded face recognition are thoroughly
discussed
A survey of face recognition techniques under occlusion
The limited capacity to recognize faces under occlusions is a long-standing problem that presents a unique challenge for face recognition systems and even for humans. The problem regarding occlusion is less covered by research when compared to other challenges such as pose variation, different expressions, etc. Nevertheless, occluded face recognition is imperative to exploit the full potential of face recognition for real-world applications. In this paper, we restrict the scope to occluded face recognition. First, we explore what the occlusion problem is and what inherent difficulties can arise. As a part of this review, we introduce face detection under occlusion, a preliminary step in face recognition. Second, we present how existing face recognition methods cope with the occlusion problem and classify them into three categories, which are 1) occlusion robust feature extraction approaches, 2) occlusion aware face recognition approaches, and 3) occlusion recovery based face recognition approaches. Furthermore, we analyze the motivations, innovations, pros and cons, and the performance of representative approaches for comparison. Finally, future challenges and method trends of occluded face recognition are thoroughly discussed
Explanation of Siamese Neural Networks for Weakly Supervised Learning
A new method for explaining the Siamese neural network (SNN) as a black-box model for weakly supervised learning is proposed under condition that the output of every subnetwork of the SNN is a vector which is accessible. The main problem of the explanation is that the perturbation technique cannot be used directly for input instances because only their semantic similarity or dissimilarity is known. Moreover, there is no an "inverse" map between the SNN output vector and the corresponding input instance. Therefore, a special autoencoder is proposed, which takes into account the proximity of its hidden representation and the SNN outputs. Its pre-trained decoder part as well as the encoder are used to reconstruct original instances from the SNN perturbed output vectors. The important features of the explained instances are determined by averaging the corresponding changes of the reconstructed instances. Numerical experiments with synthetic data and with the well-known dataset MNIST illustrate the proposed method
Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification
Deep neural networks have achieved promising progress in remote sensing (RS)
image classification, for which the training process requires abundant samples
for each class. However, it is time-consuming and unrealistic to annotate
labels for each RS category, given the fact that the RS target database is
increasing dynamically. Zero-shot learning (ZSL) allows for identifying novel
classes that are not seen during training, which provides a promising solution
for the aforementioned problem. However, previous ZSL models mainly depend on
manually-labeled attributes or word embeddings extracted from language models
to transfer knowledge from seen classes to novel classes. Besides, pioneer ZSL
models use convolutional neural networks pre-trained on ImageNet, which focus
on the main objects appearing in each image, neglecting the background context
that also matters in RS scene classification. To address the above problems, we
propose to collect visually detectable attributes automatically. We predict
attributes for each class by depicting the semantic-visual similarity between
attributes and images. In this way, the attribute annotation process is
accomplished by machine instead of human as in other methods. Moreover, we
propose a Deep Semantic-Visual Alignment (DSVA) that take advantage of the
self-attention mechanism in the transformer to associate local image regions
together, integrating the background context information for prediction. The
DSVA model further utilizes the attribute attention maps to focus on the
informative image regions that are essential for knowledge transfer in ZSL, and
maps the visual images into attribute space to perform ZSL classification. With
extensive experiments, we show that our model outperforms other
state-of-the-art models by a large margin on a challenging large-scale RS scene
classification benchmark.Comment: Published in ISPRS P&RS. The code is available at
https://github.com/wenjiaXu/RS_Scene_ZS
Thinking inside The Box: Learning Hypercube Representations for Group Recommendation
As a step beyond traditional personalized recommendation, group
recommendation is the task of suggesting items that can satisfy a group of
users. In group recommendation, the core is to design preference aggregation
functions to obtain a quality summary of all group members' preferences. Such
user and group preferences are commonly represented as points in the vector
space (i.e., embeddings), where multiple user embeddings are compressed into
one to facilitate ranking for group-item pairs. However, the resulted group
representations, as points, lack adequate flexibility and capacity to account
for the multi-faceted user preferences. Also, the point embedding-based
preference aggregation is a less faithful reflection of a group's
decision-making process, where all users have to agree on a certain value in
each embedding dimension instead of a negotiable interval. In this paper, we
propose a novel representation of groups via the notion of hypercubes, which
are subspaces containing innumerable points in the vector space. Specifically,
we design the hypercube recommender (CubeRec) to adaptively learn group
hypercubes from user embeddings with minimal information loss during preference
aggregation, and to leverage a revamped distance metric to measure the affinity
between group hypercubes and item points. Moreover, to counteract the
long-standing issue of data sparsity in group recommendation, we make full use
of the geometric expressiveness of hypercubes and innovatively incorporate
self-supervision by intersecting two groups. Experiments on four real-world
datasets have validated the superiority of CubeRec over state-of-the-art
baselines.Comment: To appear in SIGIR'2
Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed
appropriately, may deliver the best of expectations over many application sectors across the field. For this
to occur shortly in Machine Learning, the entire community stands in front of the barrier of explainability,
an inherent problem of the latest techniques brought by sub-symbolism (e.g. ensembles or Deep Neural
Networks) that were not present in the last hype of AI (namely, expert systems and rule based models).
Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is widely
acknowledged as a crucial feature for the practical deployment of AI models. The overview presented in
this article examines the existing literature and contributions already done in the field of XAI, including a
prospect toward what is yet to be reached. For this purpose we summarize previous efforts made to define
explainability in Machine Learning, establishing a novel definition of explainable Machine Learning that
covers such prior conceptual propositions with a major focus on the audience for which the explainability
is sought. Departing from this definition, we propose and discuss about a taxonomy of recent contributions
related to the explainability of different Machine Learning models, including those aimed at explaining
Deep Learning methods for which a second dedicated taxonomy is built and examined in detail. This
critical literature analysis serves as the motivating background for a series of challenges faced by XAI,
such as the interesting crossroads of data fusion and explainability. Our prospects lead toward the concept
of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI
methods in real organizations with fairness, model explainability and accountability at its core. Our
ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve
as reference material in order to stimulate future research advances, but also to encourage experts and
professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any
prior bias for its lack of interpretability.Basque GovernmentConsolidated Research Group MATHMODE - Department of Education of the Basque Government IT1294-19Spanish GovernmentEuropean Commission TIN2017-89517-PBBVA Foundation through its Ayudas Fundacion BBVA a Equipos de Investigacion Cientifica 2018 call (DeepSCOP project)European Commission 82561
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI
In the last few years, Artificial Intelligence (AI) has achieved a notable momentum that, if harnessed appropriately, may deliver the best of expectations over many application sectors across the field. For this to occur shortly in Machine Learning, the entire community stands in front of the barrier of explainability, an inherent problem of the latest techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI (namely, expert systems and rule based models). Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is widely acknowledged as a crucial feature for the practical deployment of AI models. The overview presented in this article examines the existing literature and contributions already done in the field of XAI, including a prospect toward what is yet to be reached. For this purpose we summarize previous efforts made to define explainability in Machine Learning, establishing a novel definition of explainable Machine Learning that covers such prior conceptual propositions with a major focus on the audience for which the explainability is sought. Departing from this definition, we propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at explaining Deep Learning methods for which a second dedicated taxonomy is built and examined in detail. This critical literature analysis serves as the motivating background for a series of challenges faced by XAI, such as the interesting crossroads of data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve as reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability
- âŠ