22 research outputs found
Detecting adversarial manipulation using inductive Venn-ABERS predictors
Inductive Venn-ABERS predictors (IVAPs) are a type of probabilistic predictors with the theoretical guarantee that their predictions are perfectly calibrated. In this paper, we propose to exploit this calibration property for the detection of adversarial examples in binary classification tasks. By rejecting predictions if the uncertainty of the IVAP is too high, we obtain an algorithm that is both accurate on the original test set and resistant to adversarial examples. This robustness is observed on adversarials for the underlying model as well as adversarials that were generated by taking the IVAP into account. The method appears to offer competitive robustness compared to the state-of-the-art in adversarial defense yet it is computationally much more tractable
Calibration of Natural Language Understanding Models with Venn--ABERS Predictors
Transformers, currently the state-of-the-art in natural language
understanding (NLU) tasks, are prone to generate uncalibrated predictions or
extreme probabilities, making the process of taking different decisions based
on their output relatively difficult. In this paper we propose to build several
inductive Venn--ABERS predictors (IVAP), which are guaranteed to be well
calibrated under minimal assumptions, based on a selection of pre-trained
transformers. We test their performance over a set of diverse NLU tasks and
show that they are capable of producing well-calibrated probabilistic
predictions that are uniformly spread over the [0,1] interval -- all while
retaining the original model's predictive accuracy.Comment: Accepted at the 11th Symposium on Conformal and Probabilistic
Prediction with Applications - COPA 202
Calibrated Explanations: with Uncertainty Information and Counterfactuals
While local explanations for AI models can offer insights into individual
predictions, such as feature importance, they are plagued by issues like
instability. The unreliability of feature weights, often skewed due to poorly
calibrated ML models, deepens these challenges. Moreover, the critical aspect
of feature importance uncertainty remains mostly unaddressed in Explainable AI
(XAI). The novel feature importance explanation method presented in this paper,
called Calibrated Explanations (CE), is designed to tackle these issues
head-on. Built on the foundation of Venn-Abers, CE not only calibrates the
underlying model but also delivers reliable feature importance explanations
with an exact definition of the feature weights. CE goes beyond conventional
solutions by addressing output uncertainty. It accomplishes this by providing
uncertainty quantification for both feature weights and the model's probability
estimates. Additionally, CE is model-agnostic, featuring easily comprehensible
conditional rules and the ability to generate counterfactual explanations with
embedded uncertainty quantification. Results from an evaluation with 25
benchmark datasets underscore the efficacy of CE, making it stand as a fast,
reliable, stable, and robust solution.Comment: 19 pages, 6 figures, 3 tables, submitted to journa
Computationally efficient versions of conformal predictive distributions
Conformal predictive systems are a recent modification of conformal
predictors that output, in regression problems, probability distributions for
labels of test observations rather than set predictions. The extra information
provided by conformal predictive systems may be useful, e.g., in decision
making problems. Conformal predictive systems inherit the relative
computational inefficiency of conformal predictors. In this paper we discuss
two computationally efficient versions of conformal predictive systems, which
we call split conformal predictive systems and cross-conformal predictive
systems. The main advantage of split conformal predictive systems is their
guaranteed validity, whereas for cross-conformal predictive systems validity
only holds empirically and in the absence of excessive randomization. The main
advantage of cross-conformal predictive systems is their greater predictive
efficiency.Comment: 31 pages, 14 figures, 1 table. The conference version published in
the Proceedings of COPA 2018, and the journal version is to appear in
Neurocomputin
Prescience:Probabilistic Guidance on the Retraining Conundrum for Malware Detection
Malware evolves perpetually and relies on increasingly sophisticatedattacks to supersede defense strategies. Datadrivenapproaches to malware detection run the risk of becomingrapidly antiquated. Keeping pace with malwarerequires models that are periodically enriched with freshknowledge, commonly known as retraining. In this work,we propose the use of Venn-Abers predictors for assessingthe quality of binary classification tasks as a first step towardsidentifying antiquated models. One of the key bene-fits behind the use of Venn-Abers predictors is that they areautomatically well calibrated and offer probabilistic guidanceon the identification of nonstationary populations ofmalware. Our framework is agnostic to the underlying classificationalgorithm and can then be used for building betterretraining strategies in the presence of concept drift. Resultsobtained over a timeline-based evaluation with about 90Ksamples show that our framework can identify when modelstend to become obsolete