490 research outputs found
Induction of Non-Monotonic Logic Programs to Explain Boosted Tree Models Using LIME
We present a heuristic based algorithm to induce \textit{nonmonotonic} logic
programs that will explain the behavior of XGBoost trained classifiers. We use
the technique based on the LIME approach to locally select the most important
features contributing to the classification decision. Then, in order to explain
the model's global behavior, we propose the LIME-FOLD algorithm ---a
heuristic-based inductive logic programming (ILP) algorithm capable of learning
non-monotonic logic programs---that we apply to a transformed dataset produced
by LIME. Our proposed approach is agnostic to the choice of the ILP algorithm.
Our experiments with UCI standard benchmarks suggest a significant improvement
in terms of classification evaluation metrics. Meanwhile, the number of induced
rules dramatically decreases compared to ALEPH, a state-of-the-art ILP system
Fast Private Data Release Algorithms for Sparse Queries
We revisit the problem of accurately answering large classes of statistical
queries while preserving differential privacy. Previous approaches to this
problem have either been very general but have not had run-time polynomial in
the size of the database, have applied only to very limited classes of queries,
or have relaxed the notion of worst-case error guarantees. In this paper we
consider the large class of sparse queries, which take non-zero values on only
polynomially many universe elements. We give efficient query release algorithms
for this class, in both the interactive and the non-interactive setting. Our
algorithms also achieve better accuracy bounds than previous general techniques
do when applied to sparse queries: our bounds are independent of the universe
size. In fact, even the runtime of our interactive mechanism is independent of
the universe size, and so can be implemented in the "infinite universe" model
in which no finite universe need be specified by the data curator
Classifiers With a Reject Option for Early Time-Series Classification
Early classification of time-series data in a dynamic environment is a
challenging problem of great importance in signal processing. This paper
proposes a classifier architecture with a reject option capable of online
decision making without the need to wait for the entire time series signal to
be present. The main idea is to classify an odor/gas signal with an acceptable
accuracy as early as possible. Instead of using posterior probability of a
classifier, the proposed method uses the "agreement" of an ensemble to decide
whether to accept or reject the candidate label. The introduced algorithm is
applied to the bio-chemistry problem of odor classification to build a novel
Electronic-Nose called Forefront-Nose. Experimental results on wind tunnel
test-bed facility confirms the robustness of the forefront-nose compared to the
standard classifiers from both earliness and recognition perspectives
Effective Classification using a small Training Set based on Discretization and Statistical Analysis
This work deals with the problem of producing a fast and accurate data classification, learning it from a possibly small set of records that are already classified. The proposed approach is based on the framework of the so-called Logical Analysis of Data (LAD), but enriched with information obtained from statistical considerations on the data. A number of discrete optimization problems are solved in the different steps of the procedure, but their computational demand can be controlled. The accuracy of the proposed approach is compared to that of the standard LAD algorithm, of Support Vector Machines and of Label Propagation algorithm on publicly available datasets of the UCI repository. Encouraging results are obtained and discusse
QCBA: Postoptimization of Quantitative Attributes in Classifiers based on Association Rules
The need to prediscretize numeric attributes before they can be used in
association rule learning is a source of inefficiencies in the resulting
classifier. This paper describes several new rule tuning steps aiming to
recover information lost in the discretization of numeric (quantitative)
attributes, and a new rule pruning strategy, which further reduces the size of
the classification models. We demonstrate the effectiveness of the proposed
methods on postoptimization of models generated by three state-of-the-art
association rule classification algorithms: Classification based on
Associations (Liu, 1998), Interpretable Decision Sets (Lakkaraju et al, 2016),
and Scalable Bayesian Rule Lists (Yang, 2017). Benchmarks on 22 datasets from
the UCI repository show that the postoptimized models are consistently smaller
-- typically by about 50% -- and have better classification performance on most
datasets
Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability
Despite significant effort, building models that are both interpretable and
accurate is an unresolved challenge for many pattern recognition problems. In
general, rule-based and linear models lack accuracy, while deep learning
interpretability is based on rough approximations of the underlying inference.
Using a linear combination of conjunctive clauses in propositional logic,
Tsetlin Machines (TMs) have shown competitive performance on diverse
benchmarks. However, to do so, many clauses are needed, which impacts
interpretability. Here, we address the accuracy-interpretability challenge in
machine learning by equipping the TM clauses with integer weights. The
resulting Integer Weighted TM (IWTM) deals with the problem of learning which
clauses are inaccurate and thus must team up to obtain high accuracy as a team
(low weight clauses), and which clauses are sufficiently accurate to operate
more independently (high weight clauses). Since each TM clause is formed
adaptively by a team of Tsetlin Automata, identifying effective weights becomes
a challenging online learning problem. We address this problem by extending
each team of Tsetlin Automata with a stochastic searching on the line (SSL)
automaton. In our novel scheme, the SSL automaton learns the weight of its
clause in interaction with the corresponding Tsetlin Automata team, which, in
turn, adapts the composition of the clause by the adjusting weight. We evaluate
IWTM empirically using five datasets, including a study of interpetability. On
average, IWTM uses 6.5 times fewer literals than the vanilla TM and 120 times
fewer literals than a TM with real-valued weights. Furthermore, in terms of
average F1-Score, IWTM outperforms simple Multi-Layered Artificial Neural
Networks, Decision Trees, Support Vector Machines, K-Nearest Neighbor, Random
Forest, XGBoost, Explainable Boosting Machines, and standard and real-value
weighted TMs.Comment: 20 pages, 10 figure
- …