115 research outputs found
Decision Stream: Cultivating Deep Decision Trees
Various modifications of decision trees have been extensively used during the
past years due to their high efficiency and interpretability. Tree node
splitting based on relevant feature selection is a key step of decision tree
learning, at the same time being their major shortcoming: the recursive nodes
partitioning leads to geometric reduction of data quantity in the leaf nodes,
which causes an excessive model complexity and data overfitting. In this paper,
we present a novel architecture - a Decision Stream, - aimed to overcome this
problem. Instead of building a tree structure during the learning process, we
propose merging nodes from different branches based on their similarity that is
estimated with two-sample test statistics, which leads to generation of a deep
directed acyclic graph of decision rules that can consist of hundreds of
levels. To evaluate the proposed solution, we test it on several common machine
learning problems - credit scoring, twitter sentiment analysis, aircraft flight
control, MNIST and CIFAR image classification, synthetic data classification
and regression. Our experimental results reveal that the proposed approach
significantly outperforms the standard decision tree learning methods on both
regression and classification tasks, yielding a prediction error decrease up to
35%
Can FCA-based Recommender System Suggest a Proper Classifier?
The paper briefly introduces multiple classifier systems and describes a new
algorithm, which improves classification accuracy by means of recommendation of
a proper algorithm to an object classification. This recommendation is done
assuming that a classifier is likely to predict the label of the object
correctly if it has correctly classified its neighbors. The process of
assigning a classifier to each object is based on Formal Concept Analysis. We
explain the idea of the algorithm with a toy example and describe our first
experiments with real-world datasets.Comment: 10 pages, 1 figure, 4 tables, ECAI 2014, workshop "What FCA can do
for "Artifficial Intelligence
- …