147,487 research outputs found
Learning using Local Membership Queries
We introduce a new model of membership query (MQ) learning, where the
learning algorithm is restricted to query points that are \emph{close} to
random examples drawn from the underlying distribution. The learning model is
intermediate between the PAC model (Valiant, 1984) and the PAC+MQ model (where
the queries are allowed to be arbitrary points).
Membership query algorithms are not popular among machine learning
practitioners. Apart from the obvious difficulty of adaptively querying
labelers, it has also been observed that querying \emph{unnatural} points leads
to increased noise from human labelers (Lang and Baum, 1992). This motivates
our study of learning algorithms that make queries that are close to examples
generated from the data distribution.
We restrict our attention to functions defined on the -dimensional Boolean
hypercube and say that a membership query is local if its Hamming distance from
some example in the (random) training data is at most . We show the
following results in this model:
(i) The class of sparse polynomials (with coefficients in R) over
is polynomial time learnable under a large class of \emph{locally smooth}
distributions using -local queries. This class also includes the
class of -depth decision trees.
(ii) The class of polynomial-sized decision trees is polynomial time
learnable under product distributions using -local queries.
(iii) The class of polynomial size DNF formulas is learnable under the
uniform distribution using -local queries in time
.
(iv) In addition we prove a number of results relating the proposed model to
the traditional PAC model and the PAC+MQ model
Properly Learning Decision Trees with Queries Is NP-Hard
We prove that it is NP-hard to properly PAC learn decision trees with
queries, resolving a longstanding open problem in learning theory (Bshouty
1993; Guijarro-Lavin-Raghavan 1999; Mehta-Raghavan 2002; Feldman 2016). While
there has been a long line of work, dating back to (Pitt-Valiant 1988),
establishing the hardness of properly learning decision trees from random
examples, the more challenging setting of query learners necessitates different
techniques and there were no previous lower bounds. En route to our main
result, we simplify and strengthen the best known lower bounds for a different
problem of Decision Tree Minimization (Zantema-Bodlaender 2000; Sieling 2003).
On a technical level, we introduce the notion of hardness distillation, which
we study for decision tree complexity but can be considered for any complexity
measure: for a function that requires large decision trees, we give a general
method for identifying a small set of inputs that is responsible for its
complexity. Our technique even rules out query learners that are allowed
constant error. This contrasts with existing lower bounds for the setting of
random examples which only hold for inverse-polynomial error.
Our result, taken together with a recent almost-polynomial time query
algorithm for properly learning decision trees under the uniform distribution
(Blanc-Lange-Qiao-Tan 2022), demonstrates the dramatic impact of distributional
assumptions on the problem.Comment: 41 pages, 10 figures, FOCS 202
Medical-based Deep Curriculum Learning for Improved Fracture Classification
International audienceAbstract. Current deep-learning-based methods do not easily integrate into clinical protocols, neither take full advantage of medical knowledge.In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracturefrom X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledgesuch as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of diculty to eachtraining sample. We demonstrate that if we start learning \easy" examples and move towards \hard", the model can reach better performance,even with fewer data. The evaluation is performed on the classification of a clinical dataset of about 1000 X-ray images. Our results show that,compared to class-uniform and random strategies, the proposed medical knowledge-based curriculum, performs up to 15% better in terms ofaccuracy, achieving the performance of experienced trauma surgeons. Keywords: Curriculum learning, multi-label classification, bone fractures, computer-aided diagnosis, medical decision tre
A System for Induction of Oblique Decision Trees
This article describes a new system for induction of oblique decision trees.
This system, OC1, combines deterministic hill-climbing with two forms of
randomization to find a good oblique split (in the form of a hyperplane) at
each node of a decision tree. Oblique decision tree methods are tuned
especially for domains in which the attributes are numeric, although they can
be adapted to symbolic or mixed symbolic/numeric attributes. We present
extensive empirical studies, using both real and artificial data, that analyze
OC1's ability to construct oblique trees that are smaller and more accurate
than their axis-parallel counterparts. We also examine the benefits of
randomization for the construction of oblique decision trees.Comment: See http://www.jair.org/ for an online appendix and other files
accompanying this articl
- …