1,112 research outputs found
A System for Induction of Oblique Decision Trees
This article describes a new system for induction of oblique decision trees.
This system, OC1, combines deterministic hill-climbing with two forms of
randomization to find a good oblique split (in the form of a hyperplane) at
each node of a decision tree. Oblique decision tree methods are tuned
especially for domains in which the attributes are numeric, although they can
be adapted to symbolic or mixed symbolic/numeric attributes. We present
extensive empirical studies, using both real and artificial data, that analyze
OC1's ability to construct oblique trees that are smaller and more accurate
than their axis-parallel counterparts. We also examine the benefits of
randomization for the construction of oblique decision trees.Comment: See http://www.jair.org/ for an online appendix and other files
accompanying this articl
Deep Learning without Poor Local Minima
In this paper, we prove a conjecture published in 1989 and also partially
address an open problem announced at the Conference on Learning Theory (COLT)
2015. With no unrealistic assumption, we first prove the following statements
for the squared loss function of deep linear neural networks with any depth and
any widths: 1) the function is non-convex and non-concave, 2) every local
minimum is a global minimum, 3) every critical point that is not a global
minimum is a saddle point, and 4) there exist "bad" saddle points (where the
Hessian has no negative eigenvalue) for the deeper networks (with more than
three layers), whereas there is no bad saddle point for the shallow networks
(with three layers). Moreover, for deep nonlinear neural networks, we prove the
same four statements via a reduction to a deep linear model under the
independence assumption adopted from recent work. As a result, we present an
instance, for which we can answer the following question: how difficult is it
to directly train a deep model in theory? It is more difficult than the
classical machine learning models (because of the non-convexity), but not too
difficult (because of the nonexistence of poor local minima). Furthermore, the
mathematically proven existence of bad saddle points for deeper models would
suggest a possible open problem. We note that even though we have advanced the
theoretical foundations of deep learning and non-convex optimization, there is
still a gap between theory and practice.Comment: In NIPS 2016. Selected for NIPS oral presentation (top 2%
submissions). ---- The final NIPS 2016 version: the results remain the sam
Flow Navigation by Smart Microswimmers via Reinforcement Learning
Smart active particles can acquire some limited knowledge of the fluid
environment from simple mechanical cues and exert a control on their preferred
steering direction. Their goal is to learn the best way to navigate by
exploiting the underlying flow whenever possible. As an example, we focus our
attention on smart gravitactic swimmers. These are active particles whose task
is to reach the highest altitude within some time horizon, given the
constraints enforced by fluid mechanics. By means of numerical experiments, we
show that swimmers indeed learn nearly optimal strategies just by experience. A
reinforcement learning algorithm allows particles to learn effective strategies
even in difficult situations when, in the absence of control, they would end up
being trapped by flow structures. These strategies are highly nontrivial and
cannot be easily guessed in advance. This Letter illustrates the potential of
reinforcement learning algorithms to model adaptive behavior in complex flows
and paves the way towards the engineering of smart microswimmers that solve
difficult navigation problems.Comment: Published on Physical Review Letters (April 12, 2017
- …