122 research outputs found
Which Learning Algorithms Can Generalize Identity-Based Rules to Novel Inputs?
We propose a novel framework for the analysis of learning algorithms that
allows us to say when such algorithms can and cannot generalize certain
patterns from training data to test data. In particular we focus on situations
where the rule that must be learned concerns two components of a stimulus being
identical. We call such a basis for discrimination an identity-based rule.
Identity-based rules have proven to be difficult or impossible for certain
types of learning algorithms to acquire from limited datasets. This is in
contrast to human behaviour on similar tasks. Here we provide a framework for
rigorously establishing which learning algorithms will fail at generalizing
identity-based rules to novel stimuli. We use this framework to show that such
algorithms are unable to generalize identity-based rules to novel inputs unless
trained on virtually all possible inputs. We demonstrate these results
computationally with a multilayer feedforward neural network.Comment: 6 pages, accepted abstract at COGSCI 201
Diversities and the Geometry of Hypergraphs
The embedding of finite metrics in has become a fundamental tool for
both combinatorial optimization and large-scale data analysis. One important
application is to network flow problems in which there is close relation
between max-flow min-cut theorems and the minimal distortion embeddings of
metrics into . Here we show that this theory can be generalized
considerably to encompass Steiner tree packing problems in both graphs and
hypergraphs. Instead of the theory of metrics and minimal distortion
embeddings, the parallel is the theory of diversities recently introduced by
Bryant and Tupper, and the corresponding theory of diversities and
embeddings which we develop here.Comment: 19 pages, no figures. This version: further small correction
Constant distortion embeddings of Symmetric Diversities
Diversities are like metric spaces, except that every finite subset, instead
of just every pair of points, is assigned a value. Just as there is a theory of
minimal distortion embeddings of finite metric spaces into , there is a
similar, yet undeveloped, theory for embedding finite diversities into the
diversity analogue of spaces. In the metric case, it is well known that
an -point metric space can be embedded into with
distortion. For diversities, the optimal distortion is unknown. Here, we
establish the surprising result that symmetric diversities, those in which the
diversity (value) assigned to a set depends only on its cardinality, can be
embedded in with constant distortion.Comment: 14 pages, 3 figure
The Relation between Approximation in Distribution and Shadowing in Molecular Dynamics
Molecular dynamics refers to the computer simulation of a material at the
atomic level. An open problem in numerical analysis is to explain the apparent
reliability of molecular dynamics simulations. The difficulty is that
individual trajectories computed in molecular dynamics are accurate for only
short time intervals, whereas apparently reliable information can be extracted
from very long-time simulations. It has been conjectured that long molecular
dynamics trajectories have low-dimensional statistical features that accurately
approximate those of the original system. Another conjecture is that numerical
trajectories satisfy the shadowing property: that they are close over long time
intervals to exact trajectories but with different initial conditions. We prove
that these two views are actually equivalent to each other, after we suitably
modify the concept of shadowing. A key ingredient of our result is a general
theorem that allows us to take random elements of a metric space that are close
in distribution and embed them in the same probability space so that they are
close in a strong sense. This result is similar to the Strassen-Dudley Theorem
except that a mapping is provided between the two random elements. Our results
on shadowing are motivated by molecular dynamics but apply to the approximation
of any dynamical system when initial conditions are selected according to a
probability measure.Comment: 21 pages, final version accepted in SIAM Dyn Sy
- β¦