1,558 research outputs found
Classification and Geometry of General Perceptual Manifolds
Perceptual manifolds arise when a neural population responds to an ensemble
of sensory signals associated with different physical features (e.g.,
orientation, pose, scale, location, and intensity) of the same perceptual
object. Object recognition and discrimination requires classifying the
manifolds in a manner that is insensitive to variability within a manifold. How
neuronal systems give rise to invariant object classification and recognition
is a fundamental problem in brain theory as well as in machine learning. Here
we study the ability of a readout network to classify objects from their
perceptual manifold representations. We develop a statistical mechanical theory
for the linear classification of manifolds with arbitrary geometry revealing a
remarkable relation to the mathematics of conic decomposition. Novel
geometrical measures of manifold radius and manifold dimension are introduced
which can explain the classification capacity for manifolds of various
geometries. The general theory is demonstrated on a number of representative
manifolds, including L2 ellipsoids prototypical of strictly convex manifolds,
L1 balls representing polytopes consisting of finite sample points, and
orientation manifolds which arise from neurons tuned to respond to a continuous
angle variable, such as object orientation. The effects of label sparsity on
the classification capacity of manifolds are elucidated, revealing a scaling
relation between label sparsity and manifold radius. Theoretical predictions
are corroborated by numerical simulations using recently developed algorithms
to compute maximum margin solutions for manifold dichotomies. Our theory and
its extensions provide a powerful and rich framework for applying statistical
mechanics of linear classification to data arising from neuronal responses to
object stimuli, as well as to artificial deep networks trained for object
recognition tasks.Comment: 24 pages, 12 figures, Supplementary Material
Analysis of approximate nearest neighbor searching with clustered point sets
We present an empirical analysis of data structures for approximate nearest
neighbor searching. We compare the well-known optimized kd-tree splitting
method against two alternative splitting methods. The first, called the
sliding-midpoint method, which attempts to balance the goals of producing
subdivision cells of bounded aspect ratio, while not producing any empty cells.
The second, called the minimum-ambiguity method is a query-based approach. In
addition to the data points, it is also given a training set of query points
for preprocessing. It employs a simple greedy algorithm to select the splitting
plane that minimizes the average amount of ambiguity in the choice of the
nearest neighbor for the training points. We provide an empirical analysis
comparing these two methods against the optimized kd-tree construction for a
number of synthetically generated data and query sets. We demonstrate that for
clustered data and query sets, these algorithms can provide significant
improvements over the standard kd-tree construction for approximate nearest
neighbor searching.Comment: 20 pages, 8 figures. Presented at ALENEX '99, Baltimore, MD, Jan
15-16, 199
Symmetric confidence regions and confidence intervals for normal map formulations of stochastic variational inequalities
Stochastic variational inequalities (SVI) model a large class of equilibrium
problems subject to data uncertainty, and are closely related to stochastic
optimization problems. The SVI solution is usually estimated by a solution to a
sample average approximation (SAA) problem. This paper considers the normal map
formulation of an SVI, and proposes a method to build asymptotically exact
confidence regions and confidence intervals for the solution of the normal map
formulation, based on the asymptotic distribution of SAA solutions. The
confidence regions are single ellipsoids with high probability. We also discuss
the computation of simultaneous and individual confidence intervals
Accelerated Parameter Estimation with DALE
We consider methods for improving the estimation of constraints on a
high-dimensional parameter space with a computationally expensive likelihood
function. In such cases Markov chain Monte Carlo (MCMC) can take a long time to
converge and concentrates on finding the maxima rather than the often-desired
confidence contours for accurate error estimation. We employ DALE (Direct
Analysis of Limits via the Exterior of ) for determining confidence
contours by minimizing a cost function parametrized to incentivize points in
parameter space which are both on the confidence limit and far from previously
sampled points. We compare DALE to the nested sampling algorithm
implemented in MultiNest on a toy likelihood function that is highly
non-Gaussian and non-linear in the mapping between parameter values and
. We find that in high-dimensional cases DALE finds the same
confidence limit as MultiNest using roughly an order of magnitude fewer
evaluations of the likelihood function. DALE is open-source and available
at https://github.com/danielsf/Dalex.git
- …