Classification and Geometry of General Perceptual Manifolds

Perceptual manifolds arise when a neural population responds to an ensemble of sensory signals associated with different physical features (e.g., orientation, pose, scale, location, and intensity) of the same perceptual object. Object recognition and discrimination requires classifying the manifolds in a manner that is insensitive to variability within a manifold. How neuronal systems give rise to invariant object classification and recognition is a fundamental problem in brain theory as well as in machine learning. Here we study the ability of a readout network to classify objects from their perceptual manifold representations. We develop a statistical mechanical theory for the linear classification of manifolds with arbitrary geometry revealing a remarkable relation to the mathematics of conic decomposition. Novel geometrical measures of manifold radius and manifold dimension are introduced which can explain the classification capacity for manifolds of various geometries. The general theory is demonstrated on a number of representative manifolds, including L2 ellipsoids prototypical of strictly convex manifolds, L1 balls representing polytopes consisting of finite sample points, and orientation manifolds which arise from neurons tuned to respond to a continuous angle variable, such as object orientation. The effects of label sparsity on the classification capacity of manifolds are elucidated, revealing a scaling relation between label sparsity and manifold radius. Theoretical predictions are corroborated by numerical simulations using recently developed algorithms to compute maximum margin solutions for manifold dichotomies. Our theory and its extensions provide a powerful and rich framework for applying statistical mechanics of linear classification to data arising from neuronal responses to object stimuli, as well as to artificial deep networks trained for object recognition tasks.Comment: 24 pages, 12 figures, Supplementary Material

On the accuracy of solving confluent Prony systems

In this paper we consider several nonlinear systems of algebraic equations which can be called "Prony-type". These systems arise in various reconstruction problems in several branches of theoretical and applied mathematics, such as frequency estimation and nonlinear Fourier inversion. Consequently, the question of stability of solution with respect to errors in the right-hand side becomes critical for the success of any particular application. We investigate the question of "maximal possible accuracy" of solving Prony-type systems, putting stress on the "local" behavior which approximates situations with low absolute measurement error. The accuracy estimates are formulated in very simple geometric terms, shedding some light on the structure of the problem. Numerical tests suggest that "global" solution techniques such as Prony's algorithm and ESPRIT method are suboptimal when compared to this theoretical "best local" behavior

Short-Term Memory in Orthogonal Neural Networks

We study the ability of linear recurrent networks obeying discrete time dynamics to store long temporal sequences that are retrievable from the instantaneous state of the network. We calculate this temporal memory capacity for both distributed shift register and random orthogonal connectivity matrices. We show that the memory capacity of these networks scales with system size.Comment: 4 pages, 4 figures, to be published in Phys. Rev. Let

Swelling of particle-encapsulating random manifolds

We study the statistical mechanics of a closed random manifold of fixed area and fluctuating volume, encapsulating a fixed number of noninteracting particles. Scaling analysis yields a unified description of such swollen manifolds, according to which the mean volume gradually increases with particle number, following a single scaling law. This is markedly different from the swelling under fixed pressure difference, where certain models exhibit criticality. We thereby indicate when the swelling due to encapsulated particles is thermodynamically inequivalent to that caused by fixed pressure. The general predictions are supported by Monte Carlo simulations of two particle-encapsulating model systems -- a two-dimensional self-avoiding ring and a three-dimensional self-avoiding fluid vesicle. In the former the particle-induced swelling is thermodynamically equivalent to the pressure-induced one whereas in the latter it is not.Comment: 8 pages, 6 figure