On the Sample Complexity of Predictive Sparse Coding

The goal of predictive sparse coding is to learn a representation of examples as sparse linear combinations of elements from a dictionary, such that a learned hypothesis linear in the new representation performs well on a predictive task. Predictive sparse coding algorithms recently have demonstrated impressive performance on a variety of supervised tasks, but their generalization properties have not been studied. We establish the first generalization error bounds for predictive sparse coding, covering two settings: 1) the overcomplete setting, where the number of features k exceeds the original dimensionality d; and 2) the high or infinite-dimensional setting, where only dimension-free bounds are useful. Both learning bounds intimately depend on stability properties of the learned sparse encoder, as measured on the training sample. Consequently, we first present a fundamental stability result for the LASSO, a result characterizing the stability of the sparse codes with respect to perturbations to the dictionary. In the overcomplete setting, we present an estimation error bound that decays as \tilde{O}(sqrt(d k/m)) with respect to d and k. In the high or infinite-dimensional setting, we show a dimension-free bound that is \tilde{O}(sqrt(k^2 s / m)) with respect to k and s, where s is an upper bound on the number of non-zeros in the sparse code for any training data point.Comment: Sparse Coding Stability Theorem from version 1 has been relaxed considerably using a new notion of coding margin. Old Sparse Coding Stability Theorem still in new version, now as Theorem 2. Presentation of all proofs simplified/improved considerably. Paper reorganized. Empirical analysis showing new coding margin is non-trivial on real dataset

Glassy dynamics in granular compaction

Two models are presented to study the influence of slow dynamics on granular compaction. It is found in both cases that high values of packing fraction are achieved only by the slow relaxation of cooperative structures. Ongoing work to study the full implications of these results is discussed.Comment: 12 pages, 9 figures; accepted in J. Phys: Condensed Matter, proceedings of the Trieste workshop on 'Unifying concepts in glass physics

Multi-particle structures in non-sequentially reorganized hard sphere deposits

We have examined extended structures, bridges and arches, in computer generated, non-sequentially stabilized, hard sphere deposits. The bridges and arches have well defined distributions of sizes and shapes. The distribution functions reflect the contraints associated with hard particle packing and the details of the restructuring process. A subpopulation of string-like bridges has been identified. Bridges are fundamental microstructural elements in real granular systems and their sizes and shapes dominate considerations of structural properties and flow instabilities such as jamming.Comment: 9 pages, 7 figure

A two-species model of a two-dimensional sandpile surface: a case of asymptotic roughening

We present and analyze a model of an evolving sandpile surface in (2 + 1) dimensions where the dynamics of mobile grains ({\rho}(x, t)) and immobile clusters (h(x, t)) are coupled. Our coupling models the situation where the sandpile is flat on average, so that there is no bias due to gravity. We find anomalous scaling: the expected logarithmic smoothing at short length and time scales gives way to roughening in the asymptotic limit, where novel and non-trivial exponents are found.Comment: 7 Pages, 6 Figures; Granular Matter, 2012 (Online

Interacting Multiple Model-Feedback Particle Filter for Stochastic Hybrid Systems

In this paper, a novel feedback control-based particle filter algorithm for the continuous-time stochastic hybrid system estimation problem is presented. This particle filter is referred to as the interacting multiple model-feedback particle filter (IMM-FPF), and is based on the recently developed feedback particle filter. The IMM-FPF is comprised of a series of parallel FPFs, one for each discrete mode, and an exact filter recursion for the mode association probability. The proposed IMM-FPF represents a generalization of the Kalmanfilter based IMM algorithm to the general nonlinear filtering problem. The remarkable conclusion of this paper is that the IMM-FPF algorithm retains the innovation error-based feedback structure even for the nonlinear problem. The interaction/merging process is also handled via a control-based approach. The theoretical results are illustrated with the aid of a numerical example problem for a maneuvering target tracking application

Competition and cooperation:aspects of dynamics in sandpiles

In this article, we review some of our approaches to granular dynamics, now well known to consist of both fast and slow relaxational processes. In the first case, grains typically compete with each other, while in the second, they cooperate. A typical result of {\it cooperation} is the formation of stable bridges, signatures of spatiotemporal inhomogeneities; we review their geometrical characteristics and compare theoretical results with those of independent simulations. {\it Cooperative} excitations due to local density fluctuations are also responsible for relaxation at the angle of repose; the {\it competition} between these fluctuations and external driving forces, can, on the other hand, result in a (rare) collapse of the sandpile to the horizontal. Both these features are present in a theory reviewed here. An arena where the effects of cooperation versus competition are felt most keenly is granular compaction; we review here a random graph model, where three-spin interactions are used to model compaction under tapping. The compaction curve shows distinct regions where 'fast' and 'slow' dynamics apply, separated by what we have called the {\it single-particle relaxation threshold}. In the final section of this paper, we explore the effect of shape -- jagged vs. regular -- on the compaction of packings near their jamming limit. One of our major results is an entropic landscape that, while microscopically rough, manifests {\it Edwards' flatness} at a macroscopic level. Another major result is that of surface intermittency under low-intensity shaking.Comment: 36 pages, 23 figures, minor correction

Shaking a Box of Sand

We present a simple model of a vibrated box of sand, and discuss its dynamics in terms of two parameters reflecting static and dynamic disorder respectively. The fluidised, intermediate and frozen (`glassy') dynamical regimes are extensively probed by analysing the response of the packing fraction to steady, as well as cyclic, shaking, and indicators of the onset of a `glass transition' are analysed. In the `glassy' regime, our model is exactly solvable, and allows for the qualitative description of ageing phenomena in terms of two characteristic lengths; predictions are also made about the influence of grain shape anisotropy on ageing behaviour.Comment: Revised version. To appear in Europhysics Letter

Improved Perturbation Theory for Improved Lattice Actions

We study a systematic improvement of perturbation theory for gauge fields on the lattice; the improvement entails resumming, to all orders in the coupling constant, a dominant subclass of tadpole diagrams. This method, originally proposed for the Wilson gluon action, is extended here to encompass all possible gluon actions made of closed Wilson loops; any fermion action can be employed as well. The effect of resummation is to replace various parameters in the action (coupling constant, Symanzik coefficients, clover coefficient) by ``dressed'' values; the latter are solutions to certain coupled integral equations, which are easy to solve numerically. Some positive features of this method are: a) It is gauge invariant, b) it can be systematically applied to improve (to all orders) results obtained at any given order in perturbation theory, c) it does indeed absorb in the dressed parameters the bulk of tadpole contributions. Two different applications are presented: The additive renormalization of fermion masses, and the multiplicative renormalization Z_V (Z_A) of the vector (axial) current. In many cases where non-perturbative estimates of renormalization functions are also available for comparison, the agreement with improved perturbative results is significantly better as compared to results from bare perturbation theory.Comment: 17 pages, 3 tables, 6 figure

On random graphs and the statistical mechanics of granular matter

The dynamics of spins on a random graph with ferromagnetic three-spin interactions is used to model the compaction of granular matter under a series of taps. Taps are modelled as the random flipping of a small fraction of the spins followed by a quench at zero temperature. We find that the density approached during a logarithmically slow compaction - the random-close-packing density - corresponds to a dynamical phase transition. We discuss the the role of cascades of successive spin-flips in this model and link them with density-noise power fluctuations observed in recent experiments.Comment: minor changes, to appear in EP