54,087 research outputs found
Stochastic Optimization for Deep CCA via Nonlinear Orthogonal Iterations
Deep CCA is a recently proposed deep neural network extension to the
traditional canonical correlation analysis (CCA), and has been successful for
multi-view representation learning in several domains. However, stochastic
optimization of the deep CCA objective is not straightforward, because it does
not decouple over training examples. Previous optimizers for deep CCA are
either batch-based algorithms or stochastic optimization using large
minibatches, which can have high memory consumption. In this paper, we tackle
the problem of stochastic optimization for deep CCA with small minibatches,
based on an iterative solution to the CCA objective, and show that we can
achieve as good performance as previous optimizers and thus alleviate the
memory requirement.Comment: in 2015 Annual Allerton Conference on Communication, Control and
Computin
Canonical and non-canonical equilibrium distribution
We address the problem of the dynamical foundation of non-canonical
equilibrium. We consider, as a source of divergence from ordinary statistical
mechanics, the breakdown of the condition of time scale separation between
microscopic and macroscopic dynamics. We show that this breakdown has the
effect of producing a significant deviation from the canonical prescription. We
also show that, while the canonical equilibrium can be reached with no apparent
dependence on dynamics, the specific form of non-canonical equilibrium is, in
fact, determined by dynamics. We consider the special case where the thermal
reservoir driving the system of interest to equilibrium is a generator of
intermittent fluctuations. We assess the form of the non-canonical equilibrium
reached by the system in this case. Using both theoretical and numerical
arguments we demonstrate that Levy statistics are the best description of the
dynamics and that the Levy distribution is the correct basin of attraction. We
also show that the correct path to non-canonical equilibrium by means of
strictly thermodynamic arguments has not yet been found, and that further
research has to be done to establish a connection between dynamics and
thermodynamics.Comment: 13 pages, 6 figure
- …