1,140 research outputs found
Neuroevolution on the Edge of Chaos
Echo state networks represent a special type of recurrent neural networks.
Recent papers stated that the echo state networks maximize their computational
performance on the transition between order and chaos, the so-called edge of
chaos. This work confirms this statement in a comprehensive set of experiments.
Furthermore, the echo state networks are compared to networks evolved via
neuroevolution. The evolved networks outperform the echo state networks,
however, the evolution consumes significant computational resources. It is
demonstrated that echo state networks with local connections combine the best
of both worlds, the simplicity of random echo state networks and the
performance of evolved networks. Finally, it is shown that evolution tends to
stay close to the ordered side of the edge of chaos.Comment: To appear in Proceedings of the Genetic and Evolutionary Computation
Conference 2017 (GECCO '17
Differentiable Game Mechanics
Deep learning is built on the foundational guarantee that gradient descent on
an objective function converges to local minima. Unfortunately, this guarantee
fails in settings, such as generative adversarial nets, that exhibit multiple
interacting losses. The behavior of gradient-based methods in games is not well
understood -- and is becoming increasingly important as adversarial and
multi-objective architectures proliferate. In this paper, we develop new tools
to understand and control the dynamics in n-player differentiable games.
The key result is to decompose the game Jacobian into two components. The
first, symmetric component, is related to potential games, which reduce to
gradient descent on an implicit function. The second, antisymmetric component,
relates to Hamiltonian games, a new class of games that obey a conservation law
akin to conservation laws in classical mechanical systems. The decomposition
motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding
stable fixed points in differentiable games. Basic experiments show SGA is
competitive with recently proposed algorithms for finding stable fixed points
in GANs -- while at the same time being applicable to, and having guarantees
in, much more general cases.Comment: JMLR 2019, journal version of arXiv:1802.0564
Explaining Latent Factor Models for Recommendation with Influence Functions
Latent factor models (LFMs) such as matrix factorization achieve the
state-of-the-art performance among various Collaborative Filtering (CF)
approaches for recommendation. Despite the high recommendation accuracy of
LFMs, a critical issue to be resolved is the lack of explainability. Extensive
efforts have been made in the literature to incorporate explainability into
LFMs. However, they either rely on auxiliary information which may not be
available in practice, or fail to provide easy-to-understand explanations. In
this paper, we propose a fast influence analysis method named FIA, which
successfully enforces explicit neighbor-style explanations to LFMs with the
technique of influence functions stemmed from robust statistics. We first
describe how to employ influence functions to LFMs to deliver neighbor-style
explanations. Then we develop a novel influence computation algorithm for
matrix factorization with high efficiency. We further extend it to the more
general neural collaborative filtering and introduce an approximation algorithm
to accelerate influence analysis over neural network models. Experimental
results on real datasets demonstrate the correctness, efficiency and usefulness
of our proposed method
- …