8,200 research outputs found
Evolving stochastic learning algorithm based on Tsallis entropic index
In this paper, inspired from our previous algorithm, which was based on the theory of Tsallis statistical mechanics, we develop a new evolving stochastic learning algorithm for neural networks. The new algorithm combines deterministic and stochastic search steps by employing a different adaptive stepsize for each network weight, and applies a form of noise that is characterized by the nonextensive entropic index q, regulated by a weight decay term. The behavior of the learning algorithm can be made more stochastic or deterministic depending on the trade off between the temperature T and the q values. This is achieved by introducing a formula that defines a time-dependent relationship between these two important learning parameters. Our experimental study verifies that there are indeed improvements in the convergence speed of this new evolving stochastic learning algorithm, which makes learning faster than using the original Hybrid Learning Scheme (HLS). In addition, experiments are conducted to explore the influence of the entropic index q and temperature T on the convergence speed and stability of the proposed method
Scaling Algorithms for Unbalanced Transport Problems
This article introduces a new class of fast algorithms to approximate
variational problems involving unbalanced optimal transport. While classical
optimal transport considers only normalized probability distributions, it is
important for many applications to be able to compute some sort of relaxed
transportation between arbitrary positive measures. A generic class of such
"unbalanced" optimal transport problems has been recently proposed by several
authors. In this paper, we show how to extend the, now classical, entropic
regularization scheme to these unbalanced problems. This gives rise to fast,
highly parallelizable algorithms that operate by performing only diagonal
scaling (i.e. pointwise multiplications) of the transportation couplings. They
are generalizations of the celebrated Sinkhorn algorithm. We show how these
methods can be used to solve unbalanced transport, unbalanced gradient flows,
and to compute unbalanced barycenters. We showcase applications to 2-D shape
modification, color transfer, and growth models
Faster Convex Optimization: Simulated Annealing with an Efficient Universal Barrier
This paper explores a surprising equivalence between two seemingly-distinct
convex optimization methods. We show that simulated annealing, a well-studied
random walk algorithms, is directly equivalent, in a certain sense, to the
central path interior point algorithm for the the entropic universal barrier
function. This connection exhibits several benefits. First, we are able improve
the state of the art time complexity for convex optimization under the
membership oracle model. We improve the analysis of the randomized algorithm of
Kalai and Vempala by utilizing tools developed by Nesterov and Nemirovskii that
underly the central path following interior point algorithm. We are able to
tighten the temperature schedule for simulated annealing which gives an
improved running time, reducing by square root of the dimension in certain
instances. Second, we get an efficient randomized interior point method with an
efficiently computable universal barrier for any convex set described by a
membership oracle. Previously, efficiently computable barriers were known only
for particular convex sets
- …