47 research outputs found
Sparse Randomized Shortest Paths Routing with Tsallis Divergence Regularization
This work elaborates on the important problem of (1) designing optimal
randomized routing policies for reaching a target node t from a source note s
on a weighted directed graph G and (2) defining distance measures between nodes
interpolating between the least cost (based on optimal movements) and the
commute-cost (based on a random walk on G), depending on a temperature
parameter T. To this end, the randomized shortest path formalism (RSP,
[2,99,124]) is rephrased in terms of Tsallis divergence regularization, instead
of Kullback-Leibler divergence. The main consequence of this change is that the
resulting routing policy (local transition probabilities) becomes sparser when
T decreases, therefore inducing a sparse random walk on G converging to the
least-cost directed acyclic graph when T tends to 0. Experimental comparisons
on node clustering and semi-supervised classification tasks show that the
derived dissimilarity measures based on expected routing costs provide
state-of-the-art results. The sparse RSP is therefore a promising model of
movements on a graph, balancing sparse exploitation and exploration in an
optimal way
On the Dual Formulation of Boosting Algorithms
We study boosting algorithms from a new perspective. We show that the
Lagrange dual problems of AdaBoost, LogitBoost and soft-margin LPBoost with
generalized hinge loss are all entropy maximization problems. By looking at the
dual problems of these boosting algorithms, we show that the success of
boosting algorithms can be understood in terms of maintaining a better margin
distribution by maximizing margins and at the same time controlling the margin
variance.We also theoretically prove that, approximately, AdaBoost maximizes
the average margin, instead of the minimum margin. The duality formulation also
enables us to develop column generation based optimization algorithms, which
are totally corrective. We show that they exhibit almost identical
classification results to that of standard stage-wise additive boosting
algorithms but with much faster convergence rates. Therefore fewer weak
classifiers are needed to build the ensemble using our proposed optimization
technique.Comment: 16 pages. To publish/Published in IEEE Transactions on Pattern
Analysis and Machine Intelligence, 201
25 Years of Self-Organized Criticality: Solar and Astrophysics
Shortly after the seminal paper {\sl "Self-Organized Criticality: An
explanation of 1/f noise"} by Bak, Tang, and Wiesenfeld (1987), the idea has
been applied to solar physics, in {\sl "Avalanches and the Distribution of
Solar Flares"} by Lu and Hamilton (1991). In the following years, an inspiring
cross-fertilization from complexity theory to solar and astrophysics took
place, where the SOC concept was initially applied to solar flares, stellar
flares, and magnetospheric substorms, and later extended to the radiation belt,
the heliosphere, lunar craters, the asteroid belt, the Saturn ring, pulsar
glitches, soft X-ray repeaters, blazars, black-hole objects, cosmic rays, and
boson clouds. The application of SOC concepts has been performed by numerical
cellular automaton simulations, by analytical calculations of statistical
(powerlaw-like) distributions based on physical scaling laws, and by
observational tests of theoretically predicted size distributions and waiting
time distributions. Attempts have been undertaken to import physical models
into the numerical SOC toy models, such as the discretization of
magneto-hydrodynamics (MHD) processes. The novel applications stimulated also
vigorous debates about the discrimination between SOC models, SOC-like, and
non-SOC processes, such as phase transitions, turbulence, random-walk
diffusion, percolation, branching processes, network theory, chaos theory,
fractality, multi-scale, and other complexity phenomena. We review SOC studies
from the last 25 years and highlight new trends, open questions, and future
challenges, as discussed during two recent ISSI workshops on this theme.Comment: 139 pages, 28 figures, Review based on ISSI workshops "Self-Organized
Criticality and Turbulence" (2012, 2013, Bern, Switzerland