15,949 research outputs found
Dynamic Steerable Blocks in Deep Residual Networks
Filters in convolutional networks are typically parameterized in a pixel
basis, that does not take prior knowledge about the visual world into account.
We investigate the generalized notion of frames designed with image properties
in mind, as alternatives to this parametrization. We show that frame-based
ResNets and Densenets can improve performance on Cifar-10+ consistently, while
having additional pleasant properties like steerability. By exploiting these
transformation properties explicitly, we arrive at dynamic steerable blocks.
They are an extension of residual blocks, that are able to seamlessly transform
filters under pre-defined transformations, conditioned on the input at training
and inference time. Dynamic steerable blocks learn the degree of invariance
from data and locally adapt filters, allowing them to apply a different
geometrical variant of the same filter to each location of the feature map.
When evaluated on the Berkeley Segmentation contour detection dataset, our
approach outperforms all competing approaches that do not utilize pre-training.
Our results highlight the benefits of image-based regularization to deep
networks
Time-Varying Graphs and Dynamic Networks
The past few years have seen intensive research efforts carried out in some
apparently unrelated areas of dynamic systems -- delay-tolerant networks,
opportunistic-mobility networks, social networks -- obtaining closely related
insights. Indeed, the concepts discovered in these investigations can be viewed
as parts of the same conceptual universe; and the formal models proposed so far
to express some specific concepts are components of a larger formal description
of this universe. The main contribution of this paper is to integrate the vast
collection of concepts, formalisms, and results found in the literature into a
unified framework, which we call TVG (for time-varying graphs). Using this
framework, it is possible to express directly in the same formalism not only
the concepts common to all those different areas, but also those specific to
each. Based on this definitional work, employing both existing results and
original observations, we present a hierarchical classification of TVGs; each
class corresponds to a significant property examined in the distributed
computing literature. We then examine how TVGs can be used to study the
evolution of network properties, and propose different techniques, depending on
whether the indicators for these properties are a-temporal (as in the majority
of existing studies) or temporal. Finally, we briefly discuss the introduction
of randomness in TVGs.Comment: A short version appeared in ADHOC-NOW'11. This version is to be
published in Internation Journal of Parallel, Emergent and Distributed
System
Evolving Ensemble Fuzzy Classifier
The concept of ensemble learning offers a promising avenue in learning from
data streams under complex environments because it addresses the bias and
variance dilemma better than its single model counterpart and features a
reconfigurable structure, which is well suited to the given context. While
various extensions of ensemble learning for mining non-stationary data streams
can be found in the literature, most of them are crafted under a static base
classifier and revisits preceding samples in the sliding window for a
retraining step. This feature causes computationally prohibitive complexity and
is not flexible enough to cope with rapidly changing environments. Their
complexities are often demanding because it involves a large collection of
offline classifiers due to the absence of structural complexities reduction
mechanisms and lack of an online feature selection mechanism. A novel evolving
ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in
this paper. pENsemble differs from existing architectures in the fact that it
is built upon an evolving classifier from data streams, termed Parsimonious
Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism,
which estimates a localized generalization error of a base classifier. A
dynamic online feature selection scenario is integrated into the pENsemble.
This method allows for dynamic selection and deselection of input features on
the fly. pENsemble adopts a dynamic ensemble structure to output a final
classification decision where it features a novel drift detection scenario to
grow the ensemble structure. The efficacy of the pENsemble has been numerically
demonstrated through rigorous numerical studies with dynamic and evolving data
streams where it delivers the most encouraging performance in attaining a
tradeoff between accuracy and complexity.Comment: this paper has been published by IEEE Transactions on Fuzzy System
Adaptive, locally-linear models of complex dynamics
The dynamics of complex systems generally include high-dimensional,
non-stationary and non-linear behavior, all of which pose fundamental
challenges to quantitative understanding. To address these difficulties we
detail a new approach based on local linear models within windows determined
adaptively from the data. While the dynamics within each window are simple,
consisting of exponential decay, growth and oscillations, the collection of
local parameters across all windows provides a principled characterization of
the full time series. To explore the resulting model space, we develop a novel
likelihood-based hierarchical clustering and we examine the eigenvalues of the
linear dynamics. We demonstrate our analysis with the Lorenz system undergoing
stable spiral dynamics and in the standard chaotic regime. Applied to the
posture dynamics of the nematode our approach identifies
fine-grained behavioral states and model dynamics which fluctuate close to an
instability boundary, and we detail a bifurcation in a transition from forward
to backward crawling. Finally, we analyze whole-brain imaging in
and show that the stability of global brain states changes with oxygen
concentration.Comment: 25 pages, 16 figure
The Complementary Brain: A Unifying View of Brain Specialization and Modularity
Defense Advanced Research Projects Agency and Office of Naval Research (N00014-95-I-0409); National Science Foundation (ITI-97-20333); Office of Naval Research (N00014-95-I-0657
Encoding dynamics for multiscale community detection: Markov time sweeping for the Map equation
The detection of community structure in networks is intimately related to
finding a concise description of the network in terms of its modules. This
notion has been recently exploited by the Map equation formalism (M. Rosvall
and C.T. Bergstrom, PNAS, 105(4), pp.1118--1123, 2008) through an
information-theoretic description of the process of coding inter- and
intra-community transitions of a random walker in the network at stationarity.
However, a thorough study of the relationship between the full Markov dynamics
and the coding mechanism is still lacking. We show here that the original Map
coding scheme, which is both block-averaged and one-step, neglects the internal
structure of the communities and introduces an upper scale, the `field-of-view'
limit, in the communities it can detect. As a consequence, Map is well tuned to
detect clique-like communities but can lead to undesirable overpartitioning
when communities are far from clique-like. We show that a signature of this
behavior is a large compression gap: the Map description length is far from its
ideal limit. To address this issue, we propose a simple dynamic approach that
introduces time explicitly into the Map coding through the analysis of the
weighted adjacency matrix of the time-dependent multistep transition matrix of
the Markov process. The resulting Markov time sweeping induces a dynamical
zooming across scales that can reveal (potentially multiscale) community
structure above the field-of-view limit, with the relevant partitions indicated
by a small compression gap.Comment: 10 pages, 6 figure
- …