41,319 research outputs found
Characterizing the Shape of Activation Space in Deep Neural Networks
The representations learned by deep neural networks are difficult to
interpret in part due to their large parameter space and the complexities
introduced by their multi-layer structure. We introduce a method for computing
persistent homology over the graphical activation structure of neural networks,
which provides access to the task-relevant substructures activated throughout
the network for a given input. This topological perspective provides unique
insights into the distributed representations encoded by neural networks in
terms of the shape of their activation structures. We demonstrate the value of
this approach by showing an alternative explanation for the existence of
adversarial examples. By studying the topology of network activations across
multiple architectures and datasets, we find that adversarial perturbations do
not add activations that target the semantic structure of the adversarial class
as previously hypothesized. Rather, adversarial examples are explainable as
alterations to the dominant activation structures induced by the original
image, suggesting the class representations learned by deep networks are
problematically sparse on the input space
Time-Varying Graphs and Dynamic Networks
The past few years have seen intensive research efforts carried out in some
apparently unrelated areas of dynamic systems -- delay-tolerant networks,
opportunistic-mobility networks, social networks -- obtaining closely related
insights. Indeed, the concepts discovered in these investigations can be viewed
as parts of the same conceptual universe; and the formal models proposed so far
to express some specific concepts are components of a larger formal description
of this universe. The main contribution of this paper is to integrate the vast
collection of concepts, formalisms, and results found in the literature into a
unified framework, which we call TVG (for time-varying graphs). Using this
framework, it is possible to express directly in the same formalism not only
the concepts common to all those different areas, but also those specific to
each. Based on this definitional work, employing both existing results and
original observations, we present a hierarchical classification of TVGs; each
class corresponds to a significant property examined in the distributed
computing literature. We then examine how TVGs can be used to study the
evolution of network properties, and propose different techniques, depending on
whether the indicators for these properties are a-temporal (as in the majority
of existing studies) or temporal. Finally, we briefly discuss the introduction
of randomness in TVGs.Comment: A short version appeared in ADHOC-NOW'11. This version is to be
published in Internation Journal of Parallel, Emergent and Distributed
System
- …