22,733 research outputs found
Spectral Simplicity of Apparent Complexity, Part I: The Nondiagonalizable Metadynamics of Prediction
Virtually all questions that one can ask about the behavioral and structural
complexity of a stochastic process reduce to a linear algebraic framing of a
time evolution governed by an appropriate hidden-Markov process generator. Each
type of question---correlation, predictability, predictive cost, observer
synchronization, and the like---induces a distinct generator class. Answers are
then functions of the class-appropriate transition dynamic. Unfortunately,
these dynamics are generically nonnormal, nondiagonalizable, singular, and so
on. Tractably analyzing these dynamics relies on adapting the recently
introduced meromorphic functional calculus, which specifies the spectral
decomposition of functions of nondiagonalizable linear operators, even when the
function poles and zeros coincide with the operator's spectrum. Along the way,
we establish special properties of the projection operators that demonstrate
how they capture the organization of subprocesses within a complex system.
Circumventing the spurious infinities of alternative calculi, this leads in the
sequel, Part II, to the first closed-form expressions for complexity measures,
couched either in terms of the Drazin inverse (negative-one power of a singular
operator) or the eigenvalues and projection operators of the appropriate
transition dynamic.Comment: 24 pages, 3 figures, 4 tables; current version always at
http://csc.ucdavis.edu/~cmg/compmech/pubs/sdscpt1.ht
Graph-based Semi-Supervised & Active Learning for Edge Flows
We present a graph-based semi-supervised learning (SSL) method for learning
edge flows defined on a graph. Specifically, given flow measurements on a
subset of edges, we want to predict the flows on the remaining edges. To this
end, we develop a computational framework that imposes certain constraints on
the overall flows, such as (approximate) flow conservation. These constraints
render our approach different from classical graph-based SSL for vertex labels,
which posits that tightly connected nodes share similar labels and leverages
the graph structure accordingly to extrapolate from a few vertex labels to the
unlabeled vertices. We derive bounds for our method's reconstruction error and
demonstrate its strong performance on synthetic and real-world flow networks
from transportation, physical infrastructure, and the Web. Furthermore, we
provide two active learning algorithms for selecting informative edges on which
to measure flow, which has applications for optimal sensor deployment. The
first strategy selects edges to minimize the reconstruction error bound and
works well on flows that are approximately divergence-free. The second approach
clusters the graph and selects bottleneck edges that cross cluster-boundaries,
which works well on flows with global trends
Active Semi-Supervised Learning Using Sampling Theory for Graph Signals
We consider the problem of offline, pool-based active semi-supervised
learning on graphs. This problem is important when the labeled data is scarce
and expensive whereas unlabeled data is easily available. The data points are
represented by the vertices of an undirected graph with the similarity between
them captured by the edge weights. Given a target number of nodes to label, the
goal is to choose those nodes that are most informative and then predict the
unknown labels. We propose a novel framework for this problem based on our
recent results on sampling theory for graph signals. A graph signal is a
real-valued function defined on each node of the graph. A notion of frequency
for such signals can be defined using the spectrum of the graph Laplacian
matrix. The sampling theory for graph signals aims to extend the traditional
Nyquist-Shannon sampling theory by allowing us to identify the class of graph
signals that can be reconstructed from their values on a subset of vertices.
This approach allows us to define a criterion for active learning based on
sampling set selection which aims at maximizing the frequency of the signals
that can be reconstructed from their samples on the set. Experiments show the
effectiveness of our method.Comment: 10 pages, 6 figures, To appear in KDD'1
Momentum Strategies with L1 Filter
In this article, we discuss various implementation of L1 filtering in order
to detect some properties of noisy signals. This filter consists of using a L1
penalty condition in order to obtain the filtered signal composed by a set of
straight trends or steps. This penalty condition, which determines the number
of breaks, is implemented in a constrained least square problem and is
represented by a regularization parameter ? which is estimated by a
cross-validation procedure. Financial time series are usually characterized by
a long-term trend (called the global trend) and some short-term trends (which
are named local trends). A combination of these two time scales can form a
simple model describing the process of a global trend process with some
mean-reverting properties. Explicit applications to momentum strategies are
also discussed in detail with appropriate uses of the trend configurations.Comment: 22 pages, 15 figures. Submitted to The Journal of Investment
Strategies, reference code: JOIS140227T
Evaluating Overfit and Underfit in Models of Network Community Structure
A common data mining task on networks is community detection, which seeks an
unsupervised decomposition of a network into structural groups based on
statistical regularities in the network's connectivity. Although many methods
exist, the No Free Lunch theorem for community detection implies that each
makes some kind of tradeoff, and no algorithm can be optimal on all inputs.
Thus, different algorithms will over or underfit on different inputs, finding
more, fewer, or just different communities than is optimal, and evaluation
methods that use a metadata partition as a ground truth will produce misleading
conclusions about general accuracy. Here, we present a broad evaluation of over
and underfitting in community detection, comparing the behavior of 16
state-of-the-art community detection algorithms on a novel and structurally
diverse corpus of 406 real-world networks. We find that (i) algorithms vary
widely both in the number of communities they find and in their corresponding
composition, given the same input, (ii) algorithms can be clustered into
distinct high-level groups based on similarities of their outputs on real-world
networks, and (iii) these differences induce wide variation in accuracy on link
prediction and link description tasks. We introduce a new diagnostic for
evaluating overfitting and underfitting in practice, and use it to roughly
divide community detection methods into general and specialized learning
algorithms. Across methods and inputs, Bayesian techniques based on the
stochastic block model and a minimum description length approach to
regularization represent the best general learning approach, but can be
outperformed under specific circumstances. These results introduce both a
theoretically principled approach to evaluate over and underfitting in models
of network community structure and a realistic benchmark by which new methods
may be evaluated and compared.Comment: 22 pages, 13 figures, 3 table
Spectral Simplicity of Apparent Complexity, Part II: Exact Complexities and Complexity Spectra
The meromorphic functional calculus developed in Part I overcomes the
nondiagonalizability of linear operators that arises often in the temporal
evolution of complex systems and is generic to the metadynamics of predicting
their behavior. Using the resulting spectral decomposition, we derive
closed-form expressions for correlation functions, finite-length Shannon
entropy-rate approximates, asymptotic entropy rate, excess entropy, transient
information, transient and asymptotic state uncertainty, and synchronization
information of stochastic processes generated by finite-state hidden Markov
models. This introduces analytical tractability to investigating information
processing in discrete-event stochastic processes, symbolic dynamics, and
chaotic dynamical systems. Comparisons reveal mathematical similarities between
complexity measures originally thought to capture distinct informational and
computational properties. We also introduce a new kind of spectral analysis via
coronal spectrograms and the frequency-dependent spectra of past-future mutual
information. We analyze a number of examples to illustrate the methods,
emphasizing processes with multivariate dependencies beyond pairwise
correlation. An appendix presents spectral decomposition calculations for one
example in full detail.Comment: 27 pages, 12 figures, 2 tables; most recent version at
http://csc.ucdavis.edu/~cmg/compmech/pubs/sdscpt2.ht
- …