8,174 research outputs found
Propagation Kernels
We introduce propagation kernels, a general graph-kernel framework for
efficiently measuring the similarity of structured data. Propagation kernels
are based on monitoring how information spreads through a set of given graphs.
They leverage early-stage distributions from propagation schemes such as random
walks to capture structural information encoded in node labels, attributes, and
edge information. This has two benefits. First, off-the-shelf propagation
schemes can be used to naturally construct kernels for many graph types,
including labeled, partially labeled, unlabeled, directed, and attributed
graphs. Second, by leveraging existing efficient and informative propagation
schemes, propagation kernels can be considerably faster than state-of-the-art
approaches without sacrificing predictive performance. We will also show that
if the graphs at hand have a regular structure, for instance when modeling
image or video data, one can exploit this regularity to scale the kernel
computation to large databases of graphs with thousands of nodes. We support
our contributions by exhaustive experiments on a number of real-world graphs
from a variety of application domains
Tensor Graphical Lasso (TeraLasso)
This paper introduces a multi-way tensor generalization of the Bigraphical
Lasso (BiGLasso), which uses a two-way sparse Kronecker-sum multivariate-normal
model for the precision matrix to parsimoniously model conditional dependence
relationships of matrix-variate data based on the Cartesian product of graphs.
We call this generalization the {\bf Te}nsor g{\bf ra}phical Lasso (TeraLasso).
We demonstrate using theory and examples that the TeraLasso model can be
accurately and scalably estimated from very limited data samples of high
dimensional variables with multiway coordinates such as space, time and
replicates. Statistical consistency and statistical rates of convergence are
established for both the BiGLasso and TeraLasso estimators of the precision
matrix and estimators of its support (non-sparsity) set, respectively. We
propose a scalable composite gradient descent algorithm and analyze the
computational convergence rate, showing that the composite gradient descent
algorithm is guaranteed to converge at a geometric rate to the global minimizer
of the TeraLasso objective function. Finally, we illustrate the TeraLasso using
both simulation and experimental data from a meteorological dataset, showing
that we can accurately estimate precision matrices and recover meaningful
conditional dependency graphs from high dimensional complex datasets.Comment: accepted to JRSS-
Foundational principles for large scale inference: Illustrations through correlation mining
When can reliable inference be drawn in the "Big Data" context? This paper
presents a framework for answering this fundamental question in the context of
correlation mining, with implications for general large scale inference. In
large scale data applications like genomics, connectomics, and eco-informatics
the dataset is often variable-rich but sample-starved: a regime where the
number of acquired samples (statistical replicates) is far fewer than the
number of observed variables (genes, neurons, voxels, or chemical
constituents). Much of recent work has focused on understanding the
computational complexity of proposed methods for "Big Data." Sample complexity
however has received relatively less attention, especially in the setting when
the sample size is fixed, and the dimension grows without bound. To
address this gap, we develop a unified statistical framework that explicitly
quantifies the sample complexity of various inferential tasks. Sampling regimes
can be divided into several categories: 1) the classical asymptotic regime
where the variable dimension is fixed and the sample size goes to infinity; 2)
the mixed asymptotic regime where both variable dimension and sample size go to
infinity at comparable rates; 3) the purely high dimensional asymptotic regime
where the variable dimension goes to infinity and the sample size is fixed.
Each regime has its niche but only the latter regime applies to exa-scale data
dimension. We illustrate this high dimensional framework for the problem of
correlation mining, where it is the matrix of pairwise and partial correlations
among the variables that are of interest. We demonstrate various regimes of
correlation mining based on the unifying perspective of high dimensional
learning rates and sample complexity for different structured covariance models
and different inference tasks
GPstruct: Bayesian structured prediction using Gaussian processes
We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design. We motivate the model with respect to existing approaches, among others, conditional random fields (CRFs), maximum margin Markov networks (M ^3 N), and structured support vector machines (SVMstruct), which embody only a subset of its properties. We present an inference procedure based on Markov Chain Monte Carlo. The framework can be instantiated for a wide range of structured objects such as linear chains, trees, grids, and other general graphs. As a proof of concept, the model is benchmarked on several natural language processing tasks and a video gesture segmentation task involving a linear chain structure. We show prediction accuracies for GPstruct which are comparable to or exceeding those of CRFs and SVMstruct
- …