102 research outputs found
Variational Inference for Stochastic Block Models from Sampled Data
This paper deals with non-observed dyads during the sampling of a network and
consecutive issues in the inference of the Stochastic Block Model (SBM). We
review sampling designs and recover Missing At Random (MAR) and Not Missing At
Random (NMAR) conditions for the SBM. We introduce variants of the variational
EM algorithm for inferring the SBM under various sampling designs (MAR and
NMAR) all available as an R package. Model selection criteria based on
Integrated Classification Likelihood are derived for selecting both the number
of blocks and the sampling design. We investigate the accuracy and the range of
applicability of these algorithms with simulations. We explore two real-world
networks from ethnology (seed circulation network) and biology (protein-protein
interaction network), where the interpretations considerably depends on the
sampling designs considered
Sparsity with sign-coherent groups of variables via the cooperative-Lasso
We consider the problems of estimation and selection of parameters endowed
with a known group structure, when the groups are assumed to be sign-coherent,
that is, gathering either nonnegative, nonpositive or null parameters. To
tackle this problem, we propose the cooperative-Lasso penalty. We derive the
optimality conditions defining the cooperative-Lasso estimate for generalized
linear models, and propose an efficient active set algorithm suited to
high-dimensional problems. We study the asymptotic consistency of the estimator
in the linear regression setup and derive its irrepresentable conditions, which
are milder than the ones of the group-Lasso regarding the matching of groups
with the sparsity pattern of the true parameters. We also address the problem
of model selection in linear regression by deriving an approximation of the
degrees of freedom of the cooperative-Lasso estimator. Simulations comparing
the proposed estimator to the group and sparse group-Lasso comply with our
theoretical results, showing consistent improvements in support recovery for
sign-coherent groups. We finally propose two examples illustrating the wide
applicability of the cooperative-Lasso: first to the processing of ordinal
variables, where the penalty acts as a monotonicity prior; second to the
processing of genomic data, where the set of differentially expressed probes is
enriched by incorporating all the probes of the microarray that are related to
the corresponding genes.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS520 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Inferring Multiple Graphical Structures
Gaussian Graphical Models provide a convenient framework for representing
dependencies between variables. Recently, this tool has received a high
interest for the discovery of biological networks. The literature focuses on
the case where a single network is inferred from a set of measurements, but, as
wetlab data is typically scarce, several assays, where the experimental
conditions affect interactions, are usually merged to infer a single network.
In this paper, we propose two approaches for estimating multiple related
graphs, by rendering the closeness assumption into an empirical prior or group
penalties. We provide quantitative results demonstrating the benefits of the
proposed approaches. The methods presented in this paper are embeded in the R
package 'simone' from version 1.0-0 and later
Weighted-Lasso for Structured Network Inference from Time Course Data
We present a weighted-Lasso method to infer the parameters of a first-order
vector auto-regressive model that describes time course expression data
generated by directed gene-to-gene regulation networks. These networks are
assumed to own a prior internal structure of connectivity which drives the
inference method. This prior structure can be either derived from prior
biological knowledge or inferred by the method itself. We illustrate the
performance of this structure-based penalization both on synthetic data and on
two canonical regulatory networks, first yeast cell cycle regulation network by
analyzing Spellman et al's dataset and second E. coli S.O.S. DNA repair network
by analysing U. Alon's lab data
Variational inference for sparse network reconstruction from count data
In multivariate statistics, the question of finding direct interactions can
be formulated as a problem of network inference - or network reconstruction -
for which the Gaussian graphical model (GGM) provides a canonical framework.
Unfortunately, the Gaussian assumption does not apply to count data which are
encountered in domains such as genomics, social sciences or ecology.
To circumvent this limitation, state-of-the-art approaches use two-step
strategies that first transform counts to pseudo Gaussian observations and then
apply a (partial) correlation-based approach from the abundant literature of
GGM inference. We adopt a different stance by relying on a latent model where
we directly model counts by means of Poisson distributions that are conditional
to latent (hidden) Gaussian correlated variables. In this multivariate Poisson
lognormal-model, the dependency structure is completely captured by the latent
layer. This parametric model enables to account for the effects of covariates
on the counts.
To perform network inference, we add a sparsity inducing constraint on the
inverse covariance matrix of the latent Gaussian vector. Unlike the usual
Gaussian setting, the penalized likelihood is generally not tractable, and we
resort instead to a variational approach for approximate likelihood
maximization. The corresponding optimization problem is solved by alternating a
gradient ascent on the variational parameters and a graphical-Lasso step on the
covariance matrix.
We show that our approach is highly competitive with the existing methods on
simulation inspired from microbiological data. We then illustrate on three
various data sets how accounting for sampling efforts via offsets and
integrating external covariates (which is mostly never done in the existing
literature) drastically changes the topology of the inferred network
Finite-sum optimization: Adaptivity to smoothness and loopless variance reduction
For finite-sum optimization, variance-reduced gradient methods (VR) compute
at each iteration the gradient of a single function (or of a mini-batch), and
yet achieve faster convergence than SGD thanks to a carefully crafted
lower-variance stochastic gradient estimator that reuses past gradients.
Another important line of research of the past decade in continuous
optimization is the adaptive algorithms such as AdaGrad, that dynamically
adjust the (possibly coordinate-wise) learning rate to past gradients and
thereby adapt to the geometry of the objective function. Variants such as
RMSprop and Adam demonstrate outstanding practical performance that have
contributed to the success of deep learning. In this work, we present AdaVR,
which combines the AdaGrad algorithm with variance-reduced gradient estimators
such as SAGA or L-SVRG. We assess that AdaVR inherits both good convergence
properties from VR methods and the adaptive nature of AdaGrad: in the case of
-smooth convex functions we establish a gradient complexity of
without prior knowledge of . Numerical
experiments demonstrate the superiority of AdaVR over state-of-the-art methods.
Moreover, we empirically show that the RMSprop and Adam algorithm combined with
variance-reduced gradients estimators achieve even faster convergence
- …