411 research outputs found
Bayesian Fused Lasso regression for dynamic binary networks
We propose a multinomial logistic regression model for link prediction in a
time series of directed binary networks. To account for the dynamic nature of
the data we employ a dynamic model for the model parameters that is strongly
connected with the fused lasso penalty. In addition to promoting sparseness,
this prior allows us to explore the presence of change points in the structure
of the network. We introduce fast computational algorithms for estimation and
prediction using both optimization and Bayesian approaches. The performance of
the model is illustrated using simulated data and data from a financial trading
network in the NYMEX natural gas futures market. Supplementary material
containing the trading network data set and code to implement the algorithms is
available online
Solving Multiple-Block Separable Convex Minimization Problems Using Two-Block Alternating Direction Method of Multipliers
In this paper, we consider solving multiple-block separable convex
minimization problems using alternating direction method of multipliers (ADMM).
Motivated by the fact that the existing convergence theory for ADMM is mostly
limited to the two-block case, we analyze in this paper, both theoretically and
numerically, a new strategy that first transforms a multi-block problem into an
equivalent two-block problem (either in the primal domain or in the dual
domain) and then solves it using the standard two-block ADMM. In particular, we
derive convergence results for this two-block ADMM approach to solve
multi-block separable convex minimization problems, including an improved
O(1/\epsilon) iteration complexity result. Moreover, we compare the numerical
efficiency of this approach with the standard multi-block ADMM on several
separable convex minimization problems which include basis pursuit, robust
principal component analysis and latent variable Gaussian graphical model
selection. The numerical results show that the multiple-block ADMM, although
lacks theoretical convergence guarantees, typically outperforms two-block
ADMMs
Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization
problem to characterize graphical model selection in the presence of unobserved
variables. This convex optimization problem aims to estimate an inverse
covariance matrix that can be decomposed into a sparse matrix minus a low-rank
matrix from sample data. Solving this convex optimization problem is very
challenging, especially for large problems. In this paper, we propose two
alternating direction methods for solving this problem. The first method is to
apply the classical alternating direction method of multipliers to solve the
problem as a consensus problem. The second method is a proximal gradient based
alternating direction method of multipliers. Our methods exploit and take
advantage of the special structure of the problem and thus can solve large
problems very efficiently. Global convergence result is established for the
proposed methods. Numerical results on both synthetic data and gene expression
data show that our methods usually solve problems with one million variables in
one to two minutes, and are usually five to thirty five times faster than a
state-of-the-art Newton-CG proximal point algorithm
Linearized Alternating Direction Method with Parallel Splitting and Adaptive Penalty for Separable Convex Programs in Machine Learning
Many problems in machine learning and other fields can be (re)for-mulated as
linearly constrained separable convex programs. In most of the cases, there are
multiple blocks of variables. However, the traditional alternating direction
method (ADM) and its linearized version (LADM, obtained by linearizing the
quadratic penalty term) are for the two-block case and cannot be naively
generalized to solve the multi-block case. So there is great demand on
extending the ADM based methods for the multi-block case. In this paper, we
propose LADM with parallel splitting and adaptive penalty (LADMPSAP) to solve
multi-block separable convex programs efficiently. When all the component
objective functions have bounded subgradients, we obtain convergence results
that are stronger than those of ADM and LADM, e.g., allowing the penalty
parameter to be unbounded and proving the sufficient and necessary conditions}
for global convergence. We further propose a simple optimality measure and
reveal the convergence rate of LADMPSAP in an ergodic sense. For programs with
extra convex set constraints, with refined parameter estimation we devise a
practical version of LADMPSAP for faster convergence. Finally, we generalize
LADMPSAP to handle programs with more difficult objective functions by
linearizing part of the objective function as well. LADMPSAP is particularly
suitable for sparse representation and low-rank recovery problems because its
subproblems have closed form solutions and the sparsity and low-rankness of the
iterates can be preserved during the iteration. It is also highly
parallelizable and hence fits for parallel or distributed computing. Numerical
experiments testify to the advantages of LADMPSAP in speed and numerical
accuracy.Comment: Preliminary version published on Asian Conference on Machine Learning
201
A Proximal Approach for a Class of Matrix Optimization Problems
In recent years, there has been a growing interest in mathematical models
leading to the minimization, in a symmetric matrix space, of a Bregman
divergence coupled with a regularization term. We address problems of this type
within a general framework where the regularization term is split in two parts,
one being a spectral function while the other is arbitrary. A Douglas-Rachford
approach is proposed to address such problems and a list of proximity operators
is provided allowing us to consider various choices for the fit-to-data
functional and for the regularization term. Numerical experiments show the
validity of this approach for solving convex optimization problems encountered
in the context of sparse covariance matrix estimation. Based on our theoretical
results, an algorithm is also proposed for noisy graphical lasso where a
precision matrix has to be estimated in the presence of noise. The nonconvexity
of the resulting objective function is dealt with a majorization-minimization
approach, i.e. by building a sequence of convex surrogates and solving the
inner optimization subproblems via the aforementioned Douglas-Rachford
procedure. We establish conditions for the convergence of this iterative scheme
and we illustrate its good numerical performance with respect to
state-of-the-art approaches
- …