15,213 research outputs found
Splitting Method for Support Vector Machine in Reproducing Kernel Banach Space with Lower Semi-continuous Loss Function
In this paper, we use the splitting method to solve support vector machine in
reproducing kernel Banach space with lower semi-continuous loss function. We
equivalently transfer support vector machines in reproducing kernel Banach
space with lower semi-continuous loss function to a finite-dimensional tensor
Optimization and propose the splitting method based on alternating direction
method of multipliers. By Kurdyka-Lojasiewicz inequality, the iterative
sequence obtained by this splitting method is globally convergent to a
stationary point if the loss function is lower semi-continuous and subanalytic.
Finally, several numerical performances demonstrate the effectiveness.Comment: arXiv admin note: text overlap with arXiv:2208.1252
Quantized Consensus ADMM for Multi-Agent Distributed Optimization
Multi-agent distributed optimization over a network minimizes a global
objective formed by a sum of local convex functions using only local
computation and communication. We develop and analyze a quantized distributed
algorithm based on the alternating direction method of multipliers (ADMM) when
inter-agent communications are subject to finite capacity and other practical
constraints. While existing quantized ADMM approaches only work for quadratic
local objectives, the proposed algorithm can deal with more general objective
functions (possibly non-smooth) including the LASSO. Under certain convexity
assumptions, our algorithm converges to a consensus within
iterations, where depends on the local
objectives and the network topology, and is a polynomial determined by
the quantization resolution, the distance between initial and optimal variable
values, the local objective functions and the network topology. A tight upper
bound on the consensus error is also obtained which does not depend on the size
of the network.Comment: 30 pages, 4 figures; to be submitted to IEEE Trans. Signal
Processing. arXiv admin note: text overlap with arXiv:1307.5561 by other
author
Recovery Conditions and Sampling Strategies for Network Lasso
The network Lasso is a recently proposed convex optimization method for
machine learning from massive network structured datasets, i.e., big data over
networks. It is a variant of the well-known least absolute shrinkage and
selection operator (Lasso), which is underlying many methods in learning and
signal processing involving sparse models. Highly scalable implementations of
the network Lasso can be obtained by state-of-the art proximal methods, e.g.,
the alternating direction method of multipliers (ADMM). By generalizing the
concept of the compatibility condition put forward by van de Geer and Buehlmann
as a powerful tool for the analysis of plain Lasso, we derive a sufficient
condition, i.e., the network compatibility condition, on the underlying network
topology such that network Lasso accurately learns a clustered underlying graph
signal. This network compatibility condition relates the location of the
sampled nodes with the clustering structure of the network. In particular, the
NCC informs the choice of which nodes to sample, or in machine learning terms,
which data points provide most information if labeled.Comment: nominated as student paper award finalist at Asilomar 2017. arXiv
admin note: substantial text overlap with arXiv:1704.0210
Proximal Symmetric Non-negative Latent Factor Analysis: A Novel Approach to Highly-Accurate Representation of Undirected Weighted Networks
An Undirected Weighted Network (UWN) is commonly found in big data-related
applications. Note that such a network's information connected with its nodes,
and edges can be expressed as a Symmetric, High-Dimensional and Incomplete
(SHDI) matrix. However, existing models fail in either modeling its intrinsic
symmetry or low-data density, resulting in low model scalability or
representation learning ability. For addressing this issue, a Proximal
Symmetric Nonnegative Latent-factor-analysis (PSNL) model is proposed. It
incorporates a proximal term into symmetry-aware and data density-oriented
objective function for high representation accuracy. Then an adaptive
Alternating Direction Method of Multipliers (ADMM)-based learning scheme is
implemented through a Tree-structured of Parzen Estimators (TPE) method for
high computational efficiency. Empirical studies on four UWNs demonstrate that
PSNL achieves higher accuracy gain than state-of-the-art models, as well as
highly competitive computational efficiency
Iteration Complexity Analysis of Multi-Block ADMM for a Family of Convex Minimization without Strong Convexity
The alternating direction method of multipliers (ADMM) is widely used in
solving structured convex optimization problems due to its superior practical
performance. On the theoretical side however, a counterexample was shown in [7]
indicating that the multi-block ADMM for minimizing the sum of
convex functions with block variables linked by linear constraints may
diverge. It is therefore of great interest to investigate further sufficient
conditions on the input side which can guarantee convergence for the
multi-block ADMM. The existing results typically require the strong convexity
on parts of the objective. In this paper, we present convergence and
convergence rate results for the multi-block ADMM applied to solve certain
-block convex minimization problems without requiring strong
convexity. Specifically, we prove the following two results: (1) the
multi-block ADMM returns an -optimal solution within
iterations by solving an associated perturbation to the
original problem; (2) the multi-block ADMM returns an -optimal
solution within iterations when it is applied to solve a
certain sharing problem, under the condition that the augmented Lagrangian
function satisfies the Kurdyka-Lojasiewicz property, which essentially covers
most convex optimization models except for some pathological cases.Comment: arXiv admin note: text overlap with arXiv:1408.426
- …