228 research outputs found
Estimation of Markov Chain via Rank-Constrained Likelihood
This paper studies the estimation of low-rank Markov chains from empirical
trajectories. We propose a non-convex estimator based on rank-constrained
likelihood maximization. Statistical upper bounds are provided for the
Kullback-Leiber divergence and the risk between the estimator and the
true transition matrix. The estimator reveals a compressed state space of the
Markov chain. We also develop a novel DC (difference of convex function)
programming algorithm to tackle the rank-constrained non-smooth optimization
problem. Convergence results are established. Experiments show that the
proposed estimator achieves better empirical performance than other popular
approaches.Comment: Accepted at ICML 201
Identification of Structured LTI MIMO State-Space Models
The identification of structured state-space model has been intensively
studied for a long time but still has not been adequately addressed. The main
challenge is that the involved estimation problem is a non-convex (or bilinear)
optimization problem. This paper is devoted to developing an identification
method which aims to find the global optimal solution under mild computational
burden. Key to the developed identification algorithm is to transform a
bilinear estimation to a rank constrained optimization problem and further a
difference of convex programming (DCP) problem. The initial condition for the
DCP problem is obtained by solving its convex part of the optimization problem
which happens to be a nuclear norm regularized optimization problem. Since the
nuclear norm regularized optimization is the closest convex form of the
low-rank constrained estimation problem, the obtained initial condition is
always of high quality which provides the DCP problem a good starting point.
The DCP problem is then solved by the sequential convex programming method.
Finally, numerical examples are included to show the effectiveness of the
developed identification algorithm.Comment: Accepted to IEEE Conference on Decision and Control (CDC) 201
A Unified Bregman Alternating Minimization Algorithm for Generalized DC Programming with Application to Imaging Data
In this paper, we consider a class of nonconvex (not necessarily
differentiable) optimization problems called generalized DC
(Difference-of-Convex functions) programming, which is minimizing the sum of
two separable DC parts and one two-block-variable coupling function. To
circumvent the nonconvexity and nonseparability of the problem under
consideration, we accordingly introduce a Unified Bregman Alternating
Minimization Algorithm (UBAMA) by maximally exploiting the favorable DC
structure of the objective. Specifically, we first follow the spirit of
alternating minimization to update each block variable in a sequential order,
which can efficiently tackle the nonseparablitity caused by the coupling
function. Then, we employ the Fenchel-Young inequality to approximate the
second DC components (i.e., concave parts) so that each subproblem reduces to a
convex optimization problem, thereby alleviating the computational burden of
the nonconvex DC parts. Moreover, each subproblem absorbs a Bregman proximal
regularization term, which is usually beneficial for inducing closed-form
solutions of subproblems for many cases via choosing appropriate Bregman kernel
functions. It is remarkable that our algorithm not only provides an algorithmic
framework to understand the iterative schemes of some novel existing
algorithms, but also enjoys implementable schemes with easier subproblems than
some state-of-the-art first-order algorithms developed for generic nonconvex
and nonsmooth optimization problems. Theoretically, we prove that the sequence
generated by our algorithm globally converges to a critical point under the
Kurdyka-{\L}ojasiewicz (K{\L}) condition. Besides, we estimate the local
convergence rates of our algorithm when we further know the prior information
of the K{\L} exponent.Comment: 44 pages, 7figures, 5 tables. Any comments are welcom
Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)
The implicit objective of the biennial "international - Traveling Workshop on
Interactions between Sparse models and Technology" (iTWIST) is to foster
collaboration between international scientific teams by disseminating ideas
through both specific oral/poster presentations and free discussions. For its
second edition, the iTWIST workshop took place in the medieval and picturesque
town of Namur in Belgium, from Wednesday August 27th till Friday August 29th,
2014. The workshop was conveniently located in "The Arsenal" building within
walking distance of both hotels and town center. iTWIST'14 has gathered about
70 international participants and has featured 9 invited talks, 10 oral
presentations, and 14 posters on the following themes, all related to the
theory, application and generalization of the "sparsity paradigm":
Sparsity-driven data sensing and processing; Union of low dimensional
subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph
sensing/processing; Blind inverse problems and dictionary learning; Sparsity
and computational neuroscience; Information theory, geometry and randomness;
Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?;
Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website:
http://sites.google.com/site/itwist1
A Boosted-DCA with Power-Sum-DC Decomposition for Linearly Constrained Polynomial Programs
This paper proposes a novel Difference-of-Convex (DC) decomposition for
polynomials using a power-sum representation, achieved by solving a sparse
linear system. We introduce the Boosted DCA with Exact Line Search (BDCAe) for
addressing linearly constrained polynomial programs within the DC framework.
Notably, we demonstrate that the exact line search equates to determining the
roots of a univariate polynomial in an interval, with coefficients being
computed explicitly based on the power-sum DC decompositions. The subsequential
convergence of BDCAe to critical points is proven, and its convergence rate
under the Kurdyka-Lojasiewicz property is established. To efficiently tackle
the convex subproblems, we integrate the Fast Dual Proximal Gradient (FDPG)
method by exploiting the separable block structure of the power-sum DC
decompositions. We validate our approach through numerical experiments on the
Mean-Variance-Skewness-Kurtosis (MVSK) portfolio optimization model and
box-constrained polynomial optimization problems. Comparative analysis of BDCAe
against DCA, BDCA with Armijo line search, UDCA, and UBDCA with projective DC
decomposition, alongside standard nonlinear optimization solvers FMINCON and
FILTERSD, substantiates the efficiency of our proposed approach.Comment: 39 pages, 5 figure
Solving And Applications Of Multi-Facility Location Problems
This thesis is devoted towards the study and solving of a new class of multi-facility location problems. This class is of a great theoretical interest both in variational analysis and optimization while being of high importance to a variety of practical applications. Optimization problems of this type cannot be reduced to convex programming like, the much more investigated facility location problems with only one center. In contrast, such classes of multi-facility location problems can be described by using DC (difference of convex) programming, which are significantly more involved from both theoretical and numerical viewpoints.In this thesis, we present a new approach to solve multi-facility location problems, which is based on mixed integer programming and algorithms for minimizing differences of convex (DC) functions. We then computationally implement the proposed algorithm on both artificial and real data sets and provide many numerical examples. Finally, some directions and insights for future work are detailed
- …