69,866 research outputs found
On Global Warming (Softening Global Constraints)
We describe soft versions of the global cardinality constraint and the
regular constraint, with efficient filtering algorithms maintaining domain
consistency. For both constraints, the softening is achieved by augmenting the
underlying graph. The softened constraints can be used to extend the
meta-constraint framework for over-constrained problems proposed by Petit,
Regin and Bessiere.Comment: 15 pages, 7 figures. Accepted at the 6th International Workshop on
Preferences and Soft Constraint
Optimization viewpoint on Kalman smoothing, with applications to robust and sparse estimation
In this paper, we present the optimization formulation of the Kalman
filtering and smoothing problems, and use this perspective to develop a variety
of extensions and applications. We first formulate classic Kalman smoothing as
a least squares problem, highlight special structure, and show that the classic
filtering and smoothing algorithms are equivalent to a particular algorithm for
solving this problem. Once this equivalence is established, we present
extensions of Kalman smoothing to systems with nonlinear process and
measurement models, systems with linear and nonlinear inequality constraints,
systems with outliers in the measurements or sudden changes in the state, and
systems where the sparsity of the state sequence must be accounted for. All
extensions preserve the computational efficiency of the classic algorithms, and
most of the extensions are illustrated with numerical examples, which are part
of an open source Kalman smoothing Matlab/Octave package.Comment: 46 pages, 11 figure
A Generic Path Algorithm for Regularized Statistical Estimation
Regularization is widely used in statistics and machine learning to prevent
overfitting and gear solution towards prior information. In general, a
regularized estimation problem minimizes the sum of a loss function and a
penalty term. The penalty term is usually weighted by a tuning parameter and
encourages certain constraints on the parameters to be estimated. Particular
choices of constraints lead to the popular lasso, fused-lasso, and other
generalized penalized regression methods. Although there has been a lot
of research in this area, developing efficient optimization methods for many
nonseparable penalties remains a challenge. In this article we propose an exact
path solver based on ordinary differential equations (EPSODE) that works for
any convex loss function and can deal with generalized penalties as well
as more complicated regularization such as inequality constraints encountered
in shape-restricted regressions and nonparametric density estimation. In the
path following process, the solution path hits, exits, and slides along the
various constraints and vividly illustrates the tradeoffs between goodness of
fit and model parsimony. In practice, the EPSODE can be coupled with AIC, BIC,
or cross-validation to select an optimal tuning parameter. Our
applications to generalized regularized generalized linear models,
shape-restricted regressions, Gaussian graphical models, and nonparametric
density estimation showcase the potential of the EPSODE algorithm.Comment: 28 pages, 5 figure
A New Approach to Collaborative Filtering: Operator Estimation with Spectral Regularization
We present a general approach for collaborative filtering (CF) using spectral
regularization to learn linear operators from "users" to the "objects" they
rate. Recent low-rank type matrix completion approaches to CF are shown to be
special cases. However, unlike existing regularization based CF methods, our
approach can be used to also incorporate information such as attributes of the
users or the objects -- a limitation of existing regularization based CF
methods. We then provide novel representer theorems that we use to develop new
estimation methods. We provide learning algorithms based on low-rank
decompositions, and test them on a standard CF dataset. The experiments
indicate the advantages of generalizing the existing regularization based CF
methods to incorporate related information about users and objects. Finally, we
show that certain multi-task learning methods can be also seen as special cases
of our proposed approach
Solving finite-domain linear constraints in presence of the
In this paper, we investigate the possibility of improvement of the
widely-used filtering algorithm for the linear constraints in constraint
satisfaction problems in the presence of the alldifferent constraints. In many
cases, the fact that the variables in a linear constraint are also constrained
by some alldifferent constraints may help us to calculate stronger bounds of
the variables, leading to a stronger constraint propagation. We propose an
improved filtering algorithm that targets such cases. We provide a detailed
description of the proposed algorithm and prove its correctness. We evaluate
the approach on five different problems that involve combinations of the linear
and the alldifferent constraints. We also compare our algorithm to other
relevant approaches. The experimental results show a great potential of the
proposed improvement.Comment: 28 pages, 2 figure
A Non-Local Structure Tensor Based Approach for Multicomponent Image Recovery Problems
Non-Local Total Variation (NLTV) has emerged as a useful tool in variational
methods for image recovery problems. In this paper, we extend the NLTV-based
regularization to multicomponent images by taking advantage of the Structure
Tensor (ST) resulting from the gradient of a multicomponent image. The proposed
approach allows us to penalize the non-local variations, jointly for the
different components, through various matrix norms with .
To facilitate the choice of the hyper-parameters, we adopt a constrained convex
optimization approach in which we minimize the data fidelity term subject to a
constraint involving the ST-NLTV regularization. The resulting convex
optimization problem is solved with a novel epigraphical projection method.
This formulation can be efficiently implemented thanks to the flexibility
offered by recent primal-dual proximal algorithms. Experiments are carried out
for multispectral and hyperspectral images. The results demonstrate the
interest of introducing a non-local structure tensor regularization and show
that the proposed approach leads to significant improvements in terms of
convergence speed over current state-of-the-art methods
- …