41,981 research outputs found
Doubly Robust Smoothing of Dynamical Processes via Outlier Sparsity Constraints
Coping with outliers contaminating dynamical processes is of major importance
in various applications because mismatches from nominal models are not uncommon
in practice. In this context, the present paper develops novel fixed-lag and
fixed-interval smoothing algorithms that are robust to outliers simultaneously
present in the measurements {\it and} in the state dynamics. Outliers are
handled through auxiliary unknown variables that are jointly estimated along
with the state based on the least-squares criterion that is regularized with
the -norm of the outliers in order to effect sparsity control. The
resultant iterative estimators rely on coordinate descent and the alternating
direction method of multipliers, are expressed in closed form per iteration,
and are provably convergent. Additional attractive features of the novel doubly
robust smoother include: i) ability to handle both types of outliers; ii)
universality to unknown nominal noise and outlier distributions; iii)
flexibility to encompass maximum a posteriori optimal estimators with reliable
performance under nominal conditions; and iv) improved performance relative to
competing alternatives at comparable complexity, as corroborated via simulated
tests.Comment: Submitted to IEEE Trans. on Signal Processin
Inference via low-dimensional couplings
We investigate the low-dimensional structure of deterministic transformations
between random variables, i.e., transport maps between probability measures. In
the context of statistics and machine learning, these transformations can be
used to couple a tractable "reference" measure (e.g., a standard Gaussian) with
a target measure of interest. Direct simulation from the desired measure can
then be achieved by pushing forward reference samples through the map. Yet
characterizing such a map---e.g., representing and evaluating it---grows
challenging in high dimensions. The central contribution of this paper is to
establish a link between the Markov properties of the target measure and the
existence of low-dimensional couplings, induced by transport maps that are
sparse and/or decomposable. Our analysis not only facilitates the construction
of transformations in high-dimensional settings, but also suggests new
inference methodologies for continuous non-Gaussian graphical models. For
instance, in the context of nonlinear state-space models, we describe new
variational algorithms for filtering, smoothing, and sequential parameter
inference. These algorithms can be understood as the natural
generalization---to the non-Gaussian case---of the square-root
Rauch-Tung-Striebel Gaussian smoother.Comment: 78 pages, 25 figure
Smoothing Proximal Gradient Method for General Structured Sparse Learning
We study the problem of learning high dimensional regression models
regularized by a structured-sparsity-inducing penalty that encodes prior
structural information on either input or output sides. We consider two widely
adopted types of such penalties as our motivating examples: 1) overlapping
group lasso penalty, based on the l1/l2 mixed-norm penalty, and 2) graph-guided
fusion penalty. For both types of penalties, due to their non-separability,
developing an efficient optimization method has remained a challenging problem.
In this paper, we propose a general optimization approach, called smoothing
proximal gradient method, which can solve the structured sparse regression
problems with a smooth convex loss and a wide spectrum of
structured-sparsity-inducing penalties. Our approach is based on a general
smoothing technique of Nesterov. It achieves a convergence rate faster than the
standard first-order method, subgradient method, and is much more scalable than
the most widely used interior-point method. Numerical results are reported to
demonstrate the efficiency and scalability of the proposed method.Comment: arXiv admin note: substantial text overlap with arXiv:1005.471
- …