1,389 research outputs found
First order algorithms in variational image processing
Variational methods in imaging are nowadays developing towards a quite
universal and flexible tool, allowing for highly successful approaches on tasks
like denoising, deblurring, inpainting, segmentation, super-resolution,
disparity, and optical flow estimation. The overall structure of such
approaches is of the form ; where the functional is a data fidelity term also
depending on some input data and measuring the deviation of from such
and is a regularization functional. Moreover is a (often linear)
forward operator modeling the dependence of data on an underlying image, and
is a positive regularization parameter. While is often
smooth and (strictly) convex, the current practice almost exclusively uses
nonsmooth regularization functionals. The majority of successful techniques is
using nonsmooth and convex functionals like the total variation and
generalizations thereof or -norms of coefficients arising from scalar
products with some frame system. The efficient solution of such variational
problems in imaging demands for appropriate algorithms. Taking into account the
specific structure as a sum of two very different terms to be minimized,
splitting algorithms are a quite canonical choice. Consequently this field has
revived the interest in techniques like operator splittings or augmented
Lagrangians. Here we shall provide an overview of methods currently developed
and recent results as well as some computational studies providing a comparison
of different methods and also illustrating their success in applications.Comment: 60 pages, 33 figure
Proximal methods for structured group features and correlation matrix nearness
Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura: junio de 2014Optimization is ubiquitous in real life as many of the strategies followed both by nature and
by humans aim to minimize a certain cost, or maximize a certain benefit. More specifically,
numerous strategies in engineering are designed according to a minimization problem, although
usually the problems tackled are convex with a di erentiable objective function, since these
problems have no local minima and they can be solved with gradient-based techniques. Nevertheless,
many interesting problems are not di erentiable, such as, for instance, projection problems
or problems based on non-smooth norms. An approach to deal with them can be found in
the theory of Proximal Methods (PMs), which are based on iterative local minimizations using
the Proximity Operator (ProxOp) of the terms that compose the objective function.
This thesis begins with a general introduction and a brief motivation of the work done. The state
of the art in PMs is thoroughly reviewed, defining the basic concepts from the very beginning
and describing the main algorithms, as far as possible, in a simple and self-contained way.
After that, the PMs are employed in the field of supervised regression, where regularized models
play a prominent role. In particular, some classical linear sparse models are reviewed and unified
under the point of view of regularization, namely the Lasso, the Elastic–Network, the Group
Lasso and the Group Elastic–Network. All these models are trained by minimizing an error
term plus a regularization term, and thus they fit nicely in the domain of PMs, as the structure of
the problem can be exploited by minimizing alternatively the di erent expressions that compose
the objective function, in particular using the Fast Iterative Shrinkage–Thresholding Algorithm
(FISTA). As a real-world application, it is shown how these models can be used to forecast wind
energy, where they yield both good predictions in terms of the error and, more importantly,
valuable information about the structure and distribution of the relevant features.
Following with the regularized learning approach, a new regularizer is proposed, called the
Group Total Variation, which is a group extension of the classical Total Variation regularizer
and thus it imposes constancy over groups of features. In order to deal with it, an approach to
compute its ProxOp is derived. Moreover, it is shown that this regularizer can be used directly
to clean noisy multidimensional signals (such as colour images) or to define a new linear model,
the Group Fused Lasso (GFL), which can be then trained using FISTA. It is also exemplified
how this model, when applied to regression problems, is able to provide solutions that identify
the underlying problem structure. As an additional result of this thesis, a public software
implementation of the GFL model is provided.
The PMs are also applied to the Nearest Correlation Matrix problem under observation uncertainty.
The original problem consists in finding the correlation matrix which is nearest to the
true empirical one. Some variants introduce weights to adapt the confidence given to each entry
of the matrix; with a more general perspective, in this thesis the problem is explored directly
considering uncertainty on the observations, which is formalized as a set of intervals where the
measured matrices lie. Two di erent variants are defined under this framework: a robust approach
called the Robust Nearest Correlation Matrix (which aims to minimize the worst-case
scenario) and an exploratory approach, the Exploratory Nearest Correlation Matrix (which focuses
on the best-case scenario). It is shown how both optimization problems can be solved
using the Douglas–Rachford PM with a suitable splitting of the objective functions.
The thesis ends with a brief overall discussion and pointers to further work.La optimización está presente en todas las facetas de la vida, de hecho muchas de las estrategias
tanto de la naturaleza como del ser humano pretenden minimizar un cierto coste, o maximizar
un cierto beneficio. En concreto, multitud de estrategias en ingeniería se diseñan según problemas
de minimización, que habitualmente son problemas convexos con una función objetivo
diferenciable, puesto que en ese caso no hay mínimos locales y los problemas pueden resolverse
mediante técnicas basadas en gradiente. Sin embargo, hay muchos problemas interesantes que
no son diferenciables, como por ejemplo problemas de proyección o basados en normas no suaves.
Una aproximación para abordar estos problemas son los Métodos Proximales (PMs), que
se basan en minimizaciones locales iterativas utilizando el Operador de Proximidad (ProxOp)
de los términos de la función objetivo.
La tesis comienza con una introducción general y una breve motivación del trabajo hecho. Se
revisa en profundidad el estado del arte en PMs, definiendo los conceptos básicos y describiendo
los algoritmos principales, dentro de lo posible, de forma simple y auto-contenida.
Tras ello, se emplean los PMs en el campo de la regresión supervisada, donde los modelos regularizados
tienen un papel prominente. En particular, se revisan y unifican bajo esta perspectiva
de regularización algunos modelos lineales dispersos clásicos, a saber, Lasso, Elastic–Network,
Lasso Grupal y Elastic–Network Grupal. Todos estos modelos se entrenan minimizando un término
de error y uno de regularización, y por tanto encajan perfectamente en el dominio de los
PMs, ya que la estructura del problema puede ser aprovechada minimizando alternativamente las
diferentes expresiones que componen la función objetivo, en particular mediante el Algoritmo
Fast Iterative Shrinkage–Thresholding (FISTA). Como aplicación al mundo real, se muestra que
estos modelos pueden utilizarse para predecir energía eólica, donde proporcionan tanto buenos
resultados en términos del error como información valiosa sobre la estructura y distribución de
las características relevantes.
Siguiendo con esta aproximación, se propone un nuevo regularizador, llamado Variación Total
Grupal, que es una extensión grupal del regularizador clásico de Variación Total y que por
tanto induce constancia sobre grupos de características. Para aplicarlo, se desarrolla una aproximación
para calcular su ProxOp. Además, se muestra que este regularizador puede utilizarse
directamente para limpiar señales multidimensionales ruidosas (como imágenes a color) o para
definir un nuevo modelo lineal, el Fused Lasso Grupal (GFL), que se entrena con FISTA. Se
ilustra cómo este modelo, cuando se aplica a problemas de regresión, es capaz de proporcionar
soluciones que identifican la estructura subyacente del problema. Como resultado adicional de
esta tesis, se publica una implementación software del modelo GFL.
Asimismo, se aplican los PMs al problema de Matriz de Correlación Próxima (NCM) bajo incertidumbre.
El problema original consiste en encontrar la matriz de correlación más cercana a
la empírica verdadera. Algunas variantes introducen pesos para ajustar la confianza que se da a
cada entrada de la matriz; con un carácter más general, en esta tesis se explora el problema considerando
incertidumbre en las observaciones, que se formaliza como un conjunto de intervalos
en el que se encuentran las matrices medidas. Bajo este marco se definen dos variantes: una
aproximación robusta llamada NCM Robusta (que minimiza el caso peor) y una exploratoria,
NCM Exploratoria (que se centra en el caso mejor). Ambos problemas de optimización pueden
resolverse con el PM de Douglas–Rachford y una partición adecuada de las funciones objetivo.
La tesis concluye con una discusión global y referencias a trabajo futur
Wavelet and Multiscale Methods
[no abstract available
Fast Algorithms for the computation of Fourier Extensions of arbitrary length
Fourier series of smooth, non-periodic functions on are known to
exhibit the Gibbs phenomenon, and exhibit overall slow convergence. One way of
overcoming these problems is by using a Fourier series on a larger domain, say
with , a technique called Fourier extension or Fourier
continuation. When constructed as the discrete least squares minimizer in
equidistant points, the Fourier extension has been shown shown to converge
geometrically in the truncation parameter . A fast algorithm has been described to compute Fourier extensions for the case
where , compared to for solving the dense discrete
least squares problem. We present two algorithms for
the computation of these approximations for the case of general , made
possible by exploiting the connection between Fourier extensions and Prolate
Spheroidal Wave theory. The first algorithm is based on the explicit
computation of so-called periodic discrete prolate spheroidal sequences, while
the second algorithm is purely algebraic and only implicitly based on the
theory
A fixed-point policy-iteration-type algorithm for symmetric nonzero-sum stochastic impulse control games
Nonzero-sum stochastic differential games with impulse controls offer a realistic and far-reaching modelling framework for applications within finance, energy markets, and other areas, but the difficulty in solving such problems has hindered their proliferation. Semi-analytical approaches make strong assumptions pertaining to very particular cases. To the author’s best knowledge, the only numerical method in the literature is the heuristic one we put forward in Aïd et al (ESAIM Proc Surv 65:27–45, 2019) to solve an underlying system of quasi-variational inequalities. Focusing on symmetric games, this paper presents a simpler, more precise and efficient fixed-point policy-iteration-type algorithm which removes the strong dependence on the initial guess and the relaxation scheme of the previous method. A rigorous convergence analysis is undertaken with natural assumptions on the players strategies, which admit graph-theoretic interpretations in the context of weakly chained diagonally dominant matrices. A novel provably convergent single-player impulse control solver is also provided. The main algorithm is used to compute with high precision equilibrium payoffs and Nash equilibria of otherwise very challenging problems, and even some which go beyond the scope of the currently available theory
Optimal market making under partial information and numerical methods for impulse control games with applications
The topics treated in this thesis are inherently two-fold. The first part considers the problem of a market maker who wants to optimally set bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders she receives depend not only on the spreads she quotes, but also on unobservable factors modelled by a hidden Markov chain. This stochastic control problem under partial information is solved by means of stochastic filtering, control and piecewise-deterministic Markov processes theory. The value function is characterized as the unique continuous viscosity solution of its dynamic programming equation. Afterwards, the analogous full information problem is solved and results are compared numerically through a concrete example. The optimal full information spreads are shown to be biased when the exact market regime is unknown, as the market maker needs to adjust for additional regime uncertainty in terms of P&L sensitivity and observable order ow volatility.
The second part deals with numerically solving nonzero-sum stochastic differential games with impulse controls. These offer a realistic and far-reaching modelling framework for applications within finance, energy markets and other areas, but the diffculty in solving such problems has hindered their proliferation. Semi-analytical approaches make strong assumptions pertaining very particular cases. To the author's best knowledge, there are no numerical methods available in the literature. A policy-iteration-type solver is proposed to solve an underlying system of quasi-variational inequalities, and it is validated numerically with reassuring results. In particular, it is observed that the algorithm does not enjoy global convergence and a heuristic methodology is proposed to construct initial guesses.
Eventually, the focus is put on games with a symmetric structure and a substantially improved version of the former algorithm is put forward. A rigorous convergence analysis is undertaken with natural assumptions on the players strategies, which admit graph-theoretic interpretations in the context of weakly chained diagonally dominant matrices. A provably convergent single-player impulse control solver, often outperforming classical policy iteration, is also provided. The main algorithm is used to compute with high precision equilibrium payoffs and Nash equilibria of otherwise too challenging problems, and even some for which results go beyond the scope of all the currently available theory
- …