30,177 research outputs found
Likelihood decision functions
In both classical and Bayesian approaches, statistical inference is unified and generalized by the corresponding decision theory. This is not the case for the likelihood approach to statistical inference, in spite of the manifest success of the likelihood methods in statistics. The goal of the present work is to fill this gap, by extending the likelihood approach in order to cover decision making as well. The resulting decision functions, called likelihood decision functions, generalize the usual likelihood methods (such as ML estimators and LR tests), in the sense that these methods appear as the likelihood decision functions in particular decision problems. In general, the likelihood decision functions maintain some key properties of the usual
likelihood methods, such as equivariance and asymptotic optimality. By unifying and generalizing the likelihood approach to statistical inference, the present work offers a new perspective on statistical methodology and on the connections among likelihood methods
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
We consider linear inverse problems where the solution is assumed to have a
sparse expansion on an arbitrary pre-assigned orthonormal basis. We prove that
replacing the usual quadratic regularizing penalties by weighted l^p-penalties
on the coefficients of such expansions, with 1 < or = p < or =2, still
regularizes the problem. If p < 2, regularized solutions of such l^p-penalized
problems will have sparser expansions, with respect to the basis under
consideration. To compute the corresponding regularized solutions we propose an
iterative algorithm that amounts to a Landweber iteration with thresholding (or
nonlinear shrinkage) applied at each iteration step. We prove that this
algorithm converges in norm. We also review some potential applications of this
method.Comment: 30 pages, 3 figures; this is version 2 - changes with respect to v1:
small correction in proof (but not statement of) lemma 3.15; description of
Besov spaces in intro and app A clarified (and corrected); smaller pointsize
(making 30 instead of 38 pages
Likelihood Analysis of Power Spectra and Generalized Moment Problems
We develop an approach to spectral estimation that has been advocated by
Ferrante, Masiero and Pavon and, in the context of the scalar-valued covariance
extension problem, by Enqvist and Karlsson. The aim is to determine the power
spectrum that is consistent with given moments and minimizes the relative
entropy between the probability law of the underlying Gaussian stochastic
process to that of a prior. The approach is analogous to the framework of
earlier work by Byrnes, Georgiou and Lindquist and can also be viewed as a
generalization of the classical work by Burg and Jaynes on the maximum entropy
method. In the present paper we present a new fast algorithm in the general
case (i.e., for general Gaussian priors) and show that for priors with a
specific structure the solution can be given in closed form.Comment: 17 pages, 4 figure
On a continuation approach in Tikhonov regularization and its application in piecewise-constant parameter identification
We present a new approach to convexification of the Tikhonov regularization
using a continuation method strategy. We embed the original minimization
problem into a one-parameter family of minimization problems. Both the penalty
term and the minimizer of the Tikhonov functional become dependent on a
continuation parameter.
In this way we can independently treat two main roles of the regularization
term, which are stabilization of the ill-posed problem and introduction of the
a priori knowledge. For zero continuation parameter we solve a relaxed
regularization problem, which stabilizes the ill-posed problem in a weaker
sense. The problem is recast to the original minimization by the continuation
method and so the a priori knowledge is enforced.
We apply this approach in the context of topology-to-shape geometry
identification, where it allows to avoid the convergence of gradient-based
methods to a local minima. We present illustrative results for magnetic
induction tomography which is an example of PDE constrained inverse problem
The information bottleneck method
We define the relevant information in a signal as being the
information that this signal provides about another signal y\in \Y. Examples
include the information that face images provide about the names of the people
portrayed, or the information that speech sounds provide about the words
spoken. Understanding the signal requires more than just predicting , it
also requires specifying which features of \X play a role in the prediction.
We formalize this problem as that of finding a short code for \X that
preserves the maximum information about \Y. That is, we squeeze the
information that \X provides about \Y through a `bottleneck' formed by a
limited set of codewords \tX. This constrained optimization problem can be
seen as a generalization of rate distortion theory in which the distortion
measure d(x,\x) emerges from the joint statistics of \X and \Y. This
approach yields an exact set of self consistent equations for the coding rules
X \to \tX and \tX \to \Y. Solutions to these equations can be found by a
convergent re-estimation method that generalizes the Blahut-Arimoto algorithm.
Our variational principle provides a surprisingly rich framework for discussing
a variety of problems in signal processing and learning, as will be described
in detail elsewhere
Recursive Aggregation of Estimators by Mirror Descent Algorithm with Averaging
We consider a recursive algorithm to construct an aggregated estimator from a
finite number of base decision rules in the classification problem. The
estimator approximately minimizes a convex risk functional under the
l1-constraint. It is defined by a stochastic version of the mirror descent
algorithm (i.e., of the method which performs gradient descent in the dual
space) with an additional averaging. The main result of the paper is an upper
bound for the expected accuracy of the proposed estimator. This bound is of the
order with an explicit and small constant factor, where
is the dimension of the problem and stands for the sample size. A similar
bound is proved for a more general setting that covers, in particular, the
regression model with squared loss.Comment: 29 pages; mai 200
Adaptive complexity regularization for linear inverse problems
We tackle the problem of building adaptive estimation procedures for
ill-posed inverse problems. For general regularization methods depending on
tuning parameters, we construct a penalized method that selects the optimal
smoothing sequence without prior knowledge of the regularity of the function to
be estimated. We provide for such estimators oracle inequalities and optimal
rates of convergence. This penalized approach is applied to Tikhonov
regularization and to regularization by projection.Comment: Published in at http://dx.doi.org/10.1214/07-EJS115 the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
On the Optimal Control of the Free Boundary Problems for the Second Order Parabolic Equations. I.Well-posedness and Convergence of the Method of Lines
We develop a new variational formulation of the inverse Stefan problem, where
information on the heat flux on the fixed boundary is missing and must be found
along with the temperature and free boundary. We employ optimal control
framework, where boundary heat flux and free boundary are components of the
control vector, and optimality criteria consists of the minimization of the sum
of -norm declinations from the available measurement of the temperature
flux on the fixed boundary and available information on the phase transition
temperature on the free boundary. This approach allows one to tackle situations
when the phase transition temperature is not known explicitly, and is available
through measurement with possible error. It also allows for the development of
iterative numerical methods of least computational cost due to the fact that
for every given control vector, the parabolic PDE is solved in a fixed region
instead of full free boundary problem. We prove well-posedness in Sobolev
spaces framework and convergence of discrete optimal control problems to the
original problem both with respect to cost functional and control
- …