907 research outputs found
MDL Denoising Revisited
We refine and extend an earlier MDL denoising criterion for wavelet-based
denoising. We start by showing that the denoising problem can be reformulated
as a clustering problem, where the goal is to obtain separate clusters for
informative and non-informative wavelet coefficients, respectively. This
suggests two refinements, adding a code-length for the model index, and
extending the model in order to account for subband-dependent coefficient
distributions. A third refinement is derivation of soft thresholding inspired
by predictive universal coding with weighted mixtures. We propose a practical
method incorporating all three refinements, which is shown to achieve good
performance and robustness in denoising both artificial and natural signals.Comment: Submitted to IEEE Transactions on Information Theory, June 200
Model Selection in High-Dimensional Block-Sparse Linear Regression
Model selection is an indispensable part of data analysis dealing very
frequently with fitting and prediction purposes. In this paper, we tackle the
problem of model selection in a general linear regression where the parameter
matrix possesses a block-sparse structure, i.e., the non-zero entries occur in
clusters or blocks and the number of such non-zero blocks is very small
compared to the parameter dimension. Furthermore, a high-dimensional setting is
considered where the parameter dimension is quite large compared to the number
of available measurements. To perform model selection in this setting, we
present an information criterion that is a generalization of the Extended
Bayesian Information Criterion-Robust (EBIC-R) and it takes into account both
the block structure and the high-dimensionality scenario. The analytical steps
for deriving the EBIC-R for this setting are provided. Simulation results show
that the proposed method performs considerably better than the existing
state-of-the-art methods and achieves empirical consistency at large sample
sizes and/or at high-SNR.Comment: 5 pages, 2 figure
Model Selection for Geometric Fitting: Geometric Ale and Geometric MDL
Contrasting "geometric fitting", for which the noise level is taken as the asymptotic variable, with "statistical inference", for which the number of observations is taken as the asymptotic variable, we give a new definition of the "geometric AIC" and the "geometric MDL" as the counterparts of Akaike's AIC and Rissanen's MDL. We discuss various theoretical and practical problems that emerge from our analysis. Finally, we show, doing experiments using synthetic and real images, that the geometric MDL does not necessarily outperform the geometric AIC and that the two criteria have very different characteristics
- …