A functional approach to estimation of the parameters of generalized negative binomial and gamma distributions

The generalized negative binomial distribution (GNB) is a new flexible family of discrete distributions that are mixed Poisson laws with the mixing generalized gamma (GG) distributions. This family of discrete distributions is very wide and embraces Poisson distributions, negative binomial distributions, Sichel distributions, Weibull--Poisson distributions and many other types of distributions supplying descriptive statistics with many flexible models. These distributions seem to be very promising for the statistical description of many real phenomena. GG distributions are widely applied in signal and image processing and other practical problems. The statistical estimation of the parameters of GNB and GG distributions is quite complicated. To find estimates, the methods of moments or maximum likelihood can be used as well as two-stage grid EM-algorithms. The paper presents a methodology based on the search for the best distribution using the minimization of $\ell^p$-distances and $L^p$-metrics for GNB and GG distributions, respectively. This approach, first, allows to obtain parameter estimates without using grid methods and solving systems of nonlinear equations and, second, yields not point estimates as the methods of moments or maximum likelihood do, but the estimate for the density function. In other words, within this approach the set of decisions is not a Euclidean space, but a functional space.Comment: 13 pages, 6 figures, The XXI International Conference on Distributed Computer and Communication Networks: Control, Computation, Communications (DCCN 2018

On Simple Estimators of the alpha-mu Fading Distribution

In this letter, new estimators of the alpha-mu distribution are derived based on the skewness of the logarithmic alpha-mu distribution using the moments method. This distribution has been recently proposed to model the received field strength in nonlinear propagation mediums. Therefore, simple and computationally efficient estimators are required to infer the parameters of the received signal amplitude distribution in nonlinear wireless communication propagation channels. The performance of these new estimators is compared to that obtained with the estimators calculated with the moments method of the alpha-mu distribution by solving numerically transcendental equations. These estimators are easily evaluated with simple expressions.Reig, J.; Rubio Arjona, L. (2011). On Simple Estimators of the alpha-mu Fading Distribution. IEEE Transactions on Communications. 59(12):3254-3258. doi:10.1109/TCOMM.2011.080111.090223S32543258591

A Robust Zero-point Attraction LMS Algorithm on Near Sparse System Identification

The newly proposed $l_1$ norm constraint zero-point attraction Least Mean Square algorithm (ZA-LMS) demonstrates excellent performance on exact sparse system identification. However, ZA-LMS has less advantage against standard LMS when the system is near sparse. Thus, in this paper, firstly the near sparse system modeling by Generalized Gaussian Distribution is recommended, where the sparsity is defined accordingly. Secondly, two modifications to the ZA-LMS algorithm have been made. The $l_1$ norm penalty is replaced by a partial $l_1$ norm in the cost function, enhancing robustness without increasing the computational complexity. Moreover, the zero-point attraction item is weighted by the magnitude of estimation error which adjusts the zero-point attraction force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS (DWZA-LMS) algorithm is further proposed, which shows better performance on near sparse system identification. In addition, the mean square performance of DWZA-LMS algorithm is analyzed. Finally, computer simulations demonstrate the effectiveness of the proposed algorithm and verify the result of theoretical analysis.Comment: 20 pages, 11 figure

Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions

The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both the distribution mean and the scatter (or the precision) matrix are unknown is still an open problem. In robust estimation, imposing classical constraints on the precision matrix, such as sparsity, has been limited by the non-convexity of the resulting cost function. This paper tackles these issues from an optimization viewpoint by proposing a convex formulation with well-established convergence properties. We embed our analysis in a noisy scenario where robustness is induced by modelling multiplicative perturbations. The resulting framework is flexible as it combines a variety of regularizations for the precision matrix, the mean and model perturbations. This paper presents proof of the desired theoretical properties, specifies the conditions preserving these properties for different regularization choices and designs a general proximal primal-dual optimization strategy. The experiments show a more accurate precision and covariance matrix estimation with similar performance for the mean vector parameter compared to Tyler's M-estimator. In a high-dimensional setting, the proposed method outperforms the classical GLASSO, one of its robust extensions, and the regularized Tyler's estimator

A flexible space-variant anisotropic regularisation for image restoration with automated parameter selection

We propose a new space-variant anisotropic regularisation term for variational image restoration, based on the statistical assumption that the gradients of the target image distribute locally according to a bivariate generalised Gaussian distribution. The highly flexible variational structure of the corresponding regulariser encodes several free parameters which hold the potential for faithfully modelling the local geometry in the image and describing local orientation preferences. For an automatic estimation of such parameters, we design a robust maximum likelihood approach and report results on its reliability on synthetic data and natural images. For the numerical solution of the corresponding image restoration model, we use an iterative algorithm based on the Alternating Direction Method of Multipliers (ADMM). A suitable preliminary variable splitting together with a novel result in multivariate non-convex proximal calculus yield a very efficient minimisation algorithm. Several numerical results showing significant quality-improvement of the proposed model with respect to some related state-of-the-art competitors are reported, in particular in terms of texture and detail preservation

Space-variant Generalized Gaussian Regularization for Image Restoration

We propose a new space-variant regularization term for variational image restoration based on the assumption that the gradient magnitudes of the target image distribute locally according to a half-Generalized Gaussian distribution. This leads to a highly flexible regularizer characterized by two per-pixel free parameters, which are automatically estimated from the observed image. The proposed regularizer is coupled with either the $L_2$ or the $L_1$ fidelity terms, in order to effectively deal with additive white Gaussian noise or impulsive noises such as, e.g, additive white Laplace and salt and pepper noise. The restored image is efficiently computed by means of an iterative numerical algorithm based on the alternating direction method of multipliers. Numerical examples indicate that the proposed regularizer holds the potential for achieving high quality restorations for a wide range of target images characterized by different gradient distributions and for the different types of noise considered

On the Method of Logarithmic Cumulants for Parametric Probability Density Function Estimation

Parameter estimation of probability density functions is one of the major steps in the mainframe of statistical image and signal processing. In this report we explore the properties and limitations of the recently proposed method of logarithmic cumulants (MoLC) parameter estimation approach which is an alternative to the classical maximum likelihood (ML) and method of moments (MoM) approaches. We derive the general sufficient condition of strong consistency of MoLC estimates which represents an important asymptotic property of any statistical estimator. With its help we demonstrate the strong consistency of MoLC estimates for a selection of widely used distribution families originating (but not restricted to) synthetic aperture radar (SAR) image processing. We then derive the analytical conditions of applicability of MoLC to samples generated from several distribution families in our selection. Finally, we conduct various synthetic and real data experiments to assess the comparative properties, applicability and small sample performance of MoLC notably for the generalized gamma and K family of distributions. Supervised image classification experiments are considered for medical ultrasound and remote sensing SAR imagery. The obtained results suggest MoLC to be a feasible yet not universally applicable alternative to MoM that can be considered when the direct ML approach turns out to be unfeasible.L'estimation de paramètres de fonctions de densité de probabilité est une étape majeure dans le domaine du traitement statistique du signal et des images. Dans ce rapport, nous étudions les propriétés et les limites de l'estimation de paramètres par la méthode des cumulants logarithmiques (MoLC), qui est une alternative à la fois au maximum de vraisemblance (MV) classique et à la méthode des moments. Nous dérivons la condition générale suffisante de consistance forte de l'estimation par la méthode MoLC, qui représente une propriété asymptotique importante de tout estimateur statistique. Grâce à cela, nous démontrons la consistance forte de l'estimation par la méthode MoLC pour une sélection de familles de distributions particulièrement adaptées (mais non restreintes) au traitement d'images acquises par radar à synthèse d'ouverture (RSO). Nous dérivons ensuite les conditions analytiques d'applicabilité de la méthode MoLC à des échantillons générés qui suivent les lois des différentes familles de distribution de notre sélection. Enfin, nous testons la méthode MoLC sur des données synthétiques et réelles afin de comparer les différentes propriétés inhérentes aux différents types d'images, l'applicabilité de la méthode et les effets d'un nombre restreint d'échantillons. Nous avons, en particulier, considéré les distributions gamma généralisée et K. Comme exemple d'application, nous avons réalisé des classifications supervisées d'images médicales à ultrason ainsi que d'images de télédétection acquises par des capteurs RSO. Les résultats obtenus montrent que la méthode MoLC est une bonne alternative à la méthode des moments, bien qu'elle contienne certaines limitations. Elle est particulièrement utile lorsqu'une approche directe par MV n'est pas possible

Space adaptive and hierarchical Bayesian variational models for image restoration

The main contribution of this thesis is the proposal of novel space-variant regularization or penalty terms motivated by a strong statistical rational. In light of the connection between the classical variational framework and the Bayesian formulation, we will focus on the design of highly flexible priors characterized by a large number of unknown parameters. The latter will be automatically estimated by setting up a hierarchical modeling framework, i.e. introducing informative or non-informative hyperpriors depending on the information at hand on the parameters. More specifically, in the first part of the thesis we will focus on the restoration of natural images, by introducing highly parametrized distribution to model the local behavior of the gradients in the image. The resulting regularizers hold the potential to adapt to the local smoothness, directionality and sparsity in the data. The estimation of the unknown parameters will be addressed by means of non-informative hyperpriors, namely uniform distributions over the parameter domain, thus leading to the classical Maximum Likelihood approach. In the second part of the thesis, we will address the problem of designing suitable penalty terms for the recovery of sparse signals. The space-variance in the proposed penalties, corresponding to a family of informative hyperpriors, namely generalized gamma hyperpriors, will follow directly from the assumption of the independence of the components in the signal. The study of the properties of the resulting energy functionals will thus lead to the introduction of two hybrid algorithms, aimed at combining the strong sparsity promotion characterizing non-convex penalty terms with the desirable guarantees of convex optimization

Applied Harmonic Analysis and Sparse Approximation

Efficiently analyzing functions, in particular multivariate functions, is a key problem in applied mathematics. The area of applied harmonic analysis has a significant impact on this problem by providing methodologies both for theoretical questions and for a wide range of applications in technology and science, such as image processing. Approximation theory, in particular the branch of the theory of sparse approximations, is closely intertwined with this area with a lot of recent exciting developments in the intersection of both. Research topics typically also involve related areas such as convex optimization, probability theory, and Banach space geometry. The workshop was the continuation of a first event in 2012 and intended to bring together world leading experts in these areas, to report on recent developments, and to foster new developments and collaborations