Nonparametric Identification and Estimation of Multivariate Mixtures

We study nonparametric identifiability of finite mixture models of k-variate data with M subpopulations, in which the components of the data vector are independent conditional on belonging to a subpopulation. We provide a sufficient condition for nonparametrically identifying M subpopulations when k>=3. Our focus is on the relationship between the number of values the components of the data vector can take on, and the number of identifiable subpopulations. Intuition would suggest that if the data vector can take many different values, then combining information from these different values helps identification. Hall and Zhou (2003) show, however, when k=2, two-component finite mixture models are not nonparametrically identifiable regardless of the number of the values the data vector can take. When k>=3, there emerges a link between the variation in the data vector, and the number of identifiable subpopulations: the number of identifiable subpopulations increases as the data vector takes on additional (different) values. This points to the possibility of identifying many components even when k=3, if the data vector has a continuously distributed element. Our identification method is constructive, and leads to an estimation strategy. It is not as efficient as the MLE, but can be used as the initial value of the optimization algorithm in computing the MLE. We also provide a sufficient condition for identifying the number of nonparametrically identifiable components, and develop a method for statistically testing and consistently estimating the number of nonparametrically identifiable components. We extend these procedures to develop a test for the number of components in binomial mixtures.finite mixture, binomial mixture, model selection, number of components, rank estimation

Maximum Score Type Estimators

This paper presents maximum score type estimators for linear, binomial, tobit and truncated regression models. These estimators estimate the normalized vector of slopes and do not provide the estimator of intercept, although it may appear in the model. Strong consistency is proved. In addition, in the case of truncated and tobit regression models, maximum score estimators allow restriction of the sample in order to make ordinary least squares method consistent.maximum score estimation, tobit, truncated, binomial, semiparametric

Graver Bases and Universal Gr\"obner Bases for Linear Codes

Two correspondences have been provided that associate any linear code over a finite field with a binomial ideal. In this paper, algorithms for computing their Graver bases and universal Gr\"obner bases are given. To this end, a connection between these binomial ideals and toric ideals will be established.Comment: 18 page

Group invariant inferred distributions via noncommutative probability

One may consider three types of statistical inference: Bayesian, frequentist, and group invariance-based. The focus here is on the last method. We consider the Poisson and binomial distributions in detail to illustrate a group invariance method for constructing inferred distributions on parameter spaces given observed results. These inferred distributions are obtained without using Bayes' method and in particular without using a joint distribution of random variable and parameter. In the Poisson and binomial cases, the final formulas for inferred distributions coincide with the formulas for Bayes posteriors with uniform priors.Comment: Published at http://dx.doi.org/10.1214/074921706000000563 in the IMS Lecture Notes--Monograph Series (http://www.imstat.org/publications/lecnotes.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

A Polyhedral Method to Compute All Affine Solution Sets of Sparse Polynomial Systems

To compute solutions of sparse polynomial systems efficiently we have to exploit the structure of their Newton polytopes. While the application of polyhedral methods naturally excludes solutions with zero components, an irreducible decomposition of a variety is typically understood in affine space, including also those components with zero coordinates. We present a polyhedral method to compute all affine solution sets of a polynomial system. The method enumerates all factors contributing to a generalized permanent. Toric solution sets are recovered as a special case of this enumeration. For sparse systems as adjacent 2-by-2 minors our methods scale much better than the techniques from numerical algebraic geometry

On Binomial Ideals associated to Linear Codes

Recently, it was shown that a binary linear code can be associated to a binomial ideal given as the sum of a toric ideal and a non-prime ideal. Since then two different generalizations have been provided which coincide for the binary case. In this paper, we establish some connections between the two approaches. In particular, we show that the corresponding code ideals are related by elimination. Finally, a new heuristic decoding method for linear codes over prime fields is discussed using Gr\"obner bases

Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models

Penalization of the likelihood by Jeffreys' invariant prior, or by a positive power thereof, is shown to produce finite-valued maximum penalized likelihood estimates in a broad class of binomial generalized linear models. The class of models includes logistic regression, where the Jeffreys-prior penalty is known additionally to reduce the asymptotic bias of the maximum likelihood estimator; and also models with other commonly used link functions such as probit and log-log. Shrinkage towards equiprobability across observations, relative to the maximum likelihood estimator, is established theoretically and is studied through illustrative examples. Some implications of finiteness and shrinkage for inference are discussed, particularly when inference is based on Wald-type procedures. A widely applicable procedure is developed for computation of maximum penalized likelihood estimates, by using repeated maximum likelihood fits with iteratively adjusted binomial responses and totals. These theoretical results and methods underpin the increasingly widespread use of reduced-bias and similarly penalized binomial regression models in many applied fields