The Classical Econometric Model

A compendium is presented of the various approaches that may be taken in deriving the estimators of the simultaneous-equations econometric model according to the principle of maximum likelihood. The structural equations of the model have the character both of a regression equation and of an errors-in-variables equation. This partly accounts for way in which the various approaches that have been followed appear to differ widely. In the process of achieving a synthesis of the methods of estimation, some elements that have been missing from the theory are supplied.

Algebraic statistical models

Many statistical models are algebraic in that they are defined in terms of polynomial constraints, or in terms of polynomial or rational parametrizations. The parameter spaces of such models are typically semi-algebraic subsets of the parameter space of a reference model with nice properties, such as for example a regular exponential family. This observation leads to the definition of an `algebraic exponential family'. This new definition provides a unified framework for the study of statistical models with algebraic structure. In this paper we review the ingredients to this definition and illustrate in examples how computational algebraic geometry can be used to solve problems arising in statistical inference in algebraic models

Symbolic Maximum Likelihood Estimation with Mathematica

Mathematica is a symbolic programming language that empowers the user to undertake complicated algebraic tasks. One such task is the derivation of maximum likelihood estimators, demonstrably an important topic in statistics at both the research and expository level. In this paper, a Mathematica package is provided that contains a function entitled SuperLog. This function utilises pattern-matching code that enhances Mathematica's ability to simplify expressions involving the natural logarithm of a product of algebraic terms. This enhancement to Mathematica's functionality can be of particular benefit for maximum likelihood estimation

A Bayesian information criterion for singular models

We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher-information matrices may fail to be invertible along other competing submodels. Such singular models do not obey the regularity conditions underlying the derivation of Schwarz's Bayesian information criterion (BIC) and the penalty structure in BIC generally does not reflect the frequentist large-sample behavior of their marginal likelihood. While large-sample theory for the marginal likelihood of singular models has been developed recently, the resulting approximations depend on the true parameter value and lead to a paradox of circular reasoning. Guided by examples such as determining the number of components of mixture models, the number of factors in latent factor models or the rank in reduced-rank regression, we propose a resolution to this paradox and give a practical extension of BIC for singular model selection problems

Moment Varieties of Gaussian Mixtures

The points of a moment variety are the vectors of all moments up to some order of a family of probability distributions. We study this variety for mixtures of Gaussians. Following up on Pearson's classical work from 1894, we apply current tools from computational algebra to recover the parameters from the moments. Our moment varieties extend objects familiar to algebraic geometers. For instance, the secant varieties of Veronese varieties are the loci obtained by setting all covariance matrices to zero. We compute the ideals of the 5-dimensional moment varieties representing mixtures of two univariate Gaussians, and we offer a comparison to the maximum likelihood approach.Comment: 17 pages, 2 figure