An Algorithm for Unsupervised Learning via Normal Mixture Models

: We consider the approach to unsupervised learning whereby a normal mixture model is fitted to the data by maximum likelihood. An algorithm called NMM is presented that enables the normal mixture model with either restricted or unrestricted component covariance matrices to be fitted to a given data set. The algorithm automatically handles the problem of the specification of initial values for the parameters in the iterative fitting of the model within the framework of the EM algorithm. The algorithm also has the provision to carry a test for the number of components on the basis of the likelihood ratio statistic. Keywords: Mixture models, Maximum likelihood, EM algorithm, Likelihood ratio test. Area of Interest: Concept Formation and Classification. 1 Introduction In this paper we consider the development of an algorithm for the fitting of a normal mixture model in the absence of data on entities that have been classified with respect to the components of the mixture. This is usual..

Clustering Via Normal Mixture Models

We consider a model-based approach to clustering, whereby each observation is assumed to have arisen from an underlying mixture of a finite number of distributions. The number of components in this mixture model corresponds to the number of clusters to be imposed on the data. A common assumption is to take the component distributions to be multivariate normal with perhaps some restrictions on the component covariance matrices. The model can be fitted to the data using maximum likelihood implemented via the EM algorithm. There is a number of computational issues associated with the fitting, including the specification of initial starting points for the EM algorithm and the carrying out of tests for the number of components in the final version of the model. We shall discuss some of these problems and describe an algorithm that attempts to handle them automatically. 1. INTRODUCTION In some applications of mixture models, questions related to clustering may arise only after the mixture mo..

THE EMMIX SOFTWARE FOR THE FITTING OF MIXTURES OF NORMAL AND t-COMPONENTS

We consider the fitting of normal or t-component mixturemodels to multivariate data, using maximum likelihood via the EM algorithm. This approach requires the initial specification of an initial estimate of the vector of unknown parameters, or equivalently, of an initial classification of the data with respect to the components of the mixturemodel under fit. We describe an algorithm called EMMIX that automatically undertakes this fitting, including the provision of suitable initial values if not supplied by the user. The EMMIX algorithm has several options, including the option to carry out a resampling-based test for the number of components in the mixture model

Standard errors of fitted component means of normal mixtures

In this paper use consider the problem of providing standard errors of the component means in normal mixture models fitted to univariate or multivariate data by maximum likelihood via the EM algorithm. Two methods of estimation of the standard errors are considered: the standard information-based method and the computationally-intensive bootstrap method. They are compared empirically by their application to three real data sets and by a small-scale Monte Carlo experiment