An empirical comparison of EM, SEM and MCMC performance for problematic Gaussian mixture likelihoods

We compare EM, SEM, and MCMC algorithms to estimate the parameters of the Gaussian mixture model. We focus on problems in estimation arising from the likelihood function having a sharp ridge or saddle points. We use both synthetic and empirical data with those features. The comparison includes Bayesian approaches with different prior specifications and various procedures to deal with label switching. Although the solutions provided by these stochastic algorithms are more often degenerate, we conclude that SEM and MCMC may display faster convergence and improve the ability to locate the global maximum of the likelihood function.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/47969/1/11222_2004_Article_5273887.pd

Fitting mixtures of linear regressions

In most applications, the parameters of a mixture of linear regression models are estimated by maximum likelihood using the expectation maximization (EM) algorithm. In this article, we propose the comparison of three algorithms to compute maximum likelihood estimates of the parameters of these models: the EM algorithm, the classification EM algorithm and the stochastic EM algorithm. The comparison of the three procedures was done through a simulation study of the performance (computational effort, statistical properties of estimators and goodness of fit) of these approaches on simulated data sets. Simulation results show that the choice of the approach depends essentially on the configuration of the true regression lines and the initialization of the algorithms

Novas empresas e criação de emprego: dois ensaios com modelos de mistura

JEL: C25, L26, M13, J23As novas empresas e o sector das pequenas e médias empresas constituem vectores fundamentais para o desenvolvimento das economias ocidentais. Neste contexto, o empresário responsável pela constituição de uma nova empresa desempenha um papel importante, com impacto no desempenho da economia. Com base em dados recolhidos para a economia portuguesa sobre os empresários que constituíram empresas em 2002, procede-se à construção de uma tipologia de empresários baseada nas suas motivações. Os resultados indicam a existência de três segmentos, com motivações e perfis distintos. As críticas a trabalhos anteriores são acolhidas neste estudo, através da selecção das variáveis de segmentação e da técnica estatística escolhida. O modelo de mistura com variáveis concomitantes utilizado inclui num único modelo probabilístico o processo de segmentação e caracterização dos segmentos, ao contrário do processo tradicional em duas etapas. Numa segunda fase, procede-se à avaliação do efeito das características do empresário sobre a criação de emprego, com base num modelo de mistura de regressões de Poisson. Este modelo incorpora dois aspectos inovadores: considera a população de empresários como heterogénea e admite para a variável Emprego uma distribuição de Poisson. Os resultados indicam a existência de três segmentos de empresários e sugerem efeitos positivos associados à experiência de gestão e de constituição de empresas, bem como às alianças entre empresas. O sexo e a nacionalidade dos empresários também apresentam efeitos significativos. A extensão do estudo a outras medidas de performance da empresa afigura-se importante para o aprofundamento do conhecimento nesta matéria.New businesses and the small and medium size enterprises sector are two fundamental paths for the development of western economies. In this context, the entrepreneur who starts a new business has an important role to play, which affects the performance of the economy. Data was collected for Portuguese entrepreneurs who started businesses in 2002 and is used as the input for a typology of entrepreneurs based on their motivations. Three different clusters were found, with specific motivations and profiles. Criticism on previous studies is taken into account, through the adequate selection of clustering variables and statistic model. The mixture model with concomitant variables includes in one probabilistic model the segmentation and profiling procedures, instead of the traditional two-steps process. The second part of the study concerns the assessment of the effect of entrepreneur characteristics on the employment. The mixture model of Poisson regressions used has two innovative features: accounts for heterogeneity in the population of entrepreneurs and the dependent variable – Employment – is modeled by the Poisson distribution. The results indicate three clusters and suggest positive effects of management experience and previous business start-ups, as well of business alliances, on the employment. Entrepreneur’s sex and nationality also have significant effects. The extension of the study methodology to other performance measures would be important to expand knowledge on this subject

Flexible Mixed-Effect Modeling of Functional Data, with Applications to Process Monitoring

High levels of automation in manufacturing industries are leading to data sets of increasing size and dimension. The challenge facing statisticians and field professionals is to develop methodology to help meet this demand. Functional data is one example of high-dimensional data characterized by observations recorded as a function of some continuous measure, such as time. An application considered in this thesis comes from the automotive industry. It involves a production process in which valve seats are force-fitted by a ram into cylinder heads of automobile engines. For each insertion, the force exerted by the ram is automatically recorded every fraction of a second for about two and a half seconds, generating a force profile. We can think of these profiles as individual functions of time summarized into collections of curves. The focus of this thesis is the analysis of functional process data such as the valve seat insertion example. A number of techniques are set forth. In the first part, two ways to model a single curve are considered: a b-spline fit via linear regression, and a nonlinear model based on differential equations. Each of these approaches is incorporated into a mixed effects model for multiple curves, and multivariate process monitoring techniques are applied to the predicted random effects in order to identify anomalous curves. In the second part, a Bayesian hierarchical model is used to cluster low-dimensional summaries of the curves into meaningful groups. The belief is that the clusters correspond to distinct types of processes (e.g. various types of “good” or “faulty” assembly). New observations can be assigned to one of these by calculating the probabilities of belonging to each cluster. Mahalanobis distances are used to identify new observations not belonging to any of the existing clusters. Synthetic and real data are used to validate the results

Essays on Finite Mixture Models

Finite mixture distributions are a weighted average of a ¯nite number of distributions. The latter are usually called the mixture components. The weights are usually described by a multinomial distribution and are sometimes called mixing proportions. The mixture components may be the same type of distributions with di®erent parameter values but they may also be completely di®erent distributions (Everitt and Hand, 1981; Titterington et al., 1985). Therefore, ¯nite mixture distributions are very °exible for modeling data. They are frequently used as a building block within many modern econometric models. The speci¯cation of the mixture distribution depends on the modeling problem at hand. In this thesis, we introduce new applications of ¯nite mixtures to deal with several di®erent modeling issues. Each chapter of the thesis focusses on a speci¯c modeling issue. The parameters of some of the resulting models can be estimated using standard techniques but for some of the chapters we need to develop new estimation and inference methods. To illustrate how the methods can be applied, we analyze at least one empirical data set for each approach. These data sets cover a wide range of research ¯elds, such as macroeconomics, marketing, and political science. We show the usefulness of the methods and, in some cases, the improvement over previous methods in the literature

Information Asymmetries: Three Essays in Market Microstructure

This dissertation aims at revisiting existing empirical market microstructure models for the measurement of information asymmetries on financial markets and to develop alternative estimation approaches that (1) reduce existing bias, (2) reduce data requirement, and (3) increase the applicability of these models. The first chapter of this thesis reconsiderates trade indicator models and is a joint work with Erik Theissen. More precisely, it reconsiderates the models of Madhavan Richardson, and Roomans (1997) and Huang and Stoll (1997). Trade indicator models divide the spread into an adverse selection component and remaining components. As a byproduct an estimate of the spread becomes available. It is a stylized fact that trade indicator models (e.g. Madhavan, Richardson, and Roomans (1997) and Huang and Stoll (1997)) underestimate the bid-ask spread. We argue that this negative bias is due to an endogeneity problem, which is caused by a negative correlation between the arrival of public information and trade direction. In our sample (the component stocks of the DAX 30 index) we find that the average correlation between these variables is -0.193. We develop modified estimators and show that they yield essentially unbiased spread estimates. The second and third chapters of this thesis build an entity and consider another way to measure information asymmetries on financial markets. In the second chapter, a joint work with Joachim Grammig, and Erik Theissen, we propose a methodology to estimate the probability of informed trading (PIN) that only requires data on the daily number of transactions (but not on the number of buyer-initiated and seller-initiated trades). Because maximum likelihood estimation of the model is problematic we propose a Bayesian estimation approach. We perform extensive simulations to evaluate the performance of our estimator. Our methodology increases the applicability of PIN estimation to situations in which the data necessary for trade classification is unavailable, or in which trade classification is inaccurate. The third chapter investigates information asymmetries in U.S. corporate bond markets using transaction data from the Trade Reporting and Compliance Engine (TRACE) for constituents of the S&P 500 in the first half-year of 2011. As a measurement of information asymmetry I employ the probability of informed trading (PIN) proposed by Easley, Kiefer, O’Hara, and Paperman (1996). In a cross-sectional regression of 4,155 fixed-income securities on bond characteristics, market variables, and stock statistics, I find that nearly 50% of the variation in PINs is explained. All estimated coefficients conform to expectations. While a comparison of PINs in bond and corresponding equity markets confirms prior findings of lower PINs on more active stock markets, it indicates the reverse for fixed-income securities: Less-frequently traded bonds exhibit lower PINs. These findings accord with there being lower transaction costs on less active bond markets as found by Goldstein, Hotchkiss, and Sirri (2007). However, as news probabilities for bonds from the same issuer and bonds and corresponding stocks differ significantly, I question the appropriateness of traditional models for measuring information asymmetries. The probability of informed trading might not be a suitable measure for highly fragmented markets such as the U.S. corporate bond market

Segmentation and Dimension Reduction: Exploratory and Model-Based Approaches

Representing the information in a data set in a concise way is an important part of data analysis. A variety of multivariate statistical techniques have been developed for this purpose, such as k-means clustering and principal components analysis. These techniques are often based on the principles of segmentation (partitioning the observations into distinct groups) and dimension reduction (constructing a low-dimensional representation of a data set). However, such techniques typically make no statistical assumptions on the process that generates the data; as a result, the statistical significance of the results is often unknown. In this thesis, we incorporate the modeling principles of segmentation and dimension reduction into statistical models. We thus develop new models that can summarize and explain the information in a data set in a simple way. The focus is on dimension reduction using bilinear parameter structures and techniques for clustering both modes of a two-mode data matrix. To illustrate the usefulness of the techniques, the thesis includes a variety of empirical applications in marketing, psychometrics, and political science. An important application is modeling the response behavior in surveys with rating scales, which provides novel insight into what kinds of response styles exist, and how substantive opinions vary among respondents. We find that our modeling approaches yield new techniques for data analysis that can be useful in a variety of applied fields