Finite mixtures for the modelling of heterogeneity in ordinal response

Die Modellierung von Heterogenität ist ein entscheidender Aspekt in jeder statistischen Analyse. Um ein geeignetes Modell zu finden, ist es notwendig, möglichst alle relevanten Strukturen und Einflussgrößen einzubeziehen. Die meisten statistischen Modelle können leicht beobachtete Strukturen einbinden, jedoch haben sie oft Schwierigkeiten latente Strukturen abzubilden. Misch-Modelle können Heterogenität berücksichtigen, die aus zugrunde liegenden latenten Strukturen entstehen, wie etwa die unbeobachtete Zugehörigkeit zu verschiedenen Gruppen oder unterschiedliches Antwortverhalten. Mit dieser Doktorarbeit möchte ich einen Beitrag für die Verwendung von Misch-Modellen zur Modellierung von Heterogenität bei ordinalen Zielgrößen leisten und Variablen Selektion in diesem Kontext durchführen. Zuerst konzentriere ich mich auf Heterogenität, die bei Umfragen auftritt, wenn beispielsweise die Befragten bei der Wahl einer bestimmten geordneten Kategorie unsicher sind. In diesem Fall bestehen die Misch-Modelle üblicherweise aus einer Präferenz-Komponente und einer Unsicherheits-Komponente. Ein Gewicht bestimmt die Neigung jeder Person zu einer dieser beiden Komponenten zu gehören. Das existierende CUB Modell verwendet eine verschobene Binomialverteilung für die erste und eine Gleichverteilung für die zweite Komponente. Im vorgeschlagenem CUP Modell wird die Präferenz-Komponente mit einem beliebigen ordinalen Modell wie dem kumulativen Logit Modell ersetzt, um eine höhere Flexibilität in der Präferenz-Komponente zu erreichen. Im BetaBin Modell wird das Konzept der Unsicherheit als zufällige Wahl einer Kategorie so erweitert, dass Unsicherheit auch die Tendenz zu der zentralen Kategorie und extremen Kategorien erfasst. Auf diese Weise wird die Gleichverteilung des CUP Modells durch einer flexiblere, beschränkte Beta-Binomial Verteilung ersetzt. Als zweites zeige ich, wie diskrete Cure Modelle verwendet werden können, um in der Survival-Analyse für diskrete Zeit mit Heterogenität umzugehen, die aus der unbeobachteten Zugehörigkeit zu verschiedenen Gruppen entsteht. "Cure" bezeichnet dabei den Umstand, dass eine Gruppe von Beobachtungen "geheilt ist" oder als sogenannte Langzeit-Überlebende charakterisiert ist, während die andere Gruppe dem Risiko des Ereignisses wie zum Beispiel "Eintritt von Arbeitslosigkeit" ausgesetzt ist. Die Zugehörigkeit zu dieser Gruppe ist unbekannt. Cure Modelle schätzen die Wahrscheinlichkeit zur Nicht-geheilten Population zu gehören und die Form der Survival Funktion für die Beobachtungen unter Risiko. Drittens führe ich Variablen Selektion für das CUB, CUP und das Cure Modell mit Hilfe von Penalisierung und teilweise schrittweise Selektionsverfahren durch. Die Herausforderung liegt insbesondere darin zu entscheiden, welche Variablen in welche Komponente des Misch-Modells aufgenommen werden sollen. Variablen können hier zum einen für die Schätzung der Gewichte der Komponenten und zum anderen für die Form einer oder zwei Misch-Komponenten verwendet werden. Es werden dafür spezifische Bestrafungsterme vorgestellt, die für das jeweilige Modell geeignet sind. Alle Modelle werden mit dem EM-Algorithmus geschätzt, der die unbekannte Zugehörigkeit zu einer der Komponenten als fehlende Daten behandelt. Es werden auch einige computationale Aspekte besprochen wie etwa mit der Initialisierung und der Konvergenz umzugehen ist. Die penalisierte Likelihood wird mit dem sogenannten FISTA Algorithmus geschätzt, da die Ableitungen der penalisierten Likelihood nicht existieren. Es werden sowohl Simulations-Studien als auch reelle Daten verwendet, um die Nützlichkeit der neuen Ansätze aufzuzeigen.Modelling heterogeneity is a crucial aspect of every statistical analysis. To find a reasonable model, it is necessary to include all relevant structures and explanatory variables. Most statistical models can easily include observed patterns but have often difficulties in dealing with latent structures. Mixture models can account for heterogeneity which arise from latent underlying structures, for example, the unobserved membership to different groups or different response styles. In this thesis, I contribute to the use of mixture models to model heterogeneity in ordinal response and perform variable selection in this context. First, I focus on heterogeneity, which occurs in surveys when, for instance, respondents are uncertain about choosing a certain ordered category. In this case, the mixture model traditionally consists of a preference component and an uncertainty component. A weight determines the propensity of each person belonging to one of these components. The traditional CUB model uses a shifted binomial distribution for the first and a uniform distribution for the later component. In the proposed CUP model, the preference component is replaced by any ordinal model, such as the cumulative logit model or the adjacent category model, to achieve more flexibility in the preference component. In the BetaBin model, the concept of uncertainty, understood as a random choice of a category, is extended in such a way that uncertainty can also capture the tendency to the middle and extreme categories. Thus, the uniform distribution of the CUP model is replaced by a more flexible restricted beta-binomial distribution. Second, I show how discrete cure models can be used for dealing with heterogeneity in the survival analysis for discrete time arising from the unobserved membership to different groups. "Cure" refers to the fact that one group of observations is "cured" or characterized as long-term survivors, while the other group is exposed to the risk of the event such as the "occurrence of unemployment". The membership to this group is unknown. Cure models estimate the probability for belonging to the non-cured population and the shape of the survival function of the observations under risk. Third, I perform variable selection for the CUB, the CUP and the cure model using penalization techniques and to some extend stepwise selection procedures. In particular, the challenge is to decide which variables should be included in which component of the mixture model. On the one hand, variables can be used to estimate the weights of the components and on the other hand, for the shape of one or two mixture components. Therefore, specific penalty terms are presented which are appropriate for the particular model. All models are estimated with the EM-Algorithm which treats the unknown membership to the components as missing data. I also address some computational issues, for instance, how to deal with initialization and convergence. The penalized likelihood is estimated with the so-called FISTA algorithm since the derivatives of the penalized likelihood do not exist. Both simulation studies and real data applications are used to demonstrate the usefulness of the new approaches

Finite mixtures for the modelling of heterogeneity in ordinal response

Die Modellierung von Heterogenität ist ein entscheidender Aspekt in jeder statistischen Analyse. Um ein geeignetes Modell zu finden, ist es notwendig, möglichst alle relevanten Strukturen und Einflussgrößen einzubeziehen. Die meisten statistischen Modelle können leicht beobachtete Strukturen einbinden, jedoch haben sie oft Schwierigkeiten latente Strukturen abzubilden. Misch-Modelle können Heterogenität berücksichtigen, die aus zugrunde liegenden latenten Strukturen entstehen, wie etwa die unbeobachtete Zugehörigkeit zu verschiedenen Gruppen oder unterschiedliches Antwortverhalten. Mit dieser Doktorarbeit möchte ich einen Beitrag für die Verwendung von Misch-Modellen zur Modellierung von Heterogenität bei ordinalen Zielgrößen leisten und Variablen Selektion in diesem Kontext durchführen. Zuerst konzentriere ich mich auf Heterogenität, die bei Umfragen auftritt, wenn beispielsweise die Befragten bei der Wahl einer bestimmten geordneten Kategorie unsicher sind. In diesem Fall bestehen die Misch-Modelle üblicherweise aus einer Präferenz-Komponente und einer Unsicherheits-Komponente. Ein Gewicht bestimmt die Neigung jeder Person zu einer dieser beiden Komponenten zu gehören. Das existierende CUB Modell verwendet eine verschobene Binomialverteilung für die erste und eine Gleichverteilung für die zweite Komponente. Im vorgeschlagenem CUP Modell wird die Präferenz-Komponente mit einem beliebigen ordinalen Modell wie dem kumulativen Logit Modell ersetzt, um eine höhere Flexibilität in der Präferenz-Komponente zu erreichen. Im BetaBin Modell wird das Konzept der Unsicherheit als zufällige Wahl einer Kategorie so erweitert, dass Unsicherheit auch die Tendenz zu der zentralen Kategorie und extremen Kategorien erfasst. Auf diese Weise wird die Gleichverteilung des CUP Modells durch einer flexiblere, beschränkte Beta-Binomial Verteilung ersetzt. Als zweites zeige ich, wie diskrete Cure Modelle verwendet werden können, um in der Survival-Analyse für diskrete Zeit mit Heterogenität umzugehen, die aus der unbeobachteten Zugehörigkeit zu verschiedenen Gruppen entsteht. "Cure" bezeichnet dabei den Umstand, dass eine Gruppe von Beobachtungen "geheilt ist" oder als sogenannte Langzeit-Überlebende charakterisiert ist, während die andere Gruppe dem Risiko des Ereignisses wie zum Beispiel "Eintritt von Arbeitslosigkeit" ausgesetzt ist. Die Zugehörigkeit zu dieser Gruppe ist unbekannt. Cure Modelle schätzen die Wahrscheinlichkeit zur Nicht-geheilten Population zu gehören und die Form der Survival Funktion für die Beobachtungen unter Risiko. Drittens führe ich Variablen Selektion für das CUB, CUP und das Cure Modell mit Hilfe von Penalisierung und teilweise schrittweise Selektionsverfahren durch. Die Herausforderung liegt insbesondere darin zu entscheiden, welche Variablen in welche Komponente des Misch-Modells aufgenommen werden sollen. Variablen können hier zum einen für die Schätzung der Gewichte der Komponenten und zum anderen für die Form einer oder zwei Misch-Komponenten verwendet werden. Es werden dafür spezifische Bestrafungsterme vorgestellt, die für das jeweilige Modell geeignet sind. Alle Modelle werden mit dem EM-Algorithmus geschätzt, der die unbekannte Zugehörigkeit zu einer der Komponenten als fehlende Daten behandelt. Es werden auch einige computationale Aspekte besprochen wie etwa mit der Initialisierung und der Konvergenz umzugehen ist. Die penalisierte Likelihood wird mit dem sogenannten FISTA Algorithmus geschätzt, da die Ableitungen der penalisierten Likelihood nicht existieren. Es werden sowohl Simulations-Studien als auch reelle Daten verwendet, um die Nützlichkeit der neuen Ansätze aufzuzeigen.Modelling heterogeneity is a crucial aspect of every statistical analysis. To find a reasonable model, it is necessary to include all relevant structures and explanatory variables. Most statistical models can easily include observed patterns but have often difficulties in dealing with latent structures. Mixture models can account for heterogeneity which arise from latent underlying structures, for example, the unobserved membership to different groups or different response styles. In this thesis, I contribute to the use of mixture models to model heterogeneity in ordinal response and perform variable selection in this context. First, I focus on heterogeneity, which occurs in surveys when, for instance, respondents are uncertain about choosing a certain ordered category. In this case, the mixture model traditionally consists of a preference component and an uncertainty component. A weight determines the propensity of each person belonging to one of these components. The traditional CUB model uses a shifted binomial distribution for the first and a uniform distribution for the later component. In the proposed CUP model, the preference component is replaced by any ordinal model, such as the cumulative logit model or the adjacent category model, to achieve more flexibility in the preference component. In the BetaBin model, the concept of uncertainty, understood as a random choice of a category, is extended in such a way that uncertainty can also capture the tendency to the middle and extreme categories. Thus, the uniform distribution of the CUP model is replaced by a more flexible restricted beta-binomial distribution. Second, I show how discrete cure models can be used for dealing with heterogeneity in the survival analysis for discrete time arising from the unobserved membership to different groups. "Cure" refers to the fact that one group of observations is "cured" or characterized as long-term survivors, while the other group is exposed to the risk of the event such as the "occurrence of unemployment". The membership to this group is unknown. Cure models estimate the probability for belonging to the non-cured population and the shape of the survival function of the observations under risk. Third, I perform variable selection for the CUB, the CUP and the cure model using penalization techniques and to some extend stepwise selection procedures. In particular, the challenge is to decide which variables should be included in which component of the mixture model. On the one hand, variables can be used to estimate the weights of the components and on the other hand, for the shape of one or two mixture components. Therefore, specific penalty terms are presented which are appropriate for the particular model. All models are estimated with the EM-Algorithm which treats the unknown membership to the components as missing data. I also address some computational issues, for instance, how to deal with initialization and convergence. The penalized likelihood is estimated with the so-called FISTA algorithm since the derivatives of the penalized likelihood do not exist. Both simulation studies and real data applications are used to demonstrate the usefulness of the new approaches

Mixture Models for Ordinal Responses with a Flexible Uncertainty Component

In classical mixture models for ordinal data with an uncertainty component the uniform distribution is used to model indecision. In the approach proposed here the discrete uniform distribution is replaced by a more exible distribution, which is centered in the middle of the response categories.The resulting model allows to distinguish between a tendency to middle categories and a tendency to extreme categories. By linking these preferences to explanatory variables one can investigate which persons show a tendency to these response styles. It is demonstrated that severe bias might occur if inadvertently the uniform distribution is used to model uncertainty. An application to attitudes on the performance of health services illustrates the advantages of the more exible model

Dealing with Heterogeneity in Discrete Survival Analysis using the Cure Model

Cure models are able to model heterogeneity which arises from two subgroups with different hazards. One subgroup is characterized as long-term survivors with a hazard equal to zero, while the other subgroup is at-risk of the event. While cure models for continuous time are well established, cure models for discrete time points are rarely prevalent. In this article I describe discrete cure models, how they are defined, estimated and can be applied to real data. I propose to use penalization techniques to stabilize the model estimation, to smooth the baseline and to perform variable selection. The methods are illustrated on data about criminal recidivism and applied to data about breast cancer. As one result patients with no positive lymph nodes, a very small tumor, which can be well differentiated from healthy cells and with ethnicity which is neither black or white have the best estimated chances to belong to the long-term survivors of breast cancer

Bland-White-Garland syndrome and atrial septal defect—: Rare Association and diagnostic challenge

Summary: We report on a 40-year-old woman referred for evaluation of a cardiac murmur and dyspnea on exertion. The electrocardiogram (ECG) showed incomplete right bundle branch block, and echocardiography revealed a large atrial septal defect (ASD, ostium secundum type) with dilated right-sided heart chambers. At cardiac catheterization, a large left-to-right shunt (78% of the pulmonary blood flow) was found, and surprisingly, the additional diagnosis of anomalous origin of the left coronary artery from pulmonary artery (ALCAPA) was established. After ASD closure and left coronary artery ligation with implantation of a vein graft to the left anterior descending artery, she had an uneventful 18-years follow-up. We discuss the interaction of the two associated conditions, and based on the herein reported unusual combination, we highlight typical features of non-invasive examinations including auscultation, ECG, and echocardiography in adult patients with ALCAP

Mixture Models for Ordinal Responses to Account for Uncertainty of Choice

In CUB models the uncertainty of choice is explicitly modelled as a Combination of discrete Uniform and shifted Binomial random variables. The basic concept to model the response as a mixture of a deliberate choice of a response category and an uncertainty component that is represented by a uniform distribution on the response categories is extended to a much wider class of models. The deliberate choice can in particular be determined by classical ordinal response models as the cumulative and adjacent categories model. Then one obtains the traditional and flexible models as special cases when the uncertainty component is irrelevant. It is shown that the effect of explanatory variables is underestimated if the uncertainty component is neglected in a cumulative type mixture model. Visualization tools for the effects of variables are proposed and the modelling strategies are evaluated by use of real data sets. It is demonstrated that the extended class of models frequently yields better fit than classical ordinal response models without an uncertainty component

Uncertainty in Issue Placements and Spatial Voting

Empirical applications of spatial voting approaches frequently rely on ordinal policy scales to measure the policy preferences of voters and their perceptions about party or candidate platforms. Even though it is well known that these placements are affected by uncertainty, only a few empirical voter choice models incorporate uncertainty into the choice rule. In this manuscript, we develop a two-stage approach to further the understanding of how uncertainty impacts on spatial issue voting. First, we model survey responses to ordinal policy scales where specific response styles capture the uncertainty structure in issue placements. At the second stage, we model voter choice and use the placements adjusted for the detected uncertainty as predictors in calculating spatial proximity. We apply the approach to the 2016 US presidential election and study voter preferences and perceptions of the two major candidate platforms on the traditional liberal-conservative scale and three specific issues. Our approach gives insights into how voters attribute issue positions and spatial voting behavior, and performs better than a voter choice model without accounting for uncertainty measured by AIC

Variable Selection in Mixture Models with an Uncertainty Component

Mixture Models as CUB and CUP models provide the opportunity to model discrete human choices as a combination of a preference and an uncertainty structure. In CUB models the preference is represented by shifted binomial random variables and the uncertainty by a discrete uniform distribution. CUP models extend this concept by using ordinal response models as the cumulative model for the preference structure. To reduce model complexity we propose variable selection via group lasso regularization. The approach is developed for CUB and CUP models and compared to a stepwise selection. Both simulated data and survey data are used to investigate the performance of the selection procedures. It is demonstrated that variable selection by regularization yields stable parameter estimates and easy-to-interpret results in both model components and provides a data-driven method for model selection in mixture models with an uncertainty component

Mixture Models for Ordinal Responses to Account for Uncertainty of Choice

In CUB models the uncertainty of choice is explicitly modelled as a Combination of discrete Uniform and shifted Binomial random variables. The basic concept to model the response as a mixture of a deliberate choice of a response category and an uncertainty component that is represented by a uniform distribution on the response categories is extended to a much wider class of models. The deliberate choice can in particular be determined by classical ordinal response models as the cumulative and adjacent categories model. Then one obtains the traditional and flexible models as special cases when the uncertainty component is irrelevant. It is shown that the effect of explanatory variables is underestimated if the uncertainty component is neglected in a cumulative type mixture model. Visualization tools for the effects of variables are proposed and the modelling strategies are evaluated by use of real data sets. It is demonstrated that the extended class of models frequently yields better fit than classical ordinal response models without an uncertainty component

Fluxons in high-impedance long Josephson junctions

The dynamics of fluxons in long Josephson junctions is a well-known example of soliton physics and allows for studying highly nonlinear relativistic electrodynamics on a microscopic scale. Such fluxons are supercurrent vortices that can be accelerated by bias current up to the Swihart velocity, which is the characteristic velocity of electromagnetic waves in the junction. We experimentally demonstrate slowing down relativistic fluxons in Josephson junctions whose bulk superconducting electrodes are replaced by thin films of a high kinetic inductance superconductor. Here, the amount of magnetic flux carried by each supercurrent vortex is significantly smaller than the magnetic flux quantum $_0$. Our data show that the Swihart velocity is reduced by about one order of magnitude compared to conventional long Josephson junctions. At the same time, the characteristic impedance is increased by an order of magnitude, which makes these junctions suitable for a variety of applications in superconducting electronics