Confidence interval estimation for a difference between two correlated intraclass correlation coefficients with variable class sizes

The intraclass correlation coefficient (ICC) has application in several fields of research. Various confidence interval methods for a single ICC are available. However, statis­ tical inference for multiple ICCs has primarily relied on hypothesis testing which cannot distinguish between statistical significance and practical importance. The focus of this thesis is to develop and evaluate confidence interval procedures for a difference between two correlated ICCs with variable class sizes. The strategy used in the thesis is to recover variance estimates needed for the confidence interval for the difference from the confidence limits for single ICCs. Simulation results show that the procedure based on inverse hyperbolic tangent transformation for single ICCs performs well. The Galton’s 1886 dataset on siblings heights is used to illustrate the methodology

Techniques for handling clustered binary data

Bibliography : leaves 143-153.Over the past few decades there has been increasing interest in clustered studies and hence much research has gone into the analysis of data arising from these studies. It is erroneous to treat clustered data, where observations within a cluster are correlated with each other, as one would treat independent data. It has been found that point estimates are not as greatly affected by clustering as are the standard deviations of the estimates. But as a consequence, confidence intervals and hypothesis testing are severely affected. Therefore one has to approach the analysis of clustered data with caution. Methods that specifically deal with correlated data have been developed. Analysis may be further complicated when the outcome variable of interest is binary rather than continuous. Methods for estimation of proportions, their variances, calculation of confidence intervals and a variety of techniques for testing the homogeneity of proportions have been developed over the years (Donner and Klar, 1993; Donner, 1989, and Rao and Scott, 1992). The methods developed within the context of experimental design generally involve incorporating the effect of clustering in the analysis. This cluster effect is quantified by the intracluster correlation and needs to be taken into account when estimating proportions, comparing proportions and in sample size calculations. In the context of observational studies, the effect of clustering is expressed by the design effect which is the inflation in the variance of an estimate that is due to selecting a cluster sample rather than an independent sample. Another important aspect of the analysis of complex sample data that is often neglected is sampling weights. One needs to recognise that each individual may not have the same probability of being selected. These weights adjust for this fact (Little et al, 1997). Methods for modelling correlated binary data have also been discussed quite extensively. Among the many models which have been proposed for analyzing binary clustered data are two approaches which have been studied and compared: the population-averaged and cluster-specific approach. The population-averaged model focuses on estimating the effect of a set of covariates on the marginal expectation of the response. One example of the population-averaged approach for parameter estimation is known as generalized estimating equations, proposed by Liang and Zeger (1986). It involves assuming that elements within a cluster are independent and then imposing a correlation structure on the set of responses. This is a useful application in longitudinal studies where a subject is regarded as a cluster. Then the parameters describe how the population-averaged response rather than a specific subject's response depends on the covariates of interest. On the other hand, cluster specific models introduce cluster to cluster variability in the model by including random effects terms, which are specific to the cluster, as linear predictors in the regression model (Neuhaus et al, 1991). Unlike the special case of correlated Gaussian responses, the parameters for the cluster specific model obtained for binary data describe different effects on the responses compared to that obtained from the population-averaged model. For longitudinal data, the parameters of a cluster-specific model describe how a specific individuals probability of a response depends on the covariates. The decision to use either of these modelling methods depends on the questions of interest. Cluster-specific models are useful for studying the effects of cluster-varying covariates and when an individual's response rather than an average population's response is the focus. The population-averaged model is useful when interest lies in how the average response across clusters changes with covariates. A criticism of this approach is that there may be no individual with the characteristics of the population-averaged model

Estimation and Inference of the Three-Level Intraclass Correlation Coefficient

Since the early 1900\u27s, the intraclass correlation coefficient (ICC) has been used to quantify the level of agreement among different assessments on the same object. By comparing the level of variability that exists within subjects to the overall error, a measure of the agreement among the different assessments can be calculated. Historically, this has been performed using subject as the only random effect. However, there are many cases where other nested effects, such as site, should be controlled for when calculating the ICC to determine the chance corrected agreement adjusted for other nested factors. We will present a unified framework to estimate both the two-level and three-level ICC for both binomial and multinomial outcomes. In addition, the corresponding standard errors and confidence intervals for both ICC measurements will be displayed. Finally, an example of the effect that controlling for site can have on ICC measures will be presented for subjects nested within genotyping plates comparing genetically determined race to patient reported race. In addition, when determining agreement on a multinomial response, the question of homogeneity of agreement of individual categories within the multinomial response is raised. One such scenario is the GO project at the University of Pennsylvania where subjects ages 8-21 were asked to rate a series of actors\u27 faces as happy, sad, angry, fearful or neutral. Methods exist to quantify overall agreement among the five responses, but only if the ICCs for each item-wise response are homogeneous. We will present a method to determine homogeneity of ICCs of the item-wise responses across a multinomial outcome and provide simulation results that demonstrate strong control of the type I error rate. This method will subsequently be extended to verify the assumptions of homogeneity of ICCs in the multinomial nested-level model to determine if the overall nested-level ICC is sufficient to describe the nested-level agreement

CONFIDENCE INTERVAL ESTIMATION FOR A DIFFERENCE BETWEEN CORRELATED INTRACLASS CORRELATION COEFFICIENTS

The intraclass correlation coefficient (ICC), an index of similarity, plays an important role in a wide range of disciplines, for example in the assessment of instrument reliability. In this case, the study design may involve recruiting a sample of subjects each of whom are assessed several times with a new device and the standard. The ICC estimates for the two devices may then be compared using a test of hypothesis. However it is well known that conclusions drawn from hypothesis testing are confounded by sample size, i.e., a significant p-value can result from a sufficiently large sample size. In such cases, a confidence interval for a difference between two ICCs is more informative since it combines point estimation and hypothesis testing into a single inference statement. The sampling distribution for the ICC is well known to be left-skewed and thus confidence limits are usually constructed using Fisher’s Z-transformation or the F- distribution. Unfortunately, such an approach is not applicable to a difference between two ICCs. The remaining alternative is to apply a simple asymptotic approach, i.e., point estimate plus/minus normal quantile multiplied by the estimate of standard error. However this method is known to perform poorly because it ignores the features of the underlying sampling distribution. In this thesis I develop a confidence interval procedure using the method of variance estimate recovery (MOVER). Specifically, the variance estimates required for the upper and lower limits of a difference are recovered from those obtained for separate ICCs. An advantage of this approach is that it provides a confidence interval that reflects the underlying sampling distribution. Simulation results show that the MOVER method performs very well in terms of overall coverage percentage and tail errors. Two data sets are used to illustrate this procedure

CONFIDENCE INTERVAL ESTIMATION FOR A DIFFERENCE BETWEEN CORRELATED INTRACLASS CORRELATION COEFFICIENTS

The intraclass correlation coefficient (ICC), an index of similarity, plays an important role in a wide range of disciplines, for example in the assessment of instrument reliability. In this case, the study design may involve recruiting a sample of subjects each of whom are assessed severe^ times with a new device and the standard. The ICC estimates for the two devices may then be compared using a test of hypothesis. However it is well known that conclusions drawn from hypothesis testing are confounded by sample size, i.e., a significant p-value can result from a sufficiently large sample size. In such cases, a confidence interval for a difference between two ICCs is more informative since it combines point estimation and hypothesis testing into a single inference statement. The sampling distribution for the ICC is well known to be left-skewed and thus confidence limits are usually constructed using Fisher’s Z-transformation or the F- distribution. Unfortunately, such an approach is not applicable to a difference between two ICCs. The remaining alternative is to apply a simple asymptotic approach, i.e., point estimate plus/minus normal quantile multiplied by the estimate of standard error. However this method is known to perform poorly because it ignores the features of the underlying sampling distribution. In this thesis I develop a confidence interval procedure using the method of variance estimate recovery (MOVER). Specifically, the variance estimates required for the upper and lower limits of a difference are iii recovered from those obtained for separate ICCs. An advantage of this approach is that it provides a confidence interval that reflects the underlying sampling distribution. Simulation results show that the MOVER method performs very well in terms of overall coverage percentage and tail errors. Two data sets are used to illustrate this procedure

Fit Index Sensitivity in Multilevel Structural Equation Modeling

Multilevel Structural Equation Modeling (MSEM) is used to estimate latent variable models in the presence of multilevel data. A key feature of MSEM is its ability to quantify the extent to which a hypothesized model fits the observed data. Several test statistics and so-called fit indices can be calculated in MSEM as is done in single-level structural equation modeling. Accordingly, problems associated with these measures in the single-level case may apply to the multilevel case and new complications may arise. Few studies, however, have examined the performance of fit indices in MSEM. Furthermore, recent findings suggest that evaluating fit at each level separately is advantageous to evaluating fit for the overall model. Therefore, the purpose of the present study was to evaluate the sensitivity of several fit indices to misspecification in the cluster-level model under varying multilevel data conditions including the intraclass correlation coefficient, sample size configuration, and severity of model misspecification. Furthermore, three methods of level-specific fit evaluation were compared. Results from a Monte Carlo simulation study suggest that fit indices are affected by the ICC of model indicators and sample size configurations in MSEM. With the exception of the SRMR, all fit indices were less sensitive to cluster-level model misspecification at low indicator ICCs, large overall sample sizes, and smaller numbers of clusters. Discrepancies in fit information between two methods of level-specific fit were observed at low ICC values. Finally, two fit indices rarely used in SEM applications revealed desirable properties in certain simulation conditions. Implications of the simulation results are discussed and a program for implementing level-specific fit evaluation in the R statistical language is provided

Validation of the SenseWear Pro 2 Armband Calorimeter to Assess Energy Expenditure of Adolescents during Various Modes of Activity

VALIDATION OF THE SENSEWEAR PRO2 ARMBAND CALORIMETER TO ASSESS ENERGY EXPENDITURE OF ADOLESCENTS DURING VARIOUS MODES OF ACTIVITYKim Crawford, PhDUniversity of Pittsburgh, 2004The primary purpose of this investigation was to examine the validity of the SenseWear® Pro 2 Armband (SAB) to assess energy expenditure during various modes of physical activity in adolescents. It was hypothesized that measures of energy expenditure during treadmill and cycle ergometer exercise would not differ between the SAB and the criterion respiratory metabolic system (RMS) when examined for female and male subjects. Twenty-four healthy adolescents completed both the cycle ergometer and treadmill exercise protocols. The primary findings of this investigation were the SAB significantly underestimated energy expenditure during cycle ergometer exercise at the low (1.53 + 0.60 kcal.min-1; P<0.001) and moderate (2.48 + 0.95 kcal.min-1; P<0.001) intensities and for total energy expenditure (19.11 + 7.43 kcal; P<0.001) in both the female and male subjects. In the treadmill exercise, there were no significant differences between measures of energy expenditure during treadmill walking at 3.0 mph, 0% incline in female and male subjects. However, the SAB significantly underestimated measures of energy expenditure at 4.0 mph, 0% grade (0.86 + 0.84 kcal.min-1; P<0.001); 4.0 mph, 5% grade (2.13 + 1.40 kcal.min-1; P<0.001); 4.5 mph, 5% grade (2.97 + 1.56 kcal.min-1; P<0.001) and for total energy expenditure (23.66 + 14.92 kcal; P<0.001) during treadmill exercise in female and male subjects.Possible mechanisms underlying the underestimation of energy expenditure by the SAB are complex but may include: the use of generalized exercise algorithms to predict all types of physical activity; possible disproportionate reliance on the two-axis accelerometer during non-weight bearing and graded exercises; the delay in body heat transfer to the skin; and the inability to account for variability in walking gait, lean body mass and fat mass. All of these factors impact on the accuracy of the SAB to accurately estimate energy expenditure. This is the first study to examine the accuracy of the SAB in adolescent subjects and is an important first step in validating SAB technology in adolescents

A Case Study of the Application of a Multilevel Growth Curve Model and the Prediction of Health Trajectories

This case study provides guidance on the application of a multilevel growth curve model and the prediction of health trajectories. Using our secondary data analysis as an example, we introduce definitions of the multilevel growth curve model, random intercept and slope, and the intraclass correlation coefficient. We discuss time centering and time metrics, marginal effects for drawing frailty trajectories, as well as multiple imputation for handling missing values