Multiple Comparison Procedures, Trimmed Means And Transformed Statistics

A modification to testing pairwise comparisons that may provide better control of Type I errors in the presence of non-normality is to use a preliminary test for symmetry which determines whether data should be trimmed symmetrically or asymmetrically. Several pairwise MCPs were investigated, employing a test of symmetry with a number of heteroscedastic test statistics that used trimmed means and Winsorized variances. Results showed improved Type I error control than competing robust statistics

Type I Error Rates Of Four Methods For Analyzing Data Collected In A Groups vs Individuals Design

Using previous work on the Behrens-Fisher problem, two approximate degrees of freedom tests, that can be used when one treatment is individually administered and one is administered to groups, were developed. Type I error rates are presented for these tests, an additional approximate degrees of freedom test developed by Myers, Dicecco, and Lorch (1981), and a mixed model test. The results indicate that the test that best controls the Type I error rate depends on the number of groups in the group-administered treatment. The mixed model test should be avoided

A Robust Root Mean Square Standardized Effect Size in One-Way Fixed-Effects ANOVA

A robust Root Mean Square Standardized Effect Size (RMSSER) was developed to address the unsatisfactory performance of the Root Mean Square Standardized Effect Size. The coverage performances of the confidence intervals (CI) for RMSSER were investigated. The coverage probabilities of the non-central F distribution-based CI for RMSSER were adequate

Coverage Performance of the Non-Central F-based and Percentile Bootstrap Confidence Intervals for Root Mean Square Standardized Effect Size in One-Way Fixed-Effects ANOVA

The coverage performance of the confidence intervals (CIs) for the Root Mean Square Standardized Effect Size (RMSSE) was investigated in a balanced, one-way, fixed-effects, between-subjects ANOVA design. The noncentral F distribution-based and the percentile bootstrap CI construction methods were compared. The results indicated that the coverage probabilities of the CIs for RMSSE were not adequate

Analyses of Unbalanced Groups-Versus-Individual Research Designs Using Three Alternative Approximate Degrees of Freedom Tests: Test Development and Type I Error Rates

Three approximate degrees of freedom quasi-F tests of treatment effectiveness were developed for use in research designs when one treatment is individually delivered and the other is delivered to individuals nested in groups of unequal size. Imbalance in the data was studied from the prospective of subject attrition. The results indicated the test that best controls the Type I error rate depends on the number of groups in the group-administered treatment but does not depend on the subject attrition rates included in the study

Type I Error Rates For A One Factor Within-Subjects Design With Missing Values

Missing data are a common problem in educational research. A promising technique, that can be implemented in SAS PROC MIXED and is therefore widely available, is to use maximum likelihood to estimate model parameters and base hypothesis tests on these estimates. However, it is not clear which test statistic in PROC MIXED performs better with missing data. The performance of the Hotelling- Lawley-McKeon and Kenward-Roger omnibus test statistics on the means for a single factor withinsubject ANOVA are compared. The results indicate that the Kenward-Roger statistic performed better in terms of keeping the Type I error close to the nominal alpha level

A Comparison Of Methods For Longitudinal Analysis With Missing Data

In a longitudinal two-group randomized trials design, also referred to as randomized parallel-groups design or split-plot repeated measures design, the important hypothesis of interest is whether there are differential rates of change over time, that is, whether there is a group by time interaction. Several analytic methods have been presented in the literature for testing this important hypothesis when data are incomplete. We studied these methods for the case in which the missing data pattern is non-monotone. In agreement with earlier work on monotone missing data patterns, our results on bias, sampling variability, Type I error and power support the use of a procedure due to Overall, Ahn, Shivakumar, and Kalburgi (1999) that can easily be implemented with SAS’s PROC MIXE

Type I Error Rates of the Kenward-Roger Adjusted Degree of Freedom F-test for a Split-Plot Design with Missing Values

The Type I error rate of the Kenward-Roger (KR) test, implemented by PROC MIXED in SAS, was assessed through a simulation study for a one between- and one within-subjects factor split-plot design with ignorable missing values and covariance heterogeneity. The KR test controlled the Type I error well under all of the simulation factors, with all estimated Type I error rates between .040 and .075. The best control was for testing the between-subjects main effect (error rates between .041 and .057) and the worst control was for the between-by-within interaction (.040 to .075). The simulated factors had very small effects on the Type I error rates, with simple effects in two-way tables no larger than .01

Assessing Treatment Effects in Randomized Longitudinal Two-Group Designs with Missing Observations

SAS’s PROC MIXED can be problematic when analyzing data from randomized longitudinal two-group designs when observations are missing over time. Overall (1996, 1999) and colleagues found a number of procedures that are effective in controlling the number of false positives (Type I errors) and are yet sensitive (powerful) to detect treatment effects. Two favorable methods incorporate time in study and baseline scores to model the missing data mechanism; one method was a single-stage PROC MIXED ANCOVA solution and the other was a two-stage endpoint analysis using the change scores as dependent scores. Because the twostage approach can lack sensitivity to detect effects for certain missing data mechanisms, in this article we examined variations of the single-stage approach under conditions not considered by Overall et al., in order to assess the generality of the procedure’s positive characteristics. The results indicate when and when not it is beneficial to include a baseline score as a covariate in the model. As well, we provide clarification regarding the merits of adopting an endpoint analysis as compared to the single-stage PROC MIXED procedure

Robust Confidence Intervals for Effect Size in the Two-Group Case

The probability coverage of intervals involving robust estimates of effect size based on seven procedures was compared for asymmetrically trimming data in an independent two-groups design, and a method that symmetrically trims the data. Four conditions were varied: (a) percentage of trimming, (b) type of nonnormal population distribution, (c) population effect size, and (d) sample size. Results indicated that coverage probabilities were generally well controlled under the conditions of nonnormality. The symmetric trimming method provided excellent probability coverage. Recommendations are provided