Sample Size in Ordinal Logistic Hierarchical Linear Modeling

Most quantitative research is conducted by randomly selecting members of a population on which to conduct a study. When statistics are run on a sample, and not the entire population of interest, they are subject to a certain amount of error. Many factors can impact the amount of error, or bias, in statistical estimates. One important factor is sample size; larger samples are more likely to minimize bias than smaller samples. Therefore, determining the necessary sample size to obtain accurate statistical estimates is a critical component of designing a quantitative study. Much research has been conducted on the impact of sample size on simple statistical techniques such as group mean comparisons and ordinary least squares regression. Less sample size research, however, has been conducted on complex techniques such as hierarchical linear modeling (HLM). HLM, also known as multilevel modeling, is used to explain and predict an outcome based on knowledge of other variables in nested populations. Ordinal logistic HLM (OLHLM) is used when the outcome variable has three or more ordered categories. While there is a growing body of research on sample size for two-level HLM utilizing a continuous outcome, there is no existing research exploring sample size for OLHLM. The purpose of this study was to determine the impact of sample size on statistical estimates for ordinal logistic hierarchical linear modeling. A Monte Carlo simulation study was used to investigate this research query. Four variables were manipulated: level-one sample size, level-two sample size, sample outcome category allocation, and predictor-criterion correlation. Statistical estimates explored include bias in level-one and level-two parameters, power, and prediction accuracy. Results indicate that, in general, holding other conditions constant, bias decreases as level-one sample size increases. However, bias increases or remains unchanged as level-two sample size increases, holding other conditions constant. Power to detect the independent variable coefficients increased as both level-one and level-two sample size increased, holding other conditions constant. Overall, prediction accuracy is extremely poor. The overall prediction accuracy rate across conditions was 47.7%, with little variance across conditions. Furthermore, there is a strong tendency to over-predict the middle outcome category

Statistical analysis of the primary outcome in acute stroke trials

Common outcome scales in acute stroke trials are ordered categorical or pseudocontinuous in structure but most have been analyzed as binary measures. The use of fixed dichotomous analysis of ordered categorical outcomes after stroke (such as the modified Rankin Scale) is rarely the most statistically efficient approach and usually requires a larger sample size to demonstrate efficacy than other approaches. Preferred statistical approaches include sliding dichotomous, ordinal, or continuous analyses. Because there is no best approach that will work for all acute stroke trials, it is vital that studies are designed with a full understanding of the type of patients to be enrolled (in particular their case mix, which will be critically dependent on their age and severity), the potential mechanism by which the intervention works (ie, will it tend to move all patients somewhat, or some patients a lot, and is a common hazard present), a realistic assessment of the likely effect size, and therefore the necessary sample size, and an understanding of what the intervention will cost if implemented in clinical practice. If these approaches are followed, then the risk of missing useful treatment effects for acute stroke will diminish

Delivering on a Promise: A Longitudinal Cohort Study of Emergent Bilinguals\u27 Academic Achievement in a Utah Dual Language Program

Emergent bilingual (EB) students are a growing population in the U.S. school system currently comprising over 10% of the total population. U.S. school districts have educated EB students using a myriad of practices, including dual language immersion (DLI). Many studies have looked at the academic achievement of native-Spanish speaking EB students, yet there is a dearth of research on DLI in medium-sized school districts in rural settings. This study focused on native-Spanish speaking EB students in a DLI program in a rural Utah district. Specifically, the study compared student academic achievement in English acquisition, English language arts, mathematics, and grade point average of EB students enrolled in DLI to EB students not enrolled in a DLI program. Student achievement data for EB students was collected from 2014-2020 (N = 1,046). Using various regression methods such as ordinal logistic regression, multiple regression, and multilevel modeling (MLM), the study sought the predictive power of DLI after controlling for gender, free and reduced lunch status, and special education enrollment. Results found that on average DLI students performed as well as or better than their non-DLI peers. MLM analyses indicated that EB students enrolled in DLI had superior growth trajectories to their non-DLI peers over time

Building school-based social capital through 'We Act - Together for Health' - a quasi-experimental study

Abstract Background Social capital has been found to be positively associated with various health and well-being outcomes amongst children. Less is known about how social capital may be generated and specifically in relation to children in the school setting. Drawing on the social cohesion approach and the democratic health educational methodology IVAC (Investigation – Vision – Action – Change) the aim of this study was to examine the effect of the Health Promoting School intervention ‘We Act – Together for Health’ on children’s cognitive social capital. Method A quasi-experimental controlled pre- and post-intervention study design was conducted with 548 participants (mean age 11.7 years). Cognitive social capital was measured as: horizontal social capital (trust and support in pupils); vertical social capital (trust and support in teachers); and a sense of belonging in the school using questions derived from the Health Behaviour in School Children study. A series of multilevel ordinal logistic regression analyses was performed for each outcome to estimate the effect of the intervention. Result The analyses showed no overall significant effect from the intervention on horizontal social capital or vertical social capital at the six-month follow-up. A negative effect was found on the sense of belonging in the school. Gender and grade appeared to be important for horizontal social capital, while grade was important for sense of belonging in the school. The results are discussed in relation to We Act’s implementation process, our conceptual framework and methodological issues and can be used to direct future research in the field. Conclusion The study finds that child participation in health education can affect the children’s sense of belonging in the school, though without sufficient management support, this may have a negative effect. With low implementation fidelity regarding the Action and Change dimension of the intervention at both the school and class level, and with measurement issues regarding the concept of social capital, more research is needed to establish a firm conclusion on the importance of the children’s active participation as a source for cognitive social capital creation in the school setting. Trial registration https://www.isrctn.com/ISRCTN8520301

Exact Approaches for Bias Detection and Avoidance with Small, Sparse, or Correlated Categorical Data

Every day, traditional statistical methodology are used world wide to study a variety of topics and provides insight regarding countless subjects. Each technique is based on a distinct set of assumptions to ensure valid results. Additionally, many statistical approaches rely on large sample behavior and may collapse or degenerate in the presence of small, spare, or correlated data. This dissertation details several advancements to detect these conditions, avoid their consequences, and analyze data in a different way to yield trustworthy results. One of the most commonly used modeling techniques for outcomes with only two possible categorical values (eg. live/die, pass/fail, better/worse, ect.) is logistic regression. While some potential complications with this approach are widely known, many investigators are unaware that their particular data does not meet the foundational assumptions, since they are not easy to verify. We have developed a routine for determining if a researcher should be concerned about potential bias in logistic regression results, so they can take steps to mitigate the bias or use a different procedure altogether to model the data. Correlated data may arise from common situations such as multi-site medical studies, research on family units, or investigations on student achievement within classrooms. In these circumstance the associations between cluster members must be included in any statistical analysis testing the hypothesis of a connection be-tween two variables in order for results to be valid. Previously investigators had to choose between using a method intended for small or sparse data while assuming independence between observations or a method that allowed for correlation between observations, while requiring large samples to be reliable. We present a new method that allows for small, clustered samples to be assessed for a relationship between a two-level predictor (eg. treatment/control) and a categorical outcome (eg. low/medium/high)

Medical psychometrics:A psychometric evaluation of Type D personality and its predictive value in medical research

Type D personality–a combination of high negative affectivity and high social inhibition–has been identified as a risk factor for adverse outcome in various patient populations. However, common methods used to establish the predictive value of Type D personality have been criticized and several recent studies were not able to replicate previous findings. To explain these inconsistencies, this interdisciplinary dissertation brings together experts from the fields of medical psychology and psychometrics. It presents a psychometric evaluation of the construct Type D personality and illustrates how it can best be modeled in medical and psychological research. Based on thousands of computer-simulated datasets, as well as empirical data from patients with various types of diseases, this dissertation shows why most published research testing a Type D personality effect should be reanalyzed using modern psychometric and statistical methods. It also presents a first attempt at this endeavor by reanalyzing various earlier published datasets, showing that coronary artery disease patients with Type D personality are at increased risk for adverse outcome

A comparison of inferential analysis methods for multilevel studies: implications for drawing conclusions in animal welfare science

Investigations comparing the behaviour and welfare of animals in different environments have led to mixed and often conflicting results. These could arise from genuine differences in welfare, poor validity of indicators, low statistical power, publication bias, or inappropriate statistical analysis. Our aim was to investigate the effects of using four approaches for inferential analysis of datasets of varying size on model outcomes and potential conclusions. We considered aggression in 864 growing pigs over six weeks as measured by ear and body injury score and relationships with: less and more enriched environments, pig's relative weight, and sex. Pigs were housed in groups of 18 in one of four pens, replicating the experiment 12 times. We applied four inferential models that either used a summary statistic approach, or else fully or partially accounted for complexities in study design. We tested models using both the full dataset (n = 864) and also using small sample sizes (n = 72). The most appropriate inferential model was a mixed effects, repeated measures model to compare ear and body score. Statistical models that did not acco unt for the correlation between repeated measures and/or the random effects from replications and pens led to spurious associations between environmental factors and indicators of aggression, which were not supported by the initial exploratory analysis. For analyses on smaller datasets (n = 72), due to the effect size and number of independent factors, there was insufficient power to determine statistically significant associations. Based on the mixed effects, repeated measures models, higher body injury scores were associated with more enrichment (coef. est. = 0.09, p = 0.02); weight (coef. est. = 0.05, p < 0.001); pen location on the right side (coef. est. = 0.08, p = 0.03) and at the front of the experimental room (coef. est. = 0.11, p = 0.003). By comparison, lower ear injury scores were associated with more enrichment (coef. est. = -0.51, p = 0.005) and pen location at the front of the experimental room (coef. est. = -0.4, p = 0.02). These observed differences support the hypothesis that injuries to the body and ears arise from different risk factors. Although calculation of the minimum required sample size prior to conducting an experiment and selection of the inferential analysis method will contribute to the validity of the study results, conflict between the outcomes will require further investigation via different methods such as sensitivity and specificity analysis