The Influence of Sample Size on Parameter Estimates in Three-Level Random-Effects Models

Kerkhoff D, Nussbeck FW. The Influence of Sample Size on Parameter Estimates in Three-Level Random-Effects Models. Frontiers in Psychology. 2019;10: 1067.In educational psychology, observational units are oftentimes nested within superordinate groups. Researchers need to account for hierarchy in the data by means of multilevel modeling, but especially in three-level longitudinal models, it is often unclear which sample size is necessary for reliable parameter estimation. To address this question, we generated a population dataset based on a study in the field of educational psychology, consisting of 3000 classrooms (level-3) with 55000 students (level-2) measured at 5 occasions (level-1), including predictors on each level and interaction effects. Drawing from this data, we realized 1000 random samples each for various sample and missing value conditions and compared analysis results with the true population parameters. We found that sampling at least 15 level-2 units each in 35 level-3 units results in unbiased fixed effects estimates, whereas higher-level random effects variance estimates require larger samples. Overall, increasing the level-2 sample size most strongly improves estimation soundness. We further discuss how data characteristics influence parameter estimation and provide specific sample size recommendations

The Influence of Sample Size on Parameter Estimates in Three-Level Random-Effects Models

In educational psychology, observational units are oftentimes nested within superordinate groups. Researchers need to account for hierarchy in the data by means of multilevel modeling, but especially in three-level longitudinal models, it is often unclear which sample size is necessary for reliable parameter estimation. To address this question, we generated a population dataset based on a study in the field of educational psychology, consisting of 3000 classrooms (level-3) with 55000 students (level-2) measured at 5 occasions (level-1), including predictors on each level and interaction effects. Drawing from this data, we realized 1000 random samples each for various sample and missing value conditions and compared analysis results with the true population parameters. We found that sampling at least 15 level-2 units each in 35 level-3 units results in unbiased fixed effects estimates, whereas higher-level random effects variance estimates require larger samples. Overall, increasing the level-2 sample size most strongly improves estimation soundness. We further discuss how data characteristics influence parameter estimation and provide specific sample size recommendations

Quality of parameter estimates in three-level random effects models

publishe

Use and impact of the open source online editor Etherpad in a psychology students’ statistics class

Bebermeier S, Kerkhoff D. Use and impact of the open source online editor Etherpad in a psychology students’ statistics class. Psychology Teaching Review. 2019;25(2):30-38

Obtaining sound intraclass correlation and variance estimates in three-level models: The role of sampling-strategies

Kerkhoff D, Nussbeck FW. Obtaining sound intraclass correlation and variance estimates in three-level models: The role of sampling-strategies. Methodology. 2022;18(1):5-23

Obtaining sound intraclass correlation and variance estimates in three-level models : The role of sampling-strategies

Three-level clustered data commonly occur in social and behavioral research and are prominently analyzed using multilevel modeling. The influence of the clustering on estimation results is assessed with the intraclass correlation coefficients (ICCs), which indicate the fraction of variance in the outcome located at each higher level. However, ICCs are prone to bias due to high requirements regarding the overall sample size and the sample size at each data level. In Monte Carlo simulations, we investigate how these sample characteristics influence the bias of the ICCs and statistical power of the variance components using robust ML-estimation. Results reveal considerable underestimation on Level-3 and the importance of the Level-3 sample size in combination with the ICC sizes. Based on our results, we derive concise sampling recommendations and discuss limits to our inferences.publishe

Evaluation of a 'Painting and Puzzles Exercise Book for Statistics' for psychology first year students

Austerschmidt K, Kerkhoff D, Bebermeier S, Hagemann A. Evaluation of a 'Painting and Puzzles Exercise Book for Statistics' for psychology first year students. Psychology Teaching Review. 2021;27(2):5-21.Statistics courses are challenging for many psychology students. We designed a ‘Painting and Puzzles Exercise Book for Statistics’ for first year psychology undergraduates, to repeat and deepen the course content and get prepared for the exam. We describe the development and characteristics of the book and report a longitudinal evaluation study. Users (N=72) rated the book positively and judged it as helpful, easy to use, and enjoyable. Students with initially higher skills, younger students and, those who graduated from school recently were more likely to use it. We matched users to non-users from the preceding and the evaluation cohort, respectively, on characteristics at study entry and found positive effects on achievement (exam grade and perceived management of course content) for users. We reason that the book is a valuable support that can be embedded in an ongoing course easily

Dyadic Coping in Foster and Biological Parents and Its Relation to Child Psychopathology

Job A-K, Kerkhoff D, Nussbeck FW, Konrad K, Heinrichs N, Lohaus A. Dyadic Coping in Foster and Biological Parents and Its Relation to Child Psychopathology. EUROPEAN JOURNAL OF HEALTH PSYCHOLOGY. 2019;26(3):71-89.This study investigated whether foster parents' reports of their dyadic coping competencies differ from biological parents, whether there are differences with regard to the temporal associations between maternal and paternal dyadic coping in the two samples, and whether parental dyadic coping competencies predict future mental health problems in children. A total of 94 foster children and 157 children living in their biological families, both samples aged 2-7 years, as well as their (foster) parents were assessed three times over a 12-month period. The mothers' and fathers' dyadic coping competencies were assessed using the Dyadic Coping Inventory (DCI). Child psychopathology was assessed using the Child Behavior Checklist (CBCL) and a standardized clinical interview (Kinder-DIPS), both mainly based on maternal report. Foster parents reported better dyadic coping competencies across the three assessments than did biological parents. There were no significant differences with regard to the temporal associations between mothers' and fathers' report over time between the two samples. Cross-lagged panel models yielded a high within person stability across the three assessments for both, mothers and fathers (actor effects), as well as some significant interpersonal effects primarily from paternal to maternal dyadic coping (partner effects). In contrast to the expectation, mothers' and fathers' dyadic coping did not predict child mental health problems at the third assessment. The results make an important contribution to the research on dyadic coping and on how child mental health problems affect parental dyadic coping competencies and vice versa