Alternative Model-Based and Design-Based Frameworks for Inference From Samples to Populations: From Polarization to Integration

A model-based framework, due originally to R. A. Fisher, and a design-based framework, due originally to J. Neyman, offer alternative mechanisms for inference from samples to populations. We show how these frameworks can utilize different types of samples (nonrandom or random vs. only random) and allow different kinds of inference (descriptive vs. analytic) to different kinds of populations (finite vs. infinite). We describe the extent of each framework's implementation in observational psychology research. After clarifying some important limitations of each framework, we describe how these limitations are overcome by a newer hybrid model/design-based inferential framework. This hybrid framework allows both kinds of inference to both kinds of populations, given a random sample. We illustrate implementation of the hybrid framework using the High School and Beyond data set

Revealing the Form and Function of Self-Injurious Thoughts and Behaviors: A Real-Time Ecological Assessment Study among Adolescents and Young Adults

Self-injurious behaviors are among the leading causes of death worldwide. However, the basic nature of self-injurious thoughts and behaviors (SITBs) is not well understood because prior studies have relied on long-term, retrospective, aggregate, self-report assessment methods. The authors used ecological momentary assessment methods to measure suicidal and nonsuicidal SITBs as they naturally occur in real time. Participants were 30 adolescents and young adults with a recent history of self-injury who completed signal- and event-contingent assessments on handheld computers over a 14-day period, resulting in the collection of data on 1,262 thought and behavior episodes. Participants reported an average of 5.0 thoughts of nonsuicidal self-injury (NSSI) per week, most often of moderate intensity and short duration (1–30 min), and 1.6 episodes of NSSI per week. Suicidal thoughts occurred less frequently (1.1 per week), were of longer duration, and led to self-injurious behavior (i.e., suicide attempts) less often. Details are reported about the contexts in which SITBs most often occur (e.g., what participants were doing, who they were with, and what they were feeling before and after each episode). This study provides a first glimpse of how SITBs are experienced in everyday life and has significant implications for scientific and clinical work on self-injurious behaviors.Psycholog

Explicating the Conditions Under Which Multilevel Multiple Imputation Mitigates Bias Resulting from Random Coefficient-Dependent Missing Longitudinal Data

Random coefficient dependent (RCD) missingness is a non-ignorable mechanism through which missing data can arise in longitudinal designs. RCD, for which we cannot test, is a problematic form of missingness that occurs if subject-specific random effects correlate with propensity for missingness or dropout. Particularly when covariate missingness is a problem, investigators typically handle missing longitudinal data by using single-level multiple imputation procedures implemented with long-format data, which ignores within-person dependency entirely, or implemented with wide-format (i.e., multivariate) data, which ignores some aspects of within-person dependency. When either of these standard approaches to handling missing longitudinal data is used, RCD missingness leads to parameter bias and incorrect inference. We explain why multilevel multiple imputation (MMI) should alleviate bias induced by a RCD missing data mechanism under conditions that contribute to stronger determinacy of random coefficients. We evaluate our hypothesis with a simulation study. Three design factors are considered: intraclass correlation (ICC; ranging from .25 to .75), number of waves (ranging from 4 to 8), and percent of missing data (ranging from 20% to 50%). We find that MMI greatly outperforms the single-level wide-format (multivariate) method for imputation under a RCD mechanism. For the MMI analyses, bias was most alleviated when the ICC is high, there were more waves of data, and when there was less missing data. Practical recommendations for handling longitudinal missing data are suggested

Fitting multilevel models with ordinal outcomes: Performance of alternative specifications and methods of estimation.

Previous research has compared methods of estimation for multilevel models fit to binary data but there are reasons to believe that the results will not always generalize to the ordinal case. This paper thus evaluates (a) whether and when fitting multilevel linear models to ordinal outcome data is justified and (b) which estimator to employ when instead fitting multilevel cumulative logit models to ordinal data, Maximum Likelihood (ML) or Penalized Quasi-Likelihood (PQL). ML and PQL are compared across variations in sample size, magnitude of variance components, number of outcome categories, and distribution shape. Fitting a multilevel linear model to ordinal outcomes is shown to be inferior in virtually all circumstances. PQL performance improves markedly with the number of ordinal categories, regardless of distribution shape. In contrast to binary data, PQL often performs as well as ML when used with ordinal data. Further, the performance of PQL is typically superior to ML when the data includes a small to moderate number of clusters (i.e., ≤ 50 clusters)

Revealing the form and function of self-injurious thoughts and behaviors: A real-time ecological assessment study among adolescents and young adults.

Self-injurious behaviors are among the leading causes of death worldwide. However, the basic nature of self-injurious thoughts and behaviors (SITBs) is not well-understood because prior studies have relied on long-term, retrospective, aggregate, self-report assessment methods. We used ecological momentary assessment methods to measure suicidal and non-suicidal SITBs as they naturally occur in real-time. Participants were 30 adolescents and young adults with a recent history of self-injury who completed signal- and event-contingent assessments on handheld computers over a 14-day period, resulting in the collection of data on 1262 thought and behavior episodes. Participants reported an average of 5.0 thoughts of nonsuicidal self-injury (NSSI) per week, most often of moderate intensity and short duration (1–30 minutes), and 1.6 episodes of NSSI per week. Suicidal thoughts occurred less frequently (1.1 per week), were of longer duration, and led to self-injurious behavior (i.e., suicide attempts) less often. Details are reported about the contexts in which SITBs most often occur (e.g., what participants were doing, who they were with, and what they were feeling before and after each episode). This study provides a first glimpse of how SITBs are experienced in everyday life and has significant implications for scientific and clinical work on self-injurious behaviors

Evaluating Group-Based Interventions When Control Participants Are Ungrouped

Individually randomized treatments are often administered within a group setting. As a consequence, outcomes for treated individuals may be correlated due to provider effects, common experiences within the group, and/or informal processes of socialization. In contrast, it is often reasonable to regard outcomes for control participants as independent, given that these individuals are not placed into groups. Although this kind of design is common in intervention research, the statistical models applied to evaluate the treatment effects are usually inconsistent with the resulting data structure, potentially leading to biased inferences. This article presents an alternative model that explicitly accounts for the fact that only treated participants are grouped. In addition to providing a useful test of the overall treatment effect, this approach also permits one to formally determine the extent to which treatment effects vary over treatment groups and whether there is evidence that individuals within treatment groups become similar to one another. This strategy is demonstrated with data from the Reconnecting Youth program for high school students at risk of school failure and behavioral disorders

Longitudinal dimensionality of adolescent psychopathology: Testing the differentiation hypothesis

The differentiation hypothesis posits that the underlying liability distribution for psychopathology is of low dimensionality in young children, inflating diagnostic comorbidity rates, but increases in dimensionality with age. This hypothesis not been adequately tested with longitudinal psychiatric symptom data

Randomized controlled trial of a family cognitive-behavioral preventive intervention for children of depressed parents.

A family cognitive-behavioral preventive intervention for parents with a history of depression and their 9–15-year-old children was compared with a self-study written information condition in a randomized clinical trial (n = 111 families). Outcomes were assessed at postintervention (2 months), after completion of 4 monthly booster sessions (6 months), and at 12-month follow-up. Children were assessed by child reports on depressive symptoms, internalizing problems, and externalizing problems; by parent reports on internalizing and externalizing problems; and by child and parent reports on a standardized diagnostic interview. Parent depressive symptoms and parent episodes of major depression also were assessed. Evidence emerged for significant differences favoring the family group intervention on both child and parent outcomes; strongest effects for child outcomes were found at the 12-month assessment with medium effect sizes on most measures. Implications for the prevention of adverse outcomes in children of depressed parents are highlighted

r2mlm: An R Package Calculating R-Squared Measures for Multilevel Models

Multilevel models are used ubiquitously in the social and behavioural sciences and effect sizes are critical for contextualizing results. A general framework of R-squared effect size measures for multilevel models has only recently been developed. Rights and Sterba (2019) distinguished each source of explained variance for each possible kind of outcome variance. Though researchers have long desired a comprehensive and coherent approach to computing R-squared measures for multilevel models, the use of this framework has a steep learning curve. The purpose of this tutorial is to introduce and demonstrate using a new R package – r2mlm – that automates the intensive computations involved in implementing the framework and provides accompanying graphics to visualize all multilevel R-squared measures together. We use accessible illustrations with open data and code to demonstrate how to use and interpret the R package output