Causal Mediation Analysis in Single Case Experimental Designs: Introduction to the Special Issue.

This special issue of Evaluation and the Health Professions is dedicated to methods for causal mediation analysis in Single Case Experimental Designs (SCEDs). Mediation analysis is used to identify intermediate variables that transmit the effect of the independent variable on the outcome. Until recently, mediation analysis was mostly confined to between-subjects designs and panel studies with few exceptions. Consequently, most of the developments in causal mediation analysis have also been restricted to such designs. In applied health research, SCEDs have been used to evaluate total effects of treatments on outcomes of interest. Providing researchers with the methods for evaluating causal indirect effects for individual participants can lead to important improvements in diagnosis, treatment, and prevention. This special issue includes articles that describe advanced quantitative methods for testing mediators in SCEDs, propose and test approaches that allow for relaxing statistical assumptions that may not hold in real data, and illustrate mediation analysis for a single participant in real and simulated SCEDs data

Small Sample Meta-Analyses: Exploring heterogeneity using MetaForest

Meta-analyses often suffer from two related problems: A small sample of studies, and many between-studies differences that might influence the effect size. Power is typically too low to adequately account for these between-study differences using meta-regression. Researchers risk overfitting: Capturing noise in the data, rather than true effects. This chapter introduces MetaForest: A machine-learning-based approach for identifying relevant moderators in meta-analysis. MetaForest is robust to overfitting, handles many moderators, and captures non-linear effects and higher-order interactions. This chapter discusses the problems with small samples and many moderators, introduces MetaForest as a small sample solution, and provides a tutorial example analysis

Testing replication with small samples: Applications to ANOVA

Findings based on small samples can offer important insights, but original small sample findings should be replicated before strong conclusions can be drawn. In this chapter, we present four common replication research questions: (1) whether the new effect size is similar to the original effect size, (2) whether the new effect size differs from the original effect size, (3) whether the conclusions based on new results differ from the original conclusions, and (4) what the effect size is in the population. For each of these research questions, we discuss appropriate evaluation methods: replication Bayes factors, confidence intervals, prediction intervals, the prior predictive p-value, and bias-corrected meta-analysis methods. Each method is illustrated for the replication of an ANOVA and associated post hoc t-test. Annotated R-code for all analyses is provided with the chapter

Bayesian Versus Frequentist Estimation for Structural Equation Models in Small Sample Contexts : A Systematic Review

In small sample contexts, Bayesian estimation is often suggested as a viable alternative to frequentist estimation, such as maximum likelihood estimation. Our systematic literature review is the first study aggregating information from numerous simulation studies to present an overview of the performance of Bayesian and frequentist estimation for structural equation models with small sample sizes. We conclude that with small samples, the use of Bayesian estimation with diffuse default priors can result in severely biased estimates–the levels of bias are often even higher than when frequentist methods are used. This bias can only be decreased by incorporating prior information. We therefore recommend against naively using Bayesian estimation when samples are small, and encourage researchers to make well-considered decisions about all priors. For this purpose, we provide recommendations on how to construct thoughtful priors

Bayesian Versus Frequentist Estimation for Structural Equation Models in Small Sample Contexts : A Systematic Review

In small sample contexts, Bayesian estimation is often suggested as a viable alternative to frequentist estimation, such as maximum likelihood estimation. Our systematic literature review is the first study aggregating information from numerous simulation studies to present an overview of the performance of Bayesian and frequentist estimation for structural equation models with small sample sizes. We conclude that with small samples, the use of Bayesian estimation with diffuse default priors can result in severely biased estimates–the levels of bias are often even higher than when frequentist methods are used. This bias can only be decreased by incorporating prior information. We therefore recommend against naively using Bayesian estimation when samples are small, and encourage researchers to make well-considered decisions about all priors. For this purpose, we provide recommendations on how to construct thoughtful priors

Adjustment for unmeasured confounding through informative priors for the confounder-outcome relation

Background: Observational studies of medical interventions or risk factors are potentially biased by unmeasured confounding. In this paper we propose a Bayesian approach by defining an informative prior for the confounder-outcome relation, to reduce bias due to unmeasured confounding. This approach was motivated by the phenomenon that the presence of unmeasured confounding may be reflected in observed confounder-outcome relations being unexpected in terms of direction or magnitude. Methods: The approach was tested using simulation studies and was illustrated in an empirical example of the relation between LDL cholesterol levels and systolic blood pressure. In simulated data, a comparison of the estimated exposure-outcome relation was made between two frequentist multivariable linear regression models and three Bayesian multivariable linear regression models, which varied in the precision of the prior distributions. Simulated data contained information on a continuous exposure, a continuous outcome, and two continuous confounders (one considered measured one unmeasured), under various scenarios. Results: In various scenarios the proposed Bayesian analysis with an correctly specified informative prior for the confounder-outcome relation substantially reduced bias due to unmeasured confounding and was less biased than the frequentist model with covariate adjustment for one of the two confounding variables. Also, in general the MSE was smaller for the Bayesian model with informative prior, compared to the other models. Conclusions: As incorporating (informative) prior information for the confounder-outcome relation may reduce the bias due to unmeasured confounding, we consider this approach one of many possible sensitivity analyses of unmeasured confounding

Introduction to Bayesian Statistics

In this brief introductory chapter, we sought to inform readers new to Bayesian statistics about the fundamental concepts in Bayesian analyses. The most important take-home messages to remember are that in Bayesian statistics, the analysis starts with an explicit formulation of prior beliefs that are updated with the observed data to obtain a posterior distribution. The posterior distribution is then used to make inferences about probable values of a given parameter (or set of parameters). Furthermore, Bayes Factors allow for comparison of non-nested models, and it is possible to compute the amount of support for the null hypothesis, which cannot be done in the frequentist framework. Subsequent chapters in this volume make use of Bayesian methods for obtaining posteriors of parameters of interest, as well as Bayes Factors

Attention Deficit Hyperactivity Disorder, Aggression, and Illicit Stimulant Use: Is This Self-Medication?

This study compares adults with and without attention deficit hyperactivity disorder (ADHD) on measures of direct and displaced aggression and illicit drug use. Three hundred ninety-six adults were administered the Wender Utah Rating Scale, the Risk Behavior Assessment, the Aggression Questionnaire (AQ), and the Displaced Aggression Questionnaire (DAQ). Those with ADHD were higher on all scales of the AQ and DAQ, were younger at first use of amphetamines, and were more likely to have ever used crack and amphetamines. A Structural Equation Model found a significant interaction in that for those with medium and high levels of verbal aggression, ADHD predicts crack and amphetamine. Follow-up logistic regression models suggest that blacks self-medicate with crack and whites and Hispanics self-medicate with amphetamine when they have ADHD and verbal aggression

Attention Deficit Hyperactivity Disorder, Aggression, and Illicit Stimulant Use: Is This Self-Medication?

This study compares adults with and without attention deficit hyperactivity disorder (ADHD) on measures of direct and displaced aggression and illicit drug use. Three hundred ninety-six adults were administered the Wender Utah Rating Scale, the Risk Behavior Assessment, the Aggression Questionnaire (AQ), and the Displaced Aggression Questionnaire (DAQ). Those with ADHD were higher on all scales of the AQ and DAQ, were younger at first use of amphetamines, and were more likely to have ever used crack and amphetamines. A Structural Equation Model found a significant interaction in that for those with medium and high levels of verbal aggression, ADHD predicts crack and amphetamine. Follow-up logistic regression models suggest that blacks self-medicate with crack and whites and Hispanics self-medicate with amphetamine when they have ADHD and verbal aggression