Probability and visual aids for assessing intervention effectiveness in single-case designs: A field test

Single-case data analysis still relies heavily on visual inspection and, at the same time, it is not clear to what extent the results of different quantitative procedures converge in identifying an intervention effect and its magnitude when applied to the same data; this is the type of evidence provided here for two procedures. One of the procedures, included due to the importance of providing objective criteria to visual analysts, is a visual aid fitting and projecting split-middle trend while taking into account data variability. The other procedure converts several different metrics into probabilities making their results comparable. In the present study we study to what extend these two procedures coincide in the magnitude of intervention effect taking place in a set of studies stemming from a recent meta-analysis. The procedures concur to a greater extent with the values of the indices computed and with each other and, to a lesser extent, with our own visual analysis. For the distinctions smaller and larger effects the probability-based approach seems somewhat better suited. Moreover, the results of the field test suggest that the latter is a reasonably good mechanism for translating different metrics into similar labels. User friendly R code is provided for promoting the use of the visual aid, together with a quantification based on nonoverlap and the label provided by the probability approach

Reprint of “The Single-Case Reporting Guideline In BEhavioural interventions (SCRIBE) 2016: explanation and elaboration”

There is substantial evidence that research studies reported in the scientific literature do not provide adequate information so that readers know exactly what was done and what was found. This problem has been addressed by the development of reporting guidelines which tell authors what should be reported and how it should be described. Many reporting guidelines are now available for different types of research designs. There is no such guideline for one type of research design commonly used in the behavioral sciences, the single-case experimental design (SCED). The present study addressed this gap. This report describes the Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016, which is a set of 26 items that authors need to address when writing about SCED research for publication in a scientific journal. Each item is described, a rationale for its inclusion is provided, and examples of adequate reporting taken from the literature are quoted. It is recommended that the SCRIBE 2016 is used by authors preparing manuscripts describing SCED research for publication, as well as journal reviewers and editors who are evaluating such manuscripts.Published versio

The Effect of Embedding the Restricted Interests of Students with Autism Spectrum Disorder in Text on Reading Comprehension

Reading comprehension deficits are common for students with Autism Spectrum Disorder (ASD) and the rise in prevalence of this disorder has resulted in an increased demand for evidence-based strategies for teaching reading comprehension to this population. Research has found an increase in desired behaviors when a restricted interest (RI) or interests, a diagnostic feature of ASD, is utilized in intervention techniques. In a pilot study, El Zein, Solis, Lang, and Kim (2016) found that embedding the RI of a student with ASD in text increased that student’s reading comprehension performance. The current study further investigated the effect of embedding the RI of students with ASD in text on reading comprehension performance by replicating the pilot study and examining the impact of frequency of RI embedded with two high school students with ASD using a single-subject, multi-element research design. Neither participant showed an in increase in the number of relevant words shared during oral retell and only one participant showed an increase in the percent of correctly answered reading comprehension questions. Frequency of RI embedded in text did not impact reading comprehension performance. Results suggest that there are limitations to the results of the pilot study completed by El Zein et al. (2016) and indicate potential variables that may impact the effect of embedding the RI of students with ASD in text on reading comprehension. Findings are discussed in regards to directions for future research

Smart designs for smart technologies: research challenges and emerging solutions for scientist-practitioners within e-mental health

Consumers are increasingly likely to access various forms of e-mental health, and there is considerable danger that they may be exposed to untested interventions. Traditional research designs, such as the randomized controlled trial (RCT), are limited in their capacity to match the pace of development and evolving nature of e-mental health. There are a number of unique challenges associated with research into the development and use of technologically based interventions. This article discusses these challenges and examines emerging strategies that may enable clinicians to be more confident when integrating e-mental health in their practices. We argue that greater use of small sample size designs, greater collaboration and research in applied settings, as well as more focused empirical investigation during program development stages are needed. We use a research example of a Smartphone application aimed at the treatment of anxiety disorders to illustrate the procedure, value, and clinical applications of each of the emerging research designs

Conversation-based intervention for adolescents using augmentative and alternative communication

This study evaluated the effects of a conversation-based intervention on the use of verbs, personal pronouns, bound morphemes and spontaneous clauses in adolescents with cerebral palsy who use augmentative and alternative communication (AAC). Four teenage girls aged from 14 to 18 years participated in the study. After a baseline period, a conversation-based intervention was provided for each participant in the context of a personal collage building activity. The conversations were videotaped, transcribed, and analyzed using the Systematic Analysis of Language Samples (SALT; Miller & Iglesias, 2012). While the results are mixed, all 4 participants increased their use of at least one linguistic target, 3 increased their use of verbs and grammatically correct spontaneous clauses, 2 increased their use of personal pronouns, and one produced more bound morphemes during intervention than in baseline. These findings, and future research needs, are discussed

School-based Consultation to Promote Generalization of Early Childhood Educators’ Evidence-based Practices: A Meta-analysis

Early childhood educators are tasked with supporting young children’s academic, behavioral, and social-emotional through the use of evidence-based practices (EBPs). Unfortunately, early childhood educators may struggle with consistent implementation of EBPs. When this occurs, school-based consultation may be beneficial in promoting early childhood educators’ use of effective classroom management practices and interventions. A critical aspect of school-based consultation is ensuring that early childhood educators generalize strategies they have been trained to use through consultation. However, the extent to which early childhood educators generalize skills trained through consultation across settings, children, and other evidence-based practices is unclear. Therefore, the purpose of this meta-analysis was to synthesize the existing school-based consultation literature in early childhood settings to determine the extent to which early childhood educators generalize skills trained through consultation. A total of 12 studies including 39 educator consultees and 171 effects comprised our final sample. Three-level hierarchical models with robust variance estimation were used to pool both Log Response Ratios and Tau effect sizes for analyses. Overall results indicated school-based consultation had a positive and significant impact on educator generalization of EBPs (LRRi = 0.95; Tau = 0.79). Further, particular implementation supports (e.g., in situ training) significantly contributed to EBP generalization. These results suggest school-based consultation to be an effective means for promoting early childhood educator generalization of EBPs. Implications for research and practice and future research directions are discussed

The Interplay of Achievement and Achievement Motivation: Gender Differences in Math Top-Performers and Functional Relations

Achievement and achievement motivation are two central constructs in educational psychology. The interplay between these constructs is a major element in prominent theoretical frameworks such as the Situated Expectancy Value Model (SEVT; e.g., Eccles & Wigfield, 2020). Given the topic’s relevance for individuals’ further educational trajectories and life courses, it is important to obtain particularly reliable and robust results. To achieve this aim in the present doctoral thesis, I applied innovative multilevel meta-analytical approaches, using data from international large-scale assessments to investigate the interplay between achievement and achievement motivation. Using a Multilevel Individual Participant Data (IPD) Meta-Analysis, Study I examined gender differences in achievement, achievement profiles, and achievement motivation in mathematics, reading, and science in the group of top-performing math students (top 5%) across 82 countries. In addition, it was investigated to what extent gender differences in the top 5% in mathematics were moderated by cross-national variations in sociocultural factors (i.e., in specific gender equality indicators). To this end, I used data from 15-year-old students who participated in six PISA cycles. The results showed that there were on average more male than female students (40%) that scored in the top 5% in mathematics. In addition, mathematically top-performing female students’ achievement profiles were more balanced across domains, whereas mathematically top-performing male students’ achievement profiles were more mathematics-oriented. Moreover, mathematically top-performing female students reported a higher interest in the verbal domain and in human biology than male students. On the contrary, mathematically top-performing male students reported a higher interest in physics-related topics than female students (i.e., physics, motion of forces, energy transformation). The results also showed that specific gender equality indicators moderated the share of female students in the top 5% in mathematics and explained variability in achievement profiles. In Study II of this doctoral thesis, the functional relations between achievement and self-concept were systematically investigated using a Multilevel Integrative Data Analysis. The guiding research question was to examine the extent to which a nonlinear relation between achievement and self-concept can be generalized across domains, age groups, analytical approaches, and 13 countries. The analyses were based on eight cycles of PISA, TIMSS, and PIRLS. Quadratic and interrupted regression analyses showed nonlinear relations in secondary school students, demonstrating that the relations between achievement and corresponding self-concepts were weaker for lower achieving students than for higher achieving students. This suggests that lower achieving students might apply self-protective strategies to prevent negative self-evaluation. Nonlinear effects were also present in younger students, but the pattern of results was rather heterogeneous. The present doctoral thesis contributed to uncover the interplay between achievement and achievement motivation by using advanced multilevel meta-analytical approaches. Based on this work, future research is encouraged to apply such statistical tools to meta-analyze variance in individual participant data to enhance the reliability and robustness of the obtained empirical evidence on the interplay between achievement and achievement motivation.Leistung und Leistungsmotivation sind zentrale Konstrukte in der pädagogisch-psychologischen Forschung. Das Zusammenspiel von Leistung und Leistungsmotivation wird in prominenten theoretischen Rahmenmodellen wie der Erwartungs-Wert-Theorie (z.B. Eccles & Wigfield, 2020) untersucht. Aufgrund der zentralen Bedeutung dieses Themas für individuelle Bildungs- und Lebensverläufe, ist es wichtig reliable und robuste Ergebnisse zu gewinnen. Um dieses Ziel zu erreichen, wurden in der vorliegenden Dissertation innovative meta-analytische Ansätze und Daten internationaler Schulleistungsstudien verwendet, um das Zusammenspiel von Leistung und Leistungsmotivation zu untersuchen. Im Rahmen einer Multilevel Individual Participant Data (IPD) Meta-Analyse wurden in Teilstudie I Geschlechtsunterschiede in der Leistung, in Leistungsprofilen und in der Leistungsmotivation von mathematisch talentierten Schülerinnen und Schüler (Top 5%) in Mathematik, im Lesen und in Naturwissenschaften in 82 Ländern analysiert. Zudem wurde untersucht, inwiefern die Variation in den Geschlechtsunterschieden zwischen Ländern auf Unterschiede in soziokulturellen Faktoren (d.h., in spezifischen Indikatoren der Geschlechtergleichstellung) zurückzuführen ist. Hierfür wurden die Daten von 15-jährigen Mädchen und Jungen aus sechs PISA-Zyklen verwendet. Es konnte gezeigt werden, dass insgesamt weniger Mädchen in der Gruppe der Spitzenleistenden in Mathematik (Top 5%) vertreten waren (Mädchenanteil 40%) sowie Mädchen in dieser Gruppe balanciertere Leistungsprofile aufwiesen, während Jungen eher zu mathematik-orientierten Leistungsprofilen neigten. Weiterhin zeigte sich, dass mathematisch talentierte Mädchen eine höhere Motivation im verbalen Bereich sowie ein stärkeres Interesse an Humanbiologie als Jungen berichteten. Mathematisch talentierte Jungen berichteten hingegen ein größeres Interesse an den Themenbereichen Physik, Bewegung und Kräfte und Energieumwandlung als Mädchen. Zudem konnte in Teilstudie I gezeigt werden, dass spezifische Gleichstellungsindikatoren den Anteil der Schülerinnen in den Top 5% in Mathematik moderierten und Variabilität in den Leistungsprofilen erklärten. In Teilstudie II wurde der funktionale Zusammenhang zwischen Leistung und Selbstkonzept systematisch im Rahmen einer Multilevel Integrative Data Analysis untersucht. Die zentrale Forschungsfrage war, ob ein nicht-linearer Zusammenhang zwischen Leistung und Selbstkonzept über Inhaltsdomänen, Altersgruppen, Analysemethoden und 13 Länder hinweg generalisiert vorliegt. Die Analysen basierten auf acht Zyklen der PISA-, TIMSS- und PIRLS-Studien. Die Ergebnisse zeigten, dass nicht-lineare Zusammenhänge zwischen Leistung und korrespondierenden Selbstkonzepten in Mathematik und im verbalen Bereich bei Schülerinnen und Schülern der Sekundarstufe vorlagen. Dabei deuten die Ergebnisse der quadratischen Regressionen und der Interrupted Regressions darauf hin, dass der Zusammenhang für leistungsschwächere Schülerinnen und Schüler schwächer war als für leistungsstärkere Schülerinnen und Schüler. Dies könnte in der Anwendung selbstwertdienlicher Strategien begründet sein. Nicht-lineare Zusammenhänge zeigten sich auch für jüngere Schülerinnen und Schüler, jedoch war die Befundlage für diese Altersgruppe über Länder und Analysemethoden hinweg heterogener. Die vorliegende Doktorarbeit trägt mit diesen Erkenntnissen dazu bei, das Zusammenspiel von Leistung und Leistungsmotivation unter Anwendung von IPD-Meta-Analysen bzw. Integrativen Datenanalysen aufzuklären. Basierend auf dieser Arbeit wird die Bedeutung hervorgehoben, Daten auf der individuellen Personenebene zu meta-analysieren, um die Reliabilität und Robustheit von Befunden zum Zusammenspiel von Leistung und Leistungsmotivation zu erhöhen

Doctor of Philosophy

dissertationThe current study evaluated the efficacy of a parent-training program, Play Your Way to Compliance, to increase compliance rates of young children with autism spectrum disorders (ASD). Play Your Way to Compliance is an evidence-based intervention package with an errorless learning approach designed to teach behavior management skills to parents to increase child compliance. The study involved four 6-year-old males with ASD and their mothers. A noncurrent multiple-baseline, multiple-probe design was used to evaluate the intervention package. Dependent variables included compliance rates, parent fidelity, externalizing problems, parenting stress, and consumer satisfaction. Results showed significant increases in compliance rates for each of the participants. At baseline, the mean compliance rate across all participants was 28.9%. During intervention, the participants’ mean compliance rate increased 90.8%. Follow-up at 3 weeks after intervention showed that participants maintained gains in compliance. Play Your Way to Compliance includes all of the materials necessary to effectively implement research-based interventions in one package. The program represents an effective and viable method for caregivers to increase compliance in young children

Automated prompting technologies in rehabilitation and at home

Purpose - The purpose of this paper is to test the efficacy of an interactive verbal prompting technology (Guide) on supporting the morning routine. Data have already established the efficacy of such prompting during procedural tasks, but the efficacy of such prompting in tasks with procedural and motivational elements remains unexamined. Such tasks, such as getting out of bed in the morning and engaging in personal care, are often the focus of rehabilitation goals. Design/methodology/approach - A single-n study with a male (age 61) who had severe cognitive impairment and was having trouble completing the morning routine. An A-B-A'-B'-A?-B? design was used, with the intervention phase occurring both in an in-patient unit (B, B') and in the participant's own home (B?). Findings - Interactive verbal prompting technology (Guide) significantly reduced support worker prompting and number of errors in the in-patient setting and in the participant's own home. Research limitations/implications - The results suggest that interactive verbal prompting can be used to support motivational tasks such as getting out of bed and the morning routine. This study used a single subject experimental design and the results need to be confirmed in a larger sample. Originality/value - This is the first report of use of interactive verbal prompting technology to support rehabilitation of a motivational task. It is also the first study to evaluate Guide in a domestic context