Machine learning to analyze single-case data : a proof of concept

Visual analysis is the most commonly used method for interpreting data from singlecase designs, but levels of interrater agreement remain a concern. Although structured aids to visual analysis such as the dual-criteria (DC) method may increase interrater agreement, the accuracy of the analyses may still benefit from improvements. Thus, the purpose of our study was to (a) examine correspondence between visual analysis and models derived from different machine learning algorithms, and (b) compare the accuracy, Type I error rate and power of each of our models with those produced by the DC method. We trained our models on a previously published dataset and then conducted analyses on both nonsimulated and simulated graphs. All our models derived from machine learning algorithms matched the interpretation of the visual analysts more frequently than the DC method. Furthermore, the machine learning algorithms outperformed the DC method on accuracy, Type I error rate, and power. Our results support the somewhat unorthodox proposition that behavior analysts may use machine learning algorithms to supplement their visual analysis of single-case data, but more research is needed to examine the potential benefits and drawbacks of such an approach

Some characteristics and arguments in favor of a science of machine behavior analysis

Researchers and practitioners recognize four domains of behavior analysis: radical behaviorism, the experimental analysis of behavior, applied behavior analysis, and the practice of behavior analysis. Given the omnipresence of technology in every sphere of our lives, the purpose of this conceptual article is to describe and argue in favor of a ffth domain: machine behavior analysis. Machine behavior analysis is a science that examines how machines interact with and produce relevant changes in their external environment by relying on replicability, behavioral terminology, and the philosophical assumptions of behavior analysis (e.g., selectionism, determinism, parsimony) to study artifcial behavior. Arguments in favor of a science of machine behavior include the omnipresence and impact of machines on human behavior, the inability of engineering alone to explain and control machine behavior, and the need to organize a verbal community of scientists around this common issue. Regardless of whether behavior analysts agree or disagree with this proposal, I argue that the feld needs a debate on the topic. As such, the current article aims to encourage and contribute to this debate

Machine learning to analyze single-case graphs : a comparison to visual inspection

Behavior analysts commonly use visual inspection to analyze single-case graphs, but studies on its reliability have produced mixed results. To examine this issue, we compared the Type I error rate and power of visual inspection with a novel approach—machine learning. Five expert visual raters analyzed 1,024 simulated AB graphs, which differed on number of points per phase, autocorrelation, trend, variability, and effect size. The ratings were compared to those obtained by the conservative dual-criteria method and two models derived from machine learning. On average, visual raters agreed with each other on only 75% of graphs. In contrast, both models derived from machine learning showed the best balance between Type I error rate and power while producing more consistent results across different graph characteristics. The results suggest that machine learning may support researchers and practitioners in making fewer errors when analyzing single-case graphs, but replications remain necessary

SAGITTAL PLANE RESISTANCE TORQUE IN ANKLE BRACES

Ligaments of the ankle joint complex are among the most frequently damaged structures during sports and physical activity (Eils et al., 2002). One common intervention used to prevent ankle ligament injury is the application of lace-up style ankle braces. These braces, usually made of non stretch nylon materials, increase the mechanical stability at the ankle joint by restricting the allowable range of motion thereby limiting strain on joint ligaments. Ankle braces are primarily designed to restrict motion in the frontal plane to limit ankle inversion and eversion without impeding the plantar-dorsi flexion (PF) motion (Eils et al., 2002). However, studies examining the effect of bracing on ankle motion during drop jumping have found a significant reduction in sagittal plane ankle motion while braced (DiStefano et al., 2008). Previous studies have examined isolated ankle range of motion restriction around the PF axis with different brace types (e.g. Eils et al 2002), but these studies were not able to distinguish the resistance torque due to the brace alone. The purpose of the present study was to measure the passive mechanical resistance torque around the ankle PF axis generated by a range of commercially available ankle braces while moving through the sagittal plane

How many tiers do we need? Type I errors and power in multiple baseline designs

Design quality guidelines typically recommend that multiple baseline designs include at least three demonstrations of effects. Despite its widespread adoption, this recommendation does not appear grounded in empirical evidence. The main purpose of our study was to address this issue by assessing Type I error rate and power in multiple baseline designs. First, we generated 10,000 multiple baseline graphs, applied the dualcriteria method to each tier, and computed Type I error rate and power for different number of tiers showing a clear change. Second, two raters categorized the tiers for 300 multiple baseline graphs to replicate our analyses using visual inspection. When multiple baseline designs had at least three tiers and two or more of these tiers showed a clear change, the Type I error rate remained adequate (< .05) while power also reached acceptable levels (> .80). In contrast, requiring all tiers to show a clear change resulted in overly stringent conclusions (i.e., unacceptably low power). Therefore, our results suggest that researchers and practitioners should carefully consider limitations in power when requiring all tiers of a multiple baseline design to show a clear change in their analyses

Waiting for baseline stability in single-case designs : is it worth the time and effort?

Researchers and practitioners often use single-case designs (SCDs), or n-of-1 trials, to develop and validate novel treatments. Standards and guidelines have been published to provide guidance as to how to implement SCDs, but many of their recommendations are not derived from the research literature. For example, one of these recommendations suggests that researchers and practitioners should wait for baseline stability prior to introducing an independent variable. However, this recommendation is not strongly supported by empirical evidence. To address this issue, we used Monte Carlo simulations to generate graphs with fxed, response-guided, and random baseline lengths while manipulating trend and variability. Then, our analyses compared the type I error rate and power produced by two methods of analysis: the conservative dual-criteria method (a structured visual aid) and a support vector classifer (a model derived from machine learning). The conservative dual-criteria method produced fewer errors when using response-guided decision-making (i.e., waiting for stability) and random baseline lengths. In contrast, waiting for stability did not reduce decision-making errors with the support vector classifer. Our fndings question the necessity of waiting for baseline stability when using SCDs with machine learning, but the study must be replicated with other designs and graph parameters that change over time to support our results

A comparison of communication tone and responding across users and developers in two R mailing lists

The R programming language has an active community of both users and developers, which maintain mailing lists to communicate. Given their differences in training and stability, the effects of communication tone on responding may differ across these two groups. We thus compared the prevalence and characteristics of different tones in the R-help user and Rdevel developer mailing lists over a ten-year period as well as their relation to replies. Our analyses indicate that developers displayed marginally more positive and negative tones than users. Moreover, developers seemed less influenced by tone when choosing to reply to messages. Overall, our results suggest that different tones may produce small differences in responding across users and developers

Agreement between visual inspection and objective analysis methods : a replication and extension

Behavior analysts typically rely on visual inspection of single-case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual-criteria method. To replicate and extend a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual inspection of five raters, the conservative dual-criteria method, and a machine-learning algorithm (i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual-criteria method and the support vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective methods produce results consistent with visual inspection, which may support their future use

Évaluation fonctionnelle du comportement en psychoéducation : validité convergente de l’analyse fonctionnelle avec les analyses descriptive et indirecte

L’évaluation fonctionnelle du comportement et la sélection d’une intervention fonctionnelle font partie des meilleures pratiques pour réduire les comportements problématiques. Toutefois, peu d’études ont examiné les habiletés des psychoéducateurs en évaluation fonctionnelle du comportement. Dans cette étude, 54 psychoéducateurs ont analysé des transcriptions de la grille antécédent-comportement-conséquence (ACC) narrative et de l’Open-Ended Functional Assessment Interview (OEFAI) pour déterminer la fonction de comportements problématiques de quatre enfants ayant un trouble du spectre de l’autisme. Les psychoéducateurs étaient plus exacts pour identifier la fonction en utilisant l’OEFAI pour les comportements de trois des enfants et en utilisant la grille ACC pour le comportement d’un enfant. Lorsque nous avons comparé les conclusions des psychoéducateurs avec celles produites par des analystes du comportement, quatre des sept comparaisons indiquaient que les analystes du comportement étaient plus exacts tandis que les trois autres étaient non différenciées. Les résultats soulignent l’importance de bonifier l’enseignement de l’évaluation fonctionnelle du comportement chez les psychoéducateurs.Functional behavior assessment and the selection of function-based interventions are best practices in the treatment of challenging behavior. However, few studies have examined the skills of psychoeducators in functional behavior assessment. In this study, 54 psychoeducators analyzed ABC narrative recording and Open-Ended Functional Assessment Interview (OEFAI) transcripts to identify the function of challenging behavior in four children with autism spectrum disorders. Psychoeducators produced more accurate functions when using the OEFAI for the behavior of three children and when using the ABC recording for the behavior of one child. When we compared the conclusions of the psychoeducators with those produced by behavior analysts, four of seven comparisons indicated that behavior analysts were more accurate whereas the other three were non differentiated. Altogether, the results underline the importance of refining training in functional behavior assessment for psychoeducators

Using computer tablets to assess preference for videos in children with autism

Using computer tablets, we assessed preference for videos in five children with autism spectrum disorder (ASD). Then, we provided access to most preferred and less preferred videos contingent on sitting on one of two chairs within a concurrent schedule design. All participants spent consistently more time sitting on the chair associated with the video selected the most often during the preference assessment, indicating that practitioners may use the tablet-based assessment procedure to identify potential video reinforcers for children with ASD in applied settings