Prediction of mental effort derived from an automated vocal biomarker using machine learning in a large-scale remote sample

IntroductionBiomarkers of mental effort may help to identify subtle cognitive impairments in the absence of task performance deficits. Here, we aim to detect mental effort on a verbal task, using automated voice analysis and machine learning.MethodsAudio data from the digit span backwards task were recorded and scored with automated speech recognition using the online platform NeuroVocalixTM, yielding usable data from 2,764 healthy adults (1,022 male, 1,742 female; mean age 31.4 years). Acoustic features were aggregated across each trial and normalized within each subject. Cognitive load was dichotomized for each trial by categorizing trials at >0.6 of each participants' maximum span as “high load.” Data were divided into training (60%), test (20%), and validate (20%) datasets, each containing different participants. Training and test data were used in model building and hyper-parameter tuning. Five classification models (Logistic Regression, Naive Bayes, Support Vector Machine, Random Forest, and Gradient Boosting) were trained to predict cognitive load (“high” vs. “low”) based on acoustic features. Analyses were limited to correct responses. The model was evaluated using the validation dataset, across all span lengths and within the subset of trials with a four-digit span. Classifier discriminant power was examined with Receiver Operating Curve (ROC) analysis.ResultsParticipants reached a mean span of 6.34 out of 8 items (SD = 1.38). The Gradient Boosting classifier provided the best performing model on test data (AUC = 0.98) and showed excellent discriminant power for cognitive load on the validation dataset, across all span lengths (AUC = 0.99), and for four-digit only utterances (AUC = 0.95).DiscussionA sensitive biomarker of mental effort can be derived from vocal acoustic features in remotely administered verbal cognitive tests. The use-case of this biomarker for improving sensitivity of cognitive tests to subtle pathology now needs to be examined

Developing Digital Tools for Remote Clinical Research:How to Evaluate the Validity and Practicality of Active Assessments in Field Settings

The ability of remote research tools to collect granular, high-frequency data on symptoms and digital biomarkers is an important strength because it circumvents many limitations of traditional clinical trials and improves the ability to capture clinically relevant data. This approach allows researchers to capture more robust baselines and derive novel phenotypes for improved precision in diagnosis and accuracy in outcomes. The process for developing these tools however is complex because data need to be collected at a frequency that is meaningful but not burdensome for the participant or patient. Furthermore, traditional techniques, which rely on fixed conditions to validate assessments, may be inappropriate for validating tools that are designed to capture data under flexible conditions. This paper discusses the process for determining whether a digital assessment is suitable for remote research and offers suggestions on how to validate these novel tools

A comparison of machine learning methods for classification using simulation with multiple real data examples from mental health studies

Background: Recent literature on the comparison of machine learning methods has raised questions about the neutrality, unbiasedness and utility of many comparative studies. Reporting of results on favourable datasets and sampling error in the estimated performance measures based on single samples are thought to be the major sources of bias in such comparisons. Better performance in one or a few instances does not necessarily imply so on an average or on a population level and simulation studies may be a better alternative for objectively comparing the performances of machine learning algorithms. Methods: We compare the classification performance of a number of important and widely used machine learning algorithms, namely the Random Forests (RF), Support Vector Machines (SVM), Linear Discriminant Analysis (LDA) and k-Nearest Neighbour (kNN). Using massively parallel processing on high-performance supercomputers, we compare the generalisation errors at various combinations of levels of several factors: number of features, training sample size, biological variation, experimental variation, effect size, replication and correlation between features. Results: For smaller number of correlated features, number of features not exceeding approximately half the sample size, LDA was found to be the method of choice in terms of average generalisation errors as well as stability (precision) of error estimates. SVM (with RBF kernel) outperforms LDA as well as RF and kNN by a clear margin as the feature set gets larger provided the sample size is not too small (at least 20). The performance of kNN also improves as the number of features grows and outplays that of LDA and RF unless the data variability is too high and/or effect sizes are too small. RF was found to outperform only kNN in some instances where the data are more variable and have smaller effect sizes, in which cases it also provide more stable error estimates than kNN and LDA. Applications to a number of real datasets supported the findings from the simulation study

Validation of the mind excessively wandering scale and the relationship of mind wandering to impairment in adult ADHD

Objective: This study investigates excessive mind wandering (MW) in adult ADHD using a new scale: the Mind Excessively Wandering Scale (MEWS). Method: Data from two studies of adult ADHD was used in assessing the psychometric properties of the MEWS. Case-control differences in MW, the association with ADHD symptoms, and the contribution to functional impairment were investigated. Results: The MEWS functioned well as a brief measure of excessive MW in adult ADHD, showing good internal consistency (α > .9), and high sensitivity (.9) and specificity (.9) for the ADHD diagnosis, comparable with that of existing ADHD symptom rating scales. Elevated levels of MW were found in adults with ADHD, which contributed to impairment independently of core ADHD symptom dimensions. Conclusion: Findings suggest excessive MW is a common co-occurring feature of adult ADHD that has specific implications for the functional impairments experienced. The MEWS has potential utility as a screening tool in clinical practice to assist diagnostic assessment

The Rural Household Multiple Indicator Survey, data from 13,310 farm households in 21 countries

The Rural Household Multiple Indicator Survey (RHoMIS) is a standardized farm household survey approach which collects information on 758 variables covering household demographics, farm area, crops grown and their production, livestock holdings and their production, agricultural product use and variables underlying standard socio-economic and food security indicators such as the Probability of Poverty Index, the Household Food Insecurity Access Scale, and household dietary diversity. These variables are used to quantify more than 40 different indicators on farm and household characteristics, welfare, productivity, and economic performance. Between 2015 and the beginning of 2018, the survey instrument was applied in 21 countries in Central America, sub-Saharan Africa and Asia. The data presented here include the raw survey response data, the indicator calculation code, and the resulting indicator values. These data can be used to quantify on- and off-farm pathways to food security, diverse diets, and changes in poverty for rural smallholder farm households

Emotional lability, comorbidity and impairment in adults with attention-deficit hyperactivity disorder

BACKGROUND: Adults with attention-deficit hyperactivity disorder (ADHD) frequently report emotional lability (EL). However, it is not known whether EL may be accounted for by comorbid psychiatric conditions or symptoms. This study evaluates the influence of comorbid clinical symptoms on EL, and investigates the relationship between EL and impairment. METHODS: Over 500 consecutive male adult referrals at the ADHD Clinic for adults at the South London and Maudsley Hospital (U.K) were screened. 41 individuals with ADHD without comorbidity, current medication or frequent substance were identified, and compared with 47 matched healthy male control participants. Measures included IQ, clinical interview and self-reported ADHD symptoms, EL, impairment and antisocial behaviour. RESULTS: ADHD participants reported elevated EL, showing good case-control differentiation in receiver operating curve analysis. EL was most strongly predicted by hyperactivity-impulsivity rather than subsyndromal comorbid symptoms, and contributed independently to impairment in daily life. LIMITATIONS: Results may not generalise to children with ADHD, or many adults with ADHD, who are frequently affected by comorbid psychiatric conditions and substance use disorders. CONCLUSIONS: EL in adults with ADHD appears to be primarily associated with ADHD itself rather than comorbid conditions, and helps to explain some of the impairments not accounted for by classical features of the disorder. Results indicate that adults presenting with long-term problems with EL should routinely be screened for the presence of ADHD

An update on the debated association between ADHD and bipolar disorder across the lifespan

Diagnostic formulations for attention deficit hyperactivity disorder (ADHD) and bipolar disorder (BD) both include symptoms of distractibility, psychomotor agitation and talkativeness, alongside associated emotional features (irritability and emotional lability). Treatment studies suggest the importance of accurate delineation of ADHD and BD. However, boundaries between the two disorders are blurred by the introduction of broader conceptualisations of BD. This review attempts to elucidate whether associations between ADHD and BD are likely to be driven by superficial symptomatological similarities or by a more meaningful etiological relationship between the disorders. This is achieved by outlining findings on comorbidity, temporal progression of the disorders, familial co-variation, and neurobiology in ADHD and BD across the lifespan. Longitudinal studies fail to consistently show developmental trajectories between ADHD and BD. Comparative research investigating neurobiology is in its infancy, and although some similarities are seen between ADHD and BD, studies also emphasise differences between the two disorders. However, comorbidity and family studies appear to show that the two disorders occur together and aggregate in families at higher than expected rates. Furthermore close inspection of results from population studies reveals heightened co-occurrence of ADHD and BD even in the context of high comorbidity commonly noted in psychopathology. These results point towards a meaningful association between ADHD and BD, going beyond symptomatic similarities. However, future research needs to account for heterogeneity of BD, making clear distinctions between classical episodic forms of BD, and broader conceptualisations of the disorder characterised by irritability and emotional lability, when evaluating the relationship with ADHD

Normalisation of frontal theta activity following methylphenidate treatment in adult attention-deficit/hyperactivity disorder

Attention-deficit/hyperactivity disorder (ADHD) is associated with cognitive performance and functional brain changes that are sensitive to task conditions, indicating a role for dynamic impairments rather than stable cognitive deficits. Prominent hypotheses consistent with this observation are a failure to optimise brain arousal or activation states. Here we investigate cortical activation during different conditions. Using a sample of 41 non-comorbid adults with ADHD and 48 controls, we examine quantitative EEG activity during a resting state, a cued continuous performance test with flankers (CPT-OX) and the sustained attention to response task (SART). We further investigate the effects of methylphenidate in a subsample of 21 ADHD cases. Control participants showed a task-related increase in theta activity when engaged in cognitive tasks, primarily in frontal and parietal regions, which was absent in participants with ADHD. Treatment with methylphenidate resulted in normalisation of the resting state to task activation pattern. These findings suggest that ADHD in adults is associated with insufficient allocation of neuronal resources required for normal cortical activation commensurate with task demands. Further work is required to clarify the causal role of the deficit in cortical activation and provide a clearer understanding of the mechanisms involved