Description of the Method for Evaluating Digital Endpoints in Alzheimer Disease Study : Protocol for an Exploratory, Cross-sectional Study

©Jelena Curcic, Vanessa Vallejo, Jennifer Sorinas, Oleksandr Sverdlov, Jens Praestgaard, Mateusz Piksa, Mark Deurinck, Gul Erdemli, Maximilian Bügler, Ioannis Tarnanas, Nick Taptiklis, Francesca Cormack, Rebekka Anker, Fabien Massé, William Souillard-Mandar, Nathan Intrator, Lior Molcho, Erica Madero, Nicholas Bott, Mieko Chambers, Josef Tamory, Matias Shulz, Gerardo Fernandez, William Simpson, Jessica Robin, Jón G Snædal, Jang-Ho Cha, Kristin Hannesdottir. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 10.08.2022.BACKGROUND: More sensitive and less burdensome efficacy end points are urgently needed to improve the effectiveness of clinical drug development for Alzheimer disease (AD). Although conventional end points lack sensitivity, digital technologies hold promise for amplifying the detection of treatment signals and capturing cognitive anomalies at earlier disease stages. Using digital technologies and combining several test modalities allow for the collection of richer information about cognitive and functional status, which is not ascertainable via conventional paper-and-pencil tests. OBJECTIVE: This study aimed to assess the psychometric properties, operational feasibility, and patient acceptance of 10 promising technologies that are to be used as efficacy end points to measure cognition in future clinical drug trials. METHODS: The Method for Evaluating Digital Endpoints in Alzheimer Disease study is an exploratory, cross-sectional, noninterventional study that will evaluate 10 digital technologies' ability to accurately classify participants into 4 cohorts according to the severity of cognitive impairment and dementia. Moreover, this study will assess the psychometric properties of each of the tested digital technologies, including the acceptable range to assess ceiling and floor effects, concurrent validity to correlate digital outcome measures to traditional paper-and-pencil tests in AD, reliability to compare test and retest, and responsiveness to evaluate the sensitivity to change in a mild cognitive challenge model. This study included 50 eligible male and female participants (aged between 60 and 80 years), of whom 13 (26%) were amyloid-negative, cognitively healthy participants (controls); 12 (24%) were amyloid-positive, cognitively healthy participants (presymptomatic); 13 (26%) had mild cognitive impairment (predementia); and 12 (24%) had mild AD (mild dementia). This study involved 4 in-clinic visits. During the initial visit, all participants completed all conventional paper-and-pencil assessments. During the following 3 visits, the participants underwent a series of novel digital assessments. RESULTS: Participant recruitment and data collection began in June 2020 and continued until June 2021. Hence, the data collection occurred during the COVID-19 pandemic (SARS-CoV-2 virus pandemic). Data were successfully collected from all digital technologies to evaluate statistical and operational performance and patient acceptance. This paper reports the baseline demographics and characteristics of the population studied as well as the study's progress during the pandemic. CONCLUSIONS: This study was designed to generate feasibility insights and validation data to help advance novel digital technologies in clinical drug development. The learnings from this study will help guide future methods for assessing novel digital technologies and inform clinical drug trials in early AD, aiming to enhance clinical end point strategies with digital technologies. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/35442.Peer reviewe

Learning classification models of cognitive conditions from subtle behaviors in the digital Clock Drawing Test

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2015.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 91-95).The Clock Drawing Test -- a simple pencil and paper test -- has been used for more than 50 years as a screening tool to differentiate normal elderly individuals from those with cognitive impairment, and has proven useful in helping to diagnose dementias, such as Alzheimer's disease, Parkinson's disease, and other conditions. A group of hospitals and clinics have been administering the test using a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making available far more detailed data about the subject's performance. Using categorized stroke data from these drawings, we designed and computed a large collection of features, then explored the tradeoffs in performance and interpretability in classifiers built using a number of different subsets of these features and a variety of different machine learning techniques. We used traditional machine learning methods to build prediction models that achieve high accuracy. We operationalized widely used existing scoring algorithms so that we could use them as benchmarks for our models. We worked with clinicians to define guidelines for model interpretability, and constructed sparse linear models and decision lists designed to be as easy to use as scoring algorithms currently used by clinicians, but more accurate. We also extract insights from the data about the behavioral aspect of these conditions on patients. While our models will require additional testing with subjects for validation, they offer the possibility of substantial improvement in detecting cognitive impairment earlier than currently possible, a development with considerable potential impact in practice.by William Souillard-Mandar.M. Eng

DCTclock : Clinically-Interpretable and Automated Artificial Intelligence Analysis of Drawing Behavior for Capturing Cognition

Developing tools for efficiently measuring cognitive change specifically and brain health generally-whether for clinical use or as endpoints in clinical trials-is a major challenge, particularly for conditions such as Alzheimer's disease. Technology such as connected devices and advances in artificial intelligence offer the possibility of creating and deploying clinical-grade tools with high sensitivity, rapidly, cheaply, and non-intrusively. Starting from a widely-used paper and pencil cognitive status test-The Clock Drawing Test-we combined a digital input device to capture time-stamped drawing coordinates with a machine learning analysis of drawing behavior to create DCTclock™, an automated analysis of nuances in cognitive performance beyond successful task completion. Development and validation was conducted on a dataset of 1,833 presumed cognitively unimpaired and clinically diagnosed cognitively impaired individuals with varied neurological conditions. We benchmarked DCTclock against existing clock scoring systems and the Mini-Mental Status Examination, a widely-used but lengthier cognitive test, and showed that DCTclock offered a significant improvement in the detection of early cognitive impairment and the ability to characterize individuals along the Alzheimer's disease trajectory. This offers an example of a robust framework for creating digital biomarkers that can be used clinically and in research for assessing neurological function

DCTclock: Clinically-Interpretable and Automated Artificial Intelligence Analysis of Drawing Behavior for Capturing Cognition

Developing tools for efficiently measuring cognitive change specifically and brain health generally—whether for clinical use or as endpoints in clinical trials—is a major challenge, particularly for conditions such as Alzheimer's disease. Technology such as connected devices and advances in artificial intelligence offer the possibility of creating and deploying clinical-grade tools with high sensitivity, rapidly, cheaply, and non-intrusively. Starting from a widely-used paper and pencil cognitive status test—The Clock Drawing Test—we combined a digital input device to capture time-stamped drawing coordinates with a machine learning analysis of drawing behavior to create DCTclock™, an automated analysis of nuances in cognitive performance beyond successful task completion. Development and validation was conducted on a dataset of 1,833 presumed cognitively unimpaired and clinically diagnosed cognitively impaired individuals with varied neurological conditions. We benchmarked DCTclock against existing clock scoring systems and the Mini-Mental Status Examination, a widely-used but lengthier cognitive test, and showed that DCTclock offered a significant improvement in the detection of early cognitive impairment and the ability to characterize individuals along the Alzheimer's disease trajectory. This offers an example of a robust framework for creating digital biomarkers that can be used clinically and in research for assessing neurological function.</jats:p

Aging in the Digital Age : Using Technology to Increase the Reach of the Clinician Expert and Close the Gap Between Health Span and Life Span

Altres ajuts: Fundació La Caixa (LCF/PR/PR16/11110004), Institut Guttmann i Fundació Abertis, ICREA Academia 2019 award research grants i INNOBRAIN (COMRDI15-1-0017)Age-related cognitive impairment (ARCI) has a profound impact on individuals, families, health care systems, and societies at large. Evidence suggests that ARCI is the consequence of underlying brain pathology. Therefore, efforts to minimize the impact of ARCI and thus closing the gap between health span and life span, which has widened in recent years, requires early detection and timely deployment of targeted, personalized interventions. Access to clinical experts is limited and technology screening and assessment methods are thus appealing. However, as traditionally implemented patients were deprived of the benefit of personalized connection with a clinician, which is particularly critical for the prescription and to ensure the adherence to and ultimate success of therapeutic interventions. We present the concept of Intelligent Technology Therapy Assistant (ITA) as a scalable solution that increases the reach of clinical experts while sustaining the personal connection between each patient and their clinician. We illustrate ITA with the "Guttman Neuro Personal Trainer" ®, a tele-rehabilitation platform that provides neuropsychological evaluation and care, and the Barcelona Brain Health Initiative (BBHI) multimodal intervention coaching app, a mobile-based platform that provides lifestyle coaching support in domains related to brain health. In addition, we discuss the translation of these models to a large-scale enterprise with Linus Health. To this end, we conclude with a discussion of challenges and opportunities to move the field forward

Learning classification models of cognitive conditions from subtle behaviors in the digital Clock Drawing Test

The Clock Drawing Test—a simple pencil and paper test—has been used for more than 50 years as a screening tool to differentiate normal individuals from those with cognitive impairment, and has proven useful in helping to diagnose cognitive dysfunction associated with neurological disorders such as Alzheimer’s disease, Parkinson’s disease, and other dementias and conditions. We have been administering the test using a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making available far more detailed data about the subject’s performance. Using pen stroke data from these drawings categorized by our software, we designed and computed a large collection of features, then explored the tradeoffs in performance and interpretability in classifiers built using a number of different subsets of these features and a variety of different machine learning techniques. We used traditional machine learning methods to build prediction models that achieve high accuracy. We operationalized widely used manual scoring systems so that we could use them as benchmarks for our models. We worked with clinicians to define guidelines for model interpretability, and constructed sparse linear models and rule lists designed to be as easy to use as scoring systems currently used by clinicians, but more accurate. While our models will require additional testing for validation, they offer the possibility of substantial improvement in detecting cognitive impairment earlier than currently possible, a development with considerable potential impact in practice.Robert E. Wise Research and Education InstitutionUnited States. Defense Advanced Research Projects Agency (Contract D13AP00008)National Science Foundation (U.S.) (Award IIS-1404494)National Institute of Neurological Disorders and Stroke (U.S.) (Grants R01-NS17950, K23-NS60660, and R01-NS082386)National Heart, Lung, and Blood Institute (Contract N01-HC25195)National Institute on Aging (Grants R01 AG0333040, AG16492, and AG08122)National Institute of Mental Health (U.S.) (Grant RO1-MH073989)National Center for Advancing Translational Sciences (U.S.) (Clinical and Translational Science Award to the University of Florida UL1TR000064)University of Florida. Center for Movement Disorders and Neurorestoratio