Surface electromyographic control of a novel phonemic interface for speech synthesis

Many individuals with minimal movement capabilities use AAC to communicate. These individuals require both an interface with which to construct a message (e.g., a grid of letters) and an input modality with which to select targets. This study evaluated the interaction of two such systems: (a) an input modality using surface electromyography (sEMG) of spared facial musculature, and (b) an onscreen interface from which users select phonemic targets. These systems were evaluated in two experiments: (a) participants without motor impairments used the systems during a series of eight training sessions, and (b) one individual who uses AAC used the systems for two sessions. Both the phonemic interface and the electromyographic cursor show promise for future AAC applications.F31 DC014872 - NIDCD NIH HHS; R01 DC002852 - NIDCD NIH HHS; R01 DC007683 - NIDCD NIH HHS; T90 DA032484 - NIDA NIH HHShttps://www.ncbi.nlm.nih.gov/pubmed/?term=Surface+electromyographic+control+of+a+novel+phonemic+interface+for+speech+synthesishttps://www.ncbi.nlm.nih.gov/pubmed/?term=Surface+electromyographic+control+of+a+novel+phonemic+interface+for+speech+synthesisPublished versio

Computational optimization and prediction strategies for increasing communication rate in phoneme-based augmentative and alternative communication (AAC)

Up to 1.2% of the population is unable to meet daily communication needs using typical speech and may use augmentative and alternative communication (AAC) strategies to communicate, including manual sign language, facial gestures, and aided strategies such as selecting targets on an onscreen keyboard. However, for individuals whose impairments affect both speech and non-speech motor systems (e.g., spinal cord injury, amyotrophic lateral sclerosis, multiple sclerosis), their ability to use manual sign and access computer systems are impacted. AAC access methods in this population remain inherently slow and effortful (e.g., eye-tracking, head-tracking, mechanical switches). Thus, optimizing communication interfaces for alternate access methods may provide significant improvements in communication rates and quality of life. In this series of studies, we developed and evaluated methods for improving communication rates through optimization and prediction in communication interfaces. These interfaces enabled participants to select sounds (phonemes) instead of letters and were computationally optimized offline via a model of human movement in order for targets likely to be selected together to be in close proximity. Online prediction was implemented such that likely targets were dynamically enlarged. Computational simulations suggested that optimized phonemic interfaces could increase communication rates by up to 30.9% compared to random phonemic interfaces. Communication rates were empirically evaluated in 36 participants without motor impairment using an alternate computer access method to produce messages with phonemic interfaces over 12 sessions. Results suggested that optimization increased communication rates by 10.5–23.0% compared to a random phonemic interface. Prediction increased communication rates during training sessions, but was not a significant factor in communication rates during the final session. Empirical evaluations in individuals with motor impairment revealed that all participants strongly agreed that they would improve with practice, and four out of six participants strongly preferred the interface with prediction. Results of these studies suggest that optimized and predictive phonemic interfaces may provide increased communication rates for individuals with motor impairments affecting both oral communication and computer access. Methods for dynamically enlarging targets may also be applicable to other (non-phonemic) interfaces to increase communication rates. Further research is needed to fully translate these results into clinical practice.2020-10-24T00:00:00

Evaluating Camera Mouse as a computer access system for augmentative and alternative communication in cerebral palsy: a case study

PUPRPOSE: Individuals with disabilities, who do not have reliable motor control to manipulate a standard computer mouse, require alternate access methods for complete computer access and for communication as well. The Camera Mouse system visually tracks the movement of selected facial features using a camera to directly control the mouse pointer of a computer. Current research suggests that this system can successfully provide a means of computer access and communication for individuals with motor impairments. However, there are no existing data on the efficacy of the software’s communication output capabilities. The goal of this case study is to provide a comprehensive evaluation of Camera Mouse as a computer access method for Augmentative and Alternative Communication (AAC) for an individual with cerebral palsy, who prefers to use her unintelligible dysarthric speech to communicate her desires and thoughts despite having access to a traditional AAC system. METHOD: The current study compared the Camera Mouse system, the Tobii PCEye Mini (a popular commercially available eye tracking device) paired with speech generating technology, and natural speech using a variety of tasks in a single dysarthric speaker. Tasks consisted of two questionnaires designed to measure psychosocial impact and satisfaction with assistive technology, two sentence intelligibility tasks that were judged by 4 unfamiliar listeners, and two language samples designed to measure expressive language. Each task was completed three times—once for each communication modality in question: natural speech, Camera Mouse-to-speech system, and Tobii eye tracker-to- speech system. Participant responses were recorded and transcribed. RESULTS: Data were analyzed in terms of psychosocial effects, user satisfaction, communication efficiency (using intelligibility and rate), and various measures of expressive output ability, to determine which modality offered the highest communicative aptitude. Measures showed that when paired with an orthographic selection interface and speech-generating device, the Camera Mouse and Tobii eye tracker resulted in greatly increased intelligibility. However, natural speech was superior to assistive technology options in all other measures, including psychosocial impact, satisfaction, communication efficiency, and several expressive language components. Though results indicate that use of the Tobii eye tracker resulted in a slightly higher rate and intelligibility, the participant reported increased satisfaction and psychosocial impact when using the novel Camera Mouse access system. CONCLUSION: This study is the first to provide quantitative information regarding the efficiency, psychosocial impact, user satisfaction, and expressive language capabilities of Camera Mouse as a computer access system for AAC. This study shows promising results for Camera Mouse as a functional access system for individuals with disabilities and for future AAC applications as well.2018-08-28T00:00:00

The effects of augmentative and alternative communication cursor click modality on language complexity and user perceptions

PURPOSE: Surface electromyography (sEMG) provides an alternative method for individuals with severe motor impairments to use the voluntary contractions of sparred musculature as inputs into an alternative and augmentative communication (AAC) device. Current research suggests that individuals with typical motor control prefer a sEMG-based click mechanism over a dwell-based click mechanism to operate an on-screen cursor. However, there is no existing data on the effects of cursor click modality on language production in AAC users with motor impairments. The goal of this study was to evaluate the communicative abilities of individuals with neuromuscular disorders when using an AAC device with two different cursor click modalities. METHOD: Twelve individuals with neuromuscular disorders produced synthetic language samples via an on-screen keyboard using an sEMG/accelerometer system with two different click modalities: dwell-based clicking and sEMG-based clicking. A third language sample via natural speech was also recorded. Language sample analysis was used to evaluate language complexity at syntactic, semantic, and ideational levels. To analyze syntactic complexity, language samples were examined for clausal density, conjunction usage, phrase expansions (noun phrase, verb phrase, and prepositional phrase), and mean length of utterance. Semantic complexity was analyzed using measures of moving-average type token ratio, abstract noun usage, metacognitive verb usage, and usage of morphologically complex words. Ideational complexity was analyzed in terms of the extent to which the responses conveyed the participant’s ideas. A questionnaire was used to measure the participants’ perceptions of usefulness for each modality. RESULTS: Mean length of utterance was shorter in the dwell-based click modality than in the sEMG-based click and natural speech modalities. In the sEMG-based click modality the majority of sentences were complex sentences, whereas simple sentences made up the majority in the dwell-based click modality. Morphologically complex word usage was used more frequently in the natural speech modality than in the sEMG-based click modality and used most frequently in the dwell-based click modality. There were no modality-specific trends for ideational complexity. Measures from the questionnaire showed that participants ranked natural speech as being more useful than either of the cursor-click modalities, but all three modalities were rated as at least somewhat useful (5 out of 7 on a rating scale of usefulness). CONCLUSION: This study is the first to evaluate the effects of cursor-click modality on the communicative abilities of individuals with neuromuscular disorders. Despite differences in language complexity on some measures, participants were able to use all three modalities to accurately respond to the language prompt with similar ideational scores. These results support both sEMG and dwell as alternative access methods for controlling a cursor-click system for individuals with neuromuscular disorders in future AAC applications