Surface electromyographic control of a novel phonemic interface for speech synthesis

Many individuals with minimal movement capabilities use AAC to communicate. These individuals require both an interface with which to construct a message (e.g., a grid of letters) and an input modality with which to select targets. This study evaluated the interaction of two such systems: (a) an input modality using surface electromyography (sEMG) of spared facial musculature, and (b) an onscreen interface from which users select phonemic targets. These systems were evaluated in two experiments: (a) participants without motor impairments used the systems during a series of eight training sessions, and (b) one individual who uses AAC used the systems for two sessions. Both the phonemic interface and the electromyographic cursor show promise for future AAC applications.F31 DC014872 - NIDCD NIH HHS; R01 DC002852 - NIDCD NIH HHS; R01 DC007683 - NIDCD NIH HHS; T90 DA032484 - NIDA NIH HHShttps://www.ncbi.nlm.nih.gov/pubmed/?term=Surface+electromyographic+control+of+a+novel+phonemic+interface+for+speech+synthesishttps://www.ncbi.nlm.nih.gov/pubmed/?term=Surface+electromyographic+control+of+a+novel+phonemic+interface+for+speech+synthesisPublished versio

Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future

Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)

A Design Engineering Approach for Quantitatively Exploring Context-Aware Sentence Retrieval for Nonspeaking Individuals with Motor Disabilities

Nonspeaking individuals with motor disabilities typically have very low communication rates. This paper proposes a design engineering approach for quantitatively exploring contextaware sentence retrieval as a promising complementary input interface, working in tandem with a word-prediction keyboard. We motivate the need for complementary design engineering methodology in the design of augmentative and alternative communication and explain how such methods can be used to gain additional design insights. We then study the theoretical performance envelopes of a context-aware sentence retrieval system, identifying potential keystroke savings as a function of the parameters of the subsystems, such as the accuracy of the underlying auto-complete word prediction algorithm and the accuracy of sensed context information under varying assumptions. We find that context-aware sentence retrieval has the potential to provide users with considerable improvements in keystroke savings under reasonable parameter assumptions of the underlying subsystems. This highlights how complementary design engineering methods can reveal additional insights into design for augmentative and alternative communication

GENDER DIFFERENCES IN PERFORMANCE MEASURES OF INDIVIDUALS WHO USE AUGMENTATIVE AND ALTERNATIVE COMMUNICATION (AAC)

Language samples from 10 adults using an augmentative and alternative communication (AAC) system were analyzed for gender differences in performance measures. Participants (5 female; 5 male) were matched on device, access method, software, experience, age, and education. Each participant was asked to describe the "cookie theft" picture from the Boston Diagnostic Aphasia Examination (BDAE; Goodglass & Kaplan, 1983). The language samples were analyzed on the following two dependent variables: frequency of Semantic Compaction™ language representation use and average communication rate. A dependent samples t-test and the equivalent non-parametric matched-pair Wilcoxon tests were conduced on both variables. The effect size and the power were also calculated and used to support the following results. There was not a significant difference in the Semantic Compaction™ dependent variable, however there was a large effect size (d=1.11). A power analysis indicated a sample size consisting of 9 pairs (4 more males and 4 more females) would increase the power to 82%. Further research with an increased sample size of 9 pairs of participants may provide more support for the current finding in relation to the use of Semantic Compaction™.No significant difference was found between the average communication rates of the genders; however the presence of a female outlier was concluded to influence these results. A dependent samples t-test was conducted on the data excluding the pair containing the outlier. The results of the dependent samples t-test indicated a significant difference between the genders in the average communication rates. Overall, for both dependent variables, the majority of males were higher on the performance measures than their paired female participants. These observations support a need for future research addressing gender differences in individuals who use AAC. Clinical implications suggest that future research is needed to determine if intervention strategies need to accommodate for differences between genders in their ability to effectively use their device to communicate as fast as they are able. Caution needs to be used when interpreting and applying these results to this population due to the limitations (i.e., small sample size and lack of control of extraneous variable) of the current study

Predictive Composition of Pictogram Messages for Users with Autism.

Communication is a basic need for every person. However, there are many people who present disabilities that prevent communication through natural language. Augmentative and alternative communication (AAC) systems, including those based on pictograms, attempt to facilitate the communication for people with this kind of difficulties. In this paper we present PictoEditor, an augmentative and alternative communication application for the composition of pictogram messages for users with autism that incorporates prediction functionalities. Although such functionalities have been widely studied in text-based augmentative and alternative communication tools, they have not been applied to pictogram based ones. The results show that prediction based on frequency of use of specific pictograms improves the immediate availability of the desired pictograms, but the improvement with prediction based on sequencing of pseudo-syntactic types of pictogram is not as clear.pre-print4500 K

Reliability of brain-computer interface language sample transcription procedures

We tested the reliability of transcribing language samples of daily brain-computer interface (BCI) communication recorded as language activity monitoring (LAM) logfiles. This study determined interrater reliability and interjudgeagreement for transcription of communication of veterans with amyotrophic lateral sclerosis using a P300-based BCI as an augmentative and alternative communication (AAC) system. KeyLAM software recorded logfiles in a universal logfile format during use of BCI-controlled email and word processing applications. These logfiles were encrypted and sent to our laboratory for decryption, transcription, and analysis. The study reports reliability results on transcription of 345 daily logfile samples. The procedure was found to be accurate across transcribers/raters. Frequency of agreement ratios of 97.6% for total number of words and 93.5% for total utterances were found as measures of interrater reliability. Interjudge agreement was 100% for both measures. The results indicated that transcribing language samples using LAM data is highly reliable and the fidelity of the process can be maintained. LAM data supported the transcription of a large number of samples that could not have been completed using audio and video recordings of AAC speakers. This demonstrated efficiency of LAM tools to measure language performance benefits to BCI research and clinical communities

Intelligent Techniques to Accelerate Everyday Text Communication

People with some form of speech- or motor-impairments usually use a high-tech augmentative and alternative communication (AAC) device to communicate with other people in writing or in face-to-face conversations. Their text entry rate on these devices is slow due to their motor abilities. Making good letter or word predictions can help accelerate the communication of such users. In this dissertation, we investigated several approaches to accelerate input for AAC users. First, considering that an AAC user is participating in a face-to-face conversation, we investigated whether performing speech recognition on the speaking-side can improve next word predictions. We compared the accuracy of three plausible microphone deployment options and the accuracy of two commercial speech recognition engines. We found that despite recognition word error rates of 7-16%, our ensemble of n-gram and recurrent neural network language models made predictions nearly as good as when they used the reference transcripts. In a user study with 160 participants, we also found that increasing number of prediction slots in a keyboard interface does not necessarily correlate to improved performance. Second, typing every character in a text message may require an AAC user more time or effort than strictly necessary. Skipping spaces or other characters may be able to speed input and reduce an AAC user\u27s physical input effort. We designed a recognizer optimized for expanding noisy abbreviated input where users often omitted spaces and mid-word vowels. We showed using neural language models for selecting conversational-style training text and for rescoring the recognizer\u27s n-best sentences improved accuracy. We found accurate abbreviated input was possible even if a third of characters was omitted. In a study where users had to dwell for a second on each key, we found sentence abbreviated input was competitive with a conventional keyboard with word predictions. Finally, AAC keyboards rely on language modeling to auto-correct noisy typing and to offer word predictions. While today language models can be trained on huge amounts of text, pre-trained models may fail to capture the unique writing style and vocabulary of individual users. We demonstrated improved performance compared to a unigram cache by adapting to a user\u27s text via language models based on prediction by partial match (PPM) and recurrent neural networks. Our best model ensemble increased keystroke savings by 9.6%

Performance assessment in brain-computer interface-based augmentative and alternative communication

Abstract A large number of incommensurable metrics are currently used to report the performance of brain-computer interfaces (BCI) used for augmentative and alterative communication (AAC). The lack of standard metrics precludes the comparison of different BCI-based AAC systems, hindering rapid growth and development of this technology. This paper presents a review of the metrics that have been used to report performance of BCIs used for AAC from January 2005 to January 2012. We distinguish between Level 1 metrics used to report performance at the output of the BCI Control Module, which translates brain signals into logical control output, and Level 2 metrics at the Selection Enhancement Module, which translates logical control to semantic control. We recommend that: (1) the commensurate metrics Mutual Information or Information Transfer Rate (ITR) be used to report Level 1 BCI performance, as these metrics represent information throughput, which is of interest in BCIs for AAC; 2) the BCI-Utility metric be used to report Level 2 BCI performance, as it is capable of handling all current methods of improving BCI performance; (3) these metrics should be supplemented by information specific to each unique BCI configuration; and (4) studies involving Selection Enhancement Modules should report performance at both Level 1 and Level 2 in the BCI system. Following these recommendations will enable efficient comparison between both BCI Control and Selection Enhancement Modules, accelerating research and development of BCI-based AAC systems.http://deepblue.lib.umich.edu/bitstream/2027.42/115465/1/12938_2012_Article_658.pd