An application of steady state visual evoked potential brain-computer interface as an augmentative alternative communication system for individuals with severe motor impairments

Thesis (M.S.)--Boston UniversityPURPOSE: Tbis study will look at the feasibility of Steady State Visually Evoked Potential (SSVEP) brain-computer interfaces (BCI) as possible augmentative and alternative communication (AAC) systems for individuals who are severely disabled such as those with Locked-in Syndrome (LIS). The study intended to test whether there is a difference in BCI performance between healthy and impaired individuals and why. Specifically, the study focused on the operational competency, such as ocular motor function, ofthe impaired individuals as it relates to performance. Further, the study also attempted to explore the contributions of environmental distracts to performance. The results oftbis investigation will provide insights valuable for future BCI-AAC development and the potential for their acceptance by the AAC and LIS communities. METHODS: The study consisted of 12 healthy adults and 5 severely disabled adults presenting with 4 different neurological disorders. Tbis study consisted to two parts. The first part was an assessment ofthe communicative abilities ofthe impaired subjects. The assessment was conducted through a video recorded interview, from which communication rates were calculated and behavioral observations of each impaired subject's communicative behaviors were made with a focus on ocular motor behavior. The second part involved testing of the SSVEP BCI. All subjects performed selection tasks from a choice of four directions in the UDLR task. For each trial, the subject was prompted to attend to a specific SSVEP stimulus. Each stimulus was selected at random to flash at one of four frequencies (12, 13, 14, or 15Hz) (Lorenz, 2012). After 4 seconds, the BCI predicted the attended cue direction (Up, Down, Left, Right). If the prediction was correct, a "thumbs-up" feedback signal was shown to the subject; a "thumbs-down" was shown for incorrect predictions. The UDLR data collected for each trial consisted of a table with two columns: one column recorded the ground truth, which was the target direction, and one column recorded the decoded, or classified direction. Two additional columns were added. One column indicated whether the subject had any ocular motor impairment with a 1 or 0. A binary logistic regression was completed to investigate the main effect of age, subject group, and ocular motor impairment with respect to BCI accuracy. Additionally, observations regarding the affect of environmental distractions were also made. [TRUNCATED

Analysis of consciousness for complete locked-in syndrome patients

This thesis presents methods for detecting consciousness in patients with complete locked-in syndrome (CLIS). CLIS patients are unable to speak and have lost all muscle movement. Externally, the internal brain activity of such patients cannot be easily perceived, but CLIS patients are considered to be still conscious and cognitively active. Detecting the current state of consciousness of CLIS patients is non-trivial, and it is difficult to ascertain whether CLIS patients are conscious or not. Thus, it is vital to develop alternative ways to re-establish communication with these patients during periods of awareness, and a possible platform is through brain–computer interface (BCI). Since consciousness is required to use BCI correctly, this study proposes a modus operandi to analyze not only in intracranial electrocorticography (ECoG) signals with greater signal-to-noise ratio (SNR) and higher signal amplitude, but also in non-invasive electroencephalography (EEG) signals. By applying three different time-domain analysis approaches sample entropy, permutation entropy, and Poincaré plot as feature extraction to prevent disease-related reductions of brainwave frequency bands in CLIS patients, and cross-validated to improve the probability of correctly detecting the conscious states of CLIS patients. Due to the lack a of 'ground truth' that could be used as teaching input to correct the outcomes, k-Means and DBSCAN these unsupervised learning methods were used to reveal the presence of different levels of consciousness for individual participation in the experiment first in locked-in state (LIS) patients with ALSFRS-R score of 0. The results of these different methods converge on the specific periods of consciousness of CLIS/LIS patients, coinciding with the period during which CLIS/LIS patients recorded communication with an experimenter. To determine methodological feasibility, the methods were also applied to patients with disorders of consciousness (DOC). The results indicate that the use of sample entropy might be helpful to detect awareness not only in CLIS/LIS patients but also in minimally conscious state (MCS)/unresponsive wakefulness syndrome (UWS) patients, and showed good resolution for both ECoG signals up to 24 hours a day and EEG signals focused on one or two hours at the time of the experiment. This thesis focus on consistent results across multiple channels to avoid compensatory effects of brain injury. Unlike most techniques designed to help clinicians diagnose and understand patients' long-term disease progression or distinguish between different disease types on the clinical scales of consciousness. The aim of this investigation is to develop a reliable brain-computer interface-based communication aid eventually to provide family members with a method for short-term communication with CLIS patients in daily life, and at the same time, this will keep patients' brains active to increase patients' willingness to live and improve their quality of life (QOL)

Functional Magnetic Resonance Imaging as an Assessment Tool in Critically Ill Patients

Little is known about whether residual cognitive function occurs in the earliest stages of brain injury. The overarching goal of the work presented in this dissertation was to elucidate the role of functional neuroimaging in assessing brain activity in critically ill patients. The overall objective was addressed in the following four empirical chapters: In Chapter 2, three versions of a hierarchically-designed auditory task were developed and their ability to detect various levels of auditory language processing was assessed in individual healthy participants. The same procedure was then applied in two acutely comatose patients. In Chapter 3, a hierarchical auditory task was employed in a heterogeneous cohort of acutely comatose patients. The results revealed that the level of auditory processing in coma may be predictive of subsequent functional recovery. In Chapter 4, two mental imagery paradigms were utilized to assess covert command-following in coma. The findings demonstrate, for the first time, preserved awareness in an acutely comatose patient. In Chapter 5, functional neuroimaging techniques were used for covert communication with two completely locked-in, critically ill patients. The results suggest that this methodology could be used as an augmentative communication tool to allow patients to be involved in their own medical decision-making. Taken together, the proceeding chapters of this work demonstrate that functional neuroimaging can detect preserved cognitive functions in some acutely comatose patients, which has both diagnostic and prognostic relevance. Moreover, these techniques may be extended even further to be used as a communication tool in critically ill patients

Conversational affective social robots for ageing and dementia support

Socially assistive robots (SAR) hold significant potential to assist older adults and people with dementia in human engagement and clinical contexts by supporting mental health and independence at home. While SAR research has recently experienced prolific growth, long-term trust, clinical translation and patient benefit remain immature. Affective human-robot interactions are unresolved and the deployment of robots with conversational abilities is fundamental for robustness and humanrobot engagement. In this paper, we review the state of the art within the past two decades, design trends, and current applications of conversational affective SAR for ageing and dementia support. A horizon scanning of AI voice technology for healthcare, including ubiquitous smart speakers, is further introduced to address current gaps inhibiting home use. We discuss the role of user-centred approaches in the design of voice systems, including the capacity to handle communication breakdowns for effective use by target populations. We summarise the state of development in interactions using speech and natural language processing, which forms a baseline for longitudinal health monitoring and cognitive assessment. Drawing from this foundation, we identify open challenges and propose future directions to advance conversational affective social robots for: 1) user engagement, 2) deployment in real-world settings, and 3) clinical translation

The role of simulation in developing and designing applications for 2-class motor imagery brain-computer interfaces

A Brain-Computer Interface (BCI) can be used by people with severe physical disabilities such as Locked-in Syndrome (LiS) as a channel of input to a computer. The time-consuming nature of setting up and using a BCI, together with individual variation in performance and limited access to end users makes it difficult to employ techniques such as rapid prototyping and user centred design (UCD) in the design and development of applications. This thesis proposes a design process which incorporates the use of simulation tools and techniques to improve the speed and quality of designing BCI applications for the target user group. Two different forms of simulation can be distinguished: offline simulation aims to make predictions about a user’s performance in a given application interface given measures of their baseline control characteristics, while online simulation abstracts properties of inter- action with a BCI system which can be shown to, or used by, a stakeholder in real time. Simulators that abstract properties of BCI control at different levels are useful for different purposes. Demonstrating the use of offline simulation, Chapter 3 investigates the use of finite state machines (FSMs) to predict the time to complete tasks given a particular menu hierarchy, and compares offline predictions of task performance with real data in a spelling task. Chapter 5 aims to explore the possibility of abstracting a user’s control characteristics from a typical calibration task to predict performance in a novel control paradigm. Online simulation encompasses a range of techniques from low-fidelity prototypes built using paper and cardboard, to computer simulation models that aim to emulate the feel of control of using a BCI without actually needing to put on the BCI cap. Chapter 4 details the develop- ment and evaluation of a high fidelity BCI simulator that models the control characteristics of a BCI based on the motor-imagery (MI) paradigm. The simulation tools and techniques can be used at different stages of the application design process to reduce the level of involvement of end users while at the same time striving to employ UCD principles. It is argued that prioritising the level of involvement of end users at different stages in the design process is an important strategy for design: end user input is paramount particularly at the initial user requirements stage where the goals that are important for the end user of the application can be ascertained. The interface and specific interaction techniques can then be iteratively developed through both real and simulated BCI with people who have no or less severe physical disabilities than the target end user group, and evaluations can be carried out with end users at the final stages of the process. Chapter 6 provides a case study of using the simulation tools and techniques in the development of a music player application. Although the tools discussed in the thesis specifically concern a 2-class Motor Imagery BCI which uses the electroencephalogram (EEG) to extract brain signals, the simulation principles can be expected to apply to a range of BCI systems

Toward an Imagined Speech-Based Brain Computer Interface Using EEG Signals

Individuals with physical disabilities face difficulties in communication. A number of neuromuscular impairments could limit people from using available communication aids, because such aids require some degree of muscle movement. This makes brain–computer interfaces (BCIs) a potentially promising alternative communication technology for these people. Electroencephalographic (EEG) signals are commonly used in BCI systems to capture non-invasively the neural representations of intended, internal and imagined activities that are not physically or verbally evident. Examples include motor and speech imagery activities. Since 2006, researchers have become increasingly interested in classifying different types of imagined speech from EEG signals. However, the field still has a limited understanding of several issues, including experiment design, stimulus type, training, calibration and the examined features. The main aim of the research in this thesis is to advance automatic recognition of imagined speech using EEG signals by addressing a variety of issues that have not been solved in previous studies. These include (1)improving the discrimination between imagined speech versus non-speech tasks, (2) examining temporal parameters to optimise the recognition of imagined words and (3) providing a new feature extraction framework for improving EEG-based imagined speech recognition by considering temporal information after reducing within-session temporal non-stationarities. For the discrimination of speech versus non-speech, EEG data was collected during the imagination of randomly presented and semantically varying words. The non-speech tasks involved attention to visual stimuli and resting. Time-domain and spatio-spectral features were examined in different time intervals. Above-chance-level classification accuracies were achieved for each word and for groups of words compared to the non-speech tasks. To classify imagined words, EEG data related to the imagination of five words was collected. In addition to words classification, the impacts of experimental parameters on classification accuracy were examined. The optimization of these parameters is important to improve the rate and speed of recognizing unspoken speech in on-line applications. These parameters included using different training sizes, classification algorithms, feature extraction in different time intervals and the use of imagination time length as classification feature. Our extensive results showed that Random Forest classifier with features extracted using Discrete Wavelet Transform from 4 seconds fixed time frame EEG yielded that highest average classification of 87.93% in classification of five imagined words. To minimise within class temporal variations, a novel feature extraction framework based on dynamic time warping (DTW) was developed. Using linear discriminant analysis as the classifier, the proposed framework yielded an average 72.02% accuracy in the classification of imagined speech versus silence and 52.5% accuracy in the classification of five words. These results significantly outperformed a baseline configuration of state-of-the art time-domain features

Use of Task-Relevant Spoken Word Stimuli in an Auditory Brain-Computer Interface

Auditory brain-computer interfaces (aBCI) may be an effective solution for communication in cases of severely locked-in, late stage ALS (Lou Gehrig’s disease) and upper spinal cord injury patients who are otherwise not candidates for implanted electrodes. Feasibility of auditory BCI has been shown for both healthy participants, (Hill et al., 2004), and impaired populations (Sellers and Donchin, 2006). (Hill et al., 2014) found similar BCI performance in healthy participants and those with locked-in syndrome in a paradigm comparing words to pure tone stimuli. Additional BCI research has explored variations to augment P300 signals for use in speller paradigms, including more meaningful auditory stimuli (Klobassa et al., 2009; Furdea et al., 2009; Simon et al., 2014). It has been recognized in these studies that end users would much prefer natural sounds over a repeated tone stimulus. All of these systems required an association of sound with target stimuli, typically enforced by a visual support matrix. These systems would not be usable by the target end users of an auditory BCI. At- tempts to remove the need for visual referencing by investigating a BCI system with serial presentation of spoken letter streams as stimuli (Hoehne and Tangermann, 2014) or spoken words (Ferracuti et al., 2013) has met with limited success but presents a potential high speed communication solutions. The present study highlights a method of using BCI task relevant spoken word stimuli to eliminate visually presented references. By utilizing spoken word stimuli, a BCI system could utilize a range of stimuli equivalent to the size of the users vocabulary and provide faster communication out- put than spelling systems. As a control, spoken word stimuli that have no task specific relevance are also tested. Stimuli audio-spatial cues have shown significant improvements in aBCI performance (Käthner et al., 2013; Schreuder et al., 2011). The present study specifically evaluates the potential improvements to BCI performance of semantic and audio-spatial relevance by eliciting auditory oddball P300 responses to task relevant directional stimuli (spoken words: ‘front’, ‘back’, ‘left’, ‘right’). Participants completed several trials of a motivational game with directionally relevant targets over two experimental sessions. Offline analysis of training data was accomplished to evaluate the impact of stimulus characteristics on BCI performance. Questionnaire results on workload, motivation and system usability accurately reflected participant’s BCI performance. A behavioral button press study was utilized to further investigate the influence of spatial cues used in the paradigm, but also highlighted differences in the semantic relevance of the stimuli. Behavioral results correlated with BCI performance. The results of this study indicate task relevant stimuli are a viable option for eliminating artificial and visual stimulus references. This study’s results highlight several considerations for future auditory BCI studies, including: classifier selection, hearing threshold importance, aid of behavioral correlates to BCI performance and use of spatially separated spoken word stimuli

A ciência da leitura e a produção acadêmica: caminhos trilhados

Linguistics focuses on the different phenomena of language. In macrolinguistics areas, there is Psycholinguistics. This subfield researches (de)coding processes of messages with verbal codes. Thus, one of its influential fields of activity is reading. Reading is one of the most complex information processing tasks. It begins with the graphemes decoding and it finishes with the text comprehension. Regarding the assessment of reading, there are several exams and large-scale tests, such as Pisa, Saeb (Aneb and Anresc/Prova Brasil), ENEM. Alarming statistics come with the indicators from these evaluative instruments. There are, among Brazilians, low levels of reading comprehension and marked functional illiteracy rate. Therefore, this study aimed to research what scientific communication has shared in terms of knowledge about reading. Specifically, the objectives were synthesize, considering the psycholinguistic approach of reading research, studies and research with the most recurrent theme in the reading field evidenced from the electronic communication, in order to investigate the dimensions and limitations of knowledge about this subject. For this, through WebQualis system, Qualis A1 and A2 scientific journals with electronic format and with focuses/scopes related to reading from the areas of (1) Language Arts/Linguistics, (2) Psychology and (3) Education were selected. With the selected journals and through Capes Journals Portal, all their volumes and issues from 2011 to 2015 were analyzed. With this, scientific articles related to reading were mapped. With the mapped articles abstracts, the recurrent themes in reading in the scientific production were observed. Finally, with the full articles that had the recurrent theme, the researches results were integrated, synthesizing and pondering about them. With a critical-reflexive assessment of the data, relevant information was found. First, on one hand, it was noted that the reading has achieved a stable and upward space through the electronic communication. On the other one, it was checked that the contributions of Psychology have a great influence in reading and comprehension research. Second, it was shown that the most frequent theme in electronic productions is comprehension. Finally, with the synthesis, it was found that, increasingly, comprehension topics related to reading neurobiological aspects were empirical and directly investigated. In addition, there are several studies that propose reading teaching methods as well as strategies for improving the comprehension, including the use of TICs. Moreover, it was found that many research results are limited. This is because the comprehension involves several components – cognitive processes and skills. Researches often focus attention on one or the other component of it only, and each research fixes a specific methodology design and that vary considerabably. Regarding the assessment of reading, many of the methodological apparatus tasks evaluate only the product of comprehension and not its process. In other words, built mental representations are evaluated and not how the encoding of this text occurred. Therefore, in short, both the researches advancement in the comprehension field and several limitations were observed.A Linguística atém-se aos mais diferentes fenômenos da língua(gem). Nos domínios macrolinguísticos, há a Psicolinguística. Essa subárea tem como foco de investigação os processos de (de)codificação de mensagens de códigos verbais. Assim, um de seus influentes campos de atuação é o de leitura. A leitura é uma das tarefas de processamento de informações mais complexas. Ela tem como princípio a decodificação grafêmica e como fim a compreensão textual. Em relação à avaliação da leitura, existem diversos testes e provas em larga escala, como o Pisa, o Saeb (Aneb e Anresc/Prova Brasil), o ENEM. Com os indicadores desses instrumentos avaliativos, vêm estatísticas alarmantes. Há, entre os brasileiros, baixos níveis de compreensão leitora e acentuado índice de analfabetismo funcional. Por conseguinte, este trabalho pretendeu investigar o que a comunicação científica tem compartilhado em termos de conhecimento sobre leitura. Especificamente, objetivou-se sintetizar, considerando a abordagem psicolinguística de investigação da leitura, estudos e pesquisas cuja temática evidenciada da comunicação eletrônica fosse a mais recorrente no campo da leitura, a fim de investigar dimensões e limitações do conhecimento a respeito dessa temática. Para isso, selecionaram-se, por meio do sistema WebQualis, periódicos científicos Qualis A1 e A2 em formato eletrônico e com focos/escopos relacionados à leitura, das áreas de (1) Letras/Linguística, (2) Psicologia e (3) Educação. Com os periódicos selecionados e por meio do Portal de Periódicos Capes, analisaram-se todos os seus volumes e números de 2011 a 2015, a fim de mapear artigos científicos com assunto em leitura. Com os resumos dos artigos mapeados, evidenciaram-se temáticas mais recorrentes na produção científica em leitura. Por fim, dos artigos completos cuja temática era a mais recorrente, integraram-se resultados das pesquisas, fazendo-se uma análise, com fins de síntese e reflexão. Da apreciação crítico-reflexiva dos dados, constataram-se relevantes informações. Em primeiro lugar, de um lado, observou-se que a leitura tem conquistado um estável e ascendente espaço em meio à comunicação eletrônica. De outro, demonstrou-se que contribuições da Psicologia têm forte influência na pesquisa de leitura e compreensão. Em segundo, evidenciou-se que a compreensão é a temática mais frequente nas produções eletrônicas. Por fim, com a síntese, constatou-se que, cada vez mais, se investiga empírica e diretamente facetas da compreensão em relação às bases neurobiológicas da leitura. Igualmente, há diversas pesquisas que propõem metodologias de ensino da leitura, bem como estratégias para a melhoria da compreensão, incluindo a utilização das TICs. Além disso, concluiu-se que muitos resultados de pesquisas são limitados. Isso porque a compreensão envolve diversos componentes – processos cognitivos e habilidades. E as pesquisas, muitas vezes, apenas focam a atenção em um ou em outro componente, além de definirem específicos e variados designs de metodologia. Em relação à avaliação da leitura, muitas das tarefas do aparato metodológico das pesquisas apenas avaliam o produto da compreensão e não o seu processo. Ou seja, avaliam-se representações mentais construídas e não como ocorreu a codificação desse texto na mente do leitor. Por conseguinte, em suma, tanto o avanço de pesquisas no campo de compreensão quanto, também, diversas limitações ficaram evidentes