Advancing Electromyographic Continuous Speech Recognition: Signal Preprocessing and Modeling

Speech is the natural medium of human communication, but audible speech can be overheard by bystanders and excludes speech-disabled people. This work presents a speech recognizer based on surface electromyography, where electric potentials of the facial muscles are captured by surface electrodes, allowing speech to be processed nonacoustically. A system which was state-of-the-art at the beginning of this book is substantially improved in terms of accuracy, flexibility, and robustness

Advancing Electromyographic Continuous Speech Recognition: Signal Preprocessing and Modeling

Speech is the natural medium of human communication, but audible speech can be overheard by bystanders and excludes speech-disabled people. This work presents a speech recognizer based on surface electromyography, where electric potentials of the facial muscles are captured by surface electrodes, allowing speech to be processed nonacoustically. A system which was state-of-the-art at the beginning of this book is substantially improved in terms of accuracy, flexibility, and robustness

Applications of EMG in Clinical and Sports Medicine

This second of two volumes on EMG (Electromyography) covers a wide range of clinical applications, as a complement to the methods discussed in volume 1. Topics range from gait and vibration analysis, through posture and falls prevention, to biofeedback in the treatment of neurologic swallowing impairment. The volume includes sections on back care, sports and performance medicine, gynecology/urology and orofacial function. Authors describe the procedures for their experimental studies with detailed and clear illustrations and references to the literature. The limitations of SEMG measures and methods for careful analysis are discussed. This broad compilation of articles discussing the use of EMG in both clinical and research applications demonstrates the utility of the method as a tool in a wide variety of disciplines and clinical fields

Interfaces de fala silenciosa multimodais para português europeu com base na articulação

Doutoramento conjunto MAPi em InformáticaThe concept of silent speech, when applied to Human-Computer Interaction (HCI), describes a system which allows for speech communication in the absence of an acoustic signal. By analyzing data gathered during different parts of the human speech production process, Silent Speech Interfaces (SSI) allow users with speech impairments to communicate with a system. SSI can also be used in the presence of environmental noise, and in situations in which privacy, confidentiality, or non-disturbance are important. Nonetheless, despite recent advances, performance and usability of Silent Speech systems still have much room for improvement. A better performance of such systems would enable their application in relevant areas, such as Ambient Assisted Living. Therefore, it is necessary to extend our understanding of the capabilities and limitations of silent speech modalities and to enhance their joint exploration. Thus, in this thesis, we have established several goals: (1) SSI language expansion to support European Portuguese; (2) overcome identified limitations of current SSI techniques to detect EP nasality (3) develop a Multimodal HCI approach for SSI based on non-invasive modalities; and (4) explore more direct measures in the Multimodal SSI for EP acquired from more invasive/obtrusive modalities, to be used as ground truth in articulation processes, enhancing our comprehension of other modalities. In order to achieve these goals and to support our research in this area, we have created a multimodal SSI framework that fosters leveraging modalities and combining information, supporting research in multimodal SSI. The proposed framework goes beyond the data acquisition process itself, including methods for online and offline synchronization, multimodal data processing, feature extraction, feature selection, analysis, classification and prototyping. Examples of applicability are provided for each stage of the framework. These include articulatory studies for HCI, the development of a multimodal SSI based on less invasive modalities and the use of ground truth information coming from more invasive/obtrusive modalities to overcome the limitations of other modalities. In the work here presented, we also apply existing methods in the area of SSI to EP for the first time, noting that nasal sounds may cause an inferior performance in some modalities. In this context, we propose a non-invasive solution for the detection of nasality based on a single Surface Electromyography sensor, conceivable of being included in a multimodal SSI.O conceito de fala silenciosa, quando aplicado a interação humano-computador, permite a comunicação na ausência de um sinal acústico. Através da análise de dados, recolhidos no processo de produção de fala humana, uma interface de fala silenciosa (referida como SSI, do inglês Silent Speech Interface) permite a utilizadores com deficiências ao nível da fala comunicar com um sistema. As SSI podem também ser usadas na presença de ruído ambiente, e em situações em que privacidade, confidencialidade, ou não perturbar, é importante. Contudo, apesar da evolução verificada recentemente, o desempenho e usabilidade de sistemas de fala silenciosa tem ainda uma grande margem de progressão. O aumento de desempenho destes sistemas possibilitaria assim a sua aplicação a áreas como Ambientes Assistidos. É desta forma fundamental alargar o nosso conhecimento sobre as capacidades e limitações das modalidades utilizadas para fala silenciosa e fomentar a sua exploração conjunta. Assim, foram estabelecidos vários objetivos para esta tese: (1) Expansão das linguagens suportadas por SSI com o Português Europeu; (2) Superar as limitações de técnicas de SSI atuais na deteção de nasalidade; (3) Desenvolver uma abordagem SSI multimodal para interação humano-computador, com base em modalidades não invasivas; (4) Explorar o uso de medidas diretas e complementares, adquiridas através de modalidades mais invasivas/intrusivas em configurações multimodais, que fornecem informação exata da articulação e permitem aumentar a nosso entendimento de outras modalidades. Para atingir os objetivos supramencionados e suportar a investigação nesta área procedeu-se à criação de uma plataforma SSI multimodal que potencia os meios para a exploração conjunta de modalidades. A plataforma proposta vai muito para além da simples aquisição de dados, incluindo também métodos para sincronização de modalidades, processamento de dados multimodais, extração e seleção de características, análise, classificação e prototipagem. Exemplos de aplicação para cada fase da plataforma incluem: estudos articulatórios para interação humano-computador, desenvolvimento de uma SSI multimodal com base em modalidades não invasivas, e o uso de informação exata com origem em modalidades invasivas/intrusivas para superar limitações de outras modalidades. No trabalho apresentado aplica-se ainda, pela primeira vez, métodos retirados do estado da arte ao Português Europeu, verificando-se que sons nasais podem causar um desempenho inferior de um sistema de fala silenciosa. Neste contexto, é proposta uma solução para a deteção de vogais nasais baseada num único sensor de eletromiografia, passível de ser integrada numa interface de fala silenciosa multimodal

Closed-loop approaches for innovative neuroprostheses

The goal of this thesis is to study new ways to interact with the nervous system in case of damage or pathology. In particular, I focused my effort towards the development of innovative, closed-loop stimulation protocols in various scenarios: in vitro, ex vivo, in vivo

REDUCTION OF SKIN STRETCH INDUCED MOTION ARTIFACTS IN ELECTROCARDIOGRAM MONITORING USING ADAPTIVE FILTERING

Cardiovascular disease (CVD) is the leading cause of death in many regions worldwide, accounting for nearly one third of global deaths in 2001. Wearable electrocardiographic cardiovascular monitoring devices have contributed to reduce CVD mortality and cost by enabling the diagnosis of conditions with infrequent symptoms, the timely detection of critical signs that can be precursor to sudden cardiac death, and the long-term assessment/monitoring of symptoms, risk factors, and the effects of therapy. However, the effectiveness of ambulatory electrocardiography to improve the treatment of CVD can be significantly impaired by motion artifacts which can cause misdiagnoses, inappropriate treatment decisions, and trigger false alarms. Skin stretch associated with patient motion is a main source of motion artifact in current ECG monitors. A promising approach to reduce motion artifact is the use of adaptive filtering that utilizes a measured reference input correlated with the motion artifact to extract noise from the ECG signal. Previous attempts to apply adaptive filtering to electrocardiography have employed either electrode deformation or acceleration, body acceleration, or skin/electrode impedance as a reference input, and were not successful at reducing motion artifacts in a consistent and reproducible manner. This has been essentially attributed to the lack of correlation between the reference input selected and the induced noise. In this study, motion artifacts are adaptively filtered by using skin strain as the reference signal. Skin strain is measured non-invasively using a light emitting diode (LED) and an optical sensor incorporated in an ECG electrode. The optical strain sensor is calibrated on animal skin samples and finally in-vivo, in terms of sensitivity and measurement range. Skin stretch induced artifacts are extracted in-vivo using adaptive filters. The system and method are tested for different individuals and under various types of ambulatory conditions with the noise reduction performance quantified

Mental task classification using single-electrode brain computer interfaces

In the recent years, the field of Human-Computer Interaction (HCI) has greatly evolved to involve new and exciting interaction paradigms that allow users to interact with their environment and with technology in a more intuitive and ergonomic way. These interaction paradigms include voice, touch, virtual reality, and more recently, the brain. A brain-computer interface (BCI) is a an interface system allowing users to control devices without using the normal output pathways of peripherals, instead, by using neural activity generated in the brain. BCIs have a huge potential in a multitude of fields, all the way from providing users with severe motor disabilities with means for interaction with the external world, to entertainment, gaming, user state monitoring, and self-tracking systems. The potentials of BCI have sparked the interest of researchers, gaming markets and healthcare providers more and more in the recent years. The is due to the emergence of new commercial lightweight, low cost Electroencephalograph (EEG) equipment that made it possible to create more portable and usable BCI systems and expanded their fields of application. This Master thesis aims to explore the state of the art commercial BCI as well as the uses and challenges related to them. Commercially available EEG equipment, namely the Neurosky Brainband and Neurosky Mindset, will be investigated. User tests will be carried out to investigate whether such equipment with low accuracy and low cost can be used to recognize various mental activities. This would be performed by first collecting a dataset of brain signals during performing a set of mental tasks, which is one of the contributions of this project, followed by applying a set of signal processing algorithms, then exploring various classification techniques to classify the collected signals

In Vivo Imaging to Characterize Dynamic Tissue Responses after Neural Electrode Implantation

Implantable neural electrodes are promising technologies to restore motor, sensory, and cognitive function in many neural pathologies through brain-computer interfacing (BCI). Many BCI applications require electrode implantation within neural tissue to resolve and/or modulate the physiological activity of individual neurons via electrical recording and stimulation. This invasive implantation leads to acute and long-term deterioration of both the electrode device as well as the neurons surrounding the device. Ultimately, damage to the electrode and neural tissue results in electrode recording failure within the first years after implantation. Many strategies to improve BCI longevity focus on mitigating tissue damage through improving neuronal survival or reducing inflammatory activity around implants. Despite incremental improvements, electrode failure persists as an obstacle to wide-spread clinical deployment of BCIs. This can be partly attributed to an incomplete understanding of the biological correlates of recording performance. These correlates have largely been identified through post-mortem histological staining, which cannot capture dynamic changes in cellular physiology and morphology. In the following dissertation, we use longitudinal two-photon in vivo imaging to quantify how neurons, microglia, and meningeal immune cells are affected by an intracortical electrode during and after implantation in mouse cortex. We go beyond conventional histological techniques to show the time-course of neuronal injury and microglial recruitment after implantation. Neuronal injury occurs instantaneously, with prolonged, high calcium levels evident in neurons within 100 µm of implants. Microglial activation occurs within minutes of implantation and subsequent microglial encapsulation of electrodes can be modulated by bioactive surface coatings. Within the first day post-implant, there is high trafficking of peripheral immune cells through venules at the surface of the brain as well as along the electrode’s shank at the surface of the brain. Over the next month, calcium activity in neurons increases while the collagenous meningeal tissues at the surface of the brain thicken. We further show that meningeal thickening can have profound implications for devices implanted into non-human primates as well. In sum, these results define new potential therapeutic targets and windows that could improve the longevity of implantable neural electrodes