Neuromechanical Modelling of Articulatory Movements from Surface Electromyography and Speech Formants

Speech articulation is produced by the movements of muscles in the larynx, pharynx, mouth and face. Therefore speech shows acoustic features as formants which are directly related with neuromotor actions of these muscles. The first two formants are strongly related with jaw and tongue muscular activity. Speech can be used as a simple and ubiquitous signal, easy to record and process, either locally or on e-Health platforms. This fact may open a wide set of applications in the study of functional grading and monitoring neurodegenerative diseases. A relevant question, in this sense, is how far speech correlates and neuromotor actions are related. This preliminary study is intended to find answers to this question by using surface electromyographic recordings on the masseter and the acoustic kinematics related with the first formant. It is shown in the study that relevant correlations can be found among the surface electromyographic activity (dynamic muscle behavior) and the positions and first derivatives of the first formant (kinematic variables related to vertical velocity and acceleration of the joint jaw and tongue biomechanical system). As an application example, it is shown that the probability density function associated to these kinematic variables is more sensitive than classical features as Vowel Space Area (VSA) or Formant Centralization Ratio (FCR) in characterizing neuromotor degeneration in Parkinson's Disease.This work is being funded by Grants TEC2016-77791-C4-4-R from the Ministry of Economic Affairs and Competitiveness of Spain, Teka-Park 55 02 CENIE-0348_CIE_6_E POCTEP (InterReg Programme) and 16-30805A, SIX Research Center (CZ.1.05/2.1.00/03.0072), and LO1401 from the Czech Republic Government

Neuromechanical modelling of articulatory movements from surface electromyography and speech formants

Speech articulation is produced by the movements of muscles in the larynx, pharynx, mouth and face. Therefore speech shows acoustic features as formants which are directly related with neuromotor actions of these muscles. The first two formants are strongly related with jaw and tongue muscular activity. Speech can be used as a simple and ubiquitous signal, easy to record and process, either locally or on e-Health platforms. This fact may open a wide set of applications in the study of functional grading and monitoring neurodegenerative diseases. A relevant question, in this sense, is how far speech correlates and neuromotor actions are related. This preliminary study is intended to find answers to this question by using surface electromyographic recordings on the masseter and the acoustic kinematics related with the first formant. It is shown in the study that relevant correlations can be found among the surface electromyographic activity (dynamic muscle behavior) and the positions and first derivatives of the first formant (kinematic variables related to vertical velocity and acceleration of the joint jaw and tongue biomechanical system). As an application example, it is shown that the probability density function associated to these kinematic variables is more sensitive than classical features as Vowel Space Area (VSA) or Formant Centralization Ratio (FCR) in characterizing neuromotor degeneration in Parkinson’s Disease

Unveiling the impact of neuromotor disorders on speech: a structured approach combining biomechanical fundamentals and statistical machine learning

Speech has been shown to convey clinically useful information in the study of Neurodegenerative Disorders (NDs), such as Parkinson’s Disease (PD). Traditionally the use of speech as an exploratory tool in People with Parkinson’s (PwP) has focused on the estimation of acoustic characteristics and their study at face value, analysing the physio-acoustical markers and using them as features for the differentiation between Healthy Controls (HC) and PwP. The present work takes a step further, given the intricate interoperation between neuromotor activity, responsible for both planning and driving the system, and the production of the acoustic speech signal; by the study of speech, this relationship may be properly exploited and analysed, providing a non-invasive method for the diagnosis, analysis, and observation of NDs. This work aims to introduce a working model that is capable of linking both domains and serves as a projection tool to provide insights about a speaker’s neuromotor state. This is based on a review of the neurophysiological background of the structure and function of the nervous system, and a review of the main nervous system dysfunctions involved in PD and other related neuromotor disorders. The role of the respiratory, phonatory, and articulatory systems is reviewed in the production of voice and speech under normal and pathological circumstances. This setting might allow for speech to be considered a useful trait within the precision medicine framework, as it provides a personal biometric marker that is innate and easy to elicit, can be recorded remotely with inexpensive equipment, is non-invasive, cost-effective, and easy to process. The problem can be divided into two main categories: firstly, a binary detection task distinguishing between healthy controls and individuals with NDs based on the projection model and phonatory estimates; secondly, a progression and tracking task providing a set of quantitative indices that enable clinically interpretable scores. This study aims to define a set of features and models that help to characterise hypokinetic dysarthria (HD). These incorporate the neuroscientific knowhow semantically and quantitatively to be used in clinical decision support tools that provide mechanistic insight on the processes involved in speech production, incorporating into the algorithmic element neuromotor considerations that add to better interpretability, consequently leading to improved clinical decisions and diagnosis. An overview of the acoustic signal processing algorithms for use in speech articulation and phonation system inversion regarding neuromotor disorder assessment is provided. An algorithmic methodology for model inversion and exploration has been proposed for the functional characterization and system inversion of each subsystem involved under the neuro-biomechanical foundations exposed before. A description of the vocal fold biomechanics using the glottal source, and formant dynamics provides the base for specific mapping to articulation kinematics. The statistical methods used in performance evaluation are based on three-way comparisons and transversal and longitudinal assessment by classical hypothesis testing. Three related experimental studies are shown to empirically illustrate the potential of phonation and articulation analysis: the characterization of PD from glottal biomechanics based on the amplitude distributions of the glottal flow and on the vocal fold body stiffness in assessing the efficiency of transcranial magnetic stimulation, and the description of PD dysarthria through an articulation projection model. The results from the biomechanical analysis of phonation showed that the behaviour of glottal source amplitude distributions from PD and healthy controls using three-way comparisons and hierarchical clustering were essentially distinguishable from those from normative young participants with the best accuracy scores produced by SVM classifiers of 94.8% (males) and 92.2% (females). Nevertheless, PD participants were barely separable from age-matched controls, possibly pointing to confounding factors due to age. The outcomes from using vocal fold stiffness in assessing the efficiency of transcranial magnetic stimulation showed mixed results, as some PD participants reflected clear improvements in phonation stability after stimulation, whereas some others did not. Some cases of sham controls experienced also minor improvements of unknown origin, possibly expressing a placebo effect. The overall results on the efficiency of stimulation showed an accuracy global score of 67% over the 18 cases studied. The results from articulation projection modelling showed the possibility of formulating personalised models for PD and control participants to transform acoustic formant dynamics into articulation kinematics. This might open the possibility of characterising PD dysarthria based on speech audio records. The most remarkable findings of the study include the determination of the glottal source amplitude distribution behaviour of normative and PD participants; the impact of age effects in phonation as a confounding factor in neuromotor disorder characterization; the importance of ensuring that the classification of speech dysarthria is based on principles that can be explained and interpreted; the need of taking into account the effects of medication when framing new classification experiments; the potential of using EEG-band decomposition to analyse vocal fold stiffness correlates, as well as the possibility of using these descriptions in longitudinal monitoring of treatment efficiency; the feasibility of establishing a relationship between acoustic and kinematic variables by projection model inversion; and the potential of these descriptions for estimating neuromotor activities in midbrain related to phonation and articulation activity. The most important outcome to be brought forth from the thesis is that the methodology used throughout the project uses a bottom-up approach based on speech model inversion at the acoustical, biomechanical, and neuromotor levels allowing to estimate glottal signals, biomechanical correlates, and neuromotor activity from speech alone, establishing a common neuromechanical characterisation framework on its own

Procesos cognitivos y afectivos en adultos mayores medidos por medio del habla

Tesis por compendio de publicaciones[ES] El análisis automático del habla es una técnica que permite extraer información lingüística objetivamente de la señal de sonido emitida al hablar. En la producción de habla se interrelacionan numerosos sistemas para seleccionar y planificar el mensaje, dotarlo de una estructura adecuada, y enviar las señales neuromusculares a los órganos implicados en la producción del sonido. Estos procesos determinan las características de la onda sonora que se emite, de modo que el análisis del habla ha sido utilizado para detectar diversas alteraciones que afectan a estos sistemas lingüísticos como son la demencia tipo Alzheimer y el deterioro cognitivo leve. Esta tesis indaga sobre cómo se alteran los parámetros del habla en adultos mayores afectados por diferentes deterioros cognitivos y/o afectivos. Podemos distinguir en este trabajo dos fases: En la primera se llevan a cabo dos estudios con el objetivo de hallar factores cognitivos previos a los cambios en el habla durante el proceso de envejecimiento. En la segunda, se realizan cuatro estudios que tratan de obtener combinaciones de parámetros del habla susceptibles de ser utilizados como algoritmos en la detección de varios trastornos mediante la manipulación del proceso utilizado para elicitar habla. En el primer estudio se concluye que diversos parámetros del habla relacionados con la duración, el ritmo, las frecuencias y el análisis espectral, sufren cambios que se relacionan con el estado cognitivo general y que de hecho podrían ser sensibles a varias etapas de un deterioro. En el segundo, se examina si esos parámetros se explican mediante procesos cognitivos específicos, encontrando una relación con el acceso lingüístico a la memoria semántica, al léxico y la función ejecutiva. A continuación, los resultados demuestran que dentro de las personas con deterioro cognitivo leve podría haber perfiles de habla correspondientes a aquellos cuya causa subyacente es la Enfermedad de Alzheimer, y que podrían ser identificados a través de medidas del ritmo del habla. Se han utilizado dos tareas, una de lectura y otra de fluidez verbal, que permitan por medio del análisis del habla detectar con un nivel de éxito aceptable a personas con deterioro cognitivo leve y/o Alzheimer. Finalmente, se trató de extender el método del análisis del habla a la detección de depresión en mayores como un primer paso hacia el diagnóstico diferencial de depresión y demencia. [EN] Automatic speech analysis is a technique that allows linguistic information to be extracted objectively from the sound signal emitted during the act of speaking. Numerous systems are interrelated in speech production to select and plan the message, provide it with an appropriate structure, and send the neuromuscular signals to the organs involved in sound production. These processes determine the characteristics of the sound wave that is emitted, thus, speech analysis has been used to detect various disorders involving these systems, such as Alzheimer's dementia and mild cognitive impairment. This thesis investigates the alteration of speech parameters in older adults affected by cognitive and/or affective impairments. We can distinguish two phases in this work. In the first one, two studies are carried out with the aim of finding cognitive factors of speech changes during the aging process. In the second, four studies are carried out in an attempt to obtain combinations of speech parameters that can be used as algorithms in the detection of various disorders by manipulating the process used to elicit speech. The first study concludes that various speech parameters related to duration, rhythm, frequencies and spectral analysis undergo changes that are related to general cognitive state and may in fact be sensitive to various stages of impairment. In the second, we examine whether these parameters are explained by specific cognitive processes, finding a relationship with linguistic access to semantic memory, lexicon and executive function. Next, the results show that within people with mild cognitive impairment there could be speech profiles corresponding to those whose underlying cause is Alzheimer's disease and that could be identified through measures of rhythm. Two tasks, reading and verbal fluency, are proposed. Speech analysis on these tasks can be used with an acceptable level of success to detect people with mild cognitive impairment and/or Alzheimer's disease. Finally, an attempt was made to extend the method to the detection of depression in older adults as a first step towards the differential diagnosis of depression and dementia

Models and Analysis of Vocal Emissions for Biomedical Applications

The International Workshop on Models and Analysis of Vocal Emissions for Biomedical Applications (MAVEBA) came into being in 1999 from the particularly felt need of sharing know-how, objectives and results between areas that until then seemed quite distinct such as bioengineering, medicine and singing. MAVEBA deals with all aspects concerning the study of the human voice with applications ranging from the neonate to the adult and elderly. Over the years the initial issues have grown and spread also in other aspects of research such as occupational voice disorders, neurology, rehabilitation, image and video analysis. MAVEBA takes place every two years always in Firenze, Italy. This edition celebrates twenty years of uninterrupted and succesfully research in the field of voice analysis