Emotion Recognition in Low-Resource Settings:An Evaluation of Automatic Feature Selection Methods

Research in automatic affect recognition has seldom addressed the issue of computational resource utilization. With the advent of ambient intelligence technology which employs a variety of low-power, resource-constrained devices, this issue is increasingly gaining interest. This is especially the case in the context of health and elderly care technologies, where interventions may rely on monitoring of emotional status to provide support or alert carers as appropriate. This paper focuses on emotion recognition from speech data, in settings where it is desirable to minimize memory and computational requirements. Reducing the number of features for inductive inference is a route towards this goal. In this study, we evaluate three different state-of-the-art feature selection methods: Infinite Latent Feature Selection (ILFS), ReliefF and Fisher (generalized Fisher score), and compare them to our recently proposed feature selection method named `Active Feature Selection' (AFS). The evaluation is performed on three emotion recognition data sets (EmoDB, SAVEE and EMOVO) using two standard acoustic paralinguistic feature sets (i.e. eGeMAPs and emobase). The results show that similar or better accuracy can be achieved using subsets of features substantially smaller than the entire feature set. A machine learning model trained on a smaller feature set will reduce the memory and computational resources of an emotion recognition system which can result in lowering the barriers for use of health monitoring technology

Macrostructural Changes of the Acoustic Radiation in Humans with Hearing Loss and Tinnitus Revealed with Fixel-Based Analysis

Age-related hearing loss is the most prevalent sensory impairment in the older adult population and is related to noise-induced damage or age-related deterioration of the peripheral auditory system. Hearing loss may affect the central auditory pathway in the brain, which is a continuation of the peripheral auditory system located in the ear. A debilitating symptom that frequently co-occurs with hearing loss is tinnitus. Strikingly, investigations into the impact of acquired hearing loss, with and without tinnitus, on the human central auditory pathway are sparse. This study used diffusion-weighted imaging (DWI) to investigate changes in the largest central auditory tract, the acoustic radiation, related to hearing loss and tinnitus. Participants with hearing loss, with and without tinnitus, and a control group were included. Both conventional diffusion tensor analysis and higher-order fixel-based analysis were applied. The fixel-based analysis was used as a novel framework providing insight into the axonal density and macrostructural morphologic changes of the acoustic radiation in hearing loss and tinnitus. The results show tinnitus-related atrophy of the left acoustic radiation near the medial geniculate body. This finding may reflect a decrease in myelination of the auditory pathway, instigated by more profound peripheral deafferentation or reflecting a preexisting marker of tinnitus vulnerability. Furthermore, age was negatively correlated with the axonal density in the bilateral acoustic radiation. This loss of fiber density with age may contribute to poorer speech understanding observed in older adults. SIGNIFICANCE STATEMENT Age-related hearing loss is the most prevalent sensory impairment in the older adult population. Older individuals are subject to the cumulative effects of aging and noise exposure on the auditory system. A debilitating symptom that frequently co-occurs with hearing loss is tinnitus: the perception of a phantom sound. In this large DWI-study, we provide evidence that in hearing loss, the additional presence of tinnitus is related to degradation of the acoustic radiation. Additionally, older age was related to axonal loss in the acoustic radiation. It appears that older adults have the aggravating circumstances of age, hearing loss, and tinnitus on central auditory processing, which may partly be because of the observed deterioration of the acoustic radiation with age

Improved Emotion Recognition Using Gaussian Mixture Model and Extreme Learning Machine in Speech and Glottal Signals

Recently, researchers have paid escalating attention to studying the emotional state of an individual from his/her speech signals as the speech signal is the fastest and the most natural method of communication between individuals. In this work, new feature enhancement using Gaussian mixture model (GMM) was proposed to enhance the discriminatory power of the features extracted from speech and glottal signals. Three different emotional speech databases were utilized to gauge the proposed methods. Extreme learning machine (ELM) and k-nearest neighbor (kNN) classifier were employed to classify the different types of emotions. Several experiments were conducted and results show that the proposed methods significantly improved the speech emotion recognition performance compared to research works published in the literature

Predictive cognition in dementia: the case of music

The clinical complexity and pathological diversity of neurodegenerative diseases impose immense challenges for diagnosis and the design of rational interventions. To address these challenges, there is a need to identify new paradigms and biomarkers that capture shared pathophysiological processes and can be applied across a range of diseases. One core paradigm of brain function is predictive coding: the processes by which the brain establishes predictions and uses them to minimise prediction errors represented as the difference between predictions and actual sensory inputs. The processes involved in processing unexpected events and responding appropriately are vulnerable in common dementias but difficult to characterise. In my PhD work, I have exploited key properties of music – its universality, ecological relevance and structural regularity – to model and assess predictive cognition in patients representing major syndromes of frontotemporal dementia – non-fluent variant PPA (nfvPPA), semantic-variant PPA (svPPA) and behavioural-variant FTD (bvFTD) - and Alzheimer’s disease relative to healthy older individuals. In my first experiment, I presented patients with well-known melodies containing no deviants or one of three types of deviant - acoustic (white-noise burst), syntactic (key-violating pitch change) or semantic (key-preserving pitch change). I assessed accuracy detecting melodic deviants and simultaneously-recorded pupillary responses to these deviants. I used voxel-based morphometry to define neuroanatomical substrates for the behavioural and autonomic processing of these different types of deviants, and identified a posterior temporo-parietal network for detection of basic acoustic deviants and a more anterior fronto-temporo-striatal network for detection of syntactic pitch deviants. In my second chapter, I investigated the ability of patients to track the statistical structure of the same musical stimuli, using a computational model of the information dynamics of music to calculate the information-content of deviants (unexpectedness) and entropy of melodies (uncertainty). I related these information-theoretic metrics to performance for detection of deviants and to ‘evoked’ and ‘integrative’ pupil reactivity to deviants and melodies respectively and found neuroanatomical correlates in bilateral dorsal and ventral striatum, hippocampus, superior temporal gyri, right temporal pole and left inferior frontal gyrus. Together, chapters 3 and 4 revealed new hypotheses about the way FTD and AD pathologies disrupt the integration of predictive errors with predictions: a retained ability of AD patients to detect deviants at all levels of the hierarchy with a preserved autonomic sensitivity to information-theoretic properties of musical stimuli; a generalized impairment of surprise detection and statistical tracking of musical information at both a cognitive and autonomic levels for svPPA patients underlying a diminished precision of predictions; the exact mirror profile of svPPA patients in nfvPPA patients with an abnormally high rate of false-alarms with up-regulated pupillary reactivity to deviants, interpreted as over-precise or inflexible predictions accompanied with normal cognitive and autonomic probabilistic tracking of information; an impaired behavioural and autonomic reactivity to unexpected events with a retained reactivity to environmental uncertainty in bvFTD patients. Chapters 5 and 6 assessed the status of reward prediction error processing and updating via actions in bvFTD. I created pleasant and aversive musical stimuli by manipulating chord progressions and used a classic reinforcement-learning paradigm which asked participants to choose the visual cue with the highest probability of obtaining a musical ‘reward’. bvFTD patients showed reduced sensitivity to the consequence of an action and lower learning rate in response to aversive stimuli compared to reward. These results correlated with neuroanatomical substrates in ventral and dorsal attention networks, dorsal striatum, parahippocampal gyrus and temporo-parietal junction. Deficits were governed by the level of environmental uncertainty with normal learning dynamics in a structured and binarized environment but exacerbated deficits in noisier environments. Impaired choice accuracy in noisy environments correlated with measures of ritualistic and compulsive behavioural changes and abnormally reduced learning dynamics correlated with behavioural changes related to empathy and theory-of-mind. Together, these experiments represent the most comprehensive attempt to date to define the way neurodegenerative pathologies disrupts the perceptual, behavioural and physiological encoding of unexpected events in predictive coding terms

An examination of the language construct in NIMH's research domain criteria:Time for reconceptualization!

The National Institute of Mental Health’s Research Domain Criteria (RDoC) Initiative “calls for the development of new ways of classifying psychopathology based on dimensions of observable behavior.” As aresult of this ambitious initiative, language has been identifi d as an independent construct in the RDoC matrix. In this article, we frame language within an evolutionary and neuro- psychological context and discuss some of the limitations to the current measurements of language. Findings from genomics and the neuroimaging of performance during language tasks are dis- cussed in relation to serious mental illness and within the context of caveats regarding measuring language. Indeed, the data collec- tion and analysis methods employed to assay language have been both aided and constrained by the available technologies, methodologies, and conceptual defi Consequently, differ- ent fields of language research show inconsistent defi s of language that have become increasingly broad over time. Individ- ually, they have also shown significant improvements in conceptual resolution, aswell as inexperimental and analytic techniques. More recently, language research has embraced collaborations across disciplines, notably neuroscience, cognitive science, and computa- tional linguistics and has ultimately re-defi classical ideas of language. As we move forward, the new models of language with their remarkably multifaceted constructs force a re-examination of the NIMH RDoC conceptualization of language and thus the neuroscience and genetics underlying this concept

Intelligent Advanced User Interfaces for Monitoring Mental Health Wellbeing

It has become pressing to develop objective and automatic measurements integrated in intelligent diagnostic tools for detecting and monitoring depressive states and enabling an increased precision of diagnoses and clinical decision-makings. The challenge is to exploit behavioral and physiological biomarkers and develop Artificial Intelligent (AI) models able to extract information from a complex combination of signals considered key symptoms. The proposed AI models should be able to help clinicians to rapidly formulate accurate diagnoses and suggest personalized intervention plans ranging from coaching activities (exploiting for example serious games), support networks (via chats, or social networks), and alerts to caregivers, doctors, and care control centers, reducing the considerable burden on national health care institutions in terms of medical, and social costs associated to depression cares

Computational Language Assessment in patients with speech, language, and communication impairments

Speech, language, and communication symptoms enable the early detection, diagnosis, treatment planning, and monitoring of neurocognitive disease progression. Nevertheless, traditional manual neurologic assessment, the speech and language evaluation standard, is time-consuming and resource-intensive for clinicians. We argue that Computational Language Assessment (C.L.A.) is an improvement over conventional manual neurological assessment. Using machine learning, natural language processing, and signal processing, C.L.A. provides a neuro-cognitive evaluation of speech, language, and communication in elderly and high-risk individuals for dementia. ii. facilitates the diagnosis, prognosis, and therapy efficacy in at-risk and language-impaired populations; and iii. allows easier extensibility to assess patients from a wide range of languages. Also, C.L.A. employs Artificial Intelligence models to inform theory on the relationship between language symptoms and their neural bases. It significantly advances our ability to optimize the prevention and treatment of elderly individuals with communication disorders, allowing them to age gracefully with social engagement.Comment: 36 pages, 2 figures, to be submite