Automatic Conversion of Emotions in Speech within a Speaker Independent Framework

Emotions in speech are a fundamental part of a natural dialog. In everyday life, vocal interaction with people often implies emotions as an intrinsic part of the conversation to a greater or lesser extent. Thus, the inclusion of emotions in human-machine dialog systems is crucial to achieve an acceptable degree of naturalness in the communication. This thesis focuses on automatic emotion conversion of speech, a technique whose aim is to transform an utterance produced in neutral style to a certain emotion state in a speaker independent context. Conversion of emotions represents a challenge in the sense that emotions a affect significantly all the parts of the human vocal production system, and in the conversion process all these factors must be taken into account carefully. The techniques used in the literature are based on voice conversion approaches, with minor modifications to create the sensation of emotion. In this thesis, the idea of voice conversion systems is used as well, but the usual regression process is divided in a two-step procedure that provides additional speaker normalization to remove the intrinsic speaker dependency of this kind of systems, using vocal tract length normalization as a pre-processing technique. In addition, a new method to convert the duration trend of the utterance and the intonation contour is proposed, taking into account the contextual information

Detecting error related negativity using EEG potentials generated during simulated brain computer interaction

2014 Summer.Includes bibliographical references.Error related negativity (ERN) is one of the components of the Event-Related Potential (ERP) observed during stimulus based tasks. In order to improve the performance of a brain computing interface (BCI) system, it is important to capture the ERN, classify the trials as correct or incorrect and feed this information back to the system. The objective of this study was to investigate techniques to detect presence of ERN in trials. In this thesis, features based on averaged ERP recordings were used to classify incorrect from correct actions. One feature selection technique coupled with four classification methods were used and compared in this work. Data were obtained from healthy subjects who performed an interaction experiment and the presence of ERN indicating incorrect responses was studied. Using suitable classifiers trained on data recorded earlier, the average recognition rate of correct and erroneous trials was reported and analyzed. The significance of selecting a subset of features to reduce the data dimensionality and to improve the classification performance was explored and discussed. We obtained success rates as high as 72% using a highly compact feature subset

EAWOP Small Group Meeting 2016 on Non-linear Dynamics in Work and Organizational Psychology: To Non-linear Modelling … and Beyond

EAWOP Small Group Meeting. 17th – 18th October 2016, University of Barcelona, Barcelona, SpainThe main objective of this meeting is to foster debate and knowledge sharing among scholars interested in going beyond the generalized linear modelling. Very often we find, as part of the limitation of empirical researches, statements describing that data could have been also analysed taking advantage of nonlinear methods. However, the application of non-linear models in our field is not as common as it could be.EAWOP / UB / PsicoSAO & Gobierno de España - Ministerio de Economía y Competitividad, project number PSI2013-44854-

A Likelihood-Ratio Based Forensic Voice Comparison in Standard Thai

This research uses a likelihood ratio (LR) framework to assess the discriminatory power of a range of acoustic parameters extracted from speech samples produced by male speakers of Standard Thai. The thesis aims to answer two main questions: 1) to what extent the tested linguistic-phonetic segments of Standard Thai perform in forensic voice comparison (FVC); and 2) how such linguistic-phonetic segments are profitably combined through logistic regression using the FoCal Toolkit (Brümmer, 2007). The segments focused on in this study are the four consonants /s, ʨh, n, m/ and the two diphthongs [ɔi, ai]. First of all, using the alveolar fricative /s/, two different sets of features were compared in terms of their performance in FVC. The first comprised the spectrum-based distributional features of four spectral moments, namely mean, variance, skew and kurtosis; the second consisted of the coefficients of the Discrete Cosine Transform (DCTs) applied to a spectrum. As DCTs were found to perform better, they were subsequently used to model the consonant spectrum of the remaining consonants. The consonant spectrum was extracted at the center point of the /s, ʨh, n, m/ consonants with a Hamming window of 31.25 msec. For the diphthongs [ɔi] - [nɔi L] and [ai] - [mai HL], the cubic polynomials fitted to the F2 and F1-F3 formants were tested separately. The quadratic polynomials fitted to the tonal F0 contours of [ɔi] - [nɔi L] and [ai] - [mai HL] were tested as well. Long-term F0 distribution (LTF0) was also trialed. The results show the promising discriminatory power of the Standard Thai acoustic features and segments tested in this thesis. The main findings are as follows. 1. The fricative /s/ performed better with the DCTs (Cllr = 0.70) than with the spectral moments (Cllr = 0.92). 2. The nasals /n, m/ (Cllr = 0.47) performed better than the affricate /tɕh/ (Cllr = 0.54) and the fricative /s/ (Cllr = 0.70) when their DCT coefficients were parameterized. 3. F1-F3 trajectories (Cllr = 0.42 and Cllr = 0.49) outperformed F2 trajectory (Cllr = 0.69 and Cllr = 0.67) for both diphthongs [ɔi] and [ai]. 4. F1-F3 trajectories of the diphthong [ɔi] (Cllr = 0.42) outperformed those of [ai] (Cllr = 0.49). 5. Tonal F0 (Cllr = 0.52) outperformed LTF0 (Cllr = 0.74). 6. Overall, better results were obtained when DCTs of /n/ - [na: HL] and /n/ - [nɔi L] were fused. (Cllr = 0.40 with the largest consistent-with-fact SSLog10LR = 2.53). In light of the findings, we can conclude that Standard Thai is generally amenable to FVC, especially when linguistic-phonetic segments are being combined; it is recommended that the latter procedure be followed when dealing with forensically realistic casework

Conic optimization with applications in finance and approximation theory

This dissertation explores conic optimization techniques with applications in the fields of finance and approximation theory. One of the most general types of conic optimization problems is the so-called generalized moment problem (GMP), which plays a fundamental part in this work. While being a powerful modeling framework, the GMP is notoriously difficult to solve. Semidefinite programming problems (SDPs) can be used to define approximation hierarchies for the GMP. The thesis includes an analysis of an interior point algorithm for SDPs, as well as a convergence analysis of an approximation hierarchy for the GMP defined over special sets. Additionally, the dissertation investigates the problem of pricing options that depend on multiple underlyings, which can be modeled as a GMP. Finally, the dissertation applies tools from conic optimization to address a classical question in approximation theory

Personality and Academic Performance in College

Despite mounting evidence for the role of personality in predicting college level academic performance, there are aspects of this association that are still unexplained. With a sample of U.S. undergraduates at a large Appalachian university, this study sought to further establish what is already known about the association between personality and grade point average, credits earned, and retention rates by testing for both linear and quadratic effects. Results showed linear positive effects of conscientiousness, negative linear effects of openness and nonlinear effects of neuroticism for GPA. However, personality traits were not associated with either retention or credits earned. These findings suggest that neuroticism may be predictive of GPA in ways previous research has not uncovered and suggests certain personality traits may be curvilinearly associated with GPA. Moreover, these findings should encourage administrators and researchers to understand how to foster certain personality traits in college students

Autonomic Nervous System Dynamics for Mood and Emotional-State Recognition: Wearable Systems, Modeling, and Advanced Biosignal Processing

This thesis aims at investigating how electrophysiological signals related to the autonomic nervous system (ANS) dynamics could be source of reliable and effective markers for mood state recognition and assessment of emotional responses. In-depth methodological and applicative studies of biosignals such as electrocardiogram, electrodermal response, and respiration activity along with information coming from the eyes (gaze points and pupil size variation) were performed. Supported by the current literature, I found that nonlinear signal processing techniques play a crucial role in understanding the underlying ANS physiology and provide important quantifiers of cardiovascular control dynamics with prognostic value in both healthy subjects and patients. Two main applicative scenarios were identified: the former includes a group of healthy subjects who was presented with sets of images gathered from the International Affective Picture System hav- ing five levels of arousal and five levels of valence, including both a neutral reference level. The latter was constituted by bipolar patients who were followed for a period of 90 days during which psychophysical evaluations were performed. In both datasets, standard signal processing techniques as well as nonlinear measures have been taken into account to automatically and accurately recognize the elicited levels of arousal and valence and mood states, respectively. A novel probabilistic approach based on the point-process theory was also successfully applied in order to model and characterize the instantaneous ANS nonlinear dynamics in both healthy subjects and bipolar patients. According to the reported evidences on ANS complex behavior, experimental results demonstrate that an accurate characterization of the elicited affective levels and mood states is viable only when non- linear information are retained. Moreover, I demonstrate that the instantaneous ANS assessment is effective in both healthy subjects and patients. Besides mathematics and signal processing, this thesis also contributes to pragmatic issues such as emotional and mood state mod- eling, elicitation, and noninvasive ANS monitoring. Throughout the dissertation, a critical review on the current state-of-the-art is reported leading to the description of dedicated experimental protocols, reliable mood models, and novel wearable systems able to perform ANS monitoring in a naturalistic environment

Phenotype Extraction: Estimation and Biometrical Genetic Analysis of Individual Dynamics

Within-person data can exhibit a virtually limitless variety of statistical patterns, but it can be difficult to distinguish meaningful features from statistical artifacts. Studies of complex traits have previously used genetic signals like twin-based heritability to distinguish between the two. This dissertation is a collection of studies applying state-space modeling to conceptualize and estimate novel phenotypic constructs for use in psychiatric research and further biometrical genetic analysis. The aims are to: (1) relate control theoretic concepts to health-related phenotypes; (2) design statistical models that formally define those phenotypes; (3) estimate individual phenotypic values from time series data; (4) consider hierarchical methods for biometrical genetic analysis of individual phenotypic variation

Fine Art Pattern Extraction and Recognition

This is a reprint of articles from the Special Issue published online in the open access journal Journal of Imaging (ISSN 2313-433X) (available at: https://www.mdpi.com/journal/jimaging/special issues/faper2020)

On the Recognition of Emotion from Physiological Data

This work encompasses several objectives, but is primarily concerned with an experiment where 33 participants were shown 32 slides in order to create ‗weakly induced emotions‘. Recordings of the participants‘ physiological state were taken as well as a self report of their emotional state. We then used an assortment of classifiers to predict emotional state from the recorded physiological signals, a process known as Physiological Pattern Recognition (PPR). We investigated techniques for recording, processing and extracting features from six different physiological signals: Electrocardiogram (ECG), Blood Volume Pulse (BVP), Galvanic Skin Response (GSR), Electromyography (EMG), for the corrugator muscle, skin temperature for the finger and respiratory rate. Improvements to the state of PPR emotion detection were made by allowing for 9 different weakly induced emotional states to be detected at nearly 65% accuracy. This is an improvement in the number of states readily detectable. The work presents many investigations into numerical feature extraction from physiological signals and has a chapter dedicated to collating and trialing facial electromyography techniques. There is also a hardware device we created to collect participant self reported emotional states which showed several improvements to experimental procedure