A Novel Markovian Framework for Integrating Absolute and Relative Ordinal Emotion Information

There is growing interest in affective computing for the representation and prediction of emotions along ordinal scales. However, the term ordinal emotion label has been used to refer to both absolute notions such as low or high arousal, as well as relation notions such as arousal is higher at one instance compared to another. In this paper, we introduce the terminology absolute and relative ordinal labels to make this distinction clear and investigate both with a view to integrate them and exploit their complementary nature. We propose a Markovian framework referred to as Dynamic Ordinal Markov Model (DOMM) that makes use of both absolute and relative ordinal information, to improve speech based ordinal emotion prediction. Finally, the proposed framework is validated on two speech corpora commonly used in affective computing, the RECOLA and the IEMOCAP databases, across a range of system configurations. The results consistently indicate that integrating relative ordinal information improves absolute ordinal emotion prediction.Comment: This work has been submitted to IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Deciphering Entrepreneurial Pitches: A Multimodal Deep Learning Approach to Predict Probability of Investment

Acquiring early-stage investments for the purpose of developing a business is a fundamental aspect of the entrepreneurial process, which regularly entails pitching the business proposal to potential investors. Previous research suggests that business viability data and the perception of the entrepreneur play an important role in the investment decision-making process. This perception of the entrepreneur is shaped by verbal and non-verbal behavioral cues produced in investor-entrepreneur interactions. This study explores the impact of such cues on decisions that involve investing in a startup on the basis of a pitch. A multimodal approach is developed in which acoustic and linguistic features are extracted from recordings of entrepreneurial pitches to predict the likelihood of investment. The acoustic and linguistic modalities are represented using both hand-crafted and deep features. The capabilities of deep learning models are exploited to capture the temporal dynamics of the inputs. The findings show promising results for the prediction of the likelihood of investment using a multimodal architecture consisting of acoustic and linguistic features. Models based on deep features generally outperform hand-crafted representations. Experiments with an explainable model provide insights about the important features. The most predictive model is found to be a multimodal one that combines deep acoustic and linguistic features using an early fusion strategy and achieves an MAE of 13.91

A Personalized, Uncertainty-Aware, Trustworthy Algorithm for Effective Pain Assessment using Biosignals

Automatic pain assessment algorithms are used to improve pain assessment and assist subsequent pain treatment and management for patients without healthcare provider supervision. This thesis proposes a new pain assessment framework called "A Personalized, Uncertainty-Aware, Trustworthy Algorithm for Effective Pain Assessment using Biosignals." The framework takes into account the uncertainty of the data itself and the strong subjectivity of the pain experience, utilizing heart rate variability analysis to assess data uncertainty and test time adaptation to deal with distribution drift. It considers that pain data is imperfect, that there are data-label inconsistencies, and that the personalization of pain prediction algorithms is important. Our aim is to create complete frameworks for automated pain assessment that reduce the complexity of algorithms while predicting well. We collected experimental pain data and data from real pain patients, including post-surgical patients and women in labor. Through experiments and analyses, the framework outperforms state-of-the-art methods