A semi-supervised learning approach for acoustic-prosodic personality perception in under-resourced domains

Automatic personality analysis has gained attention in the last years as a fundamental dimension in human-To-human and human-To-machine interaction. However, it still suffers from limited number and size of speech corpora for specific domains, such as the assessment of children's personality. This paper investigates a semi-supervised training approach to tackle this scenario. We devise an experimental setup with age and language mismatch and two training sets: A small labeled training set from the Interspeech 2012 Personality Sub-challenge, containing French adult speech labeled with personality OCEAN traits, and a large unlabeled training set of Portuguese children's speech. As test set, a corpus of Portuguese children's speech labeled with OCEAN traits is used. Based on this setting, we investigate a weak supervision approach that iteratively refines an initial model trained with the labeled data-set using the unlabeled data-set. We also investigate knowledge-based features, which leverage expert knowledge in acoustic-prosodic cues and thus need no extra data. Results show that, despite the large mismatch imposed by language and age differences, it is possible to attain improvements with these techniques, pointing both to the benefits of using a weak supervision and expert-based acoustic-prosodic features across age and language

Affective analysis of customer service calls

This paper presents an affective and acoustic-prosodic analysis of a call-center corpus (700 phone calls with corresponding customer satisfaction levels). Our main goal is to understand how customers’ satisfaction correlates to the acoustic-prosodic and affective information (emotions and personality traits) of the interactions. A subset of 30 calls was manually annotated with emotions (frustrated vs.neutral) and personality traits (Big-Five model). Results on automatic satisfaction prediction from acoustic-prosodic features show a number of very informative linguistic knowledge-based features, especially pitch and energy ranges. The affective analysis also provides encouraging results, relating low/high satisfaction levels with the presence/absence of customer frustration. Concerning personality, customers tend to express signs of anxiety and nervousness, while agents are generally perceived as extroverted and open.info:eu-repo/semantics/publishedVersio

Uma abordagem de aprendizagem semissupervisionada para a classificação automática de personalidade baseada em pistas acústico-prosódicas

Automatic personality analysis has gained great attention in the last years as a fundamental dimension in human-machine interactions. However, the development of this technology in some domains, such as the classification of children’s personality, has been hindered by the limited number and size of the available speech corpora due to ethical concerns on collecting such corpora. To circumvent the lack of data, we have investigated the application of a semi-supervised training approach that makes use of heterogeneous (age and language mismatches) and partially non-labelled data sets. Namely, preliminary personality models trained using a small labelled data set with French speaking adults are iteratively refined using a larger unlabeled set of Portuguese children’s speech, whereas a labelled corpus of Portuguese children is used for evaluation. We also investigated speech representations based on prior linguistic knowledge on acoustic-prosodic clues for personality classification tasks and have analysed their relevance in the assessment of each personality trait. The results point out to the potential of applying semi-supervised learning approaches with heterogeneous data sets to overcome the lack of labelled data in under-resourced domains, and to the existence of acousticprosodic clues shared by speakers with different languages and ages, which allows for the classification of personality independently of these variables.info:eu-repo/semantics/acceptedVersio