Deep Emotion Recognition in Textual Conversations: A Survey

While Emotion Recognition in Conversations (ERC) has seen a tremendous advancement in the last few years, new applications and implementation scenarios present novel challenges and opportunities. These range from leveraging the conversational context, speaker and emotion dynamics modelling, to interpreting common sense expressions, informal language and sarcasm, addressing challenges of real time ERC, recognizing emotion causes, different taxonomies across datasets, multilingual ERC to interpretability. This survey starts by introducing ERC, elaborating on the challenges and opportunities pertaining to this task. It proceeds with a description of the emotion taxonomies and a variety of ERC benchmark datasets employing such taxonomies. This is followed by descriptions of the most prominent works in ERC with explanations of the Deep Learning architectures employed. Then, it provides advisable ERC practices towards better frameworks, elaborating on methods to deal with subjectivity in annotations and modelling and methods to deal with the typically unbalanced ERC datasets. Finally, it presents systematic review tables comparing several works regarding the methods used and their performance. The survey highlights the advantage of leveraging techniques to address unbalanced data, the exploration of mixed emotions and the benefits of incorporating annotation subjectivity in the learning phase

Survey of deep representation learning for speech emotion recognition

Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual eort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated \textit{deep representation learning} where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER

The Impact of Psycholinguistic Patterns in Discriminating between Fake News Spreaders and Fact Checkers

[EN] Fake news is a threat to society. A huge amount of fake news is posted every day on social networks which is read, believed and sometimes shared by a number of users. On the other hand, with the aim to raise awareness, some users share posts that debunk fake news by using information from fact-checking websites. In this paper, we are interested in exploring the role of various psycholinguistic characteristics in differentiating between users that tend to share fake news and users that tend to debunk them. Psycholinguistic characteristics represent the different linguistic information that can be used to profile users and can be extracted or inferred from users¿ posts. We present the CheckerOrSpreader model that uses a Convolution Neural Network (CNN) to differentiate between spreaders and checkers of fake news. The experimental results showed that CheckerOrSpreader is effective in classifying a user as a potential spreader or checker. Our analysis showed that checkers tend to use more positive language and a higher number of terms that show causality compared to spreaders who tend to use a higher amount of informal language, including slang and swear words.The works of Anastasia Giachanou and Daniel Oberski were funded by the Dutch Research Council (grant VI.Vidi.195.152). The work of Paolo Rosso was in the framework of the XAI-DisInfodemics project on eXplainable AI for disinformation and conspiracy detection during infodemics (PLEC2021-007681), funded by the Spanish Ministry of Science and Innovation, as well as IBERIFIER, the Iberian Digital Media Research and Fact-Checking Hub funded by the European Digital Media Observatory (2020-EU-IA0252).Giachanou, A.; Ghanem, BHH.; Rissola, EA.; Rosso, P.; Crestani, F.; Oberski, D. (2022). The Impact of Psycholinguistic Patterns in Discriminating between Fake News Spreaders and Fact Checkers. Data & Knowledge Engineering. 138:1-15. https://doi.org/10.1016/j.datak.2021.10196011513

An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era

Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. But the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions -- aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesised utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology which underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In the present overview, we outline ongoing trends and summarise state-of-the-art approaches in an attempt to provide a comprehensive overview of this exciting field.Comment: Submitted to the Proceedings of IEE

The relation among aging, dopamine-regulating genes, and neurocognition

When people are getting old, they often feel increasingly harder to concentrate, and become slower and more inflexible during tasks that involve focused attention, information maintenance in the face of distractions, and when fast switching according to changing goals is required. These cognitive functions are collectively referred to as working memory (WM). Both cross-sectional and longitudinal studies have reported WM impairment in aging. Moreover, aging is accompanied by alterations in brain structure, brain function, and dopaminergic neurotransmission. This thesis sought to link WM to brain structure, brain function, and dopamine (DA)-related genes in large samples of younger and older adults. The chief aims were to provide neural and genetic evidence to increase our understanding of the mechanisms of age-related deficits in WM. The DRD2/ANKK1-Taq1A polymorphism has been associated with DA D2 receptor densities in caudate. In study I, we investigated the effects of this polymorphism on grey-matter (GM) volume in striatum in older adults, and examined whether the genetic effect interacts with age. Results showed that the A allele of the DRD2/ANKK1-Taq1A polymorphism was associated with smaller GM volume in caudate and this effect was only observed in older adults (>72 years). The DRD2-C957T polymorphism has been linked to DA D2 receptor densities in both striatum and extrastriatal areas, such as in prefrontal cortex (PFC). In study II, we investigated the genetic effects of two DRD2 polymorphisms on WM functioning and examined how these effects may interact with adult age. In comparing younger and older adults, we found that the old had lower caudate activity in a highly demanding WM task. In addition, there were single and joint genetic effects of the two DRD2 polymorphisms on WM performance and frontostriatal brain activity. The genetic effects on brain function were observed in older, but not in younger adults, suggesting magnified genetic effects in aging. In study III, we related white-matter integrity with WM performance in a large sample across a wide age range. Results demonstrated that WM was associated with white-matter integrity in multiple tracts, indicating that WM functioning relies on global structural connections among multiple brain regions. Moreover, white-matter integrity could partially account for the age-related difference in WM. The COMT-Val158Met polymorphism has been associated with PFC DA levels. In this study, we found genetic effects of COMT on white-matter microstructure, suggesting a relation between dopaminergic function and white-matter integrity. In study IV, we investigated changes of white-matter integrity and WM performance using longitudinal data. We found that white-matter integrity declined across 10 years in the whole sample (25-80 years) and the decline was greater for older than for younger adults, reflecting a non-linear relation between age and white matter. More importantly, we found change – change associations of white-matter integrity and WM performance in several tracts including genu and body of corpus callosum and superior longitudinal fasciculus, suggesting that impaired WM performance in aging might reflect age-related decrease of white-matter integrity in these tracts. Collectively, these studies demonstrate age-related differences and changes in brain structure and brain function associated with impaired WM performance in aging. The findings support and extend previous work on the roles of DA in WM functioning and brain integrity in aging, and contribute to our understanding of neural and genetic correlates of WM, and how these are affected in aging

Artificial intelligence applications in marketing: the chatbot of the Department of Economics and Management "Marco Fanno”

openL'intelligenza artificiale (AI) offre numerose applicazioni nel marketing, ma allo stesso tempo ci sono diverse limitazioni da considerare nella sua adozione. Dopo la prima parte di analisi generale delle applicazioni e degli aspetti negativi dell'AI e dei chatbot, la tesi si concentra sul caso dell'implementazione di un chatbot da parte del Dipartimento di Economia e Management “Marco Fanno” dell'Università di Padova. La domanda di ricerca è volta a capire se il chatbot implementato dal Dipartimento sia stato efficace nell'alleggerire e supportare il lavoro dell'ufficio amministrativo e nel rispondere alle domande degli studenti. A tal fine, il documento analizza se il numero di email è diminuito dopo l'introduzione del chatbot. Inoltre è stato svolto un questionario per valutare l'esperienza che gli studenti del Dipartimento hanno avuto con il chatbot di ateneo. Il sondaggio ha anche chiesto agli studenti quali servizi vorrebbero che il chatbot aggiungesse a quelli attuali. Inoltre, è stata condotta un'analisi economica su benefici e costi per valutare se il chatbot genererà un risultato economico positivo. Questo studio consente di valutare l'impatto che un chatbot potrebbe avere nel campo dell'istruzione. In particolare, può fornire informazioni alle università sul fatto che un chatbot possa migliorare il coinvolgimento con gli studenti, liberare il personale da compiti ripetitivi e generare benefici economici netti nel lungo periodo. Il questionario stesso è stato condotto attraverso un sondaggio web su Google Forms e un sondaggio attraverso un chatbot. In questo modo ho anche analizzato quale dei due metodi sia il più efficace per condurre un'indagine. Alcune prove rivelano come i sondaggi condotti attraverso un chatbot possano portare a risposte più accurate da parte degli intervistati. Confrontando i risultati ottenuti della due modalità di sondaggio ho potuto verificare queste evidenze con un nuovo campione di partecipanti, gli studenti di Economia. I risultati della tesi non hanno mostrato prove chiare del fatto che il chatbot consentisse di ridurre il numero di e-mail. Ma si suggerisce un'indagine su un periodo più lungo. Successivamente i risultati hanno evidenziato un buon apprezzamento degli studenti per il chatbot e hanno suggerito l'introduzione di notifiche push che ricordano delle scadenze universitarie come le tasse. La stima dell'analisi costi-benefici prevedeva un risultato netto positivo su tre anni con un ROI del 29%. Inoltre, il sondaggio chatbot ha parzialmente confermato la tendenza ad ottenere risposte più accurate rispetto ad un classico sondaggio web.Artificial intelligence (AI) offers numerous applications in marketing, but at the same time, there are several limitations to consider in its adoption. After the first part about a general analysis of the applications and negative aspects of AI and chatbots, the thesis focuses on the case of the implementation of a chatbot by the Department of Economics and Management “Marco Fanno” of the University of Padua. The research question turns towards understanding whether the chatbot implemented by the Department was effective in easing and supporting the work of the administrative office and answering students questions. For this purpose, the paper analyses if the number of emails is decreased after the chatbot introduction. In addition, a questionnaire was carried out to evaluate the experience that the students of the Department have had with the university chatbot. The survey also asked students what services they would like the chatbot to add to their current ones. Moreover, an economic analysis on benefits and costs was conducted to estimate whether the chatbot will generate a positive outcome. This study allows evaluating the impact a chatbot could have in the education field. In particular, it can provide insight to universities on whether a chatbot could enhance the engagement with students, offload staff from repetitive tasks and generate net economic benefits in the long period. The questionnaire itself was conducted through a web survey on Google Forms and a chatbot survey. In this way, it could also be verified which of the two methods is the most effective to conduct a survey. Some evidence finds how chatbot surveys can lead to less satisfactory answers by respondents. Comparing the two survey results, I can verify these past findings with a different sample of participants, the students of Economics. The results did not show clear evidence of whether the chatbot allowed reducing the number of emails. But an investigation over a longer period is suggested. Then, findings highlighted a good appreciation of students for the chatbot and suggested the introduction of push notifications that remember university deadlines such as taxes. The estimation of the benefits-cost analysis forecasted a net positive outcome over three years with an ROI of 29%. Also, the chatbot survey partially confirmed the encouraging finding in reducing satisficing by respondents.