Feature selection for automatic analysis of emotional response based on nonlinear speech modeling suitable for diagnosis of Alzheimer׳s disease

Alzheimer׳s disease (AD) is the most common type of dementia among the elderly. This work is part of a larger study that aims to identify novel technologies and biomarkers or features for the early detection of AD and its degree of severity. The diagnosis is made by analyzing several biomarkers and conducting a variety of tests (although only a post-mortem examination of the patients’ brain tissue is considered to provide definitive confirmation). Non-invasive intelligent diagnosis techniques would be a very valuable diagnostic aid. This paper concerns the Automatic Analysis of Emotional Response (AAER) in spontaneous speech based on classical and new emotional speech features: Emotional Temperature (ET) and fractal dimension (FD). This is a pre-clinical study aiming to validate tests and biomarkers for future diagnostic use. The method has the great advantage of being non-invasive, low cost, and without any side effects. The AAER shows very promising results for the definition of features useful in the early diagnosis of AD

Speech technologies for the audiovisual and multimedia interaction environments

361 p

Speech emotion recognition through statistical classification

O propósito desta dissertação é a discussão do reconhecimento de emoção na voz. Para este fim, criou-se uma base de dados validada de discurso emocional simulado Português, intitulada European Portuguese Emotional Discourse Database (EPEDD) e foram operados algoritmos de classificação estatística nessa base de dados. EPEDD é uma base de dados simulada, caracterizada por pequenos discursos (5 frases longas, 5 frases curtas e duas palavras), todos eles pronunciados por 8 atores—ambos os sexos igualmente representados—em 9 diferentes emoções (raiva, alegria, nojo, excitação, apatia, medo, surpresa, tristeza e neutro), baseadas no modelo de emoções de Lövheim. Concretizou-se uma avaliação de 40% da base de dados por avaliadores inexperientes, filtrando 60% dos pequenos discursos, com o intuito de criar uma base de dados validada. A base de dados completa contem 718 instâncias, enquanto que a base de dados validada contém 116 instâncias. A qualidade média de representação teatral, numa escala de a 5 foi avaliada como 2,3. A base de dados validada é composta por discurso emocional cujas emoções são reconhecidas com uma taxa média de 69,6%, por avaliadores inexperientes. A raiva tem a taxa de reconhecimento mais elevada com 79,7%, enquanto que o nojo, a emoção cuja taxa de reconhecimento é a mais baixa, consta com 40,5%. A extração de características e a classificação estatística foi realizada respetivamente através dos softwares Opensmile e Weka. Os algoritmos foram operados na base dados original e na base de dados avaliada, tendo sido obtidos os melhores resultados através de SVMs, respetivamente com 48,7% e 44,0%. A apatia obteve a taxa de reconhecimento mais elevada com 79,0%, enquanto que a excitação obteve a taxa de reconhecimento mais baixa com 32,9%.The purpose of this dissertation is to discuss speech emotion recognition. It was created a validated acted Portuguese emotional speech database, named European Portuguese Emotional Discourse Database (EPEDD), and statistical classification algorithms have been applied on it. EPEDD is an acted database, featuring 12 utterances (2 single-words, 5 short sentences and 5 long sentences) per actor and per emotion, 8 actors, both genders equally represented, and 9 emotions (anger, joy, disgust, excitement, fear, apathy, surprise, sadness and neutral), based on Lövheim’s emotion model. We had 40% of the database evaluated by unexperienced evaluators, enabling us to produce a validated one, filtering 60% of the evaluated utterances. The full database contains 718 instances, while the validated one contains 116 instances. The average acting quality of the original database was evaluated, in a scale from 1 to 5, as 2,3. The validated database is composed by emotional utterances that have their emotions recognized on average at a 69,6% rate, by unexperienced judges. Anger had the highest recognition rate at 79,7%, while disgust had the lowest recognition rate at 40,5%. Feature extraction and statistical classification algorithms were performed respectively applying Opensmile and Weka software. Statistical classification algorithms operated in the full database and in the validated one, best results being obtained by SVMs, respectively the emotion recognition rates being 48,7% and 44,0%. Apathy had the highest recognition rate: 79.0%, while excitement had the lowest emotion recognition rate: 32.9%

Towards a framework for socially interactive robots

250 p.En las últimas décadas, la investigación en el campo de la robótica social ha crecido considerablemente. El desarrollo de diferentes tipos de robots y sus roles dentro de la sociedad se están expandiendo poco a poco. Los robots dotados de habilidades sociales pretenden ser utilizados para diferentes aplicaciones; por ejemplo, como profesores interactivos y asistentes educativos, para apoyar el manejo de la diabetes en niños, para ayudar a personas mayores con necesidades especiales, como actores interactivos en el teatro o incluso como asistentes en hoteles y centros comerciales.El equipo de investigación RSAIT ha estado trabajando en varias áreas de la robótica, en particular,en arquitecturas de control, exploración y navegación de robots, aprendizaje automático y visión por computador. El trabajo presentado en este trabajo de investigación tiene como objetivo añadir una nueva capa al desarrollo anterior, la capa de interacción humano-robot que se centra en las capacidades sociales que un robot debe mostrar al interactuar con personas, como expresar y percibir emociones, mostrar un alto nivel de diálogo, aprender modelos de otros agentes, establecer y mantener relaciones sociales, usar medios naturales de comunicación (mirada, gestos, etc.),mostrar personalidad y carácter distintivos y aprender competencias sociales.En esta tesis doctoral, tratamos de aportar nuestro grano de arena a las preguntas básicas que surgen cuando pensamos en robots sociales: (1) ¿Cómo nos comunicamos (u operamos) los humanos con los robots sociales?; y (2) ¿Cómo actúan los robots sociales con nosotros? En esa línea, el trabajo se ha desarrollado en dos fases: en la primera, nos hemos centrado en explorar desde un punto de vista práctico varias formas que los humanos utilizan para comunicarse con los robots de una maneranatural. En la segunda además, hemos investigado cómo los robots sociales deben actuar con el usuario.Con respecto a la primera fase, hemos desarrollado tres interfaces de usuario naturales que pretenden hacer que la interacción con los robots sociales sea más natural. Para probar tales interfaces se han desarrollado dos aplicaciones de diferente uso: robots guía y un sistema de controlde robot humanoides con fines de entretenimiento. Trabajar en esas aplicaciones nos ha permitido dotar a nuestros robots con algunas habilidades básicas, como la navegación, la comunicación entre robots y el reconocimiento de voz y las capacidades de comprensión.Por otro lado, en la segunda fase nos hemos centrado en la identificación y el desarrollo de los módulos básicos de comportamiento que este tipo de robots necesitan para ser socialmente creíbles y confiables mientras actúan como agentes sociales. Se ha desarrollado una arquitectura(framework) para robots socialmente interactivos que permite a los robots expresar diferentes tipos de emociones y mostrar un lenguaje corporal natural similar al humano según la tarea a realizar y lascondiciones ambientales.La validación de los diferentes estados de desarrollo de nuestros robots sociales se ha realizado mediante representaciones públicas. La exposición de nuestros robots al público en esas actuaciones se ha convertido en una herramienta esencial para medir cualitativamente la aceptación social de los prototipos que estamos desarrollando. De la misma manera que los robots necesitan un cuerpo físico para interactuar con el entorno y convertirse en inteligentes, los robots sociales necesitan participar socialmente en tareas reales para las que han sido desarrollados, para así poder mejorar su sociabilida

Detection of Sarcasm and Nastiness: New Resources for Spanish Language

The main goal of this work is to provide the cognitive computing community with valuable resources to analyze and simulate the intentionality and/or emotions embedded in the language employed in social media. Specifically, it is focused on the Spanish language and online dialogues, leading to the creation of SOFOCO (Spanish Online Forums Corpus). It is the first Spanish corpus consisting of dialogic debates extracted from social media and it is annotated by means of crowdsourcing in order to carry out automatic analysis of subjective language forms, like sarcasm or nastiness. Furthermore, the annotators were also asked about the context need when taking a decision. In this way, the users’ intentions and their behavior inside social networks can be better understood and more accurate text analysis is possible. An analysis of the annotation results is carried out and the reliability of the annotations is also explored. Additionally, sarcasm and nastiness detection results (around 0.76 F-Measure in both cases) are also reported. The obtained results show the presented corpus as a valuable resource that might be used in very diverse future work.This study was partially funded by the Spanish Government (TIN2014-54288-C4-4-R and TIN2017-85854-C4-3-R) by the European Unions’s H2020 program under grant 769872 and by the National Science Foundation of USA (NSF CISE R1 #1202668

COMO SE PLANIFICAM, TRATAM, ANALISAM E INTERPRETAM NARRATIVAS, SEGUNDO A ABORDAGEM COMPREENSIVA-QUALITATIVA?

This paper derives from our PhD research in Sociology on socio-identitarian requalification processes (processos de requalificação socio-identitária, henceforth PRSI) of Portuguese women who migrated to the Basque Country (San Sebastian area). To this end, we co-composed 31 exemplary case accounts, following the question: “What are the logics of action and what identitarian strategies are adopted by women who, facing the social experience of disqualification, engage in their ‘socio-identitarian requalification’?†We focus and organize this paper on the exposition of the main procedures and specific forms of the qualitative analysis, as we applied it at the ‘the processes of socio-identitarian requalification (PRSI) - how we called, since 2008, the analytical-understanding model of social trajectories of requalification from poverty conditions. (cf. Toscano 2015, 2017, 2018). So, in this paper, we explain in depth how we organize the 4 methodological acts which compose this 'analytical-understanding model processes of socio-identitarian requalification' that we have been developing since 2008 in our analysis of trajectories for social change ('leaving' so called poverty conditions). Therefore, after a brief mention of the Tool-Problematics, which constitutes the base for our research, in point 1.1 (1st act: theoretical-conceptual-epistemological roots), we focus on specific procedures, such as: - point 2: justifying the co-composition of accounts: operationalizing principles, procedures and criteria for the selection of exemplary cases and composition of narratives (2nd act, steps 2-3); - point 3: planning and co-composing the biographical process (2nd act, Steps 4a-4b; 6 stages); - point 4 (3rd act), 1st level of theorization: writing down speech through transcription-translation (step 5; stages 7-8) and transposition-rearrangement (analysis units, operation mode, discursive levels, account axes, rules/kinds of annotation - steps 6-7, stages 9-12); - point 5 (4th act), 2nd level of theorization: interpretation and theorizing composition in emergence (steps 8-9, stages 13-14, 7 operations); and finally, point 6, brief conclusions..O presente texto decorre da nossa pesquisa de doutoramento em sociologia sobre processos de requalificação sócio-identitária (futuramente abreviado por prsi) de mulheres portuguesas migrantes no País Basco (zona de San Sebastian). Para tal, co-construímos 31 relatos de casos exemplares, seguindo a pergunta: «Que lógicas de ação e tipos de estratégias identitárias constroem as mulheres que, face à experiência social da desqualificação, se implicam na sua ‘requalificação sócio-identitária’?» (cf. Toscano [2015, 2017, 2018). De forma mais aprofundada, o que se explicita no presente texto são os 4 actos metodológicos deste ‘modelo analítico-compreensivo processos de requalificação sócio-identitária’ que vimos desenvolvendo, desde 2008, na análise de trajectórias de mudança social (‘saída’ de condições ditas de pobreza). Assim, após uma breve menção da Problemática-Utensílio onde se enraizou a pesquisa no ponto 1.1 (1º. acto: raízes teórico-conceptuais-epistemológicas), neste artigo focamo-nos em procedimentos metodológicos específicos, tais como: - ponto 2: fundamentar a co-construção de relatos: operacionalizar princípios, procedimentos e critérios para seleccionar casos exemplares e construir narrativas (2.º acto, passos 2-3); - ponto 3: planificar e co-construir o processo biográfico (2.º acto, Passos 4a-4b; 6 etapas); - ponto 4 (3.º acto), 1.º nível de teorização: escrever a oralidade pelas operações de transcrição-tradução (passo 5; etapas 7.ª-8.ª) e transposição-rearranjo (unidades de análise, modo de operar, níveis discursivos, eixos do relato, regras/tipos de anotações - passos 6-7, etapas 9.ª-12.ª); - ponto 5 (4.º acto), 2.º nível de teorização: interpretação e construção teorizante em emergência (passos 8-9, etapas 13.ª-14.ª, 7 operações); e, enfim, ponto 6: breves conclusões

Automatic Identification of Emotional Information in Spanish TV Debates and Human-Machine Interactions

Automatic emotion detection is a very attractive field of research that can help build more natural human–machine interaction systems. However, several issues arise when real scenarios are considered, such as the tendency toward neutrality, which makes it difficult to obtain balanced datasets, or the lack of standards for the annotation of emotional categories. Moreover, the intrinsic subjectivity of emotional information increases the difficulty of obtaining valuable data to train machine learning-based algorithms. In this work, two different real scenarios were tackled: human–human interactions in TV debates and human–machine interactions with a virtual agent. For comparison purposes, an analysis of the emotional information was conducted in both. Thus, a profiling of the speakers associated with each task was carried out. Furthermore, different classification experiments show that deep learning approaches can be useful for detecting speakers’ emotional information, mainly for arousal, valence, and dominance levels, reaching a 0.7F1-score.The research presented in this paper was conducted as part of the AMIC and EMPATHIC projects, which received funding from the Spanish Minister of Science under grants TIN2017-85854-C4-3-R and PDC2021-120846-C43 and from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 769872. The first author also received a PhD scholarship from the University of the Basque Country UPV/EHU, PIF17/310

Application of singing synthesis techniquest to bertsolaritza

237 p

Application of singing synthesis techniquest to bertsolaritza

237 p