193 research outputs found

    Real-time generation and adaptation of social companion robot behaviors

    Get PDF
    Social robots will be part of our future homes. They will assist us in everyday tasks, entertain us, and provide helpful advice. However, the technology still faces challenges that must be overcome to equip the machine with social competencies and make it a socially intelligent and accepted housemate. An essential skill of every social robot is verbal and non-verbal communication. In contrast to voice assistants, smartphones, and smart home technology, which are already part of many people's lives today, social robots have an embodiment that raises expectations towards the machine. Their anthropomorphic or zoomorphic appearance suggests they can communicate naturally with speech, gestures, or facial expressions and understand corresponding human behaviors. In addition, robots also need to consider individual users' preferences: everybody is shaped by their culture, social norms, and life experiences, resulting in different expectations towards communication with a robot. However, robots do not have human intuition - they must be equipped with the corresponding algorithmic solutions to these problems. This thesis investigates the use of reinforcement learning to adapt the robot's verbal and non-verbal communication to the user's needs and preferences. Such non-functional adaptation of the robot's behaviors primarily aims to improve the user experience and the robot's perceived social intelligence. The literature has not yet provided a holistic view of the overall challenge: real-time adaptation requires control over the robot's multimodal behavior generation, an understanding of human feedback, and an algorithmic basis for machine learning. Thus, this thesis develops a conceptual framework for designing real-time non-functional social robot behavior adaptation with reinforcement learning. It provides a higher-level view from the system designer's perspective and guidance from the start to the end. It illustrates the process of modeling, simulating, and evaluating such adaptation processes. Specifically, it guides the integration of human feedback and social signals to equip the machine with social awareness. The conceptual framework is put into practice for several use cases, resulting in technical proofs of concept and research prototypes. They are evaluated in the lab and in in-situ studies. These approaches address typical activities in domestic environments, focussing on the robot's expression of personality, persona, politeness, and humor. Within this scope, the robot adapts its spoken utterances, prosody, and animations based on human explicit or implicit feedback.Soziale Roboter werden Teil unseres zukünftigen Zuhauses sein. Sie werden uns bei alltäglichen Aufgaben unterstützen, uns unterhalten und uns mit hilfreichen Ratschlägen versorgen. Noch gibt es allerdings technische Herausforderungen, die zunächst überwunden werden müssen, um die Maschine mit sozialen Kompetenzen auszustatten und zu einem sozial intelligenten und akzeptierten Mitbewohner zu machen. Eine wesentliche Fähigkeit eines jeden sozialen Roboters ist die verbale und nonverbale Kommunikation. Im Gegensatz zu Sprachassistenten, Smartphones und Smart-Home-Technologien, die bereits heute Teil des Lebens vieler Menschen sind, haben soziale Roboter eine Verkörperung, die Erwartungen an die Maschine weckt. Ihr anthropomorphes oder zoomorphes Aussehen legt nahe, dass sie in der Lage sind, auf natürliche Weise mit Sprache, Gestik oder Mimik zu kommunizieren, aber auch entsprechende menschliche Kommunikation zu verstehen. Darüber hinaus müssen Roboter auch die individuellen Vorlieben der Benutzer berücksichtigen. So ist jeder Mensch von seiner Kultur, sozialen Normen und eigenen Lebenserfahrungen geprägt, was zu unterschiedlichen Erwartungen an die Kommunikation mit einem Roboter führt. Roboter haben jedoch keine menschliche Intuition - sie müssen mit entsprechenden Algorithmen für diese Probleme ausgestattet werden. In dieser Arbeit wird der Einsatz von bestärkendem Lernen untersucht, um die verbale und nonverbale Kommunikation des Roboters an die Bedürfnisse und Vorlieben des Benutzers anzupassen. Eine solche nicht-funktionale Anpassung des Roboterverhaltens zielt in erster Linie darauf ab, das Benutzererlebnis und die wahrgenommene soziale Intelligenz des Roboters zu verbessern. Die Literatur bietet bisher keine ganzheitliche Sicht auf diese Herausforderung: Echtzeitanpassung erfordert die Kontrolle über die multimodale Verhaltenserzeugung des Roboters, ein Verständnis des menschlichen Feedbacks und eine algorithmische Basis für maschinelles Lernen. Daher wird in dieser Arbeit ein konzeptioneller Rahmen für die Gestaltung von nicht-funktionaler Anpassung der Kommunikation sozialer Roboter mit bestärkendem Lernen entwickelt. Er bietet eine übergeordnete Sichtweise aus der Perspektive des Systemdesigners und eine Anleitung vom Anfang bis zum Ende. Er veranschaulicht den Prozess der Modellierung, Simulation und Evaluierung solcher Anpassungsprozesse. Insbesondere wird auf die Integration von menschlichem Feedback und sozialen Signalen eingegangen, um die Maschine mit sozialem Bewusstsein auszustatten. Der konzeptionelle Rahmen wird für mehrere Anwendungsfälle in die Praxis umgesetzt, was zu technischen Konzeptnachweisen und Forschungsprototypen führt, die in Labor- und In-situ-Studien evaluiert werden. Diese Ansätze befassen sich mit typischen Aktivitäten in häuslichen Umgebungen, wobei der Schwerpunkt auf dem Ausdruck der Persönlichkeit, dem Persona, der Höflichkeit und dem Humor des Roboters liegt. In diesem Rahmen passt der Roboter seine Sprache, Prosodie, und Animationen auf Basis expliziten oder impliziten menschlichen Feedbacks an

    Context-Based Personalisation in Neural Machine Translation of Dialogue

    Get PDF
    Neural machine translation (NMT) has revolutionised automatic translation and has been instrumental in saving costs and improvements in productivity within the translation industry. However, contemporary NMT systems are still primarily designed to translate isolated sentences, disregarding crucial contextual information in the process. This lack of context awareness frequently leads to assumptions about the most likely interpretation of the source text, potentially propagating harmful biases learned from the training data, such as assuming that the average participant in a conversation is male. In the dialogue domain, where the meaning of an utterance may vary depending on what was said before, the environment, the individuals involved, their relationship, and more, translations produced by context-agnostic systems often fall short in capturing the nuances of specific characters or situations. This thesis expands the understanding of and explores the potential applications of contextual NMT with focus on personalisation. Our methods challenge the prevailing context-agnostic strategy in machine translation and seek to address the aforementioned issues. Our research suggests that by integrating existing information into the translation process we can enhance the quality of translation hypotheses. Additionally, we demonstrate that one type of information can be effectively leveraged to enable manipulation of another. Our experiments involve adapting machine translation systems to individual speakers and productions, focusing on combinations of their individual characteristics rather than relying on discrete labels. We also explore personalisation of language models based on context information expressed in this way: to personalise a model for a particular character, we use a combination of their traits. These personalised language models are then used in an evaluation scenario where the context specificity of machine translation hypotheses is expressed as the pointwise mutual information between the proposed text and its original context. Finally, our best personalised NMT system is thoroughly evaluated in a professional multi-modal setting of translating subtitles for TV series on two language pairs: English-to-German and English-to-French. Throughout the thesis, we report on experiments with various types of context in a setting of translation between English and a range of European languages. Our chosen domain is dialogue extracted from TV series and films, due to the availability of context-rich datasets, as well as the potential practical application of this research to the work of the industrial partner to this PhD, ZOO Digital. Our research tackles five primary challenges: (i) direct incorporation of extra-textual information into neural machine translation systems, (ii) zero-shot and few-shot control of this information, (iii) reference-free evaluation and analysis of contextual NMT, (iv) personalisation of language models (LMs) and NMT systems using rich sets of speaker and film metadata annotations, and (v) human evaluation of machine translation in a professional post-editing setting. By addressing these challenges, this thesis aims to enhance machine translation in dialogue by ensuring translations are better suited to the specific characters, addressees, and contextual factors involved. The research contributes to the advancement of NMT systems that can effectively account for the personalised nature of dialogue

    Makrokonstruktionen

    Get PDF
    The study investigates adverbial structures in spoken French that combine three or more discursive elements in a complex way. These structures are modeled in accordance with the terms of construction grammar as “macro constructions.” Drawing upon an extensive corpus, this study analyzes them with regard to their local emergence in interaction and their sedimentation

    The European Pilgrimage Routes for promoting sustainable and quality tourism in rural areas

    Get PDF
    The International Conference the European Pilgrimage Routes for promoting sustainable and quality tourism in rural areas took place December 4 to 6, 2014 in Firenze (Italy) and was organized by the Department of Agricultural, Food and Forestry Systems – University of Florence in collaboration with the Tuscany Region, the Department for Life Quality Studies and Department of Agricultural Sciences – University of Bologna, the Italian Association of Agricultural Engineering and the European Association of the Francigena Way. The Conference involving 150 experts from 18 countries and was divided into five areas of discussion: conservation and evolution of the landscape along the routes; life quality and social impact; tourism and local development; sustainability in the rural areas; tools and methods for building a tourist attraction

    Konstruktionssemantik

    Get PDF
    Construction semantics combines usage-based construction grammar and frame semantics, as set out in the FrameNet project. This volume models the semantic properties of grammatical constructions and their instances by utilizing frames. It demonstrates the usability of this approach for the constructicographical documentation of constructions as well as its empirical application by looking at three German reflexive constructions

    Promocijas darbs

    Get PDF
    Elektroniskā versija nesatur pielikumusPromocijas darba “Nacionālās identitātes veidošana un atspoguļojums Baltijas valstu prezidentu runās – korpusā balstīta kritiskā diskursa analīze” mērķis ir izpētīt, kā Baltijas valstu prezidentu runās atspoguļojas nacionālās identitātes diskursīvās konstrukcijas, proti, kādi valodas un diskursa makro- un mikrostruk-tūru elementi ir lietoti prezidentu retorikā, kādas ir to funkcijas un potenciālā ietekme uz runas mērķauditoriju. Izmantojot kvalitatīvo un kvantitatīvo metožu sinerģiju jeb korpusu pieeju un kritiskās diskursa analīzes vēsturisko pieeju, pētījumā veikta ne vien detalizēta runu satura, tematisko lauku, diskursīvo stratēģiju un lingvistisko paņēmienu analīze, bet arī analizēti korpusos balstītie statistiskie dati, kas palīdz detalizētāk izprast katra prezidenta lingvistisko pro-filu un lingvistisko paņēmienu izvēli. Veiktā komponentu analīze apliecina katra prezidenta multiplo identitāšu lingvistiskās iezīmes. Papildus veikta teorētisko avotu izpēte par pētījumā iekļautajiem aktuālajiem tematiem, kas veido prezi-dentu runu sociālpolitisko un vēsturisko kontekstu; pētījumā ir veiktas intervijas ar prezidentiem un prezidentu padomniekiem; savukārt, lai noskaidrotu pre-zidentu runu eksplicītos un implicītos mērķus un arī to potenciālo ietekmi uz klausītāju nacionālās identitātes veidošanu, darbā veiktas un apkopotas Latvijas iedzīvotāju viedokļu aptaujas.Atslēgvārdi: prezidentu runas, Baltijas valstis, nacionālā identitāte, kritiskās diskursa studijas, korpuslingvistikaThe goal of the dissertation ‘Construction and Representation of National Identity in the Speeches of the Presidents of the Baltic States: Corpus-Assisted Critical Discourse Analysis’ is to investigate the discursive construction of national identities in the presidential speeches of the Baltic States as well as their functions and potential impact on the target audience. By applying the synergy of qualitative and quantitative methods – corpus approach and the Discourse-Historical Approach to Critical Discourse Analysis, the study not only analyses the content of the speeches, including their thematic areas, discursive strategies, and linguistic means of realisation of these strategies but also provides statistical data and presents the analysis of the corpus data offering a detailed and objective insight into the individual linguistic profiles of the Presidents, their lexical choices, which point to the linguistic features of multiple identities constructed in the speeches. Additionally, the theoretical sources that pertain to understanding the socio-political and historical context influencing the content of the selected speeches have also been analysed, and interviews with the Presidents and their advisors, as well as opinion surveys with the target audience have been conducted to investigate the explicit and implicit goals of the speeches as well as their potential effect.Key words: presidential speeches, Baltic States, national identity, Critical Discourse Studies, Corpus Linguistic

    Proceedings of the VIIth GSCP International Conference

    Get PDF
    The 7th International Conference of the Gruppo di Studi sulla Comunicazione Parlata, dedicated to the memory of Claire Blanche-Benveniste, chose as its main theme Speech and Corpora. The wide international origin of the 235 authors from 21 countries and 95 institutions led to papers on many different languages. The 89 papers of this volume reflect the themes of the conference: spoken corpora compilation and annotation, with the technological connected fields; the relation between prosody and pragmatics; speech pathologies; and different papers on phonetics, speech and linguistic analysis, pragmatics and sociolinguistics. Many papers are also dedicated to speech and second language studies. The online publication with FUP allows direct access to sound and video linked to papers (when downloaded)

    Situated grounding and understanding of structured low-resource expert data

    Get PDF
    Conversational agents are becoming more widespread, varying from social to goaloriented to multi-modal dialogue systems. However, for systems with both visual and spatial requirements, such as situated robot planning, developing accurate goaloriented dialogue systems can be extremely challenging, especially in dynamic environments, such as underwater or first responders. Furthermore, training data-driven algorithms in these domains is challenging due to the esoteric nature of the interaction, which requires expert input. We derive solutions for creating a collaborative multi-modal conversational agent for setting high-level mission goals. We experiment with state-of-the-art deep learning models and techniques and create a new data-driven method (MAPERT) that is capable of processing language instructions by grounding the necessary elements using various types of input data (vision from a map, text and other metadata). The results show that, depending on the task, the accuracy of data-driven systems can vary dramatically depending on the type of metadata and the attention mechanisms that are used. Finally, we are dealing with low-resource expert data and this inspired the use of the Continual Learning and Human In The Loop methodology with encouraging results

    Mind and Matter

    Get PDF
    Do brains create material reality in thinking processes or is it the other way around, with things shaping the mind? Where is the location of meaning-making? How do neural networks become established by means of multimodal pattern replications, and how are they involved in conceptualization? How are resonance textures within cellular entities extended in the body and the mind by means of mirroring processes? In which ways do they correlate to consciousness and self-consciousness? Is it possible to explain out-of-awareness unconscious processes? What holds together the relationship between experiential reality, bodily processes like memory, reason, or imagination, and sign-systems and simulation structures like metaphor and metonymy visible in human language? This volume attempts to answer some of these questions

    Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020

    Get PDF
    On behalf of the Program Committee, a very warm welcome to the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020). This edition of the conference is held in Bologna and organised by the University of Bologna. The CLiC-it conference series is an initiative of the Italian Association for Computational Linguistics (AILC) which, after six years of activity, has clearly established itself as the premier national forum for research and development in the fields of Computational Linguistics and Natural Language Processing, where leading researchers and practitioners from academia and industry meet to share their research results, experiences, and challenges
    corecore