1,203 research outputs found
A Survey of Personality, Persona, and Profile in Conversational Agents and Chatbots
We present a review of personality in neural conversational agents (CAs),
also called chatbots. First, we define Personality, Persona, and Profile. We
explain all personality schemes which have been used in CAs, and list models
under the scheme(s) which they use. Second we describe 21 datasets which have
been developed in recent CA personality research. Third, we define the methods
used to embody personality in a CA, and review recent models using them.
Fourth, we survey some relevant reviews on CAs, personality, and related
topics. Finally, we draw conclusions and identify some research challenges for
this important emerging field.Comment: 25 pages, 6 tables, 207 reference
Chatbots with Personality Using Deep Learning
Natural Language Processing (NLP) requires the computational modelling of the complex relationships of the syntax and semantics of a language. While traditional machine learning methods are used to solve NLP problems, they cannot imitate the human ability for language comprehension. With the growth in deep learning, these complexities within NLP are easier to model, and be used to build many computer applications. A particular example of this is a chatbot, where a human user has a conversation with a computer program, that generates responses based on the user’s input. In this project, we study the methods used in building chatbots, what they lack and what can be improved
A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data
Endowing dialogue systems with personas is essential to deliver more
human-like conversations. However, this problem is still far from well explored
due to the difficulties of both embodying personalities in natural languages
and the persona sparsity issue observed in most dialogue corpora. This paper
proposes a pre-training based personalized dialogue model that can generate
coherent responses using persona-sparse dialogue data. In this method, a
pre-trained language model is used to initialize an encoder and decoder, and
personal attribute embeddings are devised to model richer dialogue contexts by
encoding speakers' personas together with dialogue histories. Further, to
incorporate the target persona in the decoding process and to balance its
contribution, an attention routing structure is devised in the decoder to merge
features extracted from the target persona and dialogue contexts using
dynamically predicted weights. Our model can utilize persona-sparse dialogues
in a unified manner during the training process, and can also control the
amount of persona-related features to exhibit during the inference process.
Both automatic and manual evaluation demonstrates that the proposed model
outperforms state-of-the-art methods for generating more coherent and persona
consistent responses with persona-sparse data.Comment: Long paper accepted at AAAI 202
Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning
Learning the underlying patterns in data goes beyond instance-based
generalization to external knowledge represented in structured graphs or
networks. Deep learning that primarily constitutes neural computing stream in
AI has shown significant advances in probabilistically learning latent patterns
using a multi-layered network of computational nodes (i.e., neurons/hidden
units). Structured knowledge that underlies symbolic computing approaches and
often supports reasoning, has also seen significant growth in recent years, in
the form of broad-based (e.g., DBPedia, Yago) and domain, industry or
application specific knowledge graphs. A common substrate with careful
integration of the two will raise opportunities to develop neuro-symbolic
learning approaches for AI, where conceptual and probabilistic representations
are combined. As the incorporation of external knowledge will aid in
supervising the learning of features for the model, deep infusion of
representational knowledge from knowledge graphs within hidden layers will
further enhance the learning process. Although much work remains, we believe
that knowledge graphs will play an increasing role in developing hybrid
neuro-symbolic intelligent systems (bottom-up deep learning with top-down
symbolic computing) as well as in building explainable AI systems for which
knowledge graphs will provide scaffolding for punctuating neural computing. In
this position paper, we describe our motivation for such a neuro-symbolic
approach and framework that combines knowledge graph and neural networks
Social Interactions in Immersive Virtual Environments: People, Agents, and Avatars
Immersive virtual environments (IVEs) have received increased popularity with applications in many fields. IVEs aim to approximate real environments, and to make users react similarly to how they would in everyday life. An important use case is the users-virtual characters (VCs) interaction. We interact with other people every day, hence we expect others to appropriately act and behave, verbally and non-verbally (i.e., pitch, proximity, gaze, turn-taking). These expectations also apply to interactions with VCs in IVEs, and this thesis tackles some of these aspects.
We present three projects that inform the area of social interactions with a VC in IVEs, focusing on non-verbal behaviours. In our first study on interactions between people, we collaborated with the Social Neuroscience group at the Institute of Cognitive Neuroscience from UCL on a dyad multi-modal interaction. This aims to understand the conversation dynamics, focusing on gaze and turn-taking. The results show that people have a higher frequency of gaze change (from averted to direct and vice versa) when they are being looked at compared to when they are not. When they are not being looked at, they are also directing their gaze to their partners more compared to when they are being looked at. Another contribution of this work is the automated method of annotating speech and gaze data.
Next, we consider agents’ higher-level non-verbal behaviours, covering social attitudes. We present a pipeline to collect data and train a machine learning (ML) model that detects social attitudes in a user-VC interaction. Here we collaborated with two game studios: Dream Reality Interaction and Maze Theory. We present a case study for the ML pipeline on social engagement recognition for the Peaky Blinders narrative VR game from Maze Theory studio. We use a reinforcement learning algorithm with imitation learning rewards and a temporal memory element. The results show that the model trained with raw data does not generalise and performs worse (60% accuracy) than the one trained with socially meaningful data (83% accuracy).
In IVEs, people embody avatars and their appearance can impact social interactions. In collaboration with Microsoft Research, we report a longitudinal study in mixed-reality on avatar appearance in real-work meetings between co-workers comparing personalised full-body realistic and cartoon avatars. The results imply that when participants use realistic avatars first, they may have higher expectations and they perceive their colleagues’ emotional states with less accuracy. Participants may also become more accustomed to cartoon avatars as time passes and the overall use of avatars may lead to less accurately perceiving negative emotions.
The work presented here contributes towards the field of detecting and generating nonverbal cues for VCs in IVEs. These are also important building blocks for creating autonomous agents for IVEs. Additionally, this work contributes to the games and work industry fields through an immersive ML pipeline for detecting social attitudes and through insights into using different avatar styles over time in real-world meetings
CERN for AGI: A Theoretical Framework for Autonomous Simulation-Based Artificial Intelligence Testing and Alignment
This paper explores the potential of a multidisciplinary approach to testing
and aligning artificial general intelligence (AGI) and LLMs. Due to the rapid
development and wide application of LLMs, challenges such as ethical alignment,
controllability, and predictability of these models have become important
research topics. This study investigates an innovative simulation-based
multi-agent system within a virtual reality framework that replicates the
real-world environment. The framework is populated by automated 'digital
citizens,' simulating complex social structures and interactions to examine and
optimize AGI. Application of various theories from the fields of sociology,
social psychology, computer science, physics, biology, and economics
demonstrates the possibility of a more human-aligned and socially responsible
AGI. The purpose of such a digital environment is to provide a dynamic platform
where advanced AI agents can interact and make independent decisions, thereby
mimicking realistic scenarios. The actors in this digital city, operated by the
LLMs, serve as the primary agents, exhibiting high degrees of autonomy. While
this approach shows immense potential, there are notable challenges and
limitations, most significantly the unpredictable nature of real-world social
dynamics. This research endeavors to contribute to the development and
refinement of AGI, emphasizing the integration of social, ethical, and
theoretical dimensions for future research.Comment: 32 pages, 4 figures, 2 table
User-Centric Interactive AI for Distributed Diffusion Model-based AI-Generated Content
Distributed Artificial Intelligence-Generated Content (AIGC) has attracted
increasing attention. However, it faces two significant challenges: how to
maximize the subjective Quality of Experience (QoE) and how to enhance the
energy efficiency, which are particularly pronounced in widely adopted
Generative Diffusion Model (GDM)-based AIGC services for image generation. In
this paper, we propose a novel user-centric Interactive AI (IAI) approach for
service management, with a distributed GDM-based AIGC framework, prioritizing
efficient and collaborative GDM deployment. Specifically, we restructure the
GDM's inference process, i.e., the denoising chain, to enable users'
semantically similar prompts to share a portion of diffusion steps.
Furthermore, to maximize the users' subjective QoE, we propose an IAI approach,
i.e., Reinforcement Learning With Large Language Models Interaction (RLLI),
which utilizes Large Language Model (LLM)-empowered generative agents to
replicate users interaction, providing real-time and subjective QoE feedback
that reflects a spectrum of user personalities. Lastly, we present the
GDM-based Deep Deterministic Policy Gradient (G-DDPG) algorithm, adapted to the
proposed RLLI framework, for effective communication and computing resource
allocation while considering user subjective personalities and dynamic wireless
environments in decision-making. Simulation results show that G-DDPG can
increase the sum QoE by 15%, compared with the conventional DDPG algorithm
Recommended from our members
A transdisciplinary study of embodiment in HCI, AI and New Media.
The aim of this thesis is to report on a transdisciplinary approach, regarding the complexity of thinking about human embodiment in relation to machine embodiment. A practical dimension of this thesis is to elicit some principles for the design and evaluation of virtual embodiment. The transdisciplinary approach suggests, firstly, that a single discipline or reality is, on its own, not sufficient to explain the complexity and dynamism of the embodied interaction between the human and machine. Secondly, the thesis argues for thinking of transdisciplinary research as a process of individuation, becoming or transduction, that is, as a process of mediation between heterogeneous approaches rather than perceiving research as a stabilized cognitive schema designed to accumulate new outcomes to the already-there reality. Arguing for going beyond the individualized approaches to embodiment, this thesis analyzes three cases where the problems that appear in one case are resolved through the analysis of the following one. Consisting of three phases, this research moves from objective scientific ¿reality¿ to more phenomenological, subjective and complex realities. The first study employs a critical review of embodied conversational agents in human¿computer interaction (HCI) in a learning context using a comparative meta-analysis. Meta-analysis was applied because most of the studies for evaluating embodiment are experimental. A learning context was selected because the number of studies is suitable for meta-analysis and the findings could be generalized to other contexts. The analysis reveals that there is no ¿persona effect¿, that is, the expected positive effect of virtual embodiment on the participant¿s affective, perceptive and cognitive measures. On the contrary, it shows the reduction of virtual embodiment to image and a lack of consideration for the participant¿s embodiment and interaction, in addition to theoretical and methodological shortcomings. The second phase solves these problems by focusing on Mark Hansen¿s phenomenological account of embodiment in new media. The investigation shows that Hansen improves on the HCI account by focusing on the participant¿s dynamic interaction with new media. Nevertheless, his views of embodied perception and affection are underpinned by a subjective patriarchal account leading to object/subject and body/work polarizations. The final phase resolves this polarization by analyzing the controversial work of Alan Turing on intelligent machinery. The research provides a different reading of the Turing Machine based on Simondon¿s concept of individuation, repositioning its materiality from the abstract non-existent to the actual-virtual realm and investigating the reasons for its abstraction. It relates the emergence of multiple human¿machine encounters in Turing¿s work to the complex counter-becoming of what it describes as ¿the Turing Machine compound¿.Ministry of Higher Education in the Sultanate of Oma
- …