Visual Dialogue State Tracking for Question Generation

GuessWhat?! is a visual dialogue task between a guesser and an oracle. The guesser aims to locate an object supposed by the oracle oneself in an image by asking a sequence of Yes/No questions. Asking proper questions with the progress of dialogue is vital for achieving successful final guess. As a result, the progress of dialogue should be properly represented and tracked. Previous models for question generation pay less attention on the representation and tracking of dialogue states, and therefore are prone to asking low quality questions such as repeated questions. This paper proposes visual dialogue state tracking (VDST) based method for question generation. A visual dialogue state is defined as the distribution on objects in the image as well as representations of objects. Representations of objects are updated with the change of the distribution on objects. An object-difference based attention is used to decode new question. The distribution on objects is updated by comparing the question-answer pair and objects. Experimental results on GuessWhat?! dataset show that our model significantly outperforms existing methods and achieves new state-of-the-art performance. It is also noticeable that our model reduces the rate of repeated questions from more than 50% to 21.9% compared with previous state-of-the-art methods.Comment: 8 pages, 4 figures, Accept-Oral by AAAI-202

Dialogue Systems Specialized in Social Influence: Systems, Methods, and Ethics

This thesis concerns the task of how to develop dialogue systems specialized in social influence and problems around deploying such systems. Dialogue systems have become widely adopted in our daily life. Most dialogue systems are primarily focused on information-seeking tasks or social companionship. However, they cannot apply strategies in complex and critical social influence tasks, such as healthy habit promotion, emotional support, etc. In this work, we formally define social influence dialogue systems to be systems that influence users’ behaviors, feelings, thoughts, or opinions through natural conversations. We also present methods to make such systems intelligible, privacy-preserving, and thus deployable in real life. Finally, we acknowledge potential ethical issues around social influence systems and propose solutions to mitigate them in Chapter 6. Social influence dialogues span various domains, such as persuasion, negotiation, and recommendation. We first propose a donation persuasion task, PERSUASIONFORGOOD, and ground our study on this persuasion task for social good. We then build a persuasive dialogue system, by refining the dialogue model for intelligibility and imitating human experts for persuasiveness, and a negotiation agent that can play the game of Diplomacy by decoupling the planning engine and the dialogue generation module to improve controllability of social influence systems. To deploy such a system in the wild, our work examines how humans perceive the AI agent’s identity, and how their perceptions impact the social influence outcome. Moreover, dialogue models are trained on conversations, where people could share personal information. This creates privacy concerns for deployment as the models may memorize private information. To protect user privacy in the training data, our work develops privacy-preserving learning algorithms to ensure deployed models are safe under privacy attacks. Finally, deployed dialogue agents have the potential to integrate human feedback to continuously improve themselves. So we propose JUICER, a framework to make use of both binary and free-form textual human feedback to augment the training data and keep improving dialogue model performance after deployment. Building social influence dialogue systems enables us to research future expert-level AI systems that are accessible via natural languages, accountable with domain knowledge, and privacy-preserving with privacy guarantees

Designing coherent and engaging open-domain conversational AI systems

Designing conversational AI systems able to engage in open-domain ‘social’ conversation is extremely challenging and a frontier of current research. Such systems are required to have extensive awareness of the dialogue context and world knowledge, the user intents and interests, requiring more complicated language understanding, dialogue management, and state and topic tracking mechanisms compared to traditional task-oriented dialogue systems. Given the wide coverage of topics in open-domain dialogue, the conversation can span multiple turns where a number of complex linguistic phenomena (e.g. ellipsis and anaphora) are present and should be resolved for the system to be contextually aware. Such systems also need to be engaging, keeping the users’ interest over long conversations. These are only some of the challenges that open-domain dialogue systems face. Therefore this thesis focuses on designing dialogue systems able to hold extensive open-domain conversations in a coherent, engaging, and appropriate manner over multiple turns. First, different types of dialogue systems architecture and design decisions are discussed for social open-domain conversations, along with relevant evaluation metrics. A modular architecture for ensemble-based conversational systems is presented, called Alana, a finalist in the Amazon Alexa Prize Challenge in 2017 and 2018, able to tackle many of the challenges for open-domain social conversation. The system combines different features such as topic tracking, contextual Natural Language understanding, entity linking, user modelling, information retrieval, and response ranking, using a rich representation of dialogue state. The thesis next analyses the performance of the 2017 system and describes the upgrades developed for the 2018 system. This leads to an analysis and comparison of the real-user data collected in both years with different system configurations, allowing assessment of the impact of different design decisions and modules. Finally, Alana was integrated into an embodied robotic platform and enhanced with the ability to also perform tasks. This system was deployed and evaluated in a shopping mall in Finland. Further analysis of the added embodiment is presented and discussed, as well as the challenges of translating open-domain dialogue systems into other languages. Data analysis of the collected real-user data shows the importance of a variety of features developed and decisions made in the design of the Alana system

Teaching as Analogous Personalization: A pragmatic inquiry into expert teachers' process for fostering synchrony in educational dialogs, in post-secondary writing

Descriptive understandings of what human learning is, and so normative expectations of what teachers can and should do as educational leaders, has shifted greatly in society over the past century. The learning metaphors have moved from mechanical transfer to organic transformation; the educational approaches have moved from behavioral response-training to social-emotional facilitating: encouraging students not merely to repeat experts but to think like members in those knowledge-based communities, not merely to mimic disciplines' methods but to participate personally in the ongoing discourse of those fields. In an immediate sense, this shift is progress. Yet, in a larger sense, it is merely cycling back to acknowledge an old and persistent thread of practical wisdom among educators: that people learn complexly as emotional-social-intellectual creatures, and so that a teacher's work is to entice interest and effort, to foster a sense of belonging and trust, and to persuade students toward personally connecting with and valuing those same integral parts of a subject-matter that the teacher has already beneficially personalized for themselves. This longstanding rhetorical and pragmatic view of a teacher's educational role is now being supported directly by empirical research that shows the sense-bound, neurologically integrated, socially attuned, identity-and-meaning motivated character of human feelings, thoughts, and dispositions. I introduce the term “analogous personalization” to capture this synthetic (experience-based, scientifically supported) understanding of teaching as complexly social-emotional, intellectual, persuasive work. I then focus on educational dialogs—specifically within post-secondary writing-based courses—as a means of exploring how expert teachers foster synchrony between their own and their students' personal connections to (i.e., emotional inclination toward, social affiliation with, intellectual/practical understanding of) subject-matter. First, this dissertation offers a synthetic overview of some emergent mind-brain-body findings, and points out the fundamental educational realities that those findings substantiate. On that foundation, it next overviews insights from the field of rhetoric-and-writing about how teachers can usefully conceptualize the learner-knowledge-environment relationship from a dialogic perspective, to achieve effective (intentional, situated, synchronous) educational exchanges. Building from those scientific and practical literatures, it offers a flexible research method for studying the pragmatic arc of an educational exchange (from teacher intention to student take-away): by using the teacher's own personal, practical, principled framework of educational ideals and approaches; comparing their stated intentions with students' stated learning experiences, and tracing the arc of that educational dialog through actual classroom recordings. Finally, it enlists this radically situated research method to analyze three expert university writing teachers' practices: their idiosyncratic understandings of a teacher's role (from their own perspective); their experience-based manner of forming learning-centered relationships with students (from my observing perspective); and their apparent, persuasive self-investment in the course's subject-matter and the students' learning (from students' perspectives). It concludes with observations about the role of a teacher's sincerity (both practiced and perceived) in developing professional expertise and achieving synchrony with students in educational exchanges

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences

Academic Libraries as Feminine and Feminist Models of Organization.

Because academic libraries are primarily staffed by women and are relatively autonomous entities in colleges and universities, they offer a unique model of workplace gendering and feminism. This qualitative, ethnographic study examined 3 small college libraries in 3 regions of the United States and explored issues of bureaucracy and gendering in these libraries. Feminist challenges to bureaucracy emerged in the areas of hierarchy, division of labor, competition and collaboration, decision-making, and communication. Feminine practice in the libraries reflected private sphere attitudes toward work (values of community, emotionality, and caring) and an affirmation of feminine roles in the workplace. The organizational cultures of these libraries affirmed flexible scheduling, emotions and friendship at work, and parenting talk and behaviors. The library workers also engaged in an ethic of care for library users and colleagues. Individuals in the organizations expressed motivations for work not based in monetary or status gain and endorsed women\u27s power in leadership roles. The gendering of libraries also placed strong masculinity outside of the norm, creating expectations for men to engage in androgynous or feminine behavior. Overall, the study gives voice to feminine and feminist practice in the workplace