793 research outputs found

    Layered evaluation of interactive adaptive systems : framework and formative methods

    Get PDF
    Peer reviewedPostprin

    CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues

    Get PDF
    Large corpora of task-based and open-domain conversational dialogues are hugely valuable in the field of data-driven dialogue systems. Crowdsourcing platforms, such as Amazon Mechanical Turk, have been an effective method for collecting such large amounts of data. However, difficulties arise when task-based dialogues require expert domain knowledge or rapid access to domain-relevant information, such as databases for tourism. This will become even more prevalent as dialogue systems become increasingly ambitious, expanding into tasks with high levels of complexity that require collaboration and forward planning, such as in our domain of emergency response. In this paper, we propose CRWIZ: a framework for collecting real-time Wizard of Oz dialogues through crowdsourcing for collaborative, complex tasks. This framework uses semi-guided dialogue to avoid interactions that breach procedures and processes only known to experts, while enabling the capture of a wide variety of interactions. The framework is available at https://github.com/JChiyah/crwizComment: 10 pages, 5 figures. To Appear in LREC 202

    Harry Potter, Ruby Slippers and Merlin: Telling the Client\u27s Story Using the Characters and Paradigm of the Archetypal Hero\u27s Journey

    Get PDF
    This Article focuses on the relationship of mythology and folklore heroes to everyday lawyering decisions regarding case theory when the audience is a judge or panel of judges rather than a jury. This Article adds to the discourse by beginning a conversation about what might be termed “applied legal storytelling.” The term pertains to ideas of how everyday lawyers can utilize elements of mythology as a persuasive technique in stories told directly to judges--either via bench trials or via legal writing documents such as briefs--on behalf of an individual client in everyday litigation. Parts II and III of this Article will review legal storytelling from a fiction writing perspective and will introduce the mythological and psychological perspective of heroes. Part IV will explain the different types of heroic archetypes and show examples of how to select the appropriate hero type for a client. Part V will outline the universal journey and show examples of how a lawsuit may fit into the client\u27s overall journey. In all but one example, the Article draws on more day-to-day lawyering scenarios than on seminal cases. Smaller cases are analyzed in order to demonstrate that lawyers can use heroic archetypes as a routine scaffold rather than as a tool reserved for only the exceptional client scenarios

    Researching interactions between humans and machines: methodological challenges

    Get PDF
    Communication scholars are increasingly concerned with interactions between humans and communicative agents. These agents, however, are considerably different from digital or social media: They are designed and perceived as life-like communication partners (i.e., as “communicative subjects”), which in turn poses distinct challenges for their empirical study. Hence, in this paper, we document, discuss, and evaluate potentials and pitfalls that typically arise for communication scholars when investigating simulated or non-simulated interactions between humans and chatbots, voice assistants, or social robots. In this paper, we focus on experiments (including pre-recorded stimuli, vignettes and the “Wizard of Oz”-technique) and field studies. Overall, this paper aims to provide guidance and support for communication scholars who want to empirically study human-machine communication. To this end, we not only compile potential challenges, but also recommend specific strategies and approaches. In addition, our reflections on current methodological challenges serve as a starting point for discussions in communication science on how meaning-making between humans and machines can be investigated in the best way possible, as illustrated in the concluding section

    Communicating Dominance in a Nonanthropomorphic Robot Using Locomotion

    Get PDF
    Dominance is a key aspect of interpersonal relationships. To what extent do nonverbal indicators related to dominance status translate to a nonanthropomorphic robot? An experiment (N = 25) addressed whether a mobile robot's motion style can influence people's perceptions of its status. Using concepts from improv theater literature, we developed two motion styles across three scenarios (robot makes lateral motions, approaches, and departs) to communicate a robot's dominance status through nonverbal expression. In agreement with the literature, participants described a motion style that was fast, in the foreground, and more animated as higher status than a motion style that was slow, in the periphery, and less animated. Participants used fewer negative emotion words to describe the robot with the purportedly high-status movements versus the purportedly low-status movements, but used more negative emotion words to describe the robot when it made departing motions that occurred in the same style. This result provides evidence that guidelines from improvisational theater for using nonverbal expression to perform interpersonal status can be applied to influence perception of a nonanthropomorphic robot's status, thus suggesting that useful models for more complicated behaviors might similarly be derived from performance literature and theory

    CGAMES'2009

    Get PDF

    A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version

    Get PDF
    During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models. In the area of dialogue systems, the trend is less obvious, and most practical systems are still built through significant engineering and expert knowledge. Nevertheless, several recent results suggest that data-driven approaches are feasible and quite promising. To facilitate research in this area, we have carried out a wide survey of publicly available datasets suitable for data-driven learning of dialogue systems. We discuss important characteristics of these datasets, how they can be used to learn diverse dialogue strategies, and their other potential uses. We also examine methods for transfer learning between datasets and the use of external knowledge. Finally, we discuss appropriate choice of evaluation metrics for the learning objective
    • 

    corecore