Exploring User Satisfaction in a Tutorial Dialogue System

Abstract User satisfaction is a common evaluation metric in task-oriented dialogue systems, whereas tutorial dialogue systems are often evaluated in terms of student learning gain. However, user satisfaction is also important for such systems, since it may predict technology acceptance. We present a detailed satisfaction questionnaire used in evaluating the BEETLE II system (REVU-NL), and explore the underlying components of user satisfaction using factor analysis. We demonstrate interesting patterns of interaction between interpretation quality, satisfaction and the dialogue policy, highlighting the importance of more finegrained evaluation of user satisfaction

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl

Teaching University Students to Read and Write

Recent government initiatives have required universities to include specific literacy and numeracy targets for the students. The authors – both members of the English discipline at Charles Sturt University – were invited to develop and run a two-semester program for all students studying to become early childhood, primary, and secondary teachers. This article outlines the nature of the two subjects which comprise the program: the first focused on reading and comprehension, the second on writing and composition. These subjects were conceived from collegial dialogues between academics in education and the humanities, and then developed from these different assumptions and starting points. Over the last five years, the shared experiences of teaching these prospective teachers has grown into a strongly coherent first year of study. This article seeks the describe the experiences of teaching literacy to first-year education students, and it is by turns hypothesising and speculative, reflective and qualitative, in its approach. In the process, this article offers colleagues across the country a reflection on the hypotheses of literacy education, some new ideas for teaching literacy, and some optimism for the future of the teaching profession, and the dignity of those who aspire to be a part of it

Positive Versus Negative Agents: The Effects of Emotions on Learning

The current study investigates the impact of affect, mood contagion, and linguistic alignment on learning during tutorial conversations between a human student and two artificial pedagogical agents. The study uses an Intelligent Tutoring System known as OperationARIES! to engage students in tutorial conversations with animated agents. In this investigation, 48 college students (N = 48) conversed with pedagogical agents as they displayed 3 different moods (i.e., positive, negative, and neutral) along with a control condition in a within-subjects design. Results indicate that the mood of the agent did not significantly impact student learning even though mood contagion did occur between the artificial agent and the human student. Learning was influenced by the student\u27s self-reported arousal level and the alignment scores that reflected a shared mental representation between the human student and the artificial agents. The results suggest that arousal and linguistic alignment during the tutorial conversations may play a role in learning

New measurement paradigms

This collection of New Measurement Paradigms papers represents a snapshot of the variety of measurement methods in use at the time of writing across several projects funded by the National Science Foundation (US) through its REESE and DR K–12 programs. All of the projects are developing and testing intelligent learning environments that seek to carefully measure and promote student learning, and the purpose of this collection of papers is to describe and illustrate the use of several measurement methods employed to achieve this. The papers are deliberately short because they are designed to introduce the methods in use and not to be a textbook chapter on each method. The New Measurement Paradigms collection is designed to serve as a reference point for researchers who are working in projects that are creating e-learning environments in which there is a need to make judgments about students’ levels of knowledge and skills, or for those interested in this but who have not yet delved into these methods

Developing Thai Learners’ CIC through Translanguaging in Oneon- One English Tutorial Sessions

This article presents a conceptual paper, which proposes a concept of employing translanguaging as a pedagogical tool to promote the learners’ CIC – Classroom Interactional Competence, which lies at the heart of learning. The aim of the concept is to bridge two practices of monolingualism between Thai teachers who teach English through Thai only and native speakers who use English only. Translanguaging reflects reality in terms of using both languages to interact to improve the interactional competence of the learners. When both the teachers and the learners translanguage in the classroom, ‘translanguaging space’ is established. This means boundary lines of the two languages are blurred and become so permeable that the learners are able to step in the space and utilize it to make their own ‘space of learning’ through interactions with the teachers. This concept implies that the more the learners interact with the teachers, the more they learn English. Thus, if the learners’ CIC develops in translanguaging classroom context, it can be argued that translanguaging promotes Thai learners’ CIC, which is seen the same thing as the progress of learning. The paper introduces the concept, reviews literature on translanguaging and CIC, discusses conceptual framework, and proposes significant issues in conducting a future study