CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts

Rationality and emotion are two fundamental elements of humans. Endowing agents with rationality and emotion has been one of the major milestones in AI. However, in the field of conversational AI, most existing models only specialize in one aspect and neglect the other, which often leads to dull or unrelated responses. In this paper, we hypothesize that combining rationality and emotion into conversational agents can improve response quality. To test the hypothesis, we focus on one fundamental aspect of rationality, i.e., commonsense, and propose CARE, a novel model for commonsense-aware emotional response generation. Specifically, we first propose a framework to learn and construct commonsense-aware emotional latent concepts of the response given an input message and a desired emotion. We then propose three methods to collaboratively incorporate the latent concepts into response generation. Experimental results on two large-scale datasets support our hypothesis and show that our model can produce more accurate and commonsense-aware emotional responses and achieve better human ratings than state-of-the-art models that only specialize in one aspect.Comment: AAAI-202

한국인 고등학생의 영어 형용사 타동결과구문 학습에서의 인공지능 챗봇 기반 교수의 효과

학위논문(박사) -- 서울대학교대학원 : 사범대학 외국어교육과(영어전공), 2022.2. 김기택.English adjectival transitive resultative constructions (VtR) are notoriously challenging for Korean L2 English learners due to their syntactic and semantic differences from their L1 counterparts. To deal with such a complex structure, like English adjectival VtR, Korean L2 English learners need instructional interventions, including explicit instructions and corrective feedback on the target structure. Human instructors are virtually incapable of offering adequate corrective feedback, as providing corrective feedback from a human teacher to hundreds of students requires excessive time and effort. To deal with the practicality problems faced by human instructors in providing corrective feedback, numerous artificial intelligence (AI) chatbots have been developed to provide foreign language learners with corrective feedback on par with human teachers. Regrettably, many currently available AI chatbots remain underdeveloped. In addition, no prior research has been conducted to assess the effectiveness of corrective feedback offered by an AI chatbot, a human instructor, or additional explicit instruction via video material. The current study examined the instructional effects of corrective feedback from an AI chatbot on Korean high school students’ comprehension and production of adjectival VtR. Also, the current study investigated whether the corrective feedback generated by the AI chatbot enables Korean L2 English learners to expand their constructional repertoire beyond instructed adjectival VtR to uninstructed prepositional VtR. To investigate these issues, text-based Facebook Messenger AI chatbots were developed by the researcher. The effectiveness of the AI chatbots’ corrective feedback was compared with that of a human instructor and with additional video material. Students were divided into four groups: three instructional groups and one control group. The instructional groups included a chatbot group, a human group, and a video group. All learners in the three instructional groups watched a 5-minute explicit instruction video on the form and meaning pairings of the adjectival VtR in English. After that, learners were divided into three groups based on their preferences for instructional types. The learners volunteered to participate in the instructional procedures with corrective feedback from a text-based AI chatbot, a human instructor, or additional explicit instruction using a 15-minute video. Moreover, they took part in three testing sessions, which included a pretest, an immediate posttest, and a delayed posttest. The control group students were not instructed, and only participated in the three testing sessions. Two tasks were used for each test session: an acceptability judgment task (AJT) and an elicited writing task (EWT). The AJT tested participants’ comprehension of instructed adjectival VtR and uninstructed prepositional VtR. The EWT examined the correct production of instructed adjectival VtR and uninstructed prepositional VtR. The results of the AJT revealed that the instructional treatment (e.g., corrective feedback from the AI chatbot or a human instructor, or additional explicit instruction from the video material) was marginally more effective at improving the comprehension of adjectival VtR than was the case with the control group. On the other hand, the instructional treatment on the adjectival VtR failed in the generalization to prepositional VtR which was not overtly instructed. In the EWT, the participants in the corrective feedback groups (e.g., the chatbot and human groups) showed a more significant increase in the correct production of the instructed adjectival VtR more so than those in the video and control groups. Furthermore, the chatbot group learners showed significantly higher production of uninstructed prepositional VtR compared to any other group participants. These findings suggest that chatbot-based instruction can help Korean high school L2 English learners comprehend and produce complex linguistic structures—namely, adjectival and prepositional VtR. Moreover, the current study has major pedagogical implications for principled frameworks for implementing AI chatbot-based instruction in the context of foreign language learning.영어 형용사 타동결과구문(English Adjectival Transitive Resultative Construction)은 한국인 영어 학습자들에게 모국어의 대응 구문이 갖는 의미 통사론적 차이로 인해 학습하기 매우 어려운 것으로 알려져 있다. 따라서 영어 형용사 타동결과구문과 같은 복잡한 구문을 학습하기 위해서, 한국인 영어 학습자들에게는 목표 구조에 대한 명시적 교수와 교정적 피드백을 포함한 교수 처치가 요구된다. 수백 명의 학습자들에게 교정적 피드백을 제공하기 위해서는 과도한 시간과 노력이 요구되기 때문에, 인간 교사가 적절한 양의 교정적 피드백을 제공한다는 것은 사실상 불가능하다. 교정적 피드백을 제공할 때 직면하는 이러한 실용성 문제를 해결하기 위하여, 외국어 학습자들에게 인간 교사와 유사한 교정 피드백을 제공할 수 있는 수많은 인공 지능(AI) 챗봇이 개발되었다. 유감스럽게도, 현재 사용 가능한 많은 외국어 학습용 인공지능 챗봇은 아직 충분히 개발되지 않은 상태에 남아있으며, 인공지능 챗봇의 교정적 피드백이 갖는 교수효과를 비교 분석한 연구는 현재 이루어지지 않은 상태다. 이러한 선행연구의 한계에 초점을 두어, 본 연구에서는 인공지능 챗봇의 교정적 피드백이 한국 고등학생의 영어 형용사 타동결과구문의 이해와 생성에 미치는 교수 효과를 살펴보았다. 또한 본 연구에서는 이러한 교수 효과가 언어적으로 관련된 다른 영어 구문의 학습에도 영향을 끼치는지를 알아보기 위해 교실에서 직접 가르치지 않았던 구문인 영어 전치사 타동결과구문(English Prepositional Transitive Resultative Construction)의 학습 양상을 알아보았다. 이를 위해, 본 연구에서는 텍스트 메시지 기반의 페이스북 메신저에서 구동되는 인공지능 챗봇을 개발하였다. 인공지능 챗봇의 교수효과 검증을 위해 본 연구에 참여한 학생들은 네 개의 집단으로 구분되었다: 세 개의 교수 집단에는 교수처치가 적용되었고, 한 개의 통제 집단에서는 교수처치가 적용되지 않았다. 교수처치가 적용된 세 개의 집단은 챗봇그룹, 인간그룹, 영상그룹으로 분류되었으며, 이들은 모두 영어로 된 형용사 타동결과구문의 형태와 의미 쌍에 대한 5분 길이의 학습 비디오를 시청함으로써 명시적 교수 처치를 받았다. 또한 비디오를 시청한 후 세 그룹의 학습자들은 교재를 통해 제공되는 언어연습자료를 해결하는 과업에 참여하였다. 다음으로 세 집단(챗봇그룹, 인간그룹, 영상그룹)은 다음과 같은 추가적 교수처치를 받았다: 챗봇그룹 학습자들은 교재 활동과 관련된 텍스트 기반 인공지능 챗봇과의 대화에 참여함으로써 오류에 대한 교정적 피드백을 받았다. 인간그룹 학습자들은 교재활동을 완수한 내용을 인간 교사에게 전송하고, 이에 대한 교정적 피드백을 받았다. 영상그룹 학습자들은 교재활동을 완수한 후 이에 대한 15분의 추가적인 명시적 교수자료를 영상으로 시청하였다. 학습자의 교수효과는 사전시험, 사후시험 및 지연 사후시험으로 검증되었다. 한편 통제 집단 학생들은 교수처치 없이 세 번의 시험에만 참여하였다. 세 차례의 시험에서는 수용성판단과제(AJT)와 유도작문과제(EWT)의 두 가지 과제가 사용되었다. 수용성판단과제를 통하여, 교수된 영어 형용사 타동결과구문과 지시되지 않은 영어 전치사 타동결과구문 대한 참가자의 이해도를 측정하였다. 유도작문과제를 통하여 교수된 영어 형용사 타동결과구문과 지시되지 않은 영어 전치사 타동결과구문을 참여자가 정확하게 산출할 수 있는지를 측정하였다. 시험의 결과는 다음과 같았다. 수용성판단과제의 경우, 교수처치가 적용된 세 집단이 통제 집단보다 형용사 타동결과구문의 이해도 향상에 약간 더 효과적인 것으로 나타났다. 하지만 형용사 타동결과구문에 대한 교수적처치는 교수되지 않은 전치사 타동결과구문으로의 학습에 영향을 주지 못하였다. 유도작문과제의 경우, 인공지능 챗봇이나 인간 교사에 의해 제공되는 교정 피드백 그룹의 참가자가 영상그룹 및 통제집단의 참가자보다 형용사 타동결과구문의 올바른 생성에 더 유의미한 영향을 미치는 것으로 드러났다. 동일한 교수 효과가 전치사 타동결과구문의 학습에서도 관측되어, 형용사 타동결과구문의 학습이 전치사 타동결과구문의 학습에 일반화가 일어났다. 본 연구는 인간 교사가 직면해야 하는 실용성 문제를 극복하고, 인공지능 챗봇이 한국인 고등학교 L2 영어 학습자가 형용사 및 전치사 타동결과구문과 같은 복잡한 언어 구조를 이해하고 생성하는 데에 인간 교사와 비견될 정도로 교정적 피드백을 제공할 수 있을 것임을 시사한다. 또한, 본 연구는 인공지능 챗봇 기반 외국어 교육의 실제적 사례 및 효과를 선도적으로 보여주었다는 점에서 의미가 있다.ABSTRACT i TABLE OF CONTENTS iii LIST OF TABLES v LIST OF FIGURES vii CHAPTER 1. INTRODUCTION 1 1.1. Statement of Problems and Objectives 1 1.2. Scope of the Research 6 1.3. Research Questions 9 1.4. Organization of the Dissertation 10 CHAPTER 2. LITERATURE REVIEW 12 2.1. Syntactic and Semantic Analysis of Korean and English Transitive Resultative Constructions 13 2.1.1. Syntactic Analysis of English Transitive Resultative Construction 13 2.1.2. Syntactic Analysis of Korean Transitive Resultative Constructions 25 2.1.3. Semantic Differences in VtR between Korean and English 46 2.1.4. Previous acquisition study on English adjectival and prepositional VtR 54 2.2. Corrective Feedback 59 2.2.1. Definition of Corrective Feedback 59 2.2.2. Types of Corrective Feedback 61 2.2.3. Noticeability in Corrective Feedback 67 2.2.4. Corrective Recast as a Stepwise Corrective Feedback 69 2.3. The AI Chatbot in Foreign Language Learning 72 2.3.1. Non-communicative Intelligent Computer Assisted Language Learning (ICALL) 73 2.3.2. AI Chatbot without Corrective Feedback 79 2.3.3. AI Chatbot with Corrective Feedback 86 2.4. Summary of the Literature Review 92 CHAPTER 3. METHODOLOGY 98 3.1. Participants 98 3.2. Target Structure 102 3.3. Procedure of the Study 106 3.4. Instructional Material Shared by the Experimental Group 107 3.4.1. General Framework of the Instructional Session 108 3.4.2. Instructional Material Shared by Experimental Groups 111 3.5. Group-specific Instructional Treatments: Post-Written Instructional Material Activities on Corrective Feedback from Chatbot, Human, and Additional Explicit Instruction via Video 121 3.5.1. Corrective Feedback from the AI Chatbot 122 3.5.2. Corrective Feedback from a Human Instructor 136 3.5.3. Additional Instruction via Video Material 139 3.6. Test 142 3.6.1. Acceptability Judgment Task (AJT) 144 3.6.2. Elicited Writing Task (EWT) 150 3.7. Statistical Analysis 152 CHAPTER 4. RESULTS AND DISCUSSIONS 154 4.1. Results of Acceptability Judgment Task (AJT) 154 4.1.1. AJT Results of Instructed Adjectival VtR 155 4.1.2. AJT Results of Uninstructed Prepositional VtR 160 4.1.3. Discussion 164 4.2. Results of Elicited Writing Task (EWT) 175 4.2.1. EWT Results for Instructed Adjectival VtR 176 4.2.2. EWT Results of Uninstructed Prepositional VtR 181 4.2.3. Further Analysis 187 4.2.4. Discussion 199 CHAPTER 5. CONCLUSION 205 5.1. Summary of the Findings and Implications 205 5.2. Limitations and Suggestions for Future Research 213 REFERENCES 217 APPENDICES 246 ABSTRACT IN KOREAN 297박

Towards collaborative dialogue in Minecraft

This dissertation describes our work in building interactive agents that can communicate with humans to collaboratively solve tasks in grounded scenarios. To investigate the challenges of building such agents, we define a novel instantiation of a situated, Minecraft-based, Collaborative Building Task in which one player (A, the Architect) is shown a target structure, denoted Target, and needs to instruct the other player (B, the Builder) to build a copy of this structure, denoted Built, in a predefined build region. While both players can interact asynchronously via a chat interface, we define the roles to be asymmetric: A can observe B and Target, but is invisible and cannot place blocks; meanwhile, B can freely place and remove blocks, but has no explicit knowledge of the target structure. Each agent requires a different set of abilities in order to be successful at this task: specifically, A's main challenges arise in the task of generating situated instructions by comparing Built and Target, while B's responsibilities focus mainly on comprehending A's situated instructions using both dialogue and world context. Both agents must be able to interact asynchronously in an evolving dialogue context and a dynamic world state within which they are embodied. In this work, we specifically examine how well end-to-end neural models can learn to be instruction givers (i.e., Architects) from a limited amount of real human-human data. In order to examine how humans complete the Collaborative Building Task, as well as use human-human data as a gold standard for training and evaluating models, we present the Minecraft Dialogue Corpus, a collection of 509 conversations and game logs. We then introduce baseline models for the challenging subtask of Architect utterance generation, and evaluate them offline, using both automated metrics and human evaluation. We show that while conditioning our model on a simple representation of the world gives our model improved ability to generate correct instructions, there are still many obvious shortcomings, and it is difficult for these models to learn the large variety of abilities needed to be successful Architects in an entirely end-to-end manner. To combat this, we show that including meaningful, structured inputs about the world and discourse state as additional inputs -- specifically, by adding oracle information about the Builder's next actions, as well as enriching our linguistic representation with Architect dialogue acts -- improves the performance of our utterance generation models. We also augment the data with shape information by pretraining 3D shape localization models on synthetically generated block configurations. Finally, we integrate Architect utterance generation models into actual Minecraft agents and evaluate them in a fully interactive setting

English Medium Instruction as a Local Practice

From the perspective of translanguaging and instruction theories, this Open Access book examines Chinese English Medium Instruction (EMI) lecturers’ linguistic and pedagogical characteristics. This book demonstrate that ‘English’ in EMI is not a monolingual issue and EMI lecturers have applied their bilingual advantages to systematically and strategically advance their pedagogy practices through a translanguaging process. This book reflects upon EMI lecturers’ culture-imbedded teaching and learning philosophies and explores the implications of local classroom practices, such as topic-centered instruction and teacher presentation through demonstration. This book argues that EMI teaching is not an approach that can reach universal consent across linguistic, cultural and educational systems; it is an approach that is exclusively contextualised in the lecturers’ closely related cultural and educational system, and restricted by the available resources. This is an open access book

English Medium Instruction as a Local Practice

From the perspective of translanguaging and instruction theories, this Open Access book examines Chinese English Medium Instruction (EMI) lecturers’ linguistic and pedagogical characteristics. This book demonstrate that ‘English’ in EMI is not a monolingual issue and EMI lecturers have applied their bilingual advantages to systematically and strategically advance their pedagogy practices through a translanguaging process. This book reflects upon EMI lecturers’ culture-imbedded teaching and learning philosophies and explores the implications of local classroom practices, such as topic-centered instruction and teacher presentation through demonstration. This book argues that EMI teaching is not an approach that can reach universal consent across linguistic, cultural and educational systems; it is an approach that is exclusively contextualised in the lecturers’ closely related cultural and educational system, and restricted by the available resources. This is an open access book

Acoustic correlates of encoded prosody in written conversation

This thesis presents an analysis of certain punctuation devices such as parenthesis, italics and emphatic spellings with respect to their acoustic correlates in read speech. The class of punctuation devices under investigation are referred to as prosodic markers. The thesis therefore presents an analysis of features of the spoken language which are represented symbolically in text. Hence it is a characterization of aspects of the spoken language which have been transcribed or symbolized in the written medium and then translated back into a spoken form by a reader. The thesis focuses in particular on the analysis of parenthesis, the examination of encoded prominence and emphasis, and also addresses the use of paralinguistic markers which signal attitude or emotion.In an effort to avoid the use of self constructed or artificial material containing arbitrary symbolic or prosodic encodings, all material used for empirical analysis was taken from examples of electronic written exchanges on the Internet, such as from electronic mail messages and from articles posted on electronic newsgroups and news bulletins. This medium of language, which is referred to here as written conversation, provides a rich source of material containing encoded prosodic markers. These occur in the form of 'smiley faces' expressing attitudes or feelings, words highlighted by a number of means such as capitalization, italics, underscore characters, or asterisks, and in the form of dashes or parentheses, which provide suggestions on how the information in a text or sentence may be structured with regard to its informational content.Chapter 2 investigates in detail the genre of written conversation with respect to its place in an emerging continuum between written and spoken language, concentrating on transcriptional devices and their function as indicators of prosody. The implications these symbolic representations bear on the task of reading, by humans as well as machines, are then examined.Chapters 3 and 4 turn to the acoustic analysis of parentheticals and emphasis markers respectively. The experimental work in this thesis is based on readings of a corpus of selected materials from written conversation with the acoustic analysis concentrating on the differences between readings of texts with prosodic markers and readings of the same texts from which prosodic markers have been removed. Finally, the effect of prosodic markers is tested in perception experiments involving both human and resynthesized utterances