Talk Like an Electrician: Student Dialogue Mimicking Behavior in an Intelligent Tutoring System

Abstract. Students entering a new field must learn to speak the specialized language of that field. Previous research using automated measures of word overlap has found that students who modify their language to align more closely to a tutor's language show larger overall learning gains. We present an alternative approach that assesses syntactic as well as lexical alignment in a corpus of human-computer tutorial dialogue. We found distinctive patterns differentiating high and low achieving students. Our high achievers were most likely to mimic their own earlier statements and rarely made mistakes when mimicking the tutor. Low achievers were less likely to reuse their own successful sentence structures, and were more likely to make mistakes when trying to mimic the tutor. We argue that certain types of mimicking should be encouraged in tutorial dialogue systems, an important future research direction

Adapting the scheduling of illustrations and graphs to learners in conceptual physics tutoring

This research investigates how to schedule multiple graphical representations in a dialogue-based conceptual physics tutor. Research on multiple graphical representations in tutoring suggests either frequently switching representations or fading from concrete to abstract representations. However, other research communities suggest that the best representation or scheduling can be dependent on various student and tutoring context factors. This thesis investigates whether these factors are important when considering a schedule of representations. Three major hypotheses are investigated. H1: that the best representational format for physics concepts is related to properties of the student and the tutoring context. H2: that it is possible to build models that predict the best representational format using student and tutoring context information. H3: that picking the representational format based upon student and tutoring context information will produce better learning gains than not considering student and tutoring context information. Additionally, this work addresses the question of whether multiple representations produce greater learning gains than a single representation (H4). A first experiment was performed to both investigate H1 and to collect data for H2. ANOVAs showed significant interaction effects in learning between low and high pretesters and between high and low spatial reasoning ability subjects, supporting the first hypothesis. Using the data collected and features describing student and tutoring context information, models were learned to predict when to show illustrations or graphs. That these models could be learned, produce meaningful rules, and outperformed a baseline supports H2. A new modeling algorithm was developed to learn these models by augmenting multiple linear regression to consider certain syntactic constraints. A third study was run to test H3 and H4 and to extrinsically evaluate the adaptive policy learned. One third of subjects had an adaptive scheduling of representations, one third a fixed alternating scheduling, and one third saw only one representation. In support of H3, subjects with high incoming knowledge sometimes perform better when receiving adaptive scheduling over an alternating scheduling, but there are also counter examples. For H4, it is not supported in general: showing only illustrations is best overall, but in some cases some subjects benefit from multiple representations

Students´ language in computer-assisted tutoring of mathematical proofs

Truth and proof are central to mathematics. Proving (or disproving) seemingly simple statements often turns out to be one of the hardest mathematical tasks. Yet, doing proofs is rarely taught in the classroom. Studies on cognitive difficulties in learning to do proofs have shown that pupils and students not only often do not understand or cannot apply basic formal reasoning techniques and do not know how to use formal mathematical language, but, at a far more fundamental level, they also do not understand what it means to prove a statement or even do not see the purpose of proof at all. Since insight into the importance of proof and doing proofs as such cannot be learnt other than by practice, learning support through individualised tutoring is in demand. This volume presents a part of an interdisciplinary project, set at the intersection of pedagogical science, artificial intelligence, and (computational) linguistics, which investigated issues involved in provisioning computer-based tutoring of mathematical proofs through dialogue in natural language. The ultimate goal in this context, addressing the above-mentioned need for learning support, is to build intelligent automated tutoring systems for mathematical proofs. The research presented here has been focused on the language that students use while interacting with such a system: its linguistic propeties and computational modelling. Contribution is made at three levels: first, an analysis of language phenomena found in students´ input to a (simulated) proof tutoring system is conducted and the variety of students´ verbalisations is quantitatively assessed, second, a general computational processing strategy for informal mathematical language and methods of modelling prominent language phenomena are proposed, and third, the prospects for natural language as an input modality for proof tutoring systems is evaluated based on collected corpora

Reflection and Learning Robustness in a Natural Language Conceptual Physics Tutoring System

This thesis investigates whether reflection after tutoring with the Itspoke qualitative physics tutoring system can improve both near and far transfer learning and retention. This question is formalized in three major hypotheses. H1: that reading a post-tutoring reflective text will improve learning compared to reading a non-reflective text. H2: that a more cohesive reflective text will produce higher learning gains for most students. And H3: that students with high domain knowledge will learn more from a less cohesive text.In addition, this thesis addresses the question of which mechanisms affect learning from a reflective text. Secondary hypotheses H4 and H5 posit that textual cohesion and student motivation, respectively, each affect learning by influencing the amount of inference performed while reading.These hypotheses were tested by asking students to read a reflective/abstractive text after tutoring with the Itspoke tutor. This text compared dialog parts in which similar physics principles had been applied to different situations. Students were randomly assigned among two experimental conditions which got ``high' or ``low' cohesion versions of this text, or a control condition which read non-reflective physics material after tutoring.The secondary hypotheses were tested using two measures of cognitive load while reading: reading speeds and a self-report measure of reading difficulty.Near and far transfer learning was measured using sets of questions that were mostly isomorphic vs. non-isomorphic the tutored problems, and retention was measured by administering both an immediate and a delayed post-test. Motivation was measured using a questionnaire.Reading a reflective text improved learning, but only for students with a middle amount of motivation, confirming H1 for that group. These students also learned more from a more cohesive reflective text, supporting H2. Cohesion also affected high and low knowledge students significantly differently, supporting H3, except that high knowledge students learned best from high, not low cohesion text.Students with higher amounts of motivation did have higher cognitive load, confirming hypothesis H5 and suggesting that they engaged the text more actively. However, secondary hypothesis H4 failed to show a role for cognitive load in explaining the learning interaction between knowledge and cohesion demonstrated in H3

Applications of Discourse Structure for Spoken Dialogue Systems

Language exhibits structure beyond the sentence level (e.g. the syntactic structure of a sentence). In particular, dialogues, either human-human or human-computer, have an inherent structure called the discourse structure. Models of discourse structure attempt to explain why a sequence of random utterances combines to form a dialogue or no dialogue at all. Due to the relatively simple structure of the dialogues that occur in the information-access domains of typical spoken dialogue systems (e.g. travel planning), discourse structure has often seen limited application in such systems. In this research, we investigate the utility of discourse structure for spoken dialogue systems in more complex domains, e.g. tutoring. This work was driven by two intuitions.First, we believed that the "position in the dialogue" is a critical information source for two tasks: performance analysis and characterization of dialogue phenomena. We define this concept using transitions in the discourse structure. For performance analysis, these transitions are used to create a number of novel factors which we show to be predictive of system performance. One of these factors informs a promising modification of our system which is implemented and compared with the original version of the system through a user study. Results show that the modification leads to objective improvements. For characterization of dialogue phenomena, we find statistical dependencies between discourse structure transitions and two dialogue phenomena which allow us to speculate where and why these dialogue phenomena occur and to better understand system behavior.Second, we believed that users will benefit from direct access to discourse structure information. We enable this through a graphical representation of discourse structure called the Navigation Map. We demonstrate the subjective and objective utility of the Navigation Map through two user studies.Overall, our work demonstrates that discourse structure is an important information source for designers of spoken dialogue systems