Using Latent Semantic Analysis to Assess Reader Strategies

We tested a computer-based procedure for assessing reader strategies that was based on verbal protocols that utilized latent semantic analysis (LSA). Students were given self-explanation-reading training (SERT), which teaches strategies that facilitate self-explanation during reading, such as elaboration based on world knowledge and bridging between text sentences. During a computerized version of SERT practice, students read texts and typed self-explanations into a computer after each sentence. The use of SERT strategies during this practice was assessed by determining the extent to which students used the information in the current sentence versus the prior text or world knowledge in their self-explanations. This assessment was made on the basis of human judgments and LSA. Both human judgments and LSA were remarkably similar and indicated that students who were not complying with SERT tended to paraphrase the text sentences, whereas students who were compliant with SERT tended to explain the sentences in terms of what they knew about the world and of information provided in the prior text context. The similarity between human judgments and LSA indicates that LSA will be useful in accounting for reading strategies in a Web-based version of SERT

RMT: A dialog-based research methods tutor with or without a head

Abstract. RMT (Research Methods Tutor) is a dialog-based tutoring system that has a dual role. Its modular architecture enables the interchange and evaluation of different tools and techniques for improving tutoring. In addition to its research goals, the system is intended to be integrated as a regular component of a term-long Research Methods in Psychology course. Despite the significant technical challenges, this may help reduce our knowledge gap about how such systems can help students with long-term use. In this paper, we describe the RMT architecture and give the results of an initial experiment that compared RMT’s animated agent “talking head ” with a text-only version of the system.

Squeezing out gaming behavior in a dialog-based ITS

Abstract. Research Methods Tutor (RMT) is a dialog-based intelligent tutoring system which has been used by students in Research Methods in Psychology classes since 2003. Students interact with RMT to reinforce what they learn in class in five different topics. In this paper, we evaluate a different population of students and replicate our prior research: despite the relatively small amount of exposure during the term to RMT compared to other course-related activities, students learn significantly more on topics covered with RMT [1]. However, we did not find the same advantage for the dialog-based tutoring mode of RMT over the CAI mode. When transcript analyses indicated that a small but significant number of students were gaming the system by entering empty or nonsense responses, we modified the tutor to require reasonable attempts. This did lead some students to reform their gaming ways. In other cases, however, it resulted in disengagement from tutoring at least temporarily because reasonable answers were not recognized