Analyzing collaborative learning processes automatically

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners’ interactions is a time consuming and effortful process. Improving automated analyses of such highly valued processes of collaborative learning by adapting and applying recent text classification technologies would make it a less arduous task to obtain insights from corpus data. This endeavor also holds the potential for enabling substantially improved on-line instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context sensitive collaborative learning support on an as-needed basis. In this article, we report on an interdisciplinary research project, which has been investigating the effectiveness of applying text classification technology to a large CSCL corpus that has been analyzed by human coders using a theory-based multidimensional coding scheme. We report promising results and include an in-depth discussion of important issues such as reliability, validity, and efficiency that should be considered when deciding on the appropriateness of adopting a new technology such as TagHelper tools. One major technical contribution of this work is a demonstration that an important piece of the work towards making text classification technology effective for this purpose is designing and building linguistic pattern detectors, otherwise known as features, that can be extracted reliably from texts and that have high predictive power for the categories of discourse actions that the CSCL community is interested in

Proceedings of QG2010: The Third Workshop on Question Generation

These are the peer-reviewed proceedings of "QG2010, The Third Workshop on Question Generation". The workshop included a special track for "QGSTEC2010: The First Question Generation Shared Task and Evaluation Challenge". QG2010 was held as part of The Tenth International Conference on Intelligent Tutoring Systems (ITS2010)

Recognizing Uncertainty in Speech

We address the problem of inferring a speaker's level of certainty based on prosodic information in the speech signal, which has application in speech-based dialogue systems. We show that using phrase-level prosodic features centered around the phrases causing uncertainty, in addition to utterance-level prosodic features, improves our model's level of certainty classification. In addition, our models can be used to predict which phrase a person is uncertain about. These results rely on a novel method for eliciting utterances of varying levels of certainty that allows us to compare the utility of contextually-based feature sets. We elicit level of certainty ratings from both the speakers themselves and a panel of listeners, finding that there is often a mismatch between speakers' internal states and their perceived states, and highlighting the importance of this distinction.Comment: 11 page

Supporting Constructive Learning with a Feedback Planner

A promising approach to constructing more effective computer tutors is implementing tutorial strategies that extend over multiple turns. This means that computer tutors must deal with (1) failure, (2) interruptions, (3) the need to revise their tactics, and (4) basic dialogue phenomena such as acknowledgment. To deal with these issues, we need to combine ITS technology with advances from robotics and computational linguistics. We can use reactive planning techniques from robotics to allow us to modify tutorial plans, adapting them to student input. Computational linguistics will give us guidance in handling communication management as well as building a reusable architecture for tutorial dialogue systems. A modular and reusable architecture is critical given the difficulty in constructing tutorial dialogue systems and the many domains to which we would like to apply them. In this paper, we propose such an architecture and discuss how a reactive planner in the context of this architecture can implement multi-turn tutorial strategies

Measurement and analysis of interactive behavior in tutoring action with children and robots

Vollmer A-L. Measurement and analysis of interactive behavior in tutoring action with children and robots. Bielefeld: Universität Bielefeld; 2011.Robotics research is increasingly addressing the issue of enabling robots to learn in social interaction. In contrast to the traditional approach by which robots are programmed by experts and prepared for and restricted to one specific purpose, they are now envisioned as general-purpose machines that should be able to carry out different tasks and thus solve various problems in everyday environments. Robots which are able to learn novel actions in social interaction with a human tutor would have many advantages. Unexperienced users could "program" new skills for a robot simply by demonstrating them. Children are able to rapidly learn in social interaction. Modifications in tutoring behavior toward children ("motionese") are assumed to assist their learning processes. Similar to small children, robots do not have much experience of the world and thus could make use of this beneficial natural tutoring behavior if it was employed, when tutoring them. To achieve this goal, the thesis provides theoretical background on imitation learning as a central field of social learning, which has received much attention in robotics and develops new interdisciplinary methods to measure interactive behavior. Based on this background, tutoring behavior is examined in adult-child, adult-adult, and adult-robot interactions by applying the developed methods. The findings reveal that the learner’s feedback is a constituent part of the natural tutoring interaction and shapes the tutor’s demonstration behavior. The work provides an insightful understanding of interactional patterns and processes. From this it derives feedback strategies for human-robot tutoring interactions, with which a robot could prompt hand movement modifications during the tutor’s action demonstration by using its gaze, enabling robots to elicit advantageous modifications of the tutor’s behavior

When Does Disengagement Correlate with Performance in Spoken Dialog Computer Tutoring?

In this paper we investigate how student disengagement relates to two performance metrics in a spoken dialog computer tutoring corpus, both when disengagement is measured through manual annotation by a trained human judge, and also when disengagement is measured through automatic annotation by the system based on a machine learning model. First, we investigate whether manually labeled overall disengagement and six different disengagement types are predictive of learning and user satisfaction in the corpus. Our results show that although students’ percentage of overall disengaged turns negatively correlates both with the amount they learn and their user satisfaction, the individual types of disengagement correlate differently: some negatively correlate with learning and user satisfaction, while others don’t correlate with eithermetric at all. Moreover, these relationships change somewhat depending on student prerequisite knowledge level. Furthermore, using multiple disengagement types to predict learning improves predictive power. Overall, these manual label-based results suggest that although adapting to disengagement should improve both student learning and user satisfaction in computer tutoring, maximizing performance requires the system to detect and respond differently based on disengagement type. Next, we present an approach to automatically detecting and responding to user disengagement types based on their differing correlations with correctness. Investigation of ourmachine learningmodel of user disengagement shows that its automatic labels negatively correlate with both performance metrics in the same way as the manual labels. The similarity of the correlations across the manual and automatic labels suggests that the automatic labels are a reasonable substitute for the manual labels. Moreover, the significant negative correlations themselves suggest that redesigning ITSPOKE to automatically detect and respond to disengagement has the potential to remediate disengagement and thereby improve performance, even in the presence of noise introduced by the automatic detection process