The effects of Domain and Collaboration Feedback on Learning in a Collaborative Intelligent Tutoring System

Abstract

We present initial results from a study comparing the effects of domain and collaboration feedback on learning within COLLECT-UML, a collaborative problem-solving ITS. Using COLLECT-UML, two students in separate physical locations (a collaborative pair) construct UML class diagrams to solve problems together. In the default version, COLLECT-UML provides both domain and collaboration feedback. In this study however, collaborative pairs were randomly assigned to one of four modes (treatment conditions) which varied the feedback presented by the system: no feedback (NF), domain feedback only (DF), collaborative feedback only (CF), and both domain and collaborative feedback (DCF). All conditions improved significantly between pre- and post-test, showing that practicing within COLLECT-UML helps learning. At a surface level, collaborative pairs in all modes had similar amounts of collaboration. The DCF mode had significantly higher learning gains than the other modes, indicating the value of receiving both domain and collaborative feedback. Surprisingly, the CF mode had the lowest learning gains (lower than NF), suggesting that, in this case, good collaboration without domain feedback could have simply reinforced erroneous domain knowledge