Cognition and Technology: Effectiveness of intelligent tutoring systems for software training

This study addresses the potential of using an intelligent tutoring system (ITS) to tutor on off-the-shelf (OTS) software. ITSs have been successfully used to tutor on a variety of learning domains, but there has been little research comparing ITS-based training on an OTS application with traditional software training approaches such as books or interactive software simulations. The work presented here includes procedures and results for Paint.NET training and evaluation using three methods: book-based, interactive simulation, and an ITS. It is reported that there were some associations between the training method and training experiences. Book-based training exhibited higher scores on both task performance and system usability perception, while better times were recorded for the simulation approach. Concept acquisition score was not found to significantly correlate with training method, however. Additionally, it was found that interactions between training mode and spatial ability or general self-efficacy (GSE) significantly affected system usability perception. It was also learned that within ITS high computer self-efficacy (CSE) learners outperformed these with low CSE on task performance measure. Similar findings were reported for simulation group where high-spatial learners recorded better training times than low-spatial learners. Overall, results indicated that four individual characteristics to succeed indicators explored in this study significantly correlated with total training time and system usability measures. It is concluded that if an ITS is to be a tutor on OTS application then further refinements are needed