Designing effective hints systems

Hint systems are designed to provide users with assistance with a complicated or difficult task. However, modern-day hint systems are frequently ineffective at assisting users, forcing them to consult external help. One flaw in current systems is that they fail to anticipate user needs as well as accommodate different types of users. In our study, we discovered key factors to designing an effective hint using a notoriously difficult puzzle game as our experimental object. We found that abstract hints are perceived as worse than no hint because they do not provide sufficient help for the player. In contrast, we found that concrete hints are perceived as more helpful, but ultimately that player experience attributes primarily depend on the characteristics of the game. In addition, we give players the choice to pick their own hint and find that it does not significantly alter player experience. We conclude our paper with a discussion on how we can apply these lessons to hints in non-game applications

Hint generation in programming tutors

Programming is increasingly recognized as a useful and important skill. Online programming courses that have appeared in the past decade have proven extremely popular with a wide audience. Learning in such courses is however not as effective as working directly with a teacher, who can provide students with immediate relevant feedback. The field of intelligent tutoring systems seeks to provide such feedback automatically. Traditionally, tutors have depended on a domain model defined by the teacher in advance. Creating such a model is a difficult task that requires a lot of knowledgeengineering effort, especially in complex domains such as programming. A potential solution to this problem is to use data-driven methods. The idea is to build the domain model by observing how students have solved an exercise in the past. New students can then be given feedback that directs them along successful solution paths. Implementing this approach is particularly challenging for programming domains, since the only directly observable student actions are not easily interpretable. We present two novel approaches to creating a domain model for programming exercises in a data-driven fashion. The first approach models programming as a sequence of textual rewrites, and learns rewrite rules for transforming programs. With these rules new student-submitted programs can be automatically debugged. The second approach uses structural patterns in programs’ abstract syntax trees to learn rules for classifying submissions as correct or incorrect. These rules can be used to find erroneous parts of an incorrect program. Both models support automatic hint generation. We have implemented an online application for learning programming and used it to evaluate both approaches. Results indicate that hints generated using either approach have a positive effect on student performance