Hint generation in programming tutors

Programming is increasingly recognized as a useful and important skill. Online programming courses that have appeared in the past decade have proven extremely popular with a wide audience. Learning in such courses is however not as effective as working directly with a teacher, who can provide students with immediate relevant feedback. The field of intelligent tutoring systems seeks to provide such feedback automatically. Traditionally, tutors have depended on a domain model defined by the teacher in advance. Creating such a model is a difficult task that requires a lot of knowledgeengineering effort, especially in complex domains such as programming. A potential solution to this problem is to use data-driven methods. The idea is to build the domain model by observing how students have solved an exercise in the past. New students can then be given feedback that directs them along successful solution paths. Implementing this approach is particularly challenging for programming domains, since the only directly observable student actions are not easily interpretable. We present two novel approaches to creating a domain model for programming exercises in a data-driven fashion. The first approach models programming as a sequence of textual rewrites, and learns rewrite rules for transforming programs. With these rules new student-submitted programs can be automatically debugged. The second approach uses structural patterns in programs’ abstract syntax trees to learn rules for classifying submissions as correct or incorrect. These rules can be used to find erroneous parts of an incorrect program. Both models support automatic hint generation. We have implemented an online application for learning programming and used it to evaluate both approaches. Results indicate that hints generated using either approach have a positive effect on student performance

The Role of Artificial Intelligence in Educating Novice Programmers

Programming is an inherently difficult skill to acquire and develop. Those who attempt to learn programming may be easily discouraged. The current landscape for computer science education does not address the needs of every novice programmer. Literature reports a discrepancy between student misconceptions and instructors’ perceptions of those misconceptions. Those who can afford a one-on-one human tutor perform on average two standard deviations better than those who learn via conventional methods, suggesting there is a need for a comparable, cheaper replacement. As a result, a number of intelligent tutoring systems have been developed for the purpose of teaching introductory programming concepts and replicating the benefits of one-on-one human tutoring. In this thesis, we analyze and discuss the literature pertaining to student misconceptions, selecting five fundamental misconception categories for introductory programming to demonstrate the effectiveness of existing intelligent tutoring systems. The features of existing intelligent tutoring systems are discussed and analyzed with respect to their effectiveness in addressing student misconceptions. Finally, we highlight the current gap in research on intelligent tutoring systems, hypothesizing the architecture and features of an ideal intelligent tutoring system for introductory programming.Electrical and Computer Engineerin

Design considerations of an intelligent tutoring system for programming languages

The overall goal of the thesis is to attempt to highlight the major topics which must be considered in the design of any Intelligent Tutoring System and to illustrate their application within the particular domain of LISP programming. There are two major sections to the thesis. The first considers the background to the educational application of computers. It examines possible roles for the computer, explores the relationship between education theory and computer-based teaching, and identifies some important links among existing Tutoring Systems. The section concludes with a summary of the design goals which an Intelligent Tutoring System should attempt to fulfill. The second section applies the design goals to the production of an Intelligent Tutoring System for programming languages. It devises a formal semantic description for programming languages and illustrates its application to tutoring. A method for modelling the learning process is introduced. Some techniques for maintaining a structured tutoring interaction are described. The work is set within the methodology of Artificial Intelligence research. Although a fully implemented tutoring system is not described, all features discussed are implemented as short programs intended to demonstrate the feasibility of the approach taken

An application of formal semantics to student modelling : an investigation in the domain of teaching Prolog

This thesis reports on research undertaken in an exploration of the use of formal semantics for student modelling in intelligent tutoring systems. The domain chosen was that of tutoring programming languages and within that domain Prolog was selected to be the target language for this exploration. The problem considered is one of how to analyse students' errors at a level which allows diagnosis to be more flexible and meaningful than is possible with the 'mal-rules' and 'bugcatalogue' approach of existing systems. The ideas put forward by Robin Milner [1980] in his Calculus of Communicating Systems (CCS) form the basis of the formalism which is proposed as a solution to this problem. Based on the findings of an empirical investigation, novices' misconceptions of control flow in Prolog was defined as a suitable area in which to explore the application of this solution. A selection of Prolog programs used in that investigation was formally described in terms of CCS. These formal descriptions were used by a production rule system to generate a number of the incomplete or faulty models of Prolog execution which were identified in the first empirical study. In a second empirical study, a machine-analysis tool, designed to be part of a diagnostic tutoring module, used these models to diagnose students' misconceptions of Prolog control flow. This initial application of CCS to student modelling showed that the models of Prolog execution generated by the system could be used successfully to detect students' misunderstandings. Results from the research reported here indicate that the use of formal semantics to model programming languages has a useful contribution to make to the task of student modelling

A workshop on the gathering of information for problem formulation

Issued as Quarterly progress reports no. [1-5], Proceedings and Final contract report, Project no. G-36-651Papers presented at the Workshop/Symposium on Human Computer Interaction, March 26 and 27, 1981, Atlanta, G

An empirically-based debugging system for novice programmers

The research described here concerns the design and construction of an empirically-based debugging aid for first-time computer users, integrated into the Open University's SOLO programming environment. Its basis is an account of the processes involved as human experts debug faulty code, which account was later found to be supported by empirical tests on human experts. The account implies that an understanding of the intentions of the programmer is not essential to successful debugging of a certain class of programs. That class comprises programs written in a database-dependent language by users who are initially completely computer-naive and who during their course become competent to write simple programs which embody one or more basic AI techniques such as recursive inference. The debugging system, called AURAC, incorporates an explicit model of the debugging strategies used by human experts. Its understanding, therefore, is of programming in general and of the SOLO environment in particular. We present in the process a broad taxonomy of naive users' errors, showing that they can be divided into types, each type requiring a different debugging approach and indicating a different degree of expertise on the part of the perpetrator. SOLO is a conveniently delimited though nonetheless rich problem domain. Also described is a new version of SOLO itself (MacSOLO) which incorporates a large number of traps for the simple errors which plague novices, thus enabling AURAC to concentrate on the more interesting programming mistakes. AURAC is intended to operate after the event rather than whilst a program is actually being written, and is able via analysis of programming cliches and of data flows to isolate errors in the user's code. Where AURAC cannot analyse, or where its analysis yields nothing useful, it describes the corresponding section of code instead, so that the user receives a coherent output. MacSOLO and AURAC together form a unified system, based upon the principles of Simplicity, Consistency and Transparency. We show how these principles were applied during the design and construction phases