Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes

Block-based programming environments are increasingly used to introduce computing concepts to beginners. However, novice students often struggle in these environments, given the conceptual and open-ended nature of programming tasks. To effectively support a student struggling to solve a given task, it is important to provide adaptive scaffolding that guides the student towards a solution. We introduce a scaffolding framework based on pop quizzes presented as multi-choice programming tasks. To automatically generate these pop quizzes, we propose a novel algorithm, PQuizSyn. More formally, given a reference task with a solution code and the student's current attempt, PQuizSyn synthesizes new tasks for pop quizzes with the following features: (a) Adaptive (i.e., individualized to the student's current attempt), (b) Comprehensible (i.e., easy to comprehend and solve), and (c) Concealing (i.e., do not reveal the solution code). Our algorithm synthesizes these tasks using techniques based on symbolic reasoning and graph-based code representations. We show that our algorithm can generate hundreds of pop quizzes for different student attempts on reference tasks from Hour of Code: Maze Challenge and Karel. We assess the quality of these pop quizzes through expert ratings using an evaluation rubric. Further, we have built an online platform for practicing block-based programming tasks empowered via pop quiz based feedback, and report results from an initial user study.Comment: Preprint. Accepted as a paper at the AIED'22 conferenc

A workshop on the gathering of information for problem formulation

Issued as Quarterly progress reports no. [1-5], Proceedings and Final contract report, Project no. G-36-651Papers presented at the Workshop/Symposium on Human Computer Interaction, March 26 and 27, 1981, Atlanta, G

Hint generation in programming tutors

Programming is increasingly recognized as a useful and important skill. Online programming courses that have appeared in the past decade have proven extremely popular with a wide audience. Learning in such courses is however not as effective as working directly with a teacher, who can provide students with immediate relevant feedback. The field of intelligent tutoring systems seeks to provide such feedback automatically. Traditionally, tutors have depended on a domain model defined by the teacher in advance. Creating such a model is a difficult task that requires a lot of knowledgeengineering effort, especially in complex domains such as programming. A potential solution to this problem is to use data-driven methods. The idea is to build the domain model by observing how students have solved an exercise in the past. New students can then be given feedback that directs them along successful solution paths. Implementing this approach is particularly challenging for programming domains, since the only directly observable student actions are not easily interpretable. We present two novel approaches to creating a domain model for programming exercises in a data-driven fashion. The first approach models programming as a sequence of textual rewrites, and learns rewrite rules for transforming programs. With these rules new student-submitted programs can be automatically debugged. The second approach uses structural patterns in programs’ abstract syntax trees to learn rules for classifying submissions as correct or incorrect. These rules can be used to find erroneous parts of an incorrect program. Both models support automatic hint generation. We have implemented an online application for learning programming and used it to evaluate both approaches. Results indicate that hints generated using either approach have a positive effect on student performance

Exploring the visualization of student behavior in interactive learning environments

My research combines Interactive Learning Environments (ILE), Educational Data Mining (EDM) and Information Visualization (Info-Vis) to inform analysts, educators and researchers about user behavior in software, specifically in CBEs, which include intelligent tutoring systems, computer aided instruction tools, and educational games. InVis is a novel visualization technique and tool I created for exploring, navigating, and understanding user interaction data. InVis reads in user-interaction data logged from students using educational systems and constructs an Interaction Network from those logs. Using this data InVis provides an interactive environment to allow instructors and education researchers to navigate and explore to build new insights and discoveries about student learning. I conducted a three-point user study, which included a quantitative task analysis, qualitative feedback, and a validated usability survey. Through this study, I show that creating an Interaction Network and visualizing it with InVis is an effective means of providing information to users about student behavior. In addition to this, I also provide four use-cases describing how InVis has been used to confirm hypotheses and debug software tutors. A major challenge in visualizing and exploring the Interaction Network is network's complexity, there are too many nodes and edges presented to understand the data efficiently. In a typical Interaction Network for twenty students, it is common to have hundreds of nodes, which to make sense of, has proven to be too many. I present a network reduction method, based on edge frequencies, which lowers the number of edges and nodes by roughly 90\\% while maintaining the most important elements of the Interaction Network. Next, I compare the results of this method with three alternative approaches and show our reduction method produces the preferred results. I also present an ordering detection method for identifying solution path redundancy because of student action orders. This method reduces the number of nodes and edges further and advances the resulting network towards the structure of a simple graph. Understanding the successful student solutions is only a portion of the behaviors we are interested in as researchers and educators using computer based educational systems, student difficulties are also important. To address areas of student difficulty, I present three different methods and two visual representations to draw the attention of the user to nodes where students had difficulty. Those methods include presenting the nodes with the highest number of successful students, the nodes with the highest number of failing students, and the expected difficulty of each state. Combined with a visual representation, these methods can draw the focus of users to potentially important nodes, which contain areas of difficulty for students. Lastly, I present the latest version of the InVis tool, which is a platform for investigating student behavior in computer based educational systems. Through the continued use of this tool, new researchers can investigate many new hypotheses, research questions and student behaviors, with the potential to facilitate a wide range of new discoveries

An Intelligent Debugging Tutor For Novice Computer Science Students

Debugging is a necessary aspect of computer science that can be difficult for novices and experienced programmers alike. This skill is mainly self-taught and is generally gained through trial and error, perhaps with some assistance from a professor or other expert figure. Novices encountering their first software defects may have few avenues open to them depending on the environment in which they are learning to program. The evident problem here is that the potential for a student to become stuck, frustrated, and/or losing confidence in their ability to pursue computer science is great. For a student to be successful when working professionally or progressing through academia they need to be able to function independently; trusting their own knowledge on par or above that of others so that their productivity does not rely on the knowledge of someone else. In order to solve this problem an Intelligent Tutoring System for teaching debugging skills to the novice utilizing Case Based Reasoning, Static Program Slicing, and the student\u27s preferred learning style was proposed. Case acquisition and automatic Exercise Generation were also explored. The system built for this research program was evaluated using novice students at the College and High School levels. Results of this evaluation produced statistically significant results at the p\u3c.05 and p\u3c.01 levels, with generated exercises exhibiting significance at the p\u3c.01 level. These results prove that the methodology chosen is a valid approach for the problem described, that the system does in fact teach students how to debug programs, and that the system is capable of successfully generating exercises on the fly