The Curious Case of Loops

Background and Context: Subgoal labeled worked examples are effective for teaching computing concepts, but the research to date has been reported in a piecemeal fashion. This paper aggregates data from three studies, including data that has not been previously reported upon, to examine more holistically the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective: By aggregating the data, we provide more statistical and explanatory power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size to be expected when using subgoal labels in programming instruction. Method: We use descriptive and inferential statistics to examine the data for the effect of subgoal labeled worked examples across different student populations and different classroom instructional designs. We specifically concentrate on the potential effect size across samples of the intervention for potential generalization. Findings: Two groups of students learning how to write loops using subgoal labeled instructional materials perform better than the others. The better performing groups were the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems, both with medium-large effect sizes. Implications: For educators wishing to improve student learning using subgoal labeled materials should either provide students with subgoal labels while having them practice with a wide range of practice problems or allow students to generate their own subgoal labels and practice problems within similar contexts

The curious case of loops

Background and Context Subgoal labeled worked examples have been extensively researched, but the research has been reported piecemeal. This paper aggregates data from three studies, including data previously unreported, to holistically examine the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective By aggregating the data, we provide more statistical power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size for subgoal labels in programming instruction. Method We use descriptive and inferential statistics to examine the data collected from different student populations and different classroom instructional designs. We concentrate on the effect size across samples of the intervention for generalization. Findings Students using two variations of subgoal labeled instructional materials perform better than the others: the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems

Let's Ask Students About Their Programs, Automatically

Students sometimes produce code that works but that its author does not comprehend. For example, a student may apply a poorly-understood code template, stumble upon a working solution through trial and error, or plagiarize. Similarly, passing an automated functional assessment does not guarantee that the student understands their code. One way to tackle these issues is to probe students' comprehension by asking them questions about their own programs. We propose an approach to automatically generate questions about student-written program code. We moreover propose a use case for such questions in the context of automatic assessment systems: after a student's program passes unit tests, the system poses questions to the student about the code. We suggest that these questions can enhance assessment systems, deepen student learning by acting as self-explanation prompts, and provide a window into students' program comprehension. This discussion paper sets an agenda for future technical development and empirical research on the topic