Design and Pilot Testing of Subgoal Labeled Worked Examples for Five Core Concepts in CS1

Subgoal learning has improved student problem-solving performance in programming, but it has been tested for only one-to-two hours of instruction at a time. Our work pioneers implementing subgoal learning throughout an entire introductory programming course. In this paper we discuss the protocol that we used to identify subgoals for core programming procedures, present the subgoal labels created for the course, and outline the subgoal-labeled instructional materials that were designed for a Java-based course. To examine the effect of subgoal labeled materials on student performance in the course, we compared quiz and exam grades between students who learned using subgoal labels and those who learned using conventional materials. Initial results indicate that learning with subgoals improves performance on early applications of concepts. Moreover, variance in performance was lower and persistence in the course was higher for students who learned with subgoals compared to those who learned with conventional materials, suggesting that learning with subgoal labels may uniquely benefit students who would normally receive low grades or dropout of the course

Using the SOLO Taxonomy to Understand Subgoal Labels Effect in CS1

is work extends previous research on subgoal labeled instructions by examining their effect across a semester-long, Java-based CS1 course. Across four quizzes, students were asked to explain in plain English the process that they would use to solve a programming problem. In this mixed methods study, we used the SOLO taxonomy to categorize student responses about problem-solving processes and compare students who learned with subgoal labels to those who did not. e use of the SOLO taxonomy classification allows us to look deeper than the mere correctness of answers to focus on the quality of the answers produced in terms of completeness of relevant concepts and explanation of relationships among concepts. Students who learned with subgoals produced higher-rated answers in terms of complexity and quality on three of four quizzes. Also, they were three times more likely to discuss issues of data type on a question about assignments and expressions than students who did not learn with subgoal labeling. is suggests that the use of subgoal labeling enabled students to gain a deeper and more complex understanding of the material presented in the course

The curious case of loops

Background and Context Subgoal labeled worked examples have been extensively researched, but the research has been reported piecemeal. This paper aggregates data from three studies, including data previously unreported, to holistically examine the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective By aggregating the data, we provide more statistical power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size for subgoal labels in programming instruction. Method We use descriptive and inferential statistics to examine the data collected from different student populations and different classroom instructional designs. We concentrate on the effect size across samples of the intervention for generalization. Findings Students using two variations of subgoal labeled instructional materials perform better than the others: the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems

Effect of Implementing Subgoals in Code.org\u27s Intro to Programming Unit in Computer Science Principles

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org\u27s Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least one-third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed

Effect of Implementing Subgoals in Code.org’s Intro to Programming unit in Computer Science Principles

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org’s Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least a third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed

The Curious Case of Loops

Background and Context: Subgoal labeled worked examples are effective for teaching computing concepts, but the research to date has been reported in a piecemeal fashion. This paper aggregates data from three studies, including data that has not been previously reported upon, to examine more holistically the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective: By aggregating the data, we provide more statistical and explanatory power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size to be expected when using subgoal labels in programming instruction. Method: We use descriptive and inferential statistics to examine the data for the effect of subgoal labeled worked examples across different student populations and different classroom instructional designs. We specifically concentrate on the potential effect size across samples of the intervention for potential generalization. Findings: Two groups of students learning how to write loops using subgoal labeled instructional materials perform better than the others. The better performing groups were the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems, both with medium-large effect sizes. Implications: For educators wishing to improve student learning using subgoal labeled materials should either provide students with subgoal labels while having them practice with a wide range of practice problems or allow students to generate their own subgoal labels and practice problems within similar contexts

The Design and Evaluation of an Educational Software Development Process for First Year Computing Undergraduates

First year, undergraduate computing students experience a series of well-known challenges when learning how to design and develop software solutions. These challenges, which include a failure to engage effectively with planning solutions prior to implementation ultimately impact upon the students’ competency and their retention beyond the first year of their studies. In the software industry, software development processes systematically guide the development of software solutions through iterations of analysis, design, implementation and testing. Industry-standard processes are, however, unsuitable for novice programmers as they require prior programming knowledge. This study investigates how a researcher-designed educational software development process could be created for novice undergraduate learners, and the impact of this process on their competence in learning how to develop software solutions. Based on an Action Research methodology that ran over three cycles, this research demonstrates how an educational software development methodology (termed FRESH) and its operationalised process (termed CADET which is a concrete implementation of the FRESH methodology), was designed and implemented as an educational tool for enhancing student engagement and competency in software development. Through CADET, students were reframed as software developers who understand the value in planning and developing software solutions, and not as programmers who prematurely try to implement solutions. While there remain opportunities to further enhance the technical sophistication of the process as it is implemented in practice, CADET enabled the software development steps of analysis and design to be explicit elements of developing software solutions, rather than their more typically implicit inclusion in introductory CS courses. The research contributes to the field of computing education by exploring the possibilities of – and by concretely generating – an appropriate scaffolded methodology and process; by illustrating the use of computational thinking and threshold concepts in software development; and by providing a novel evaluation framework (termed AKM-SOLO) to aid in the continuous improvement of educational processes and courses by measuring student learning experiences and competencies