Subgoals Help Students Solve Parsons Problems

We report on a study that used subgoal labels to teach students how to write while loops with a Parsons problem learning assessment. Subgoal labels were used to aid learning of programming while not overloading students\u27 cognitive abilities. We wanted to compare giving learners subgoal labels versus asking learners to generate subgoal labels. As an assessment for learning we asked students to solve a Parsons problem – to place code segments in the correct order. We found that students who were given subgoal labels performed statistically better than the groups that did not receive subgoal labels or were asked to generate subgoal labels. We conclude that a low cognitive load assessment, Parsons problems, can be more sensitive to student learning gains than traditional code generation problems

The Curious Case of Loops

Background and Context: Subgoal labeled worked examples are effective for teaching computing concepts, but the research to date has been reported in a piecemeal fashion. This paper aggregates data from three studies, including data that has not been previously reported upon, to examine more holistically the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective: By aggregating the data, we provide more statistical and explanatory power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size to be expected when using subgoal labels in programming instruction. Method: We use descriptive and inferential statistics to examine the data for the effect of subgoal labeled worked examples across different student populations and different classroom instructional designs. We specifically concentrate on the potential effect size across samples of the intervention for potential generalization. Findings: Two groups of students learning how to write loops using subgoal labeled instructional materials perform better than the others. The better performing groups were the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems, both with medium-large effect sizes. Implications: For educators wishing to improve student learning using subgoal labeled materials should either provide students with subgoal labels while having them practice with a wide range of practice problems or allow students to generate their own subgoal labels and practice problems within similar contexts

The curious case of loops

Background and Context Subgoal labeled worked examples have been extensively researched, but the research has been reported piecemeal. This paper aggregates data from three studies, including data previously unreported, to holistically examine the effect of subgoal labeled worked examples across three student populations and across different instructional designs. Objective By aggregating the data, we provide more statistical power for somewhat surprising yet replicable results. We discuss which results generalize across populations, focusing on a stable effect size for subgoal labels in programming instruction. Method We use descriptive and inferential statistics to examine the data collected from different student populations and different classroom instructional designs. We concentrate on the effect size across samples of the intervention for generalization. Findings Students using two variations of subgoal labeled instructional materials perform better than the others: the group that was given the subgoal labels with farther transfer between worked examples and practice problems and the group that constructed their own subgoal labels with nearer transfer between worked examples and practice problems

Learning Loops: A Replication Study Illuminates Impact of HS Courses

A recent study about the effectiveness of subgoal labeling in an introductory computer science programming course both supported previous research and produced some puzzling results. In this study, we replicate the experiment with a different student population to determine if the results are repeatable. We also gave the experimental task to students in a follow-on course to explore if they had indeed mastered the programming concept. We found that the previous puzzling results were repeated. In addition, for the novice programmers, we found a statistically significant difference in performance based on whether the student had previous programming courses in high school. However, this performance difference disappears in a follow-on course after all students have taken an introductory computer science programming course. The results of this study have implications for how quickly students are evaluated for mastery of knowledge and how we group students in introductory programming courses

Learning Loops: A Replication Study Illuminates Impact of HS Courses

A recent study about the effectiveness of subgoal labeling in an introductory computer science programming course both supported previous research and produced some puzzling results. In this study, we replicate the experiment with a different student population to determine if the results are repeatable. We also gave the experimental task to students in a follow-on course to explore if they had indeed mastered the programming concept. We found that the previous puzzling results were repeated. In addition, for the novice programmers, we found a statistically significant difference in performance based on whether the student had previous programming courses in high school. However, this performance difference disappears in a follow-on course after all students have taken an introductory computer science programming course. The results of this study have implications for how quickly students are evaluated for mastery of knowledge and how we group students in introductory programming courses

Effect of Implementing Subgoals in Code.org\u27s Intro to Programming Unit in Computer Science Principles

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org\u27s Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least one-third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed

Effect of Implementing Subgoals in Code.org’s Intro to Programming unit in Computer Science Principles

The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org’s Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least a third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed

How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment

As Large Language Models (LLMs) gain in popularity, it is important to understand how novice programmers use them. We present a thematic analysis of 33 learners, aged 10-17, independently learning Python through 45 code-authoring tasks using Codex, an LLM-based code generator. We explore several questions related to how learners used these code generators and provide an analysis of the properties of the written prompts and the generated code. Specifically, we explore (A) the context in which learners use Codex, (B) what learners are asking from Codex, (C) properties of their prompts in terms of relation to task description, language, and clarity, and prompt crafting patterns, (D) the correctness, complexity, and accuracy of the AI-generated code, and (E) how learners utilize AI-generated code in terms of placement, verification, and manual modifications. Furthermore, our analysis reveals four distinct coding approaches when writing code with an AI code generator: AI Single Prompt, where learners prompted Codex once to generate the entire solution to a task; AI Step-by-Step, where learners divided the problem into parts and used Codex to generate each part; Hybrid, where learners wrote some of the code themselves and used Codex to generate others; and Manual coding, where learners wrote the code themselves. The AI Single Prompt approach resulted in the highest correctness scores on code-authoring tasks, but the lowest correctness scores on subsequent code-modification tasks during training. Our results provide initial insight into how novice learners use AI code generators and the challenges and opportunities associated with integrating them into self-paced learning environments. We conclude with various signs of over-reliance and self-regulation, as well as opportunities for curriculum and tool development.Comment: 12 pages, Peer-Reviewed, Accepted for publication in the proceedings of the 2023 ACM Koli Calling International Conference on Computing Education Researc

ne-Course for Learning Programming

Difficulties in learning programming are a constant concern in engineering courses. In many research studies involving the learning programming must of the solutions presented, from the beginning of the first programming languages, was to apply different type of problems analysis. Literature relating to the understanding of nature of learning programming skills has been focused explicitly on the teaching methodology and few of them focus on abilities, characteristics and knowledge acquired over the life cycle of learning programming in each student. Most of the students enrolled in engineering courses, where programming is a crucial competence, never had the opportunity to develop skills of computational thinking. In this paper, we focus our work on the learning programming developing and applying a set of exercises where students with more difficulties can express and develop their skills in computational thinking. In order to understand some programming students difficulties we have create a set of exercises, and apply it to a pre-programming course, that allows teachers to understand how students analyse and comprehend aspects such as visualization, spatial interpretation and physical manipulation. This paper also reports on results obtained from a class experiment where Memory Transfer Language was used by students to learn programming. All the exercises must be resolved without any type of technology, designed as a ne-course (no electronic course) for learning programming