Introductory programming in higher education: A systematic literature review

A systematic literature review was performed on 33 papers obtained from the ACM, IEEE and Sciencedirect databases, in order to understand in depth, the introductory programming discipline (CS1) in higher education. Recently published works have been covered, providing an overview of the teaching-learning process of introductory programming and enabling to find out whether the research developed by universities worldwide is in line with the proposals made by ACM/IEEE-CS group for computer courses, regarding the transition to the competency-based model. The results show that the new techniques/technologies currently used in software development, as an example of agile methodology, has influenced the teaching-learning process of CS1 together with methods such as visual programming and e-learning. The analyzed papers discuss the importance of developing not only technical, but also social skills, corroborating that methodologies used in introductory programming courses need to focus on preparing students for an increasingly competitive market, associating new skills with technical aspects.This work is supported by CIEd – Research Centre on Education, Institute of Education, University of Minh

Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code

We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code, from introductory and intermediate programming courses at the postsecondary level. This emerging technology stirs countless discussions of its potential uses (e.g., exercise generation, code explanation) as well as misuses in programming education (e.g., cheating). However, the capabilities of GPT models and their limitations to reason about and/or analyze code in educational settings have been under-explored. We evaluated several OpenAI's GPT models on formative and summative MCQ assessments from three Python courses (530 questions). We found that MCQs containing code snippets are not answered as successfully as those that only contain natural language. While questions requiring to fill-in a blank in the code or completing a natural language statement about the snippet are handled rather successfully, MCQs that require analysis and/or reasoning about the code (e.g., what is true/false about the snippet, or what is its output) appear to be the most challenging. These findings can be leveraged by educators to adapt their instructional practices and assessments in programming courses, so that GPT becomes a valuable assistant for a learner as opposed to a source of confusion and/or potential hindrance in the learning process.Comment: 12 page

Automatic subject-based contextualisation of programming assignment lists.

As programming must be learned by doing, introductory programming course learners need to solve many problems, e.g., on systems such as ’Online Judges’. However, as such courses are often compulsory for non-Computer Science (nonCS) undergraduates, this may cause difficulties to learners that do not have the typical intrinsic motivation for programming as CS students do. In this sense, contextualised assignment lists, with programming problems related to the students’ major, could enhance engagement in the learning process. Thus, students would solve programming problems related to their academic context, improving their comprehension of the applicability and importance of programming. Nonetheless, preparing these contextually personalised programming assignments for classes for different courses is really laborious and would increase considerably the instructors’/monitors’ workload. Thus, this work aims, for the first time, to the best of our knowledge, to automatically classify the programming assignments in Online Judges based on students’ academic contexts by proposing a new context taxonomy, as well as a comprehensive pipeline evaluation methodology of cutting edge competitive Natural Language Processing (NLP). Our comprehensive methodology pipeline allows for comparing state of the art data augmentation, classifiers, beside NLP approaches. The context taxonomy created contains 23 subject matters related to the non-CS majors, representing thus a challenging multi-classification problem. We show how even on this problem, our comprehensive pipeline evaluation methodology allows us to achieve an accuracy of 95.2%, which makes it possible to automatically create contextually personalised program assignments for non-CS with a minimal error rate (4.8%)

Code Complexity in Introductory Programming Courses

Instructors of introductory programming courses would benefit from having a metric for evaluating the sophistication of student code. Since introductory programming courses pack a wide spectrum of topics in a short timeframe, student code changes quickly, raising questions of whether existing software complexity metrics effectively reflect student growth as reflected in their code. We investigate code produced by over 800 students in two different Python-based CS1 courses to determine if frequently used code quality and complexity metrics (e.g., cyclomatic and Halstead complexities) or metrics based on length and syntactic complexity are more effective as a heuristic for gauging students' progress through a course. We conclude that the traditional metrics do not correlate well with time passed in the course. In contrast, metrics based on syntactic complexity and solution size correlate strongly with time in the course, suggesting that they may be more appropriate for evaluating how student code evolves in a course context.Instructors of introductory programming courses would benefit from having a metric for evaluating the sophistication of student code. Since introductory programming courses pack a wide spectrum of topics in a short timeframe, student code changes quickly, raising questions of whether existing software complexity metrics effectively reflect student growth as reflected in their code. We investigate code produced by over 800 students in two different Python-based CS1 courses to determine if frequently used code quality and complexity metrics (e.g., cyclomatic and Halstead complexities) or metrics based on length and syntactic complexity are more effective as a heuristic for gauging students' progress through a course. We conclude that the traditional metrics do not correlate well with time passed in the course. In contrast, metrics based on syntactic complexity and solution size correlate strongly with time in the course, suggesting that they may be more appropriate for evaluating how student code evolves in a course context.Peer reviewe

Computing as the 4th “R”: a general education approach to computing education

Computing and computation are increasingly pervading our lives, careers, and societies - a change driving interest in computing education at the secondary level. But what should define a "general education" computing course at this level? That is, what would you want every person to know, assuming they never take another computing course? We identify possible outcomes for such a course through the experience of designing and implementing a general education university course utilizing best-practice pedagogies. Though we nominally taught programming, the design of the course led students to report gaining core, transferable skills and the confidence to employ them in their future. We discuss how various aspects of the course likely contributed to these gains. Finally, we encourage the community to embrace the challenge of teaching general education computing in contrast to and in conjunction with existing curricula designed primarily to interest students in the field

The abstraction transition taxonomy: developing desired learning outcomes through the lens of situated cognition

We report on a post-hoc analysis of introductory programming lecture materials. The purpose of this analysis is to identify what knowledge and skills we are asking students to acquire, as situated in the activity, tools, and culture of what programmers do and how they think. The specific materials analyzed are the 133 Peer Instruction questions used in lecture to support cognitive apprenticeship -- honoring the situated nature of knowledge. We propose an Abstraction Transition Taxonomy for classifying the kinds of knowing and practices we engage students in as we seek to apprentice them into the programming world. We find students are asked to answer questions expressed using three levels of abstraction: English, CS Speak, and Code. Moreover, many questions involve asking students to transition between levels of abstraction within the context of a computational problem. Finally, by applying our taxonomy in classifying a range of introductory programming exams, we find that summative assessments (including our own) tend to emphasize a small range of the skills fostered in students during the formative/apprenticeship phase

A Study of Non-computing Majors\u27 Growth Mindset, Self-efficacy and Perceived CS Relevance in CS1

As the demand for programming skills in today’s job market is rapidly increasing for disciplines outside of computing, CS courses have experienced spikes in enrollment for non-majors. Students in disciplines including art, design and biological sciences are now often required to take introductory CS courses. Previous research has shown the role of growth mindset, self-efficacy and relevance in student success within CS but such metrics are largely unknown for non-majors. In this thesis, we surveyed non-majors in CS1 at Cal Poly, San Luis Obispo during the early and late weeks of the quarter to gain insights on their growth mindset, their self-efficacy and the perceived relevance of the course to their lives. In our analysis, we discovered that non-majors’ levels of growth mindset and of self-efficacy decreased throughout the duration of CS1 with additional differences by gender. However, non-majors largely found that the material covered in CS1 was highly relevant to their academic and professional careers despite being challenged by it. These findings provide important insights into the experiences of non-majors learning to code and can help better serve a more diverse population of students

Contemporary developments in teaching and learning introductory programming: Towards a research proposal

The teaching and learning of introductory programming in tertiary institutions is problematic. Failure rates are high and the inability of students to complete small programming tasks at the completion of introductory units is not unusual. The literature on teaching programming contains many examples of changes in teaching strategies and curricula that have been implemented in an effort to reduce failure rates. This paper analyses contemporary research into the area, and summarises developments in the teaching of introductory programming. It also focuses on areas for future research which will potentially lead to improvements in both the teaching and learning of introductory programming. A graphical representation of the issues from the literature that are covered in the document is provided in the introduction

Dual Modality Code Explanations for Novices: Unexpected Results

The research in both cognitive load theory and multimedia principles for learning indicates presenting information using both diagrams and accompanying audio explanations yields better learning performance than using diagrams with text explanations. While this is a common practice in introductory programming courses, often called live coding, it has yet to be empirically tested. This paper reports on an experiment to determine if auditory explanations of code result in improved learning performance over written explanations. Students were shown videos explaining short code segments one of three ways: text only explanations, auditory only explanations, or both text and auditory explanations, thus replicating experiments from other domains. The results from this study do not support the findings from other disciplines and we offer explanations for why this may be the case