Let's Ask Students About Their Programs, Automatically

Students sometimes produce code that works but that its author does not comprehend. For example, a student may apply a poorly-understood code template, stumble upon a working solution through trial and error, or plagiarize. Similarly, passing an automated functional assessment does not guarantee that the student understands their code. One way to tackle these issues is to probe students' comprehension by asking them questions about their own programs. We propose an approach to automatically generate questions about student-written program code. We moreover propose a use case for such questions in the context of automatic assessment systems: after a student's program passes unit tests, the system poses questions to the student about the code. We suggest that these questions can enhance assessment systems, deepen student learning by acting as self-explanation prompts, and provide a window into students' program comprehension. This discussion paper sets an agenda for future technical development and empirical research on the topic

Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests

Background and Context: Over the past year, large language models (LLMs) have taken the world by storm. In computing education, like in other walks of life, many opportunities and threats have emerged as a consequence. Objectives: In this article, we explore such opportunities and threats in a specific area: responding to student programmers' help requests. More specifically, we assess how good LLMs are at identifying issues in problematic code that students request help on. Method: We collected a sample of help requests and code from an online programming course. We then prompted two different LLMs (OpenAI Codex and GPT-3.5) to identify and explain the issues in the students' code and assessed the LLM-generated answers both quantitatively and qualitatively. Findings: GPT-3.5 outperforms Codex in most respects. Both LLMs frequently find at least one actual issue in each student program (GPT-3.5 in 90% of the cases). Neither LLM excels at finding all the issues (GPT-3.5 finding them 57% of the time). False positives are common (40% chance for GPT-3.5). The advice that the LLMs provide on the issues is often sensible. The LLMs perform better on issues involving program logic rather than on output formatting. Model solutions are frequently provided even when the LLM is prompted not to. LLM responses to prompts in a non-English language are only slightly worse than responses to English prompts. Implications: Our results continue to highlight the utility of LLMs in programming education. At the same time, the results highlight the unreliability of LLMs: LLMs make some of the same mistakes that students do, perhaps especially when formatting output as required by automated assessment systems. Our study informs teachers interested in using LLMs as well as future efforts to customize LLMs for the needs of programming education.Comment: 13 pages, 1 figure. To be published in Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 (ICER '23 V1

Conceptions and Misconceptions about Computational Thinking among Italian Primary School Teachers

International audienceMany advanced countries are recognizing more and more the importance of teaching computing, in some cases even as early as in primary school. "Computational thinking" is the term often used to denote the conceptual core of computer science or "the way a computer scientist thinks", as Wing put it. Such term - given also the lack of a widely accepted definition - has become a "buzzword" meaning different things to different people. We investigated the Italian primary school teachers' conceptions about computational thinking by analyzing the results of a survey (N=972) conducted in the context of "Programma il Futuro" project. Teachers have been asked to provide a definition of computational thinking and to answer three additional related closed-ended questions. The analysis shows that, while almost half of teachers (43.4%) have included in their definitions some fundamental elements of computational thinking, very few (10.8%) have been able to provide an acceptably complete definition. On a more positive note, the majority is aware that computational thinking is not characterized by coding or by the use of information technology

Fostering Program Comprehension in Novice Programmers - Learning Activities and Learning Trajectories

This working group asserts that Program Comprehension (ProgComp) plays a critical part in the process of writing programs. For example, this paper is written from a basic draft that was edited and revised until it clearly presented our idea. Similarly, a program is written incrementally, with each step tested, debugged and extended until the program achieves its goal. Novice programmers should develop program comprehension skills as they learn to code so that they are able both to read and reason about code created by others, and to reflect on their code when writing, debugging or extending it. To foster such competencies our group identified two main goals: (g1) to collect and define learning activities that explicitly address key components of program comprehension and (g2) to define tentative theoretical learning trajectories that will guide teachers as they select and sequence those learning activities in their CS0/CS1/CS2 or K-12 courses. The WG has completed the first goal and laid down a strong foundation towards the second goal as presented in this report. After a thorough literature review, a detailed description of the Block Model is provided, as this model has been used with a dual purpose, to classify and present an extensive list of ProgComp tasks, and to describe a possible learning trajectory for a complex task, covering different cells of the Block Model matrix. The latter is intended to help instructors to decompose complex tasks and identify which aspects of ProgComp are being fostered

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Normal growth and intravascular volume status with good metabolic control during peritoneal dialysis in infancy

Peer reviewe

Ox - työkalu Java-ohjelmointiharjoitusten testaukseen

Ohjelmien testaaminen on tärkeä osa aloittelevankin ohjelmoijan kompetenssia. Testaaminen tuleekin ottaa huomioon jo ensimmäiseltä ohjelmoinnin alkeiskurssilta lähtien. Vahvasti olio-ohjelmointiin nojautuvalla ohjelmoinnin alkeiskurssilla testauksen liittäminen luontevaksi osaksi ohjelmointiprosessia ei kuitenkaan ole aivan helppoa. Harjoitustehtävissä laadittavien luokkien koekäyttö edellyttää käyttöliittymää, mutta käyttöliittymäohjelmointi on opiskelijoille aluksi vaivalloista ja mahdollisesti jopa oppimisen kannalta haitallista. Myös yksikkötestaukseen tarkoitetut kehysohjelmistot ovat aloittelijalle turhan monimutkaisia. Tässä työssä tehdään katsaus menettelytapoihin ja apuohjelmiin, jotka sopivat olio- ohjelmointilähtöisillä ohjelmoinnin alkeiskursseilla harjoitustehtävien ratkaisuiksi kirjoitettujen Java-kielisten luokkien koekäyttöön testausmielessä. Uutena ratkaisuna toteutetaan Ox-niminen ohjelmisto, joka mahdollistaa luokkien testaamisen vaivattomasti ja soveltuu ominaisuuksiltaan hyvin opiskelijamääriltään suurille ohjelmoinnin alkeiskursseille. Oxia on koekäytetty alkeiskursseilla Teknillisessä korkeakoulussa. Opettajien havainnot ja opiskelijapalaute puhuvat sen puolesta, että työkalu istuu hyvin kyseisille kursseille

A Roles-Based Approach to Variable-Oriented Programming

Delocalized variable plans pose problems for novice programmers trying to read and write programs. Variable-oriented programming is a programming paradigm that emphasizes the importance of variable-related plans, and localizes actions pertaining to each variable together in one place in the program code. This paper revisits the idea of variable-oriented programming and shows how it can be founded on roles of variables: stereotypes of variable use suitable for teaching to novices. The paper sketches out how variable-oriented, roles-based programming could be implemented using either a new programming language or a framework built on an existing language. The possible applications, merits, and problems of a roles-based approach, and variable-oriented programming in general, are discussed. This paper points toward possible research directions for the future and provides a basis for further discussions of variable-oriented, roles-based programming