30 research outputs found
Let's Ask Students About Their Programs, Automatically
Students sometimes produce code that works but that its author does not
comprehend. For example, a student may apply a poorly-understood code template,
stumble upon a working solution through trial and error, or plagiarize.
Similarly, passing an automated functional assessment does not guarantee that
the student understands their code. One way to tackle these issues is to probe
students' comprehension by asking them questions about their own programs. We
propose an approach to automatically generate questions about student-written
program code. We moreover propose a use case for such questions in the context
of automatic assessment systems: after a student's program passes unit tests,
the system poses questions to the student about the code. We suggest that these
questions can enhance assessment systems, deepen student learning by acting as
self-explanation prompts, and provide a window into students' program
comprehension. This discussion paper sets an agenda for future technical
development and empirical research on the topic
Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests
Background and Context: Over the past year, large language models (LLMs) have
taken the world by storm. In computing education, like in other walks of life,
many opportunities and threats have emerged as a consequence.
Objectives: In this article, we explore such opportunities and threats in a
specific area: responding to student programmers' help requests. More
specifically, we assess how good LLMs are at identifying issues in problematic
code that students request help on.
Method: We collected a sample of help requests and code from an online
programming course. We then prompted two different LLMs (OpenAI Codex and
GPT-3.5) to identify and explain the issues in the students' code and assessed
the LLM-generated answers both quantitatively and qualitatively.
Findings: GPT-3.5 outperforms Codex in most respects. Both LLMs frequently
find at least one actual issue in each student program (GPT-3.5 in 90% of the
cases). Neither LLM excels at finding all the issues (GPT-3.5 finding them 57%
of the time). False positives are common (40% chance for GPT-3.5). The advice
that the LLMs provide on the issues is often sensible. The LLMs perform better
on issues involving program logic rather than on output formatting. Model
solutions are frequently provided even when the LLM is prompted not to. LLM
responses to prompts in a non-English language are only slightly worse than
responses to English prompts.
Implications: Our results continue to highlight the utility of LLMs in
programming education. At the same time, the results highlight the
unreliability of LLMs: LLMs make some of the same mistakes that students do,
perhaps especially when formatting output as required by automated assessment
systems. Our study informs teachers interested in using LLMs as well as future
efforts to customize LLMs for the needs of programming education.Comment: 13 pages, 1 figure. To be published in Proceedings of the 2023 ACM
Conference on International Computing Education Research V.1 (ICER '23 V1
Conceptions and Misconceptions about Computational Thinking among Italian Primary School Teachers
International audienceMany advanced countries are recognizing more and more the importance of teaching computing, in some cases even as early as in primary school. "Computational thinking" is the term often used to denote the conceptual core of computer science or "the way a computer scientist thinks", as Wing put it. Such term - given also the lack of a widely accepted definition - has become a "buzzword" meaning different things to different people. We investigated the Italian primary school teachers' conceptions about computational thinking by analyzing the results of a survey (N=972) conducted in the context of "Programma il Futuro" project. Teachers have been asked to provide a definition of computational thinking and to answer three additional related closed-ended questions. The analysis shows that, while almost half of teachers (43.4%) have included in their definitions some fundamental elements of computational thinking, very few (10.8%) have been able to provide an acceptably complete definition. On a more positive note, the majority is aware that computational thinking is not characterized by coding or by the use of information technology
Fostering Program Comprehension in Novice Programmers - Learning Activities and Learning Trajectories
This working group asserts that Program Comprehension (ProgComp) plays a critical part in the process of writing programs. For example, this paper is written from a basic draft that was edited and revised until it clearly presented our idea. Similarly, a program is written incrementally, with each step tested, debugged and extended until the program achieves its goal. Novice programmers should develop program comprehension skills as they learn to code so that they are able both to read and reason about code created by others, and to reflect on their code when writing, debugging or extending it. To foster such competencies our group identified two main goals: (g1) to collect and define learning activities that explicitly address key components of program comprehension and (g2) to define tentative theoretical learning trajectories that will guide teachers as they select and sequence those learning activities in their CS0/CS1/CS2 or K-12 courses. The WG has completed the first goal and laid down a strong foundation towards the second goal as presented in this report. After a thorough literature review, a detailed description of the Block Model is provided, as this model has been used with a dual purpose, to classify and present an extensive list of ProgComp tasks, and to describe a possible learning trajectory for a complex task, covering different cells of the Block Model matrix. The latter is intended to help instructors to decompose complex tasks and identify which aspects of ProgComp are being fostered
Introductory programming: a systematic literature review
As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming.
This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research
Normal growth and intravascular volume status with good metabolic control during peritoneal dialysis in infancy
Peer reviewe
Ox - työkalu Java-ohjelmointiharjoitusten testaukseen
Ohjelmien testaaminen on tärkeä osa aloittelevankin ohjelmoijan kompetenssia.
Testaaminen tuleekin ottaa huomioon jo ensimmäiseltä ohjelmoinnin alkeiskurssilta lähtien.
Vahvasti olio-ohjelmointiin nojautuvalla ohjelmoinnin alkeiskurssilla testauksen liittäminen luontevaksi osaksi ohjelmointiprosessia ei kuitenkaan ole aivan helppoa.
Harjoitustehtävissä laadittavien luokkien koekäyttö edellyttää käyttöliittymää, mutta käyttöliittymäohjelmointi on opiskelijoille aluksi vaivalloista ja mahdollisesti jopa oppimisen kannalta haitallista.
Myös yksikkötestaukseen tarkoitetut kehysohjelmistot ovat aloittelijalle turhan monimutkaisia.
Tässä työssä tehdään katsaus menettelytapoihin ja apuohjelmiin, jotka sopivat olio- ohjelmointilähtöisillä ohjelmoinnin alkeiskursseilla harjoitustehtävien ratkaisuiksi kirjoitettujen Java-kielisten luokkien koekäyttöön testausmielessä.
Uutena ratkaisuna toteutetaan Ox-niminen ohjelmisto, joka mahdollistaa luokkien testaamisen vaivattomasti ja soveltuu ominaisuuksiltaan hyvin opiskelijamääriltään suurille ohjelmoinnin alkeiskursseille.
Oxia on koekäytetty alkeiskursseilla Teknillisessä korkeakoulussa.
Opettajien havainnot ja opiskelijapalaute puhuvat sen puolesta, että työkalu istuu hyvin kyseisille kursseille
A Roles-Based Approach to Variable-Oriented Programming
Delocalized variable plans pose problems for novice programmers trying to
read and write programs. Variable-oriented programming is a programming paradigm
that emphasizes the importance of variable-related plans, and localizes actions
pertaining to each variable together in one place in the program code. This paper
revisits the idea of variable-oriented programming and shows how it can be founded on
roles of variables: stereotypes of variable use suitable for teaching to novices. The paper
sketches out how variable-oriented, roles-based programming could be implemented
using either a new programming language or a framework built on an existing language.
The possible applications, merits, and problems of a roles-based approach, and
variable-oriented programming in general, are discussed. This paper points toward
possible research directions for the future and provides a basis for further discussions of
variable-oriented, roles-based programming