Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Investigating novice programming mistakes: educator beliefs vs. student data

Educators often form opinions on which programming mistakes novices make most often - for example, in Java: "they always confuse equality with assignment", or "they always call methods with the wrong types". These opinions are generally based solely on personal experience. We report a study to determine if programming educators form a consensus about which Java programming mistakes are the most common. We used the Blackbox data set to check whether the educators' opinions matched data from over 100,000 students - and checked whether this agreement was mediated by educators' experience. We found that educators formed only a weak consensus about which mistakes are most frequent, that their rankings bore only a moderate correspondence to the students in the Blackbox data, and that educators' experience had no effect on this level of agreement. These results raise questions about claims educators make regarding which errors students are most likely to commit

Programming Process, Patterns and Behaviors: Insights from Keystroke Analysis of CS1 Students

With all the experiences and knowledge, I take programming as granted. But learning to program is still difficult for a lot of introductory programming students. This is also one of the major reasons for a high attrition rate in CS1 courses. If instructors were able to identify struggling students then effective interventions can be taken to help them. This thesis is a research done on programming process data that can be collected non-intrusively from CS1 students when they are programming. The data and their findings can be leveraged in understanding students’ thought process, detecting patterns and identifying behaviors that could possibly help instructors to identify struggling students, help them and design better courses

Evaluating neural networks as a method for identifying students in need of assistance

© 2017 ACM. Course instructors need to be able to identify students in need of assistance as early in the course as possible. Recent work has suggested that machine learning approaches applied to snapshots of small programming exercises may be an effective solution to this problem. However, these results have been obtained using data from a single institution, and prior work using features extracted from student code has been highly sensitive to differences in context. This work provides two contributions: first, a partial reproduction of previously published results, but in a different context, and second, an exploration of the efficacy of neural networks in solving this problem. Our findings confirm the importance of two features (the number of steps required to solve a problem and the correctness of key problems), indicate that machine learning techniques are relatively stable across contexts (both across terms in a single course and across courses), and suggest that neural network based approaches are as effective as the best Bayesian and decision tree methods. Furthermore, neural networks can be tuned to be reliably pessimistic, so they may serve a complementary role in solving the problem of identifying students who need assistance

Predicting Performance in an Introductory Programming Course by Logging and Analyzing Student Programming Behavior

The high failure rates of many programming courses means there is a need to identify struggling students as early as possible. Prior research has focused upon using a set of tests to assess the use of a student's demographic, psychological and cognitive traits as predictors of performance. But these traits are static in nature, and therefore fail to encapsulate changes in a student's learning progress over the duration of a course. In this paper we present a new approach for predicting a student's performance in a programming course, based upon analyzing directly logged data, describing various aspects of their ordinary programming behavior. An evaluation using data logged from a sample of 45 programming students at our University, showed that our approach was an excellent early predictor of performance, explaining 42.49% of the variance in coursework marks - double the explanatory power when compared to the closest related technique in the literature

Syntax errors identification from compiler error messages using ML techniques

Compiler error messages facilitate software development and debugging by providing cause and location of the error but due to various compiler bugs and inconsistencies it often fails its purpose and negatively affect performance of both novice and experienced programmers. An errant semicolon or brace can result in many errors reported throughout the program. This study tries to statistically analyze open source code base to predict real errors from different type of compiler error messages. It also tries to auto-fix these errors. At the high level, this study handles two cases (1) when one error is present in code, (2) when two different errors are present in the code. We start with collecting different type of random error messages for both the cases by random error generation in C projects. We developed different models using document clustering, probabilistic topic modeling and multi-label classification algorithms for training and predicting real errors using collected error messages for both the cases. Our empirical evaluation on open-source projects has shown that our model correctly predicts the real error in almost 95% cases, when only one error exists in program. In case of two errors, model correctly predicts at least one error in almost 91% cases and both the errors in almost 39% cases

Software Engineering in the IoT Context: Characteristics, Challenges, and Enabling Strategies

L'abstract è presente nell'allegato / the abstract is in the attachmen

Understanding novice programmer behavior on introductory courses - Learning analytics approach

It is not easy to learn programming. This is why increasing theoretical and practical knowledge in programming education benefits both the educators as well as the students. To allow the students to gain maximal benefit from their studies, the educator must be able to recognize the students who are struggling with learning programming. Learning analytics provides a possible solution to this problem. This thesis demonstrates a novel method to model programmer behavior by using Markov Models. Programming fulfills the Markov property, because the success of the next attempt to compile or execute code is not influenced by the previous attempts; only by the current skill level of the programmer. The model is built using a state machine, which consists of states representing the different phases of the programming process. The state machine contains eight different states and 29different state transition possibilities. A Markov chain corresponding to a specific student can be computed using this state machine and then used with, for example machine learning algorithms. The data for this thesis was collected from a total of five different introductory programming courses, which used either the Java or Python programming languages. The dataset contains 1174 unique students, who made 544 835 total submissions to411 unique assignments. All programming courses were given in Turku, during2017-2021.This thesis provides a theoretical basis for modeling students (Markov Models) and offers a practical method to model students using Markov Models. This thesis only applies unsupervised machine learning methods to the data, specifically the K-Means clustering algorithm. However, supervised methods may also be used. The usefulness of the model is demonstrated by clustering students into three statistically similar clusters: students who perform well, average and poorly. The model is also applied to recognize the programming language used, based only on the transitions within the state machine.--- Ohjelmoinnin oppiminen ei ole helppoa. Tästä syystä ohjelmoinnin opetuksenteoreettinen ja käytännön edistäminen hyödyttää paitsi nykyisin ohjelmointia opettavia, myös opiskelijoita. Jotta opiskelijat voivat saavuttaa maksimaalisenhyödyn opiskelustaan, opettajan täytyy voida tunnistaa ne opiskelijat, joille ohjelmoinnin opiskelu tuottaa hankaluuksia. Oppimisanalytiikka tarjoaa tähän mahdollisuuden. Tämä väitöskirja esittelee tavan mallintaa ohjelmoinnin opiskelijoidenkäyttäytymistä käyttämällä Markovin malleja. Ohjelmoijan käyttäytyminen toteuttaa Markovin ominaisuuden, sillä ohjelmoijan koodin ajoyrityksen onnistumiseen vaikuttaa ainoastaan ohjelmoijan senhetkinen taitotaso; aikaisemmilla yrityksillä ei ole vaikutusta tuleviin kertoihin. Malli rakennetaan käyttämällä tilakonetta, jonka jokainen tila vastaa ohjelmointiprosessin vaihetta. Tilakoneessa on yhteensä kahdeksan eri tilaa ja 29 erilaista tilan muutosmahdollisuutta. Tilakoneesta lasketaan opiskelijaa vastaava Markovin ketju, mitä voidaan käyttää esimerkiksi koneoppimisalgoritmien kanssa. Dataa tähän väitöskirjaan kerättiin yhteensä viidestä ohjelmoinninperuskurssista, joissa käytettiin joko Java- tai Python-ohjelmointikieltä. Opiskelijoita kursseilla oli yhteensä 1174. Opiskelijat tekivät yhteensä 544-835 ohjelmointitehtävän palautusta 411 ohjelmointitehtävään. Kaikki ohjelmointikurssit pidettiin Turussa, vuosina 2017-2021 Tämä väitöskirja tarjoaa teoreettisen pohjan ohjelmoinnin opiskelijoidenmallintamiseen (Markovin mallit) ja tarjoaa menetelmän, jolla Markovin malleja käyttämällä voi mallintaa ohjelmoinnin opiskelijoita. Malliin sovelletaan vain ohjaamattomia koneoppimismenetelmiä, erityisesti K-Means clustering -algoritmia. Tässä väitöskirjassa osoitan myös teoreettisen mallin muutamia käytännönsovelluksia luokittelemalla opiskelijoita samoja ominaisuuksia sisältäviin luokkiin. Malli opetetaan erottelemaan opiskelijat kolmeen ryhmään: hyvin, keskiverrosti ja huonosti pärjääviin. Mallia sovelletaan onnistuneesti myös tunnistamaan käytetty ohjelmointikieli käyttämällä vain tilakoneen tilasiirtymiä

Novice Java Programming Mistakes: Large-Scale Data vs. Educator Beliefs

Teaching is the process of conveying knowledge and skills to learners. It involves preventing misunderstandings or correcting misconceptions that learners have acquired. Thus, effective teaching relies on solid knowledge of the discipline, but also a good grasp of where learners are likely to trip up or misunderstand. In programming, there is much opportunity for misunderstanding, and the penalties are harsh: failing to produce the correct syntax for a program, for example, can completely prevent any progress in learning how to program. Because programming is inherently computer-based, we have an opportunity to automatically observe programming behaviour -- more closely even than an educator in the room at the time. By observing students' programming behaviour, and surveying educators, we can ask: do educators have an accurate understanding of the mistakes that students are likely to make? In this study, we combined two years of the Blackbox dataset (with more than 900 thousand users and almost 100 million compilation events) with a survey of 76 educators to investigate which mistakes students make while learning to program Java, and whether the educators could make an accurate estimate of which mistakes were most common. We find that educators' estimates do not agree with one another or the student data, and discuss the implications of these results