CodEval

Grading coding assignments call for a lot of work. There are numerous aspects of the code that need to be checked, such as compilation errors, runtime errors, the number of test cases passed or failed, and plagiarism. Automated grading tools for programming assignments can be used to help instructors and graders in evaluating the programming assignments quickly and easily. Creating the assignment on Canvas is again a time taking process and can be automated. We developed CodEval, which instantly grades the student assignment submitted on Canvas and provides feedback to the students. It also uploads, creates, and edits assignments, thereby making the whole experience streamlined and quick for instructors and students. It is simple to use, easily integrated with the learning management system, and has a low learning curve. This report shows the background, implementation, and results of using CodEval for programming courses

Analysis of Students' Programming Knowledge and Error Development

Programmieren zu lernen ist für viele eine große Herausforderung, da es unterschiedliche Fähigkeiten erfordert. Man muss nicht nur die Programmiersprache und deren Konzepte kennen, sondern es erfordert auch spezifisches Domänenwissen und eine gewisse Problemlösekompetenz. Wissen darüber, wie sich die Programmierkenntnisse Studierender entwickeln und welche Schwierigkeiten sie haben, kann dabei helfen, geeignete Lehrstrategien zu entwickeln. Durch die immer weiter steigenden Studierendenzahlen wird es jedoch zunehmend schwieriger für Lehrkräfte, die Bedürfnisse, Probleme und Schwierigkeiten der Studierenden zu erkennen. Das Ziel dieser Arbeit ist es, Einblick in die Entwicklung von Programmierkenntnissen der Studierenden anhand ihrer Lösungen zu Programmieraufgaben zu gewinnen. Wissen setzt sich aus sogenannten Wissenskomponenten zusammen. In dieser Arbeit fokussieren wir uns auf syntaktische Wissenskomponen, die aus abstrakten Syntaxbäumen abgeleitet werden können, und semantische Wissenskomponenten, die durch sogenannte Variablenrollen repräsentiert werden. Da Wissen an sich nicht direkt messbar ist, werden häufig Skill-Modelle verwendet, um den Kenntnissstand abzuschätzen. Jedoch hat die Programmierdomäne ihre eigenen speziellen Eigenschaften, die bei der Wahl eines geeigneten Skill-Modells berücksichtigt werden müssen. Eine der Haupteigenschaften in der Programmierung ist, dass die Wissenskomponenten nicht unabhängig voneinander sind. Aus diesem Grund schlagen wir ein dynamisches Bayesnetz (DBN) als Skill-Modell vor, da es erlaubt, diese Abhängigkeiten explizit zu modellieren. Neben derWahl eines passenden Skill-Modells, müssen auch bestimmte Meta-Parameter wie beispielsweise die Granularität der Wissenkomponenten festgelegt werden. Daher evaluieren wir, wie sich die Wahl von Meta-Parameters auf die Vorhersagequalität von Skill-Modellen auswirkt und wie diese Meta-Parameter gewählt werden sollten. Wir nutzen das DBN, um Lernkurven für jede Wissenskomponenten zu ermitteln und daraus Implikationen für die Lehre abzuleiten. Nicht nur das Wissen von Studierenden, sondern auch deren “Falsch”-Wissen ist von Bedeutung. Deswegen untersuchen wir zunächst manuell sämtliche Programmierfehler der Studierenden und bestimmen deren Häufigkeit, Dauer und Wiederkehrrate. Wir unterscheiden dabei zwischen den Fehlerkategorien syntaktisch, konzeptuell, strategisch, Nachlässigkeit, Fehlinterpretation und Domäne und schauen, wie sich die Fehler über die Zeit entwickeln. Außerdem verwenden wir k-means-Clustering um potentielle Muster in der Fehlerentwicklung zu finden. Die Ergebnisse unserer Fallstudien sind vielversprechend. Wir können zeigen, dass die Wahl der Meta-Parameter einen großen Einfluss auf die Vorhersagequalität von Modellen hat. Außerdem ist unser DBN vergleichbar leistungsstark wie andere Skill-Modelle, ist gleichzeitig aber besser zu interpretieren. Die Lernkurven der Wissenskomponenten und die Analyse der Programmierfehler liefern uns wertvolle Erkenntnisse, die der Kursverbesserung helfen können, z.B. dass die Studierenden mehr Übungsaufgaben benötigen oder mit welchen Konzepten sie Schwierigkeiten haben.Learning to program is a hard task since it involves different types of specialized knowledge. You do not only need knowledge about the programming language and its concepts, but also knowledge from the problem domain and general problem solving abilities. Knowing how students develop programming knowledge and where they struggle, may help in the development of suitable teaching strategies. However, the ever increasing number of students makes it more and more difficult for educators to identify students’ needs, problems, and deficiencies. The goal of the thesis is to gain insights into students programming knowledge development based on their solutions to programming exercises. Knowledge is composed of so called knowledge components (KCs). In this thesis, we focus on KCs on a syntactic level, which can be derived from abstract systax trees, e.g., loops, comparison, etc., and semantic level, represented by so called roles of variables. Since knowledge is not directly measurable, skill models are an often used for the estimation of knowledge. But, the programming domain has its own characteristics which have to be considered when selecting an appropriate skill model. One of the main characteristics of the programming domain are the dependencies between KCs. Hence, we propose and evaluate a Dynamic Bayesian Network (DBN) for skill modeling which allows to model that dependencies explicitly. Besides the choice of a concrete model, also certain metaparameters like, e.g., the granularity level of KCs, has to be set when designing a skill model. Therefore, we evaluate how meta-parameterization affects the prediction performance of skill models and which meta-parameters to choose. We use the DBN to create learning curves for each KC and deduce implications for teaching from them. But not only students knowledge but also their “mal-knowledge” is of importance. Therefore, we manually inspect students’ programming errors and determine the error’s frequency, duration, and re-occurrence. We distinguish between the error categories syntactic, conceptual, strategic, sloppiness, misinterpretation, and domain and analyze how the errors change over time. Moreover, we use k-means clustering to identify different patterns in the development of programming errors. The results of our case studies are promising. We show that the correct metaparameterization has a huge effect on the prediction performance of skill models. In addition, our DBN performs as well as the other skill models while providing better interpretability. The learning curves of KCs and the analysis of programming errors provide valuable information which can be used for course improvement, e.g., that students require more practice opportunities or are struggling with certain concepts.2022-02-0

Full report on challenges with learning to program and problem solve: an analysis of first year undergraduate Open University distance learning students' online discussions

Students who study problem solving and programming (in a language such as Python) at University level encounter a range of challenges, from low-level issues with code that won't compile to misconceptions about the threshold concepts and skills. The current study complements existing findings on errors, misconceptions, difficulties and challenges obtained from students after-the-fact through instruments such as questionnaires and interviews. In our study, we analysed the posts from students of a large cohort (1500) of first-year University distance learning students to an online 'Python help forum' - recording issues and discussions as the students encountered specific challenges. Posts were coded in terms of topics, and subsequently thematically grouped into python-related, problem solving/generic programming related, and module specific. This report documents the full set of topics and the statistics for each of them. We also provide examples from the forum discussions which illustrate the topics that were identified

Proceedings of The Rust-Edu Workshop

The 2022 Rust-Edu Workshop was an experiment. We wanted to gather together as many thought leaders we could attract in the area of Rust education, with an emphasis on academic-facing ideas. We hoped that productive discussions and future collaborations would result. Given the quick preparation and the difficulties of an international remote event, I am very happy to report a grand success. We had more than 27 participants from timezones around the globe. We had eight talks, four refereed papers and statements from 15 participants. Everyone seemed to have a good time, and I can say that I learned a ton. These proceedings are loosely organized: they represent a mere compilation of the excellent submitted work. I hope you’ll find this material as pleasant and useful as I have. Bart Massey 30 August 202

Membrane computing with water

We introduce water tank systems as a new class of membrane systems inspired by a decentrally controlled circulation of water or other liquids throughout cells called tanks and capillaries called pipes. To our best knowledge, this is the first proposal addressing the behavioural principle of floating and stored water for modelling of information processing in terms of membrane computing. The volume of water within a tank stands for a non-negative rational value when acting in an analogue computation or it can be interpreted in a binary manner by distinction of “(nearly) full” or “(nearly) empty”. Water tanks might be interconnected by pipes for directed transport of water. Each pipe can be equipped with valves which in turn either fully open or fully close the hosting pipe according to permanent measurements whether the filling level in a dedicated water tank exceeds a certain threshold or not. We demonstrate dedicated water tank systems together with simulation case studies: a ring oscillator for generation of clock signals and for iteratively making available amounts of water in a cyclic scheme, analogue arithmetics by implementation of addition, non-negative subtraction, division, and multiplication complemented by systems in binary mode for implementation of selected logic gates

A longitudinal analysis of pathways to computing careers: Defining broadening participation in computing (BPC) success with a rearview lens

Efforts to increase the participation of groups historically underrepresented in computing studies, and in the computing workforce, are well documented. It is a national effort with funding from a variety of sources being allocated to research in broadening participation in computing (BPC). Many of the BPC efforts are funded by the National Science Foundation (NSF) but as existing literature shows, the growth in representation of traditionally underrepresented minorities and women is not commensurate to the efforts and resources that have been directed toward this aim. Instead of attempting to tackle the barriers to increasing representation, this dissertation research tackles the underrepresentation problem by identifying what has worked (leveraging existing real-world data) to increase representation. This work studies the educational pathways of persons who have successfully transitioned into the computing workforce and identifies the common roadmaps that have contributed to retention, persistence, and success in attaining computing employment. Descriptive statistics, Logistic regression, Classification algorithms, Clustering, and Predictive analytics were employed, using the Stata statistical tool and Orange Data Mining tool on real-world data, to identify educational pathways that have resulted in successful employment outcomes for women and blacks in computing. The results of this analysis have highlighted key information that is capable of informing future “Broadening Participation in Computing” (BPC) efforts. This is because the information will enable researchers and decision makers to have a clearer picture of what educational choices have resulted in favorable outcomes for underrepresented minorities and women in computing; and consequently, researchers and decision makers would be able to more accurately target their BPC efforts to achieve optimal results. This knowledge can also be applied in career advising for young students who are trying to chart their path into computing, providing insight into alternative pathways

An Empirical Study on the Classification of Python Language Features Using Eye-Tracking

Python, currently one of the most popular programming languages, is an object-oriented language that also provides language feature support for other programmingparadigms, such as functional and procedural. It is not currently understood howsupport for multiple paradigms affects the ability of developers to comprehend thatcode. Understanding the predominant paradigm in code, and how developers classifythe predominant paradigm, can benefit future research in program comprehension asthe paradigm may factor into how people comprehend that code. Other researchersmay want to look at how the paradigms in the code interact with various code smells.To investigate how developers classify the predominant paradigm in Python code,we performed an empirical study while utilizing an eye-tracker. The goal was tosee if developers gaze at specific language features while classifying the predominantparadigm and debugging code samples. The study includes both qualitative andquantitative data from 29 Python developers, including their gaze fixations during thetasks. We observed that participants seem to confuse the functional and proceduralparadigms, possibly due to confusing terminology used in Python, though they dogaze at specific language features. Overall, participants took more time classifyingfunctional code. The predominant paradigm did not affect their ability to debug code,though they gave lower confidence ratings for functional code. Adviser: Robert Dye

An Exploration of Traditional and Data Driven Predictors of Programming Performance

This thesis investigates factors that can be used to predict the success or failure of students taking an introductory programming course. Four studies were performed to explore how aspects of the teaching context, static factors based upon traditional learning theories, and data-driven metrics derived from aspects of programming behaviour were related to programming performance. In the first study, a systematic review into the worldwide outcomes of programming courses revealed an average pass rate of 67.7\%. This was found to have not significantly changed over time, or to have differed based upon aspects of the teaching context, such as the programming language taught to students. The second study showed that many of the factors based upon traditional learning theories, such as learning styles, are context dependent, and fail to consistently predict programming performance when they are applied across different teaching contexts. The third study explored data-driven metrics derived from the programming behaviour of students. Analysing data logged from students using the BlueJ IDE, 10 new data-driven metrics were identified and validated on three independently gathered datasets. Weaker students were found to make a greater percentage of successive errors, and spend a greater percentage of their lab time resolving errors than stronger students. The Robust Relative algorithm was developed to hybridize four of the strongest data-driven metrics into a performance predictor. The novel relative scoring of students based upon how their resolve times for different types of errors compared to the resolve times of their peers, resulted in a predictor which could explain a large proportion of the variance in the performance of three independent cohorts, $R^2$ = 42.19\%, 43.65\% and 44.17\% - almost double the variance which could be explained by Jadud's Error Quotient metric. The fourth study situated the findings of this thesis within the wider literature, by applying meta-analysis techniques to statistically synthesise fifty years of conflicting research, such that the most important factors for learning programming could be identified. 482 results describing the effects of 116 factors on programming performance were synthesised and consolidated to form a six class theoretical framework. The results showed that the strongest predictors identified over the past fifty years are data-driven metrics based upon programming behaviour. Several of the traditional predictors were also found to be influential, suggesting that both a certain level of scientific maturity and self-concept are necessary for programming. Two thirds of the weakest predictors were based upon demographic and psychological factors, suggesting that age, gender, self-perceived abilities, learning styles, and personality traits have no relevance for programming performance. This thesis argues that factors based upon traditional learning theories struggle to consistently predict programming performance across different teaching contexts because they were not intended to be applied for this purpose. In contrast, the main advantage of using data-driven approaches to derive metrics based upon students' programming processes, is that these metrics are directly based upon the programming behaviours of students, and therefore can encapsulate such changes in their programming knowledge over time. Researchers should continue to explore data-driven predictors in the future

Efficient Use of Teaching Technologies with Programming Education

Learning and teaching programming are challenging tasks that can be facilitated by using different teaching technologies. Visualization systems are software systems that can be used to help students in forming proper mental models of executed program code. They provide different visual and textual cues that help student in abstracting the meaning of a program code or an algorithm. Students also need to constantly practice the skill of programming by implementing programming assignments. These can be automatically assessed by other computer programs but parts of the evaluation need to be assessed manually by teachers or teaching assistants.There are a lot of existing tools that provide partial solutions to the practical problems of programming courses: visualizing program code, assessing student programming submissions automatically or rubrics that help keeping manual assessment consistent. Taking these tools into use is not straightforward. To succeed, the teacher needs to find the suitable tools and properly integrate them into the course infrastructure supporting the whole learning process. As many programming courses are mass courses, it is a constant struggle between providing sufficient personal guidance and feedback while retaining a reasonable workload for the teacher.This work answers to the question "How can the teaching of programming be effectively assisted using teaching technologies?" As a solution, different learning taxonomies are presented from Computer Science perspective and applied to visualization examples so the examples could be used to better support deeper knowledge and the whole learning process within a programming course. Then, different parts of the assessment process of programming assignments are studied to find the best practices in supporting the process, especially when multiple graders are being used, to maintain objectivity, consistency and reasonable workload in the grading.The results of the work show that teaching technologies can be a valuable aid for the teacher to support the learning process of the students and to help in the practical organization of the course without hindering the learning results or personalized feedback the students receive from their assignments. This thesis presents new visualization categories that allow deeper cognitive development and examples on how to integrate them efficiently into the course infrastructure. This thesis also presents a survey of computer-assisted assessment tools and assessable features for teachers to use in their programming assignments. Finally, the concept of rubric-based assessment tools is introduced to facilitate the manual assessment part of programming assignments