Recommender Systems and Scratch: An integrated approach for enhancing computer programming learning

Learning computer programming is a challenging process. Among the current approaches for overcoming this challenge, visual programming languages (VPLs), such as Scratch, have shown very promising results for beginners. Interestingly, some higher education institutions have started to use VPLs to introduce basic programming concepts, mainly in CS1 courses. However, an important issue regarding Scratchs usage in higher education environments is that students may feel unmotivated being confronted by programming exercises that do not fulfill their individual expectations. To try and overcome this barrier, we propose CARAMBA, a Scratch extension including an exercise recommender system. Based on features, such as taste and complexity, CARAMBA is able to personalize student learning with Scratch by suitably suggesting exercises for students. An in-depth evaluation was conducted about the effects of our proposal on both the learning of basic concepts of CS1 and the overall performance of students. We adopted an equivalent pretest-posttest design with 88 college students at an Ecuadorian university. Results confirm that recommending exercises in Scratch had a positive effect on students programming learning abilities in terms of pass rates. In totality, the pass rate achieved by our proposal was over 52%, which is 8% higher than the rate achieved during a previous experience using only Scratch (without recommendation) and 21% higher than the historical results of traditional teaching (without Scratch). Furthermore, we analyzed the degree of exploitation of CARAMBA by students to portray two facts: students actually used CARAMBA and there was a significant, positive correlation between the utilization of CARAMBA and the scores obtained by the students

Computational Thinking and Literacy

Today’s students will enter a workforce that is powerfully shaped by computing. To be successful in a changing economy, students must learn to think algorithmically and computationally, to solve problems with varying levels of abstraction. These computational thinking skills have become so integrated into social function as to represent fundamental literacies. However, computer science has not been widely taught in K-12 schools. Efforts to create computer science standards and frameworks have yet to make their way into mandated course requirements. Despite a plethora of research on digital literacies, research on the role of computational thinking in the literature is sparse. This conceptual paper proposes a three dimensional framework for exploring the relationship between computational thinking and literacy through: 1) situating computational thinking in the literature as a literacy; 2) outlining mechanisms by which students’ existing literacy skills can be leveraged to foster computational thinking; and 3) elaborating ways in which computational thinking skills facilitate literacy development

Impact of Scratch on the achievements of first-year computer science students in programming in some Nigerian polytechnics

To support the advancement of modern civilisation, our institutions of higher learning must produce the right pool of professionals, who can develop innovative software. However, the teaching and learning of the first programming language (CS1) remains a great challenge for most educators and novice computer students. Indicators such as failure and attrition rates, and CS1 student engagement, continue to show that conventional pedagogy does not adequately meet the needs of some beginning CS students. For its ease in introducing novices to programming, Scratch—a visual programming environment following the constructionism philosophy of Seymour Papert—is now employed even in some higher education CS1 classes with mixed evidence of its impact. Scratch captures the constructionist agenda by its slogan: “Imagine, Program, Share.” Therefore, this study explored the impart of using a constructionist Scratch programming pedagogy on higher education CS1 students’ achievements. This study also sought to compare the impacts of the two CS1 modes: the conventional class - involving textual programming language, lectures and labs, and the constructionist Scratch inquiry-based programming class. It further aims to discover if gender, academic level, age, prior programming, and visual artistic abilities moderate the effects of programming pedagogy on students’ achievements. To realize the study’s aims, the study employed a quasi-experimental pretest-posttest nonequivalent groups design, involving four intact CS1 classes of polytechnic students (N = 418) in north-central Nigeria. The investigation was conducted in phases: a pilot (n = 236) and main (n=182) studies lasting two academic sessions, with each study comprising one experimental and one control group. In each session, learning in both modes lasted for six weeks. In both studies, purposive sampling was employed to select institutions, and selected institutions were randomly assigned to treatment groups. Instruments employed included CS1 Student Profile Questionnaire (CSPROQ) and Introductory Programming Achievement Test (IPAT). To strengthen the research design, I employed Coarsened Exact Matching (CEM) algorithm—after conducting a priori power analysis—to generate matched random samples of cases from both studies. Thus, research data employed in the analysis include: from the pilot, 41 cases in each treatment group; from the main study, 42 cases in each treatment group. Descriptive and inferential statistics were employed to find answers to research questions and test the research hypothesis. Data from both studies satisfied the requirements for statistical tests employed, i.e., t-test and ANCOVA. The alpha level used in testing hypotheses was p = 0.05. The dependent variable is the IPAT post-test score, while the independent variables are treatment, gender, age, academic achievement level, prior programming, and prior visual art. The covariate was the IPAT pretest score. Statistical analyses were conducted using SPSS version 23. The t-test results from both pilot and main studies indicated that, both programming pedagogies had significant effects on student IPAT scores, although the effect of the constructionist Scratch intervention was higher. Results from the one-way ANCOVA analysis of both pilot and main study data—while controlling for students’ IPAT pretest scores—yielded the same outcome: There was significant main effect of treatment on students’ IPAT posttest scores, although the impact was moderate. Controlling for pre test scores, analysis of the main studies data yielded no significant main effects of: gender, age, academic level, prior programming and prior visual artistic ability. The result from the main study also reveals no interaction effect of treatment, gender, academic level, age, prior programming, and prior artistic ability. While the quality of CS1 students’ performance in each session varies as their IPAT achievements show, yet the results of this research revealed a consistent pattern: Students in the constructionist Scratch class outperformed those in the conventional class, although the impart was moderate. This finding implies college students without prior programming experience can perform better in a class following a constructionist Scratch programming pedagogy. The study recommends the use of Scratch, following a constructionist pedagogy with first-year students in colleges, especially those without prior background in programmingSchool of ComputingPh. D. (Computing Education

An Exploration of Traditional and Data Driven Predictors of Programming Performance

This thesis investigates factors that can be used to predict the success or failure of students taking an introductory programming course. Four studies were performed to explore how aspects of the teaching context, static factors based upon traditional learning theories, and data-driven metrics derived from aspects of programming behaviour were related to programming performance. In the first study, a systematic review into the worldwide outcomes of programming courses revealed an average pass rate of 67.7\%. This was found to have not significantly changed over time, or to have differed based upon aspects of the teaching context, such as the programming language taught to students. The second study showed that many of the factors based upon traditional learning theories, such as learning styles, are context dependent, and fail to consistently predict programming performance when they are applied across different teaching contexts. The third study explored data-driven metrics derived from the programming behaviour of students. Analysing data logged from students using the BlueJ IDE, 10 new data-driven metrics were identified and validated on three independently gathered datasets. Weaker students were found to make a greater percentage of successive errors, and spend a greater percentage of their lab time resolving errors than stronger students. The Robust Relative algorithm was developed to hybridize four of the strongest data-driven metrics into a performance predictor. The novel relative scoring of students based upon how their resolve times for different types of errors compared to the resolve times of their peers, resulted in a predictor which could explain a large proportion of the variance in the performance of three independent cohorts, $R^2$ = 42.19\%, 43.65\% and 44.17\% - almost double the variance which could be explained by Jadud's Error Quotient metric. The fourth study situated the findings of this thesis within the wider literature, by applying meta-analysis techniques to statistically synthesise fifty years of conflicting research, such that the most important factors for learning programming could be identified. 482 results describing the effects of 116 factors on programming performance were synthesised and consolidated to form a six class theoretical framework. The results showed that the strongest predictors identified over the past fifty years are data-driven metrics based upon programming behaviour. Several of the traditional predictors were also found to be influential, suggesting that both a certain level of scientific maturity and self-concept are necessary for programming. Two thirds of the weakest predictors were based upon demographic and psychological factors, suggesting that age, gender, self-perceived abilities, learning styles, and personality traits have no relevance for programming performance. This thesis argues that factors based upon traditional learning theories struggle to consistently predict programming performance across different teaching contexts because they were not intended to be applied for this purpose. In contrast, the main advantage of using data-driven approaches to derive metrics based upon students' programming processes, is that these metrics are directly based upon the programming behaviours of students, and therefore can encapsulate such changes in their programming knowledge over time. Researchers should continue to explore data-driven predictors in the future

Reducing Withdrawal and Failure Rates in Introductory Programming with Subgoal Labeled Worked Examples

Background: Programming a computer is an increasingly valuable skill, but dropout and failure rates in introductory programming courses are regularly as high as 50%. Like many fields, programming requires students to learn complex problem-solving procedures from instructors who tend to have tacit knowledge about low-level procedures that they have automatized. The subgoal learning framework has been used in programming and other fields to breakdown procedural problem solving into smaller pieces that novices can grasp more easily, but it has only been used in shortterm interventions. In this study, the subgoal learning framework was implemented throughout a semester-long introductory programming course to explore its longitudinal effects. Of 265 students in multiple sections of the course, half received subgoal-oriented instruction while the other half received typical instruction. Results: Learning subgoals consistently improved performance on quizzes, which were formative and given within a week of learning a new procedure, but not on exams, which were summative. While exam performance was not statistically better, the subgoal group had lower variance in exam scores and fewer students dropped or failed the course than in the control group. To better understand the learning process, we examined students’ responses to open-ended questions that asked them to explain the problem-solving process. Furthermore, we explored characteristics of learners to determine how subgoal learning affected students at risk of dropout or failure. Conclusions: Students in an introductory programming course performed better on initial assessments when they received instructions that used our intervention, subgoal labels. Though the students did not perform better than the control group on exams on average, they were less likely to get failing grades or to drop the course. Overall, subgoal labels seemed especially effective for students who might otherwise struggle to pass or complete the course

Visualising the code: a study of student engagement with programming in a distance learning context

Programming is a subject that many students find difficult and it may be particularly challenging for distance learning students working largely on their own. Many ideas have been put forward in the literature to explain why students struggle with programming, including: the relative unfamiliarity of computer programming or ‘radical novelty’ (Dijkstra, 1989), cognitive load (Shaffer, 2004) and that the whole learning environment may be influential (Scott & Ghinea, 2013). This paper reports on the first phase of a project, ‘Visualising the code’, which is investigating the impact of using a visual programming language on student engagement with programming. We used as our case-study, TU100 ‘My digital life’ which is a level 1 undergraduate Computer Science module in the Open University (UK). The rationale for this work stems from the necessity of developing an introductory undergraduate module that will engage students of widely differing prior levels of experience in terms of programming and of education generally. In TU100, the module team introduced a visual programming environment, based on Scratch (MIT, 2007), called ‘Sense’ which is used in conjunction with an electronic device, the SenseBoard. We analysed the grades of 6,159 students in the final assessment across six presentations of the module to identify student performance in the programming task, as distinct from their overall performance on the module. The aim was to explore whether there was any difference between student engagement with the programming task in comparison with non-programming tasks. Early results suggest that there is no significant difference in levels of engagement between these tasks, and it appears that success, or otherwise, in one type of task is a good predictor of engagement with the other task. There are implications for networked learning of this work, given that the learning environment encompasses: the student’s own home or other space, both printed books and digital learning materials, a programming environment linked to a physical device (i.e. the SenseBoard) and communications networks that link students to their peers and to their tutors. The learning environment also includes support through face-to-face and online tutorials and other online resources, such as forums. In the next phase of the project we will analyse the textual comments made by TU100 students in the end of module survey to evaluate their views on the visual programming environment

Retention in Introductory Programming

The introductory programming course is one of the very first courses that computer science students encounter. The course is challenging not only because of the content, but also due to the challenges related to finding a place in a new community. Many have little knowledge of what to expect from university studies, some struggle to adjust their study behavior to match the expected pace, and a few simply cannot attend instruction due to e.g. family or work constraints. As a consequence, a considerable number of students end up failing the course, or pass the course with substandard knowledge. This leads to students failing to proceed in their studies at a desirable pace, to students who struggle with the subsequent courses, and to students who completely drop out from their studies. This thesis explores the issue of retention in introductory programming courses through multiple viewpoints. We first analyze how the teaching approaches reported in literature affect introductory programming course pass rates. Then, changes on the retention at the University of Helsinki are studied using two separate approaches. The first approach is the use of a contemporary variant of Cognitive Apprenticeship called the Extreme Apprenticeship method, and the second approach is the use of a massive open online course (MOOC) in programming for recruiting students before they enter their university studies. Furthermore, data from an automatic assessment system implemented for the purposes of this thesis is studied to determine how novices write their first lines of code, and what factors contribute to the feeling of difficulty in learning programming. On average, the teaching approaches described in the literature improve the course pass rates by one third. However, the literature tends to neglect the effect of intervention on the subsequent courses. In both studies at the University of Helsinki, retention improved considerably, and the students on average also fare better in subsequent courses. Finally, the data that has been gathered with the automatic assessment system provides an excellent starting point for future research.Ohjelmointi on nykyajan käsityöläistaito, jolle on akuutti tarve työelämässä. Tämän taidon opettelua harkitseva tietää harvoin, kuinka riippuvainen yhteiskuntamme on ohjelmoinnin tuotoksista eli ohjelmistoista. Ilman ohjelmointia esimerkiksi yhteydenpito, kaupankäynti, matkustaminen ja terveydenhuolto olisivat heikommalla tasolla. Puhelimet eivät toimisi, internettiä ei olisi, eikä lääketeollisuus pystyisi käsittelemään yhtä suuria datamassoja uusia parannuskeinoja etsiessä. Kukaan ei olisi kirjoittanut ohjelmaa, joka auttoi avaruuteen pääsemisessä. Väitöskirjassa tarkastellaan ohjelmoinnin opetusmenetelmiä ja niiden toimivuutta korkeakouluissa sekä esitellään kognitiiviseen oppipoikamalliin (Cognitive Apprenticeship) perustuva “ajatuskäsityöläisten” opetusmenetelmä tehostettu kisällioppiminen (Extreme Apprenticeship). Tehostetussa kisällioppimisessa oppimista edesauttava yksilöllinen ohjaus on mahdollista skaalata satoja opiskelijoita sisältäville kursseille. Väitöskirjatyössä ehdotetaan lisäksi kaikille avoimen verkkokurssin (MOOC) käyttöä yliopisto-opiskelijoiden valintaan sekä tarkastellaan tällaisen valintaväylän toimivuutta tietojenkäsittelytieteen alalla. Väitöskirja käsittelee myös ohjelmointitehtävien automaattista arviointia ja esittelee tähän tarkoitetun Test My Code -järjestelmän, jota voidaan käyttää askeleittaisten ohjeiden ja palautteen antamiseen aloitteleville ohjelmoijille sekä tiedon keräämiseen ohjelmointiprosessissa esiintyvistä ongelmista. Tällaista tietoa voidaan tutkia oppimisanalytiikan menetelmin. Väitöskirjassa tarkastellaan myös aloittelevien ohjelmoijien ensimmäisten ohjelmien kirjoittamisessa esiintyviä ongelmia sekä esitellään ohjelmointitehtävien vaikeuden ennustamiseen sopivia menetelmiä

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Understanding conceptual transfer in students learning a new programming language

There is a large literature from at least as early as 1985 on the difficulties encountered in learning programming languages, and in particular additional programming languages. This thesis concentrates on how students transfer their knowledge from their first programming language to their second. The central idea is to adapt and use theories from linguistics of how people learn second natural languages to illuminate the problems of learning second programming languages. The major claim of this thesis is that: Semantic transfer based on syntax similarities plays a role in relative novices’ conceptual transfer between programming languages; the implementation of deliberate semantic transfer interventions during relative novices’ second language learning can lead to improved conceptual transfer and understanding in learning a second programming language. This thesis uses mixed methods to investigate how students transition from procedural Python to object-oriented (OO) Java. It includes a sequence of nine research studies building on each other. First, an exploratory qualitative study is carried out on how semantic transfer in natural language applies to programming language transfer; secondly, a Model of Programming Language Transfer (MPLT) is developed based on the first study’s findings; thirdly, four quantitative studies are carried out to validate the model; fourthly, a study that collects school teachers’ views and experiences on second language learning is carried out; fifthly, a study is conducted to explore transfer interventions with students; and the last study builds and investigates a pedagogy for transfer deriving from the MPLT. The findings support the thesis claim that semantic transfer based on syntax similarities plays a role in relative novices’ conceptual transfer between programming languages. The transfer can be positive when the first programming language (PL1) and the second programming language (PL2) share similar syntax and semantics, negative when PL1 and PL2 share similar syntax but have different semantics, and there is little or no transfer when PL1 and PL2 have different syntax but share similar semantics. The results also reveal that transfer teaching interventions based on the MPLT could improve conceptual transfer and understanding in students learning a second PL. The contribution of this thesis is two-fold: First, a validated model of programming language transfer that has three categories that reflect the types of potential transfer students encounter when learning a second programming language. The model provides a unified way to measure transfer in second language learning. Second, a validated unified pedagogical guideline for promoting transfer in programming languages derived from the MPLT. Researchers, educators and curriculum designers can use these instruments to advance research, teach, and design teaching materials. First, the researchers can use the instruments to further programming language transfer research by adopting them in other programming language contexts. Second, educators can use the instruments as a guideline for improving second and subsequent programming language teaching. Lastly, Computer Science (CS) curricular designers can draw on these instruments as guidance to design teaching material that promotes transfer as students transition to new programming languages. They can also use them for teacher professional development

An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions