Promoting Programming Learning. Engagement, Automatic Assessment with Immediate Feedback in Visualizations

The skill of programming is a key asset for every computer science student. Many studies have shown that this is a hard skill to learn and the outcomes of programming courses have often been substandard. Thus, a range of methods and tools have been developed to assist students’ learning processes. One of the biggest fields in computer science education is the use of visualizations as a learning aid and many visualization based tools have been developed to aid the learning process during last few decades. Studies conducted in this thesis focus on two different visualizationbased tools TRAKLA2 and ViLLE. This thesis includes results from multiple empirical studies about what kind of effects the introduction and usage of these tools have on students’ opinions and performance, and what kind of implications there are from a teacher’s point of view. The results from studies in this thesis show that students preferred to do web-based exercises, and felt that those exercises contributed to their learning. The usage of the tool motivated students to work harder during their course, which was shown in overall course performance and drop-out statistics. We have also shown that visualization-based tools can be used to enhance the learning process, and one of the key factors is the higher and active level of engagement (see. Engagement Taxonomy by Naps et al., 2002). The automatic grading accompanied with immediate feedback helps students to overcome obstacles during the learning process, and to grasp the key element in the learning task. These kinds of tools can help us to cope with the fact that many programming courses are overcrowded with limited teaching resources. These tools allows us to tackle this problem by utilizing automatic assessment in exercises that are most suitable to be done in the web (like tracing and simulation) since its supports students’ independent learning regardless of time and place. In summary, we can use our course’s resources more efficiently to increase the quality of the learning experience of the students and the teaching experience of the teacher, and even increase performance of the students. There are also methodological results from this thesis which contribute to developing insight into the conduct of empirical evaluations of new tools or techniques. When we evaluate a new tool, especially one accompanied with visualization, we need to give a proper introduction to it and to the graphical notation used by tool. The standard procedure should also include capturing the screen with audio to confirm that the participants of the experiment are doing what they are supposed to do. By taken such measures in the study of the learning impact of visualization support for learning, we can avoid drawing false conclusion from our experiments. As computer science educators, we face two important challenges. Firstly, we need to start to deliver the message in our own institution and all over the world about the new – scientifically proven – innovations in teaching like TRAKLA2 and ViLLE. Secondly, we have the relevant experience of conducting teaching related experiment, and thus we can support our colleagues to learn essential know-how of the research based improvement of their teaching. This change can transform academic teaching into publications and by utilizing this approach we can significantly increase the adoption of the new tools and techniques, and overall increase the knowledge of best-practices. In future, we need to combine our forces and tackle these universal and common problems together by creating multi-national and multiinstitutional research projects. We need to create a community and a platform in which we can share these best practices and at the same time conduct multi-national research projects easily.Siirretty Doriast

Using theory to inform capacity-building: Bootstrapping communities of practice in computer science education research

In this paper, we describe our efforts in the deliberate creation of a community of practice of researchers in computer science education (CSEd). We understand community of practice in the sense in which Wenger describes it, whereby the community is characterized by mutual engagement in a joint enterprise that gives rise to a shared repertoire of knowledge, artefacts, and practices. We first identify CSEd as a research field in which no shared paradigm exists, and then we describe the Bootstrapping project, its metaphor, structure, rationale, and delivery, as designed to create a community of practice of CSEd researchers. Features of other projects are also outlined that have similar aims of capacity building in disciplinary-specific pedagogic enquiry. A theoretically derived framework for evaluating the success of endeavours of this type is then presented, and we report the results from an empirical study. We conclude with four open questions for our project and others like it: Where is the locus of a community of practice? Who are the core members? Do capacity-building models transfer to other disciplines? Can our theoretically motivated measures of success apply to other projects of the same nature

Computing Education Research Compiled: Keyword Trends, Building Blocks, Creators, and Dissemination

The need for organized computing education efforts dates back to the 1950s. Since then, computing education research (CER) has evolved and matured from its early initiatives and separation from mathematics education into a respectable research specialization of its own. In recent years, a number of meta-research papers, reviews, and scientometric studies have built overviews of CER from various perspectives. This paper continues that approach by offering new perspectives on the past and present state of CER: analyses of influential papers throughout the years, of the theoretical backgrounds of CER, of the institutions and authors who create CER, and finally of the top publication venues and their citation practices. The results reveal influential contributions from early curriculum guidelines to rigorous empirical research of today, the prominence of computer programming as a topic of research, evolving patterns of learning-theory usage, the dominance of high-income countries and a cluster of 52 elite institutions, and issues regarding citation practices within the central venues of dissemination.</p

Introductory Programming and the Didactic Triangle

In this paper, we use Kansanen's didactic triangle to structure and analyse research on the teaching and learning of programming. Students, teachers and course content are the three entities that form the corners of the didactic triangle. The edges of the triangle represent the relationships between these three entities. We argue that many computing educators and computing education researchers operate from within narrow views of the didactic triangle. For example, computing educators often teach programming based on how they relate to the computer, and not how the students relate to the computer. We conclude that, while research that focuses on the corners of the didactic triangle is sometimes appropriate, there needs to be more research that focuses on the edges of the triangle, and more research that studies the entire didactic triangle. © 2010, Australian Computer Society, Inc

Teaching and learning introductory programming : a model-based approach

The dissertation identifies and discusses impact of a model-based approach to teaching and learning introductory object-oriented programming both for practitioners and for computer science education research.Learning to program is notoriously difficult. This dissertation investigates ways to teach introductory object-oriented programming at the university level. It focuses on a model-based approach, describes and argues for this approach and investigates several of its aspects. It gives an overview of the research in teaching introductory programming in an objects-first way. The dissertation also investigates ways for university teachers to share and document best practices in teaching introductory object-oriented programming through pedagogical patterns. The dissertation addresses both traditional young full-time students and experienced programmers (although not in object-orientation) participating in part-time education. It examines whether the same success factors for learning programming apply to a model-based approach as to introductory programming courses in general for full-time students and gives a general overview of research in success factors for introductory programming. Some factors are the same, because students‘ math competence is positively correlated with their success. The dissertation examines how experienced programmers link a model-based programming course to their professional practices. The general answer is that the part-time students do not need to have a direct link to their specific work-practice, they expect to create the link themselves; but the teacher must be aware of the conditions facing the part-time students in industry. Furthermore, the dissertation addresses interaction patterns for part-time students learning model-based introductory programming in a net-based environment. A previously prepared solution to an exercise is found to mediate the interaction in three different ways. Design patterns have had a major impact on the quality of object-oriented software. Inspired by this, researchers have suggested pedagogical patterns for sharing best practices in teaching introductory object-oriented programming. It was expected that university teachers‘ knowledge of pedagogical patterns was limited, but this research proved that to be wrong; about half of the teachers know pedagogical patterns. One of the problems this dissertation identifies is the lack of a structuring principle for pedagogical patterns; potential users have problems identifying the correct patterns to apply. An alternative structuring principle based on a constructivist learning theory is suggested and analysed

Proceedings of the Second Program Visualization Workshop, 2002

The Program Visualization Workshops aim to bring together researchers who design and construct program visualizations and, above all, educators who use and evaluate visualizations in their teaching. The first workshop took place in July 2000 at Porvoo, Finland. The second workshop was held in cooperation with ACM SIGCSE and took place at HornstrupCentret, Denmark in June 2002, immediately following the ITiCSE 2002 Conference in Aarhus, Denmark

Efficient Use of Teaching Technologies with Programming Education

Learning and teaching programming are challenging tasks that can be facilitated by using different teaching technologies. Visualization systems are software systems that can be used to help students in forming proper mental models of executed program code. They provide different visual and textual cues that help student in abstracting the meaning of a program code or an algorithm. Students also need to constantly practice the skill of programming by implementing programming assignments. These can be automatically assessed by other computer programs but parts of the evaluation need to be assessed manually by teachers or teaching assistants.There are a lot of existing tools that provide partial solutions to the practical problems of programming courses: visualizing program code, assessing student programming submissions automatically or rubrics that help keeping manual assessment consistent. Taking these tools into use is not straightforward. To succeed, the teacher needs to find the suitable tools and properly integrate them into the course infrastructure supporting the whole learning process. As many programming courses are mass courses, it is a constant struggle between providing sufficient personal guidance and feedback while retaining a reasonable workload for the teacher.This work answers to the question "How can the teaching of programming be effectively assisted using teaching technologies?" As a solution, different learning taxonomies are presented from Computer Science perspective and applied to visualization examples so the examples could be used to better support deeper knowledge and the whole learning process within a programming course. Then, different parts of the assessment process of programming assignments are studied to find the best practices in supporting the process, especially when multiple graders are being used, to maintain objectivity, consistency and reasonable workload in the grading.The results of the work show that teaching technologies can be a valuable aid for the teacher to support the learning process of the students and to help in the practical organization of the course without hindering the learning results or personalized feedback the students receive from their assignments. This thesis presents new visualization categories that allow deeper cognitive development and examples on how to integrate them efficiently into the course infrastructure. This thesis also presents a survey of computer-assisted assessment tools and assessable features for teachers to use in their programming assignments. Finally, the concept of rubric-based assessment tools is introduced to facilitate the manual assessment part of programming assignments

Retention in Introductory Programming

The introductory programming course is one of the very first courses that computer science students encounter. The course is challenging not only because of the content, but also due to the challenges related to finding a place in a new community. Many have little knowledge of what to expect from university studies, some struggle to adjust their study behavior to match the expected pace, and a few simply cannot attend instruction due to e.g. family or work constraints. As a consequence, a considerable number of students end up failing the course, or pass the course with substandard knowledge. This leads to students failing to proceed in their studies at a desirable pace, to students who struggle with the subsequent courses, and to students who completely drop out from their studies. This thesis explores the issue of retention in introductory programming courses through multiple viewpoints. We first analyze how the teaching approaches reported in literature affect introductory programming course pass rates. Then, changes on the retention at the University of Helsinki are studied using two separate approaches. The first approach is the use of a contemporary variant of Cognitive Apprenticeship called the Extreme Apprenticeship method, and the second approach is the use of a massive open online course (MOOC) in programming for recruiting students before they enter their university studies. Furthermore, data from an automatic assessment system implemented for the purposes of this thesis is studied to determine how novices write their first lines of code, and what factors contribute to the feeling of difficulty in learning programming. On average, the teaching approaches described in the literature improve the course pass rates by one third. However, the literature tends to neglect the effect of intervention on the subsequent courses. In both studies at the University of Helsinki, retention improved considerably, and the students on average also fare better in subsequent courses. Finally, the data that has been gathered with the automatic assessment system provides an excellent starting point for future research.Ohjelmointi on nykyajan käsityöläistaito, jolle on akuutti tarve työelämässä. Tämän taidon opettelua harkitseva tietää harvoin, kuinka riippuvainen yhteiskuntamme on ohjelmoinnin tuotoksista eli ohjelmistoista. Ilman ohjelmointia esimerkiksi yhteydenpito, kaupankäynti, matkustaminen ja terveydenhuolto olisivat heikommalla tasolla. Puhelimet eivät toimisi, internettiä ei olisi, eikä lääketeollisuus pystyisi käsittelemään yhtä suuria datamassoja uusia parannuskeinoja etsiessä. Kukaan ei olisi kirjoittanut ohjelmaa, joka auttoi avaruuteen pääsemisessä. Väitöskirjassa tarkastellaan ohjelmoinnin opetusmenetelmiä ja niiden toimivuutta korkeakouluissa sekä esitellään kognitiiviseen oppipoikamalliin (Cognitive Apprenticeship) perustuva “ajatuskäsityöläisten” opetusmenetelmä tehostettu kisällioppiminen (Extreme Apprenticeship). Tehostetussa kisällioppimisessa oppimista edesauttava yksilöllinen ohjaus on mahdollista skaalata satoja opiskelijoita sisältäville kursseille. Väitöskirjatyössä ehdotetaan lisäksi kaikille avoimen verkkokurssin (MOOC) käyttöä yliopisto-opiskelijoiden valintaan sekä tarkastellaan tällaisen valintaväylän toimivuutta tietojenkäsittelytieteen alalla. Väitöskirja käsittelee myös ohjelmointitehtävien automaattista arviointia ja esittelee tähän tarkoitetun Test My Code -järjestelmän, jota voidaan käyttää askeleittaisten ohjeiden ja palautteen antamiseen aloitteleville ohjelmoijille sekä tiedon keräämiseen ohjelmointiprosessissa esiintyvistä ongelmista. Tällaista tietoa voidaan tutkia oppimisanalytiikan menetelmin. Väitöskirjassa tarkastellaan myös aloittelevien ohjelmoijien ensimmäisten ohjelmien kirjoittamisessa esiintyviä ongelmia sekä esitellään ohjelmointitehtävien vaikeuden ennustamiseen sopivia menetelmiä

Utilizing educational technology in computer science and programming courses : theory and practice

There is one thing the Computer Science Education researchers seem to agree: programming is a difficult skill to learn. Educational technology can potentially solve a number of difficulties associated with programming and computer science education by automating assessment, providing immediate feedback and by gamifying the learning process. Still, there are two very important issues to solve regarding the use of technology: what tools to use, and how to apply them? In this thesis, I present a model for successfully adapting educational technology to computer science and programming courses. The model is based on several years of studies conducted while developing and utilizing an exercise-based educational tool in various courses. The focus of the model is in improving student performance, measured by two easily quantifiable factors: the pass rate of the course and the average grade obtained from the course. The final model consists of five features that need to be considered in order to adapt technology effectively into a computer science course: active learning and continuous assessment, heterogeneous exercise types, electronic examination, tutorial-based learning, and continuous feedback cycle. Additionally, I recommend that student mentoring is provided and cognitive load of adapting the tools considered when applying the model. The features are classified as core components, supportive components or evaluation components based on their role in the complete model. Based on the results, it seems that adapting the complete model can increase the pass rate statistically significantly and provide higher grades when compared with a “traditional” programming course. The results also indicate that although adapting the model partially can create some improvements to the performance, all features are required for the full effect to take place. Naturally, there are some limits in the model. First, I do not consider it as the only possible model for adapting educational technology into programming or computer science courses. Second, there are various other factors in addition to students’ performance for creating a satisfying learning experience that need to be considered when refactoring courses. Still, the model presented can provide significantly better results, and as such, it works as a base for future improvements in computer science education.Ohjelmoinnin oppimisen vaikeus on yksi harvoja asioita, joista lähes kaikki tietojenkäsittelyn opetuksen tutkijat ovat jokseenkin yksimielisiä. Opetusteknologian avulla on mahdollista ratkaista useita ohjelmoinnin oppimiseen liittyviä ongelmia esimerkiksi hyödyntämällä automaattista arviointia, välitöntä palautetta ja pelillisyyttä. Teknologiaan liittyy kuitenkin kaksi olennaista kysymystä: mitä työkaluja käyttää ja miten ottaa ne kursseilla tehokkaasti käyttöön? Tässä väitöskirjassa esitellään malli opetusteknologian tehokkaaseen hyödyntämiseen tietojenkäsittelyn ja ohjelmoinnin kursseilla. Malli perustuu tehtäväpohjaisen oppimisjärjestelmän runsaan vuosikymmenen pituiseen kehitys- ja tutkimusprosessiin. Mallin painopiste on opiskelijoiden suoriutumisen parantamisessa. Tätä arvioidaan kahdella kvantitatiivisella mittarilla: kurssin läpäisyprosentilla ja arvosanojen keskiarvolla. Malli koostuu viidestä tekijästä, jotka on otettava huomioon tuotaessa opetusteknologiaa ohjelmoinnin kursseille. Näitä ovat aktiivinen oppiminen ja jatkuva arviointi, heterogeeniset tehtävätyypit, sähköinen tentti, tutoriaalipohjainen oppiminen sekä jatkuva palautesykli. Lisäksi opiskelijamentoroinnin järjestäminen kursseilla ja järjestelmän käyttöönottoon liittyvän kognitiivisen kuorman arviointi tukevat mallin käyttöä. Malliin liittyvät tekijät on tässä työssä lajiteltu kolmeen kategoriaan: ydinkomponentteihin, tukikomponentteihin ja arviontiin liittyviin komponentteihin. Tulosten perusteella vaikuttaa siltä, että mallin käyttöönotto parantaa kurssien läpäisyprosenttia tilastollisesti merkittävästi ja nostaa arvosanojen keskiarvoa ”perinteiseen” kurssimalliin verrattuna. Vaikka mallin yksittäistenkin ominaisuuksien käyttöönotto voi sinällään parantaa kurssin tuloksia, väitöskirjaan kuuluvien tutkimusten perusteella näyttää siltä, että parhaat tulokset saavutetaan ottamalla malli käyttöön kokonaisuudessaan. On selvää, että malli ei ratkaise kaikkia opetusteknologian käyttöönottoon liittyviä kysymyksiä. Ensinnäkään esitetyn mallin ei ole tarkoituskaan olla ainoa mahdollinen tapa hyödyntää opetusteknologiaa ohjelmoinnin ja tietojenkäsittelyn kursseilla. Toiseksi tyydyttävään oppimiskokemukseen liittyy opiskelijoiden suoriutumisen lisäksi paljon muitakin tekijöitä, jotka tulee huomioida kurssien uudelleensuunnittelussa. Esitetty malli mahdollistaa kuitenkin merkittävästi parempien tulosten saavuttamisen kursseilla ja tarjoaa sellaisena perustan entistä parempaan opetukseen