Kohti automaattisia vihjeitä visuaalisessa algoritmisimulaatiossa

Visual Algorithm Simulation (VAS) exercise is an interactive application, which teaches an algorithm or a data structure. The exercise shows the student a visual representation of a data structure with initial data. The student imitates the execution of the algorithm by interacting with the visual representation. The student's solution is graded automatically. A misconception about the algorithm being learned can manifest itself as systematic errors, which can be modelled as a new algorithm. It is assumed that a VAS exercise, which could detect automatically a misconception and give corrective feedback would further support learning. This thesis includes a literature review on algorithm misconceptions and an empirical study. Four VAS exercises of OpenDSA e-textbook are reviewed by their program code: Evaluating a postfix expression, Build heap, Quicksort, and Dijkstra's algorithm. A dataset of 1430 Build heap VAS submissions is analysed manually with ad-hoc software. The submissions are then automatically classified based on the misconceptions found. The main result extends the set of known misconceptions of the Build-heap VAS exercise. 52 percent of the submissions were correct, 17 percent were misconceptions and the rest 31 percent had a lo0gical explanation. 95 per cent of submissions classified as misconception have multiple explanations with heap size of 10. The thesis presents a Python software, which can automatically classify the known misconceptions. Theory on how to generate VAS inputs, which support detection of misconceptions is discussed. The theory is applied by improving the input generation of Dijkstra's algorithm exercise. The thesis concludes that studying misconceptions in the VAS exercises of OpenDSA currently requires exercise-dependent work. Not all OpenDSA VAS exercises record enough data for later analysis. Moreover, the player and analysis software must be written separately for each exercise. There is need to develop the OpenDSA related software libraries to produce detailed exercise recordings. It should be studied how the heap size in the Build-heap exercise affects detection of misconceptions.Visuaalinen algoritmisimulaatiotehtävä (VAS-tehtävä) on vuorovaikutteinen sovellus, joka opettaa algoritmin tai tietorakenteen. Tehtävä näyttää opiskelijalle kuvan tietorakenteesta lähtödatalla. Opiskelija mukailee algoritmin suoritusta vuorovaikuttamalla kuvaesityksen kanssa. Opiskelijan ratkaisu arvostellaan automaattisesti. Väärinkäsitys VAS-tehtävässä on opiskelijan järjestelmällinen väärinymmärrys, joka voidaan kuvata algoritmilla. On oletus, että VAS-tehtävä, joka tunnistaisi automaattisesti väärinkäsityksen ja antaisi korjaavaa palautetta, tukisi oppimista entisestään. Tämä opinnäytetyö sisältää kirjallisuustutkimuksen algoritmien väärinkäsityksistä sekä empiirisen tutkimuksen. Sähköisen OpenDSA-kirjan neljä VAS-tehtävää on tutkittu niiden ohjelmakoodiltaan: Postfix-lausekkeen evaluointi, binäärikeon rakentaminen, pikajärjestäminen ja Dijkstran algoritmi. Tietoaineisto, jossa on 1430 tallennetta Binäärikeon rakentaminen -tehtävästä, on analysoitu käsin tätä varten kehitetyllä ohjelmalla. Tallenteet on sitten automaattisesti luokiteltu löydettyjen väärinkäsitysten perusteella. Työn päätulos laajentaa binäärikeon rakentaminen -tehtävän väärinkäsityksien joukkoa. 52 prosenttia tehtäväpalautuksista oli oikein, 17 prosenttia väärinkäsityksiä ja loput 31 prosenttia voidaan selittää loogisesti. 95 prosentilla niistä palautuksista, jotka luokiteltiin väärinkäsitykseksi, oli useampi yhtä hyvä selitys, kun keon koko oli 10. Työ esittää Python-ohjelman, joka voi automaattisesti luokitella tunnettuja väärinkäsityksiä. Työ esittää myös teoriaa, kuinka tuottaa VAS-tehtävien lähtödataa siten, että se tukisi väärinkäsitysten tunnistamista. Teoriaa on sovellettu parantam alla Dijkstran algoritmi -tehtävän syötteen tuottamista. Johtopäätöksenä OpenDSA:n VAS-tehtävien väärinkäsitysten tutkiminen vaatii nykyisellään tehtäväkohtaista työtä. Kaikki OpenDSA:n VAS-tehtävät eivät tallenna riittävästi dataa myöhempää analyysiä varten. Lisäksi tehtävätoistin ja analyysiohjelma pitää kirjoittaa erikseen joka tehtävälle. On tarve kehittää OpenDSA:n ohjelmakirjastoja tuottamaan yksityiskohtaisia tehtävätallenteita. Binäärikeon rakentaminen -tehtävässä pitäisi tutkia keon koon vaikutusta väärinkäsitysten tunnistamiseen

Classifying and recognizing students' sorting algorithm implementations in a data structures and algorithms course

We discuss an instrument for recognizing and classifying algorithms (Aari) in terms of whiteboxtesting. We examined freshmen students' sorting algorithm implementations in a datastructures and algorithms course in two rounds: at the beginning of the course before thestudents received any instruction on sorting algorithms, and after taking lectures on sortingalgorithms. We evaluated the performance of Aari with the implementations of each roundseparately. The results show that the sorting algorithms (in Java), which Aari has been trainedto recognize (Insertion sort, Selection sort, Bubble sort, Quicksort and Mergesort), arerecognized with an average accuracy of about 90%. When considering all the submitted sortingalgorithm implementations (including the variations of the standard algorithms), Aari achievedan overall accuracy being 71% and 81% for the first and second round, respectively. The manual analysis of the implementations revealed that students have manymisconceptions related to sorting algorithms. For example, they include unnecessary swaps intheir Insertion or Selection sort implementations. Based on the data, we present acategorization of these types of variations. We also discuss how these categories can be used toimprove education of sorting methods, and to further develop Aari as a white-box testing tool,which gives feedback to the students on their inappropriate solutions and asks them to correctthe problem; something black-box testing is not capable of doing

A Grounded Conceptual Model for Ownership Types in Rust

Programmers learning Rust struggle to understand ownership types, Rust's core mechanism for ensuring memory safety without garbage collection. This paper describes our attempt to systematically design a pedagogy for ownership types. First, we studied Rust developers' misconceptions of ownership to create the Ownership Inventory, a new instrument for measuring a person's knowledge of ownership. We found that Rust learners could not connect Rust's static and dynamic semantics, such as determining why an ill-typed program would (or would not) exhibit undefined behavior. Second, we created a conceptual model of Rust's semantics that explains borrow checking in terms of flow-sensitive permissions on paths into memory. Third, we implemented a Rust compiler plugin that visualizes programs under the model. Fourth, we integrated the permissions model and visualizations into a broader pedagogy of ownership by writing a new ownership chapter for The Rust Programming Language, a popular Rust textbook. Fifth, we evaluated an initial deployment of our pedagogy against the original version, using reader responses to the Ownership Inventory as a point of comparison. Thus far, the new pedagogy has improved learner scores on the Ownership Inventory by an average of 9% ($N = 342, d = 0.56$).Comment: Published at OOPSLA 202

Promoting Programming Learning. Engagement, Automatic Assessment with Immediate Feedback in Visualizations

The skill of programming is a key asset for every computer science student. Many studies have shown that this is a hard skill to learn and the outcomes of programming courses have often been substandard. Thus, a range of methods and tools have been developed to assist students’ learning processes. One of the biggest fields in computer science education is the use of visualizations as a learning aid and many visualization based tools have been developed to aid the learning process during last few decades. Studies conducted in this thesis focus on two different visualizationbased tools TRAKLA2 and ViLLE. This thesis includes results from multiple empirical studies about what kind of effects the introduction and usage of these tools have on students’ opinions and performance, and what kind of implications there are from a teacher’s point of view. The results from studies in this thesis show that students preferred to do web-based exercises, and felt that those exercises contributed to their learning. The usage of the tool motivated students to work harder during their course, which was shown in overall course performance and drop-out statistics. We have also shown that visualization-based tools can be used to enhance the learning process, and one of the key factors is the higher and active level of engagement (see. Engagement Taxonomy by Naps et al., 2002). The automatic grading accompanied with immediate feedback helps students to overcome obstacles during the learning process, and to grasp the key element in the learning task. These kinds of tools can help us to cope with the fact that many programming courses are overcrowded with limited teaching resources. These tools allows us to tackle this problem by utilizing automatic assessment in exercises that are most suitable to be done in the web (like tracing and simulation) since its supports students’ independent learning regardless of time and place. In summary, we can use our course’s resources more efficiently to increase the quality of the learning experience of the students and the teaching experience of the teacher, and even increase performance of the students. There are also methodological results from this thesis which contribute to developing insight into the conduct of empirical evaluations of new tools or techniques. When we evaluate a new tool, especially one accompanied with visualization, we need to give a proper introduction to it and to the graphical notation used by tool. The standard procedure should also include capturing the screen with audio to confirm that the participants of the experiment are doing what they are supposed to do. By taken such measures in the study of the learning impact of visualization support for learning, we can avoid drawing false conclusion from our experiments. As computer science educators, we face two important challenges. Firstly, we need to start to deliver the message in our own institution and all over the world about the new – scientifically proven – innovations in teaching like TRAKLA2 and ViLLE. Secondly, we have the relevant experience of conducting teaching related experiment, and thus we can support our colleagues to learn essential know-how of the research based improvement of their teaching. This change can transform academic teaching into publications and by utilizing this approach we can significantly increase the adoption of the new tools and techniques, and overall increase the knowledge of best-practices. In future, we need to combine our forces and tackle these universal and common problems together by creating multi-national and multiinstitutional research projects. We need to create a community and a platform in which we can share these best practices and at the same time conduct multi-national research projects easily.Siirretty Doriast

Graph Theoretic Methods for the Analysis of Data in Developing Systems

A full examination of learning or developing systems requires data analysis approaches beyond the commonplace pre-/post-testing. Drawing on graph theory, three particular approaches to the analysis of data—based on adjacency matrices, affiliation networks, and edit distances—can provide additional insight into data; these methods are applied to student performance in a Calculus course. Data analysis methods based on adjacency matrices demonstrate that learning is not unidimensional, that learning progressions do not always progress monotonically toward desired understandings and also provide insight into the connection between instruction and student learning. The use of affiliation networks supports the concept development theory of Lev Vygotsky and also provides insight into how students’ prior knowledge relates to topics being studied. Careful use of edit distances indicates a likely overestimate of effect sizes in many studies, and also provides evidence that concepts are often created in an ad hoc manner. All of these have implications for curriculum and instruction, and indicate some directions for further inquiry

Empirical Evaluation of a Differentiated Assessment of Data Structures: The Role of Prerequisite Skills

There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating between fragilities in prerequisites (in particular, knowledge and skills related to introductory programming courses) and advanced topics. This paper reports on an empirical evaluation of an instrument focusing on data structures, among those proposed by the ITiCSE working group. The evaluation aimed at understanding what fragile knowledge and skills the instrument is actually able to detect and to what extent it is able to differentiate them. Our results support that the instrument is able to distinguish between some specific fragilities (e.g., value vs. reference semantics), but not all of those claimed in the original report. In addition, our findings highlight the role of relevant skills at a level between prerequisite and advanced skills, such as program comprehension and reasoning about constraints. We also suggest ways to improve the questions in the instrument, both by improving the distractors of the multiple choice questions, and by slightly changing the content or phrasing of the questions. We argue that these improvements will increase the effectiveness of the instrument in assessing prerequisites as a whole, but also to pinpoint specific fragilities

Proceedings of the Second Program Visualization Workshop, 2002

The Program Visualization Workshops aim to bring together researchers who design and construct program visualizations and, above all, educators who use and evaluate visualizations in their teaching. The first workshop took place in July 2000 at Porvoo, Finland. The second workshop was held in cooperation with ACM SIGCSE and took place at HornstrupCentret, Denmark in June 2002, immediately following the ITiCSE 2002 Conference in Aarhus, Denmark

Exploratory Study to Uncover Student Mental Models of Singly Linked Lists in the C Programming Language

In computer science, learning abstract fundamental programming concepts requiring students to understand memory management can be very difficult and lead to misunderstandings that carry on into advanced topics. This is especially true in data structures with abstract data types. Understanding how novice students think and reason about data structures is important for improving teaching and learning in computer science. Most studies focus on student misunderstandings of advanced algorithms and data structures related to topics such as heaps, binary search trees, hash tables, dynamic programming, and recursion. Whereas, fewer studies focus on more elementary data structures, such as arrays and linked lists. Since linked lists serve as a bridge to understanding more advanced data structures, we believe that it is critical to identify students’ conceptual and procedural misunderstandings earlier rather than later. Therefore, directly after learning about linked lists using the C language, we conduct semi-structured, think-aloud interviews with 11 undergraduate students to uncover their mental models about singly linked lists in C, and how they apply their knowledge about singly linked lists to understand other types of linked lists. To determine the factors that might contribute to their understanding about linked lists, at the end of the interview, we ask students about the difficulties they had while learning about linked lists, and we give students a 10-minute visual-spatial reasoning test. Using rubrics to code responses to the interview questions, we quantify the accuracy of their mental models and reveal common misunderstandings, such as confusion between the node containing a node pointer and being a node pointer, failure to create a node structure, lack of knowledge regarding typecasting malloc, and lack of attention to the importance of the NULL value. We find that none of the participants have an accurate mental model of a singly linked list in C, after learning about and implementing them in their data structures course. Even though students struggle with expressing their conceptual understanding with enough detail in their verbal responses to interview questions, their performance on the coding questions is much better. The majority of students have a good procedural understanding of how to create and use the pieces of a linked list, and they understand how to implement most operations on a singly linked list in C. However, many students continue to struggle with C syntax and the prerequisite knowledge needed to understand linked lists in C, such as pointer manipulation and memory management. While we do not find a relationship between students’ visual-spatial ability and their conceptual or procedural understanding of linked lists in C, students report that the abstract nature of pointers and relating linked lists to the real world and prior knowledge about dynamic arrays contribute the most to their difficulties with linked lists in C

Le jeu de rôle dans l'enseignement supérieur pour développer l'autonomie et la motivation des étudiants

Comprend des références bibliographiquesL'informatique est une discipline exigeante à bien des niveaux, et son enseignement a suscité beaucoup de questionnements dans le domaine de la pédagogie. Quand s'ajoutent aux difficultés d'ordre disciplinaire, des éléments de complexité plus fort encore comme l'hétérogénéité d'un groupe d'étudiants (niveaux de compétences différents, formation et parcours professionnels différents, origines ethniques et culturelles différentes), les attendus des enseignants et des étudiants sont forcément mis à mal. Cet article propose l'évaluation d'un dispositif pédagogique par le jeu de rôle ayant pour but de prendre en compte les difficultés liées à l'altérité des étudiants et de permettre de les surmonter de manière ludique et motivante, en prenant en compte les spécificités de chacun au sein du groupe

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research