Computer Science Concept Inventories: Past and Future

Concept Inventories (CIs) are assessments designed to measure student learning of core concepts. CIs have become well known for their major impact on pedagogical techniques in other sciences, especially physics. Presently, there are no widely used, validated CIs for computer science. However, considerable groundwork has been performed in the form of identifying core concepts, analyzing student misconceptions, and developing CI assessment questions. Although much of the work has been focused on CS1 and a CI has been developed for digital logic, some preliminary work on CIs is underway for other courses. This literature review examines CI work in other STEM disciplines, discusses the preliminary development of CIs in computer science, and outlines related research in computer science education that contributes to CI development

Computer Science Concept Inventories: Past and Future

Concept Inventories (CIs) are assessments designed to measure student learning of core concepts. CIs have become well known for their major impact on pedagogical techniques in other sciences, especially physics. Presently, there are no widely used, validated CIs for computer science. However, considerable groundwork has been performed in the form of identifying core concepts, analyzing student misconceptions, and developing CI assessment questions. Although much of the work has been focused on CS1 and a CI has been developed for digital logic, some preliminary work on CIs is underway for other courses. This literature review examines CI work in other STEM disciplines, discusses the preliminary development of CIs in computer science, and outlines related research in computer science education that contributes to CI development

Goal-oriented models for teaching and understanding data structures

Most computer science curricula include a compulsory course on data structures. Students are prone to memorise facts about data structures instead of understanding the essence of underlying concepts. This can be explained by the fact that learning the basics of each data structure, the difference with each other, and the adequacy of each of them to the most appropriate context of use, is far from trivial. This paper explores the idea of providing adequate levels of abstractions to describe data structures from an intentional point of view. Our hypothesis is that adopting a goal-oriented perspective could emphasise the main goals of each data structure, its qualities, and its relationships with the potential context of use. Following this hypothesis, in this paper we present the use of iStar2.0 to teach and understand data structures. We conducted a comparative quasi-experiment with undergraduate students to evaluate the effectiveness of the approach. Significant results show the great potential of goal modeling for teaching technical courses like data structures. We conclude this paper by reflecting on further teaching and conceptual modeling research to be conducted in this field.This work has been partially supported by the by the DOGO4ML Spanish research project (ref. PID2020-117191RB-I00), the Digitalization Initiative of the Canton of Zü-rich (DIZH), and ZHAW Digital.Peer ReviewedPostprint (author's final draft

Assessing How Pre-requisite Skills Affect Learning of Advanced Concepts

Students often struggle with advanced computing courses, and comparatively few studies have looked into the reasons for this. It seems that learners do not master the most basic concepts, or forget them between courses. If so, remedial practice could improve learning, but instructors rightly will not use scarce time for this without strong evidence. Based on personal observation, program tracing seems to be an important pre-requisite skill, but there is yet little research that provides evidence for this observation. To investigate this, our group will create theory-based assessments on how tracing knowledge affects learning of advanced topics, such as data structures, algorithms, and concurrency. This working group will identify relevant concepts in advanced courses, then conceptually analyze their pre-requisites and where an imagined student with some tracing difficulties would encounter barriers. The group will use this theory to create instructor-usable assessments for advanced topics that also identify issues caused by poor pre-requisite knowledge. These assessments may then be used at the start and end of advanced courses to evaluate to what extent students\u2019 difficulties with the advanced course originate from poor pre-requisite knowledge

Toward Predicting Success and Failure in CS2: A Mixed-Method Analysis

Factors driving success and failure in CS1 are the subject of much study but less so for CS2. This paper investigates the transition from CS1 to CS2 in search of leading indicators of success in CS2. Both CS1 and CS2 at the University of North Carolina Wilmington (UNCW) are taught in Python with annual enrollments of 300 and 150 respectively. In this paper, we report on the following research questions: 1) Are CS1 grades indicators of CS2 grades? 2) Does a quantitative relationship exist between CS2 course grade and a modified version of the SCS1 concept inventory? 3) What are the most challenging aspects of CS2, and how well does CS1 prepare students for CS2 from the student's perspective? We provide a quantitative analysis of 2300 CS1 and CS2 course grades from 2013--2019. In Spring 2019, we administered a modified version of the SCS1 concept inventory to 44 students in the first week of CS2. Further, 69 students completed an exit questionnaire at the conclusion of CS2 to gain qualitative student feedback on their challenges in CS2 and on how well CS1 prepared them for CS2. We find that 56% of students' grades were lower in CS2 than CS1, 18% improved their grades, and 26% earned the same grade. Of the changes, 62% were within one grade point. We find a statistically significant correlation between the modified SCS1 score and CS2 grade points. Students identify linked lists and class/object concepts among the most challenging. Student feedback on CS2 challenges and the adequacy of their CS1 preparations identify possible avenues for improving the CS1-CS2 transition.Comment: The definitive Version of Record was published in 2020 ACM Southeast Conference (ACMSE 2020), April 2-4, 2020, Tampa, FL, USA. 8 page

Kohti automaattisia vihjeitä visuaalisessa algoritmisimulaatiossa

Visual Algorithm Simulation (VAS) exercise is an interactive application, which teaches an algorithm or a data structure. The exercise shows the student a visual representation of a data structure with initial data. The student imitates the execution of the algorithm by interacting with the visual representation. The student's solution is graded automatically. A misconception about the algorithm being learned can manifest itself as systematic errors, which can be modelled as a new algorithm. It is assumed that a VAS exercise, which could detect automatically a misconception and give corrective feedback would further support learning. This thesis includes a literature review on algorithm misconceptions and an empirical study. Four VAS exercises of OpenDSA e-textbook are reviewed by their program code: Evaluating a postfix expression, Build heap, Quicksort, and Dijkstra's algorithm. A dataset of 1430 Build heap VAS submissions is analysed manually with ad-hoc software. The submissions are then automatically classified based on the misconceptions found. The main result extends the set of known misconceptions of the Build-heap VAS exercise. 52 percent of the submissions were correct, 17 percent were misconceptions and the rest 31 percent had a lo0gical explanation. 95 per cent of submissions classified as misconception have multiple explanations with heap size of 10. The thesis presents a Python software, which can automatically classify the known misconceptions. Theory on how to generate VAS inputs, which support detection of misconceptions is discussed. The theory is applied by improving the input generation of Dijkstra's algorithm exercise. The thesis concludes that studying misconceptions in the VAS exercises of OpenDSA currently requires exercise-dependent work. Not all OpenDSA VAS exercises record enough data for later analysis. Moreover, the player and analysis software must be written separately for each exercise. There is need to develop the OpenDSA related software libraries to produce detailed exercise recordings. It should be studied how the heap size in the Build-heap exercise affects detection of misconceptions.Visuaalinen algoritmisimulaatiotehtävä (VAS-tehtävä) on vuorovaikutteinen sovellus, joka opettaa algoritmin tai tietorakenteen. Tehtävä näyttää opiskelijalle kuvan tietorakenteesta lähtödatalla. Opiskelija mukailee algoritmin suoritusta vuorovaikuttamalla kuvaesityksen kanssa. Opiskelijan ratkaisu arvostellaan automaattisesti. Väärinkäsitys VAS-tehtävässä on opiskelijan järjestelmällinen väärinymmärrys, joka voidaan kuvata algoritmilla. On oletus, että VAS-tehtävä, joka tunnistaisi automaattisesti väärinkäsityksen ja antaisi korjaavaa palautetta, tukisi oppimista entisestään. Tämä opinnäytetyö sisältää kirjallisuustutkimuksen algoritmien väärinkäsityksistä sekä empiirisen tutkimuksen. Sähköisen OpenDSA-kirjan neljä VAS-tehtävää on tutkittu niiden ohjelmakoodiltaan: Postfix-lausekkeen evaluointi, binäärikeon rakentaminen, pikajärjestäminen ja Dijkstran algoritmi. Tietoaineisto, jossa on 1430 tallennetta Binäärikeon rakentaminen -tehtävästä, on analysoitu käsin tätä varten kehitetyllä ohjelmalla. Tallenteet on sitten automaattisesti luokiteltu löydettyjen väärinkäsitysten perusteella. Työn päätulos laajentaa binäärikeon rakentaminen -tehtävän väärinkäsityksien joukkoa. 52 prosenttia tehtäväpalautuksista oli oikein, 17 prosenttia väärinkäsityksiä ja loput 31 prosenttia voidaan selittää loogisesti. 95 prosentilla niistä palautuksista, jotka luokiteltiin väärinkäsitykseksi, oli useampi yhtä hyvä selitys, kun keon koko oli 10. Työ esittää Python-ohjelman, joka voi automaattisesti luokitella tunnettuja väärinkäsityksiä. Työ esittää myös teoriaa, kuinka tuottaa VAS-tehtävien lähtödataa siten, että se tukisi väärinkäsitysten tunnistamista. Teoriaa on sovellettu parantam alla Dijkstran algoritmi -tehtävän syötteen tuottamista. Johtopäätöksenä OpenDSA:n VAS-tehtävien väärinkäsitysten tutkiminen vaatii nykyisellään tehtäväkohtaista työtä. Kaikki OpenDSA:n VAS-tehtävät eivät tallenna riittävästi dataa myöhempää analyysiä varten. Lisäksi tehtävätoistin ja analyysiohjelma pitää kirjoittaa erikseen joka tehtävälle. On tarve kehittää OpenDSA:n ohjelmakirjastoja tuottamaan yksityiskohtaisia tehtävätallenteita. Binäärikeon rakentaminen -tehtävässä pitäisi tutkia keon koon vaikutusta väärinkäsitysten tunnistamiseen

Exploratory Study to Uncover Student Mental Models of Singly Linked Lists in the C Programming Language

In computer science, learning abstract fundamental programming concepts requiring students to understand memory management can be very difficult and lead to misunderstandings that carry on into advanced topics. This is especially true in data structures with abstract data types. Understanding how novice students think and reason about data structures is important for improving teaching and learning in computer science. Most studies focus on student misunderstandings of advanced algorithms and data structures related to topics such as heaps, binary search trees, hash tables, dynamic programming, and recursion. Whereas, fewer studies focus on more elementary data structures, such as arrays and linked lists. Since linked lists serve as a bridge to understanding more advanced data structures, we believe that it is critical to identify students’ conceptual and procedural misunderstandings earlier rather than later. Therefore, directly after learning about linked lists using the C language, we conduct semi-structured, think-aloud interviews with 11 undergraduate students to uncover their mental models about singly linked lists in C, and how they apply their knowledge about singly linked lists to understand other types of linked lists. To determine the factors that might contribute to their understanding about linked lists, at the end of the interview, we ask students about the difficulties they had while learning about linked lists, and we give students a 10-minute visual-spatial reasoning test. Using rubrics to code responses to the interview questions, we quantify the accuracy of their mental models and reveal common misunderstandings, such as confusion between the node containing a node pointer and being a node pointer, failure to create a node structure, lack of knowledge regarding typecasting malloc, and lack of attention to the importance of the NULL value. We find that none of the participants have an accurate mental model of a singly linked list in C, after learning about and implementing them in their data structures course. Even though students struggle with expressing their conceptual understanding with enough detail in their verbal responses to interview questions, their performance on the coding questions is much better. The majority of students have a good procedural understanding of how to create and use the pieces of a linked list, and they understand how to implement most operations on a singly linked list in C. However, many students continue to struggle with C syntax and the prerequisite knowledge needed to understand linked lists in C, such as pointer manipulation and memory management. While we do not find a relationship between students’ visual-spatial ability and their conceptual or procedural understanding of linked lists in C, students report that the abstract nature of pointers and relating linked lists to the real world and prior knowledge about dynamic arrays contribute the most to their difficulties with linked lists in C