An architecture for systematic tracking of skills and competence level progression in computer science

A typical Computer Science degree is three to five years long, consists of four to six subjects per semester, and two semesters per year. A student enrolled in such a degree is expected to learn both discipline-specific skills and transferable generic skills. These skills are to be taught in a progressive sequence through the duration of the degree. As the student progresses through the subjects and semesters of a degree, his skill portfolio and competence level for each skill is expected to grow. Effectively modeling these curriculum skills, mapping them to assessment tasks across subjects of a degree, and measuring the progression in learner competence level is, largely, still an unsolved problem. Previous work at this scale is limited. This systematic tracking of skills and competence is crucial for effective quality control and optimization of degree structures. Our main contribution is an architecture for a curriculum information management system to facilitate this systematic tracking of skill and competence level progression in a Computer Science context

The abstraction transition taxonomy: developing desired learning outcomes through the lens of situated cognition

We report on a post-hoc analysis of introductory programming lecture materials. The purpose of this analysis is to identify what knowledge and skills we are asking students to acquire, as situated in the activity, tools, and culture of what programmers do and how they think. The specific materials analyzed are the 133 Peer Instruction questions used in lecture to support cognitive apprenticeship -- honoring the situated nature of knowledge. We propose an Abstraction Transition Taxonomy for classifying the kinds of knowing and practices we engage students in as we seek to apprentice them into the programming world. We find students are asked to answer questions expressed using three levels of abstraction: English, CS Speak, and Code. Moreover, many questions involve asking students to transition between levels of abstraction within the context of a computational problem. Finally, by applying our taxonomy in classifying a range of introductory programming exams, we find that summative assessments (including our own) tend to emphasize a small range of the skills fostered in students during the formative/apprenticeship phase

Cross Case Study of an Elementary Engineering Task

Designerly play has been identified as a fundamental component of childhood learning (Baynes, 1994; Petroski, 2003). However, as students enter grade one and beyond, the increasing academic focus has resulted in the loss of opportunities for designerly play (Zhao, 2012). At the same time, there are increasing calls to increase the number, skill, and diversity of STEM workers (Brophy, Portsmore, Klein, & Rogers, 2008). The robotics based Elementary Engineering Curriculum (Heffernan, 2013) - used by students in this study - and other similar projects have the potential to increase the STEM pipeline but elementary engineering is not well-understood. Research is needed to understand how to teach engineering to students as their cognitive, motor, and social skills rapidly develop in elementary school (Alimisis, 2012; Crismond & Adams, 2012; Mead, Thomas, & Weinberg, 2012; Penner, Giles, Lehrer, & Schauble, 1997; Roth, 1996; Schunn, 2009; Wagner, 1999). The literature review and theoretical frameworks chapters of this study determined the most relevant theoretical frameworks, engineering design process models, and existing research that is relevant to a cross-sectional case study of six grade 2 and six grade 6 elementary robotics students in the context of established K-6 elementary robotics curriculum (Heffernan, 2013). Students were videotaped doing an open-ended engineering task based on LEGO robotics using talk-aloud (Ericsson & Simon, 1993) and clinical interview (Ginsburg, 1997) techniques. The engineering design processes were analyzed and compared by age and gender. Significant differences were found in final projects and engineering design process. However, the differences were not, for the most part, related to development or gender, but were related to the complexity of the ride they tried to build and the skills and structural knowledge they brought to the task. The key factors identified consisted of three executive function process skills of cognitive flexibility, causal reasoning, and planning ability, three domain specific process skills of application of mathematics and science, engineering design process skills, and design principles of stability, scale, and the structural knowledge they had of LEGO robotics, most pointedly, LEGO connection knowledge. Implications of these findings for teachers are given

Novice Programmers' Reasoning About Reversing Conditional Statements

We want undergraduate students to develop higher-order thinking skills that enable them to master program behaviour. Nonetheless, many students of both introductory and advanced programming courses appear to struggle with the abstraction required for this purpose. In particular, a recent think-aloud study showed a group of students were able to reason about and reverse the effect of assignments and vector updates, but most of them failed when asked to reverse a seemingly simple conditional statement. We have extended that study by assigning a similar task to two cohorts of novice programmers as part of their final exam paper. Students' answers, including code and short explanations, have been analysed through the lens of the SOLO taxonomy: 28% of the solutions were correct (classified as relational); an additional 23% were partially correct but failed to identify the overlap between the two paths of the conditional statement (classified as multistructural). Furthermore, the concept of reversibility and related ``low-ceiling'' tasks, such as the one discussed in this study, could be useful resources for educators to assess and develop students' understanding of program behaviour

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Enhancing comprehension in open distance learning computer programming education with visualization

This thesis describes a research project aimed at improving the tracing skills of first-year programming students enrolled for an introductory C++ course at an open distance learning institution by means of a tutorial in the form of a program visualization tool to teach the students to draw variable diagrams. The research was based on the findings from the BRACElet project (Clear, Whalley, Robbins, Philpott, Eckerdal, Laakso & Lister, 2011). A design-based research methodology was followed. To guide the process of developing the tutorial, a framework of 26 guidelines for developing and using visualization tools to teach programming was synthesized from the literature on computing education research CER, educational psychology and computer graphics. Guidelines were supplemented with reasons or explanations for their recommendation and considerations to be taken into account when using a guideline. The framework was enhanced by lessons learnt during the development and testing of the tutorial. The tutorial was tested and refined during two implementation cycles. Both cycles included quantitative and qualitative investigations. All students registered for the introductory module received the tool with their study material. For the quantitative investigations, students completed a questionnaire after using the tutorial. Through the questionnaire biographical data was acquired, the manner in which students used the tutorial and how they experienced using it. The responses to the questionnaires were statistically analysed in combination with respondents’ final marks. The statistical modelling indicated that the students’ biographical properties (a combination of level of programming experience, marks obtained for Mathematics and English in matric and first-time registration for COS1511 or not), had the biggest impact on their final marks by far. During the qualitative investigations students were eye tracked in a Human-Computer Interaction laboratory. The gaze replays in both cycles revealed that students’ reading skills impacted largely on their success, connecting with the findings from the quantitative investigations. Reflections on why the tutorial did not achieve its purpose; and why poor reading skills may have such a strong effect on learning to program, contribute some theoretical understanding as to how novices learn to program.Computer ScienceD. Phil. (Computer Science

On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers

© 2019 Association for Computing Machinery. Most previous research studies using Explain in Plain English questions have focussed on categorising the answers of novice programmers according to the SOLO taxonomy, and/or the relationship between explaining code and writing code. In this paper, we study the words used in the explanations of novice programmers. Our data is from twelve Explain in plain English questions presented to over three hundred students in an exam at the end of the students' first semester of programming. For each question, we compare the frequency of certain words used in correct answers, between students who scored a perfect twelve on all the Explain in plain English questions and students with lower scores. We report a number of statistically significant differences in word frequency between the students who answered all questions correctly and students who did not. The students who answered all twelve questions correctly tended to be more precise, more comprehensive, and more likely to choose words not explicitly in the code, but instead words that are an abstraction beyond the code

Operational levels of cognitive stage achievement and representations of cognitive structures used in mathematical problem solving by young adult prospective teachers

The differential achievement among concrete, transitional, and formal operational levels of cognitive stage in young adults was investigated. Also, the relations were tested among (a) cognitive stage achievement, (b) mathematical problem solution success, and (c) problem-solving strategies used in mathematics (spontaneous figure labeling, chosen solution strategy, and its match with actual strategy). The cognitive interviews replicated those of Piaget and Inhelder (1975) on notions of chance and probability with the use of Green's (1978) quasi-standardized procedural and methodological suggestions. The relations among cognitive achievement, success, and strategies used with math problem solving were tested with an inventory for representations of mathematical cognitive structure (Clark & Reeves, in press). Forty subjects represented prospective teachers. The mean age was 25. 5 years

The development and validation of the screening test for the early prediction of school success (STEPSS) : a screen of cognitive functioning in four- and five-year old children with varying health conditions

The purpose of the present study was to construct and validate a brief screening instrument to support parent(s) and preschool/kindergarten teachers in monitoring and screening for cognitive impairment and/or delay in preschoolers. The target population of interest is all preschoolers at-risk for poor psychosocial and school outcomes due to chronic and acute dysfunction of the central nervous system (CNS). The accessible populations of interest to the present study are pediatric cancer survivors, preschoolers with alcohol related neurodevelopmental disorder (ARND), being preterm low birth weight, and/or diagnosed with various learning disabilities. The past practice of waiting until an at-risk child experienced poor school outcomes before being referred for cognitive assessment toward tailoring an intervention is no longer defensible. For the present study, a 61-item screening instrument (18 memory items, 19 verbal ability items, 15 attention items, and 9 demographic items) was pilot tested with parents, playschool teachers, and kindergarten teachers to rate preschoolers on overt behaviours associated with cognitive functioning. A criterion-referenced framework was used to establish a performance standard and set a cut score based on a sample of 151 normally functioning preschoolers aged 4:0- to 5:11-years. The various empirical and substantive analyses conducted resulted in a revised scale of 28 items (10 memory, 11 verbal ability, and 7 attention) titled, Screening Test for the Early Prediction of School Success (STEPSS). Given the need for a future study to validate the STEPSS with clinical groups of preschoolers, the screening instrument is intended to provide the empirical evidence needed to refer at-risk preschoolers for assessment with more comprehensive cognitive batteries. Constructing and validating the STEPSS is important for two reasons: 1) to fill a gap in the types of instruments available for monitoring and assessing cognitive functioning in at-risk preschool populations; and 2) to alleviate the current delay in targeting interventions for preschoolers because of the practice of depending upon the school system to monitor and identify poor cognitive functioning