Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates

Many researchers who study the impact of computer-based assessment (CBA) focus on the affordances or complexities of CBA approaches in comparison to traditional assessment methods. This study examines how CBA approaches were configured within and between modules, and the impact of assessment design on students’ engagement, satisfaction, and pass rates. The analysis was conducted using a combination of longitudinal visualisations, correlational analysis, and fixed-effect models on 74 undergraduate modules and their 72,377 students. Our findings indicate that educators designed very different assessment strategies, which significantly influenced student engagement as measured by time spent in the virtual learning environment (VLE). Weekly analyses indicated that assessment activities were balanced with other learning activities, which suggests that educators tended to aim for a consistent workload when designing assessment strategies. Since most of the assessments were computer-based, students spent more time on the VLE during assessment weeks. By controlling for heterogeneity within and between modules, learning design could explain up to 69% of the variability in students’ time spent on the VLE. Furthermore, assessment activities were significantly related to pass rates, but no clear relation with satisfaction was found. Our findings highlight the importance of CBA and learning design to how students learn online

Pervasive Parallel And Distributed Computing In A Liberal Arts College Curriculum

We present a model for incorporating parallel and distributed computing (PDC) throughout an undergraduate CS curriculum. Our curriculum is designed to introduce students early to parallel and distributed computing topics and to expose students to these topics repeatedly in the context of a wide variety of CS courses. The key to our approach is the development of a required intermediate-level course that serves as a introduction to computer systems and parallel computing. It serves as a requirement for every CS major and minor and is a prerequisite to upper-level courses that expand on parallel and distributed computing topics in different contexts. With the addition of this new course, we are able to easily make room in upper-level courses to add and expand parallel and distributed computing topics. The goal of our curricular design is to ensure that every graduating CS major has exposure to parallel and distributed computing, with both a breadth and depth of coverage. Our curriculum is particularly designed for the constraints of a small liberal arts college, however, much of its ideas and its design are applicable to any undergraduate CS curriculum

On the domain-specificity of mindsets: The relationship between aptitude beliefs and programming practice

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2013 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Deliberate practice is important in many areas of learning, including that of learning to program computers. However, beliefs about the nature of personal traits, known as mindsets, can have a profound impact on such practice. Previous research has shown that those with a fixed mindset believe their traits cannot change; they tend to reduce their level of practice when they encounter difficulty. In contrast, those with the growth mindset believe their traits are flexible; they tend to maintain regular practice despite the level of difficulty. However, focusing on mindset as a single construct focused on intelligence may not be appropriate in the field of computer programming. Exploring this notion, a self-belief survey was distributed to undergraduate software engineering students. It revealed that beliefs about intelligence and programming aptitude formed two distinct constructs. Furthermore, the mindset for programming aptitude had greater utility in predicting software development practice, and a follow-up survey showed that it became more fixed throughout instruction. Thus, educators should consider the role of programming-specific beliefs in the design and evaluation of introductory courses in software engineering. In particular, they need to situate and contextualize the growth messages that motivate students who experience early setbacks

Computing as the 4th “R”: a general education approach to computing education

Computing and computation are increasingly pervading our lives, careers, and societies - a change driving interest in computing education at the secondary level. But what should define a "general education" computing course at this level? That is, what would you want every person to know, assuming they never take another computing course? We identify possible outcomes for such a course through the experience of designing and implementing a general education university course utilizing best-practice pedagogies. Though we nominally taught programming, the design of the course led students to report gaining core, transferable skills and the confidence to employ them in their future. We discuss how various aspects of the course likely contributed to these gains. Finally, we encourage the community to embrace the challenge of teaching general education computing in contrast to and in conjunction with existing curricula designed primarily to interest students in the field

Development and Evaluation of the Nebraska Assessment of Computing Knowledge

One way to increase the quality of computing education research is to increase the quality of the measurement tools that are available to researchers, especially measures of students’ knowledge and skills. This paper represents a step toward increasing the number of available thoroughly-evaluated tests that can be used in computing education research by evaluating the psychometric properties of a multiple-choice test designed to differentiate undergraduate students in terms of their mastery of foundational computing concepts. Classical test theory and item response theory analyses are reported and indicate that the test is a reliable, psychometrically-sound instrument suitable for research with undergraduate students. Limitations and the importance of using standardized measures of learning in education research are discussed

Motivating Students to Learn How to Write Code Using a Gamified Programming Tutor

Engagement and retention are widely acknowledged problems in computer science and more general higher education. The need to develop programming skills is increasingly ubiquitous, but especially so in computer science where it is one of the core competencies. Learning to write code is a particularly challenging skill to master, which can make retention and success even more difficult. We attempt to address student engagement within an introductory programming module by attempting to motivate students using a gamified interactive programming tutor application that provides immediate feedback on the student’s work. In this paper, we describe the design of the gamified programming tutor application, along with a related topology to characterize student engagement. We discuss the design of the software, the gamified elements, and the structured question design. We evaluate the engagement with the gamified programming tutor of two cohorts of students in the first year of a computer science programme, with over two hundred students taking part. We attempt to frame this engagement in terms of frequency, duration, and intensity of interactions, and compare these engagement metrics with module performance. Additionally, we present quantitative and qualitative data from a survey of students about their experience using the programming tutor application to demonstrate the efficacy of this approach

PeerWise - The Marmite of Veterinary Student Learning

PeerWise is a free online student-centred collaborative learning tool with which students anonymously author, answer, and evaluate multiple choice questions (MCQs). Features such as commenting on questions, rating questions and comments, and appearing on leaderboards, can encourage healthy competition, engage students in reflection and debate, and enhance their communication skills. PeerWise has been used in diverse subject areas but never previously in Veterinary Medicine. The Veterinary undergraduates at the University of Glasgow are a distinct cohort; academically gifted and often highly strategic in their learning due to time pressures and volume of course material. In 2010-11 we introduced PeerWise into 1st year Veterinary Biomolecular Sciences in the Glasgow Bachelor of Veterinary Medicine and Surgery programme. To scaffold PeerWise use, a short interactive session introduced students to the tool and to the basic principles of good MCQ authorship. Students were asked to author four and answer forty MCQs throughout the academic year. Participation was encouraged by an allocation of up to 5% of the final year mark and inclusion of studentauthored questions in the first summative examination. Our analysis focuses on engagement of the class with the\ud tool and their perceptions of its use. All 141 students in the class engaged with PeerWise and the majority contributed beyond that which was stipulated. Student engagement with PeerWise prior to a summative exam was positively correlated to exam score, yielding a relationship that was highly significant (p<0.001). Student perceptions of PeerWise were predominantly positive with explicit recognition of its value as a learning and revision tool, and more than two thirds of the class in agreement that question authoring and answering reinforced their learning. There was clear polarisation of views, however, and those students who did not like PeerWise were vociferous in their dislike, the biggest criticism being lack of moderation by staff

Early Developmental Activities and Computing Proficiency

As countries adopt computing education for all pupils from primary school upwards, there are challenging indicators: significant proportions of students who choose to study computing at universities fail the introductory courses, and the evidence for links between formal education outcomes and success in CS is limited. Yet, as we know, some students succeed without prior computing experience. Why is this? <br/><br/> Some argue for an innate ability, some for motivation, some for the discrepancies between the expectations of instructors and students, and some – simply – for how programming is being taught. All agree that becoming proficient in computing is not easy. Our research takes a novel view on the problem and argues that some of that success is influenced by early childhood experiences outside formal education. <br/><br/> In this study, we analyzed over 1300 responses to a multi-institutional and multi-national survey that we developed. The survey captures enjoyment of early developmental activities such as childhood toys, games and pastimes between the ages 0 — 8 as well as later life experiences with computing. We identify unifying features of the computing experiences in later life, and attempt to link these computing experiences to the childhood activities. <br/><br/> The analysis indicates that computing proficiency should be seen from multiple viewpoints, including both skill-level and confidence. It shows that particular early childhood experiences are linked to parts of computing proficiency, namely those related to confidence with problem solving using computing technology. These are essential building blocks for more complex use. We recognize issues in the experimental design that may prevent our data showing a link between early activities and more complex computing skills, and suggest adjustments. Ultimately, it is hoped that this line of research will feed in to early years and primary education, and thereby improve computing education for all

A vignette model for distributed teaching and learning

Computer software and telecommunication technologies are being assimilated into the education sector. At a slower pace, educational methodologies have been evolving and gradually adopted by educators. The widespread and rapid assimilation of technology may be outstripping the uptake of better pedagogical strategies. Non‐pedagogical development of content could lead to the development of legacy systems that constrain future developments. Problems have arisen with computer‐based learning (CBL) materials, such as the lack of uptake of monolithic programmes that cannot be easily changed to keep pace with natural progress or the different requirements of different teachers and institutions. Also, hypertext/hypermedia learning environments have limitations in that following predefined paths is no more interactive than page turning. These considerations require a flexible and dynamic approach for the benefit of both the teacher and student. Courses may be constructed from vignettes to meet a desired purpose and to avoid the problems of adoption for the reasons that programmes cannot easily be changed or are not designed to meet particular needs. Vignettes are small, first‐principle, first‐person, heuristic activities (which are mimetic) from which courses can be constructed Vignettes use an object‐orientated approach to the development of computer‐based learning materials. Vignettes are objects that can be manipulated via a property sheet, which enables changing the object's inherent character or behaviour. A vignette object can interact with other vignette objects to create more complex educational interactions or models. The vignette approach leads to a development concept that is horizontally distributed across disciplines rather than vertically limited to single subjects