Automatic assessment of mathematical programming exercises with Numbas

As programming has become a common feature of undergraduate mathematics degrees, there has been an increasing focus on how to teach and assess the subject to mathematicians. The potential benefits of e-assessment of basic programming exercises have many parallels with assessment in mathematics where e-assessment tools are widely used: the chance to give instant feedback to students offers an opportunity to allow students to work at their own pace, accommodating the disparate background in programming that often exists in undergraduate mathematics cohorts. And the randomisation of question content not only offers a powerful tool for practice, with students able to repeat similar problems over and over, it also can offer some protection against plagiarism in a subject where, just like a solution to some mathematical problems, student answers to identical problems are likely to be very similar. This paper considers an extension to Numbas to automatically assess programming exercises and the successful implementation of the resource in undergraduate modules using the programming languages R and Python

Developing computational mathematics provision in undergraduate mathematics degrees

Over the last ten years we have comprehensively embedded computational mathematics, and in doing so programming, into the undergraduate mathematics degree programmes at the University of Leeds. This case study discusses some of the practical, organisational and pedagogical issues we encountered, and how we addressed them

The role of automated feedback in learning to program - Puzzle pieces of ICT-supported education for sustainability

Master's thesis in Global Development and Planning (UT505

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Semi-automated assessment of programming languages for novice programmers

There has recently been an increased emphasis on the importance of learning programming languages, not only in higher education but also in secondary schools. Students of a variety of departments such as physics, mathematics and engineering have also started learning programming languages as part of their academic courses. Assessment of students programming solutions is therefore important for developing their programming skills. Many Computer Based Assessment (CBA) systems utilise multiple-choice questions (MCQ) to evaluate students performance. However, MCQs lack the ability to comprehensively assess students knowledge. Thus, other forms of programming solutions are required to assess students knowledge. This research aims to develop a semi-automated assessment framework for novice programmers, utilising a computer to support the marking process. The research also focuses on ensuring the consistency of feedback. A novel marking process model is developed based on the semi-automated assessment approach which supports a new way of marking, termed segmented marking . A study is carried out to investigate and demonstrate the feasibility of the segmented marking technique. In addition, the new marking process model is developed based on the results of the feasibility study, and two novel marking process models are presented based on segmented marking, namely the full-marking and partial-marking process models. The Case-Based Reasoning (CBR) cycle is adopted in the marking process models in order to ensure the consistency of feedback. User interfaces of the prototype marking tools (full and partial) are designed and developed based on the marking process models and the user interface design requirements. The experimental results show that the full and partial marking techniques are feasible for use in formative assessment. Furthermore, the results also highlight that the tools are capable of providing consistent and personalised feedback and that they considerably reduce markers workload

EduZinc: A tool for the creation and assessment of student learning activities in complex open, online and flexible learning environments

This article describes the development of an application for the grading and provision of feedback on educational processes. The too, named EduZinc, enables instructors to go through the complete process of creating and evaluating the activities and materials of a course. The application enables for the simultaneous management of two teaching-related aspects: (a) creation of individualized learning products (activities, tests and exams) and (b) automatic grading (for every learning product; automated creation of student, class, and competency-based reports; and delivery of personalized reports to students, instructors and tutors). The system also has a series of warnings in place to notify instructors and tutors when a student is falling behind. As a means to reward the efforts made during the course, the program keeps relevant statistics, notifying when a student is excelling in the course

A new marking technique in semi-automated assessment

The number of students learning programming languages in higher education and secondary schools has substantially increased, especially in the last decade. The increasing number of (novice) programmers makes code script assessment more important. Thus, this study proposes a new marking technique based on a semi-automated assessment approach. It advocates providing detailed and consistent feedback for novice programmers based on formative assessment. An experiment was carried out to check the feasibility of the proposed marking technique. The initial results and findings show that this is a potentially valuable approach

The Virtual Tutor: Combining Conversational Agents with Learning Analytics to support Formative Assessment in Online Collaborative Learning

The objective of this design science research project is to combine Learning Analytics data with a conversational agent communication interface, the Virtual Tutor, which is able to support formative assessment for educators and learners in online collaborative learning (OCL) environments. The main benefit for educators is providing user-adaptable Learning Analytics data requests to fit the information needs for formative assessment. Learners receive semi-automated feedback on their platform activity in form of reports, which shall trigger self-reflection processes. By extracting requirements from the potential users and deriving design principles, a conversational agent is implemented and evaluated in an online collaborative learning course. The results indicate that the Virtual Tutor reduces the task load of educators, supports formative assessment and gives scaffolded guidance to the learners by reflecting their performance, thus triggering self-reflection processes. This research provides a first step towards data supported (semi-)automated feedback systems for formative assessment in OCL courses