A computer‐aided continuous assessment system

A high‐quality assessment system should have the following attributes: rapid feedback to the students, appropriate and detailed feedback, and an effective grading system which provides an accurate overall grade as well as information which identifies the student's weak areas. As stafflstudent ratios worsen, providing such a system will become more difficult and consequently computer assistance in this task is becoming more attractive. This paper describes a Computer‐Aided Assessment (CAA) system based on a modified version of the multiple‐choice questionnaire. The CAA has been designed to be used in continuous assessment, with features that discourage plagiarism and provide appropriate feedback Over a hundred students were tested using this CAA and the results were compared with a more traditional assessment system. In addition, questionnaires were used to assess the student's reaction to the CAA. The results were highly satisfactory, and a more advanced version of the original software is under consideration

Effects of network topology on the OpenAnswer’s Bayesian model of peer assessment

The paper investigates if and how the topology of the peer assessment network can affect the performance of the Bayesian model adopted in Ope nAnswer. Performance is evaluated in terms of the comparison of predicted grades with actual teacher’s grades. The global network is built by interconnecting smaller subnetworks, one for each student, where intra subnetwork nodes represent student's characteristics, and peer assessment assignments make up inter subnetwork connections and determine evidence propagation. A possible subset of teacher graded answers is dynamically determined by suitable selec tion and stop rules. The research questions addressed are: RQ1) “does the topology (diameter) of the network negatively influence the precision of predicted grades?”̀ in the affirmative case, RQ2) “are we able to reduce the negative effects of high diameter networks through an appropriate choice of the subset of students to be corrected by the teacher?” We show that RQ1) OpenAnswer is less effective on higher diameter topologies, RQ2) this can be avoided if the subset of corrected students is chosen considering the network topology

The tertiary debate: A case study analysis of factors considered when applying for university entry by traditional age school leavers in Brisbane

Many interacting factors contribute to a student's choice of a university. This study takes a systems perspective of the choice and develops a Bayesian Network to represent and quantify these factors and their interactions. The systems model is illustrated through a small study of traditional school leavers in Australia, and highlights similarities and differences between universities' perceptions of student choices, students' perceptions of factors that they should consider and how students really make choices. The study shows the range of information that can be gained from this approach, including identification of important factors and scenario assessment

Computer‐aided assessment in statistics: The CAMPUS project

This paper describes the specification, features and implementation of computer‐aided assessment software designed primarily for conducting assessments in Statistics but equally applicable to other quantitative disciplines. The CAMPUS (Computer Aided Marking Program Using Spreadsheets) package has been developed in Microsoft Excel so as to provide a familiar computing environment for both assessors and students. The principal feature of CAMPUS is the facility for setting questions containing random elements (including random graphs), so that each student sits essentially the same questions but with different answers. This helps to eliminate plagiarism and allows a single test to be used many times, either for reassessment or additional practice. CAMPUS has been implemented in a higher‐education context but could also be used at school level

Signaling the Competencies of High School Students to Employers

[Excerpt] The fundamental cause of the low effort level of American students, parents, and voters in school elections is the absence of good signals of effort and accomplishment and the consequent lack of rewards for learning. In most other advanced countries mastery of the curriculum is assessed by examinations that are set and graded at the national or regional level. Grades on these exams signal the student\u27s achievement to employers and colleges and influence the jobs that graduates get and the universities and programs to which they are admitted. Exam results also influence school reputations and in some countries the number of students applying for admission to the school. In the United States, by contrast, students take aptitude tests that are not intended to assess the learning that has occurred in most of the classes taken in high school. The primary signals of academic achievement are diplomas awarded for time spent in school and grades and rank in class—criteria that assess achievement relative to other students in the school or classroom, not relative to an external standard

Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Teaching and Professional Fellowship Report 2007/8 : The role of E portfolios in postgraduate art and design courses

To investigate the various concepts of E portfolios and their application in Postgraduate Art and design contexts as modes of tutorial and reflexive practice, portfolio presentation and commercial promotion