Strategies and mechanisms for electronic peer review

This journal article published at the October 2000 Frontiers in Education Conference discusses strategies and mechanisms for electronic peer review. It outlines a peer-grading system for review of student assignments over the World-Wide Web called Peer Grader. The system allows authors and reviewers to communicate and authors to update their submissions. This system facilitates collaborative learning and makes it possible to break up a large project into smaller portions. The article summarizes a unique and innovative method of peer-review. Educational levels: Graduate or professional

Methods for Ordinal Peer Grading

MOOCs have the potential to revolutionize higher education with their wide outreach and accessibility, but they require instructors to come up with scalable alternates to traditional student evaluation. Peer grading -- having students assess each other -- is a promising approach to tackling the problem of evaluation at scale, since the number of "graders" naturally scales with the number of students. However, students are not trained in grading, which means that one cannot expect the same level of grading skills as in traditional settings. Drawing on broad evidence that ordinal feedback is easier to provide and more reliable than cardinal feedback, it is therefore desirable to allow peer graders to make ordinal statements (e.g. "project X is better than project Y") and not require them to make cardinal statements (e.g. "project X is a B-"). Thus, in this paper we study the problem of automatically inferring student grades from ordinal peer feedback, as opposed to existing methods that require cardinal peer feedback. We formulate the ordinal peer grading problem as a type of rank aggregation problem, and explore several probabilistic models under which to estimate student grades and grader reliability. We study the applicability of these methods using peer grading data collected from a real class -- with instructor and TA grades as a baseline -- and demonstrate the efficacy of ordinal feedback techniques in comparison to existing cardinal peer grading methods. Finally, we compare these peer-grading techniques to traditional evaluation techniques.Comment: Submitted to KDD 201

Weapons of mouse destruction: a 3D strategy for combating cut-and-paste plagiarism using the JISC Plagiarism Advisory Service

This article describes an action research project undertaken in the Business Information Technology (BIT) subject group of a post-1992 University Business School to combat the growing menace of cut-and-paste plagiarism. The authors regard plagiarism—the passing off as one’s own, the words and ideas of another—as an academic malpractice that should be deterred, detected and dealt with appropriately. We use these three themes, or dimensions, to structure our account here of an expanding portfolio of tools and techniques we have deployed over a period of three years. Recently the Joint Information Systems Committee’s (JISC) Plagiarism Advisory Service (PAS) has become central to our efforts, and whilst it is certainly useful in structuring student perceptions, detecting and highlighting sections of cut and paste, and providing professional disciplinary evidence, we draw attention to the potentially pivotal role it can play in structuring student perceptions of plagiarism. In particular, we advise that the JISC PAS is used carefully as part of a more considered approach to student plagiarism rather than as a quick and easy panacea. Pilot studies carried out across six undergraduate and postgraduate units have revealed a growing awareness, amongst both academic and student enthusiasts, of the strengths and limitations of this service. Potentially, these limitations, combined with the restricted sanctions available according to university regulations, could constitute a small risk that some students may calculate and be willing to take. We feel it is important whilst working within this framework to adopt other complimentary strategies in order to make the wholesale or part copying of another’s work an irrational choice, even for the desperate student. This article draws upon current plagiarism literature, field observations and a survey of plagiarism perceptions conducted on over 150 final year undergraduate students. We present the findings from our ongoing action research in the form of a ‘3D’ strategy that attempts to share best practice in deterring, detecting, and dealing appropriately with cut-and-paste plagiarism. Our findings indicate that students do perceive the JISC PAS as effective across all three dimensions, but this perception can be altered significantly depending upon how the service is presented as part of a broader set of strategies to combat student plagiarism. In particular, we have found that allowing students to see the comparison report output from the JISC PAS, not only heightens student anxiety regarding speculative accusations of plagiarism, but also significantly reduces their confidence in the service as a reliable and effective detection method

Evaluating an online learning community: Intellectual, social and emotional development and transformations.

Developing online learning communities is a promising pedagogical approach in online learning contexts for adult tertiary learners, but it is no easy task. Understanding how learning communities are formed and evaluating their efficacy in supporting teaching-learning involves a complex set of issues that have a bearing on the design and facilitation of successful online learning experiences. This paper presents findings of a case study of a semester-long online graduate course designed to facilitate a learning community at a New Zealand tertiary institution. It adopts a sociocultural analytical framework and argues for a multiple developmental analytical approach to evaluating learning that considers lecturer and student intellectual, social and emotional development and transformations. Implications are presented for online lecturers, course designers and institutional administrators

Evaluating Medicaid Managed Care through a Public-Private Partnership

This paper describes how a public-private partnership paved the way for an evaluation of Medicaid managed care with previously inaccessible data. Also included are (1) a brief description of Wisconsin’s Medicaid expansion program, (2) research questions and reflections on implications generated by key informant interviews, (3) some answers to these questions and their policy implications gleaned from a Medicaid enrollee survey, and (4) an assessment of the benefits and policy implications of this partnership. The partnership developed in Wisconsin is relevant for other states with large rural populations who will be facing both welfare reform and managed care.

The anonymous reviewer:the relationship between perceived expertise and the perceptions of peer feedback in higher education

Peer feedback often has positive effects on student learning processes and outcomes. However, students may not always be honest when giving and receiving peer feedback as they are likely to be biased due to peer relations, peer characteristics and personal preferences. To alleviate these biases, anonymous peer feedback was investigated in the current research. Research suggests that the expertise of the reviewer influences the perceived usefulness of the feedback. Therefore, this research investigated the relationship between expertise and the perceptions of peer feedback in a writing assignment of 41 students in higher education with a multilevel analysis. The results show that students perceive peer feedback as more adequate when knowing the reviewer perceives him/herself to have a high level of expertise. Furthermore, the results suggest that students who received feedback from a peer who perceives their expertise as closer to the reviewee’s own perceived expertise was more willing to improve his or her own assignment