Weighted coverage based reviewer assignment

Peer reviewing is a standard process for assessing the quality of submissions at academic conferences and journals. A very important task in this process is the assignment of reviewers to papers. However, achieving an appropriate assignment is not easy, because all reviewers should have similar load and the subjects of the assigned papers should be consistent with the reviewers' expertise. In this paper, we propose a generalized framework for fair reviewer assignment. We first extract the domain knowledge from the reviewers' published papers and model this knowledge as a set of topics. Then, we perform a group assignment of reviewers to papers, which is a generalization of the classic Reviewer Assignment Problem (RAP), considering the relevance of the papers to topics as weights. We study a special case of the problem, where reviewers are to be found for just one paper (Journal Assignment Problem) and propose an exact algorithm which is fast in practice, as opposed to brute-force solutions. For the general case of having to assign multiple papers, which is too hard to be solved exactly, we propose a greedy algorithm that achieves a 1/2-approximation ratio compared to the exact solution. This is a great improvement compared to the 1/3-approximation solution proposed in previous work for the simpler coverage-based reviewer assignment problem, where there are no weights on topics. We theoretically prove the approximation bound of our solution and experimentally show that it is superior to the current state-of-the-art.postprin

Strategies and mechanisms for electronic peer review

This journal article published at the October 2000 Frontiers in Education Conference discusses strategies and mechanisms for electronic peer review. It outlines a peer-grading system for review of student assignments over the World-Wide Web called Peer Grader. The system allows authors and reviewers to communicate and authors to update their submissions. This system facilitates collaborative learning and makes it possible to break up a large project into smaller portions. The article summarizes a unique and innovative method of peer-review. Educational levels: Graduate or professional

Diverse Weighted Bipartite b-Matching

Bipartite matching, where agents on one side of a market are matched to agents or items on the other, is a classical problem in computer science and economics, with widespread application in healthcare, education, advertising, and general resource allocation. A practitioner's goal is typically to maximize a matching market's economic efficiency, possibly subject to some fairness requirements that promote equal access to resources. A natural balancing act exists between fairness and efficiency in matching markets, and has been the subject of much research. In this paper, we study a complementary goal---balancing diversity and efficiency---in a generalization of bipartite matching where agents on one side of the market can be matched to sets of agents on the other. Adapting a classical definition of the diversity of a set, we propose a quadratic programming-based approach to solving a supermodular minimization problem that balances diversity and total weight of the solution. We also provide a scalable greedy algorithm with theoretical performance bounds. We then define the price of diversity, a measure of the efficiency loss due to enforcing diversity, and give a worst-case theoretical bound. Finally, we demonstrate the efficacy of our methods on three real-world datasets, and show that the price of diversity is not bad in practice

Online vs. face-to-face discussions in a web-based research methods course for postgraduate nursing students : A quasi-experimental study

Background: Web-based technologies are increasingly being used to create modes of online learning for nurses but their effect has not been assessed in nurse education. Objectives: Assess whether participation in face-to-face discussion seminars or online asynchronous discussion groups had different effects on educational attainment in a webbased course. Design: Non-randomised or quasi-experimental design with two groups – students choosing to have face-to-face discussion seminars and students choosing to have online discussions. Setting: The Core Methods module of a postgraduate research methods course. Participants: All 114 students participating in the first 2 years during which the course teaching material was delivered online. Outcome: Assignment mark for Core Methods course module. Methods: Background details of the students, their choices of modules and assignment marks were collected as part of the routine course administration. Students’ online activities were identified using the student tracking facility within WebCT. Regression models were fitted to explore the association between available explanatory variables and assignment mark. Results: Students choosing online discussions had a higher Core Methods assignment mark (mean 60.8/100) than students choosing face-to-face discussions (54.4); the difference was statistically significant (t = 3.13, df = 102, p = 0.002), although this ignores confounding variables. Among online discussion students, assignment mark was significantly correlated with the numbers of discussion messages read (Kendall’s taub = 0.22, p = 0.050) and posted (Kendall’s taub = 0.27, p = 0.017); among face-to-face discussion students, it was significantly associated with the number of non-discussion hits in WebCT (Kendall’s taub = 0.19, p = 0.036). In regression analysis, choice of discussion method, whether an MPhil/PhD student, number of non-discussion hits in WebCT, number of online discussion messages read and number posted were associated with assignment mark at the 5% level of significance when taken singly; in combination, only whether an MPhil/PhD student (p = 0.024) and number of non-discussion hits (p = 0.045) retained significance. Conclusions: This study demonstrates that a research methods course can be delivered to postgraduate healthcare students at least as successfully by an entirely online method in which students participate in online discussion as by a blended method in which students accessing web-based teaching material attend face-to-face seminar discussions. Increased online activity was associated with higher assignment marks. The study highlights new opportunities for educational research that arise from the use of virtual learning environments that routinely record the activities of learners and tutors