Ordinal evaluation and assignment problems

In many assignment problems, a set of documents such as research proposals, promotion dossiers, resumes of job applicants is assigned to a set of experts for ordinal evaluation, ranking, and classification. A desirable condition for such assignments is that every pair of documents is compared and ordered by one or more experts. This condition was modeled as an optimization problem and the number of pairs of documents was maximized for a given incidence relation between a set of documents and a set of experts using a set covering integer programming method in the literature[5]. In this paper, we use a combinatorial approach to derive lower bounds on the number of experts needed to compare all pairs of documents and describe assignments that asymptotically match these bounds. These results are not only theoretically interesting but also have practical implications in obtaining optimal assignments without using complex optimization techniques. ©2010 IEEE

Asymptotically optimal assignments in ordinal evaluations of proposals

Ankara : The Department of Computer Engineering and the Institute of Engineering and Science of Bilkent University, 2009.Thesis (Master's) -- Bilkent University, 2009.Includes bibliographical references leaves 42-44.In ordinal evaluations of proposals in peer review systems, a set of proposals is assigned to a fixed set of referees so as to maximize the number of pairwise comparisons of proposals under certain referee capacity and proposal subject constraints. The following two related problems are considered: (1) Assuming that each referee has a capacity to review k out of n proposals, 2 ≤ k ≤ n, determine the minimum number of referees needed to ensure that each pair of proposals is reviewed by at least one referee, (2) Find an assignment that meets the lower bound determined in (1). It is easy to see that one referee is both necessary and sufficient when k = n, and n(n-1)/2 referees are both necessary and sufficient when k = 2. It is shown that 6 referees are both necessary and sufficient when k = n/2. Furthermore it is shown that 11 referees are necessary and 12 are sufficient when k = n/3, and 18 referees are necessary and 20 referees are sufficient when k = n/4. A more general lower bound of n(n-1)/k(k-1) referees is also given for any k, 2 ≤ k ≤ n, and an assignment asymptotically matching this lower bound within a factor of 2 is presented. These results are not only theoretically interesting but they also provide practical methods for efficient assignments of proposals to referees.Atmaca, AbdullahM.S

Measuring Economic Journals' Citation Efficiency: A Data Envelopment Analysis Approach

This paper by using Data Envelopment Analysis (DEA) and statistical inference evaluates the citation performance of 229 economic journals. The paper categorizes the journals into four main categories (A to D) based on their efficiency levels. The results are then compared to the 27 “core economic journals” as introduced by Dimond (1989). The results reveal that after more than twenty years Diamonds’ list of “core economic journals” is still valid. Finally, for the first time the paper uses data from four well-known databases (SSCI, Scopus, RePEc, Econlit) and two quality ranking reports (Kiel Institute internals ranking and ABS quality ranking report) in a DEA setting and in order to derive the ranking of 229 economic journals. The ten economic journals with the highest citation performance are Journal of Political Economy, Econometrica, Quarterly Journal of Economics, Journal of Financial Economics, Journal of Economic Literature, American Economic Review, Review of Economic Studies, Journal of Econometrics, Journal of Finance, Brookings Papers on Economic Activity.Ranking journals; Data Envelopment Analysis; Indexing techniques; Nonparametric analysis.

A COLLABORATIVE METHOD OF AGGREGATING GROUP OPINION BASED ON INCOMPLETE PREFERENCES

Abstrac

Ordinal covering using block designs

A frequently encountered problem in peer review systems is to facilitate pairwise comparisons of a given set of documents by as few experts as possible. In [7], it was shown that, if each expert is assigned to review k documents then ⌈n(n-1)/k(k-1)⌉ experts are necessary and ⌈n(2n-k)/k 2⌉ experts are sufficient to cover all n(n-1)/2 pairs of n documents. In this paper, we show that, if √n ≤ k ≤ n/2 then the upper bound can be improved using a new assignnment method based on a particular family of balanced incomplete block designs. Specifically, the new method uses ⌈n(n+k)/k2⌉ experts where n/k is a prime power, n divides k2, and √n ≤ k ≤ n/2. When k = √n , this new method uses the minimum number of experts possible and for all other values of k, where √n < k ≤ n/2, the new upper bound is tighter than the general upper bound given in [7]. ©2010 IEEE