Evaluating the effectiveness of explanations for recommender systems : Methodological issues and empirical studies on the impact of personalization

Peer reviewedPostprin

An examination of the reliability of prestigious scholarly journals: evidence and implications for decision-makers

In universities all over the world, hiring and promotion committees regularly hear the argument: “this is important work because it is about to appear in prestigious journal X”. Moreover, those who allocate levels of research funding, such as in the multi-billion pound Research Assessment Exercise in UK universities, often come under pressure to assess research quality in a mechanical way by using journal prestige ratings. This paper’s results suggest that such tendencies are dangerous. It uses total citations over a quarter of a century as the criterion. The paper finds that it is far better to publish the best article in an issue of a medium-quality journal like the Oxford Bulletin of Economics and Statistics than to publish the worst article (or often the worst 4 articles) in an issue of a top journal like the American Economic Review. Implications are discussed

Metrics to evaluate research performance in academic institutions: A critique of ERA 2010 as applied in forestry and the indirect H2 index as a possible alternative

Excellence for Research in Australia (ERA) is an attempt by the Australian Research Council to rate Australian universities on a 5-point scale within 180 Fields of Research using metrics and peer evaluation by an evaluation committee. Some of the bibliometric data contributing to this ranking suffer statistical issues associated with skewed distributions. Other data are standardised year-by-year, placing undue emphasis on the most recent publications which may not yet have reliable citation patterns. The bibliometric data offered to the evaluation committees is extensive, but lacks effective syntheses such as the h-index and its variants. The indirect H2 index is objective, can be computed automatically and efficiently, is resistant to manipulation, and a good indicator of impact to assist the ERA evaluation committees and to similar evaluations internationally.Comment: 19 pages, 6 figures, 7 tables, appendice

Communities and patterns of scientific collaboration in Business and Management

This is the author's accepted version of this article deposited at arXiv (arXiv:1006.1788v2 [physics.soc-ph]) and subsequently published in Scientometrics October 2011, Volume 89, Issue 1, pp 381-396. The final publication is available at link.springer.com http://link.springer.com/article/10.1007%2Fs11192-011-0439-1Author's note: 17 pages. To appear in special edition of Scientometrics. Abstract on arXiv meta-data a shorter version of abstract on actual paper (both in journal and arXiv full pape

Community structure and patterns of scientific collaboration in Business and Management

This is the author's accepted version of this article deposited at arXiv (arXiv:1006.1788v2 [physics.soc-ph]) and subsequently published in Scientometrics October 2011, Volume 89, Issue 1, pp 381-396. The final publication is available at link.springer.com http://link.springer.com/article/10.1007%2Fs11192-011-0439-1Author's note: 17 pages. To appear in special edition of Scientometrics. Abstract on arXiv meta-data a shorter version of abstract on actual paper (both in journal and arXiv full pape

Academic team formation as evolving hypergraphs

This paper quantitatively explores the social and socio-semantic patterns of constitution of academic collaboration teams. To this end, we broadly underline two critical features of social networks of knowledge-based collaboration: first, they essentially consist of group-level interactions which call for team-centered approaches. Formally, this induces the use of hypergraphs and n-adic interactions, rather than traditional dyadic frameworks of interaction such as graphs, binding only pairs of agents. Second, we advocate the joint consideration of structural and semantic features, as collaborations are allegedly constrained by both of them. Considering these provisions, we propose a framework which principally enables us to empirically test a series of hypotheses related to academic team formation patterns. In particular, we exhibit and characterize the influence of an implicit group structure driving recurrent team formation processes. On the whole, innovative production does not appear to be correlated with more original teams, while a polarization appears between groups composed of experts only or non-experts only, altogether corresponding to collectives with a high rate of repeated interactions

Impact Factor: outdated artefact or stepping-stone to journal certification?

A review of Garfield's journal impact factor and its specific implementation as the Thomson Reuters Impact Factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.Comment: 25 pages, 12 figures, 6 table