Evaluating the effectiveness of explanations for recommender systems : Methodological issues and empirical studies on the impact of personalization

Peer reviewedPostprin

Community structure and patterns of scientific collaboration in Business and Management

This is the author's accepted version of this article deposited at arXiv (arXiv:1006.1788v2 [physics.soc-ph]) and subsequently published in Scientometrics October 2011, Volume 89, Issue 1, pp 381-396. The final publication is available at link.springer.com http://link.springer.com/article/10.1007%2Fs11192-011-0439-1Author's note: 17 pages. To appear in special edition of Scientometrics. Abstract on arXiv meta-data a shorter version of abstract on actual paper (both in journal and arXiv full pape

Metrics to evaluate research performance in academic institutions: A critique of ERA 2010 as applied in forestry and the indirect H2 index as a possible alternative

Excellence for Research in Australia (ERA) is an attempt by the Australian Research Council to rate Australian universities on a 5-point scale within 180 Fields of Research using metrics and peer evaluation by an evaluation committee. Some of the bibliometric data contributing to this ranking suffer statistical issues associated with skewed distributions. Other data are standardised year-by-year, placing undue emphasis on the most recent publications which may not yet have reliable citation patterns. The bibliometric data offered to the evaluation committees is extensive, but lacks effective syntheses such as the h-index and its variants. The indirect H2 index is objective, can be computed automatically and efficiently, is resistant to manipulation, and a good indicator of impact to assist the ERA evaluation committees and to similar evaluations internationally.Comment: 19 pages, 6 figures, 7 tables, appendice

How should peer-review panels behave?

Many governments wish to assess the quality of their universities. A prominent example is the UK’s new Research Excellence Framework (REF) 2014. In the REF, peer-review panels will be provided with information on publications and citations. This paper suggests a way in which panels could choose the weights to attach to these two indicators. The analysis draws in an intuitive way on the concept of Bayesian updating (where citations gradually reveal information about the initially imperfectly-observed importance of the research). Our study should not be interpreted as the argument that only mechanistic measures ought to be used in a REF

Communities and patterns of scientific collaboration in Business and Management

This is the author's accepted version of this article deposited at arXiv (arXiv:1006.1788v2 [physics.soc-ph]) and subsequently published in Scientometrics October 2011, Volume 89, Issue 1, pp 381-396. The final publication is available at link.springer.com http://link.springer.com/article/10.1007%2Fs11192-011-0439-1Author's note: 17 pages. To appear in special edition of Scientometrics. Abstract on arXiv meta-data a shorter version of abstract on actual paper (both in journal and arXiv full pape

Academic team formation as evolving hypergraphs

This paper quantitatively explores the social and socio-semantic patterns of constitution of academic collaboration teams. To this end, we broadly underline two critical features of social networks of knowledge-based collaboration: first, they essentially consist of group-level interactions which call for team-centered approaches. Formally, this induces the use of hypergraphs and n-adic interactions, rather than traditional dyadic frameworks of interaction such as graphs, binding only pairs of agents. Second, we advocate the joint consideration of structural and semantic features, as collaborations are allegedly constrained by both of them. Considering these provisions, we propose a framework which principally enables us to empirically test a series of hypotheses related to academic team formation patterns. In particular, we exhibit and characterize the influence of an implicit group structure driving recurrent team formation processes. On the whole, innovative production does not appear to be correlated with more original teams, while a polarization appears between groups composed of experts only or non-experts only, altogether corresponding to collectives with a high rate of repeated interactions

Impact Factor: outdated artefact or stepping-stone to journal certification?

A review of Garfield's journal impact factor and its specific implementation as the Thomson Reuters Impact Factor reveals several weaknesses in this commonly-used indicator of journal standing. Key limitations include the mismatch between citing and cited documents, the deceptive display of three decimals that belies the real precision, and the absence of confidence intervals. These are minor issues that are easily amended and should be corrected, but more substantive improvements are needed. There are indications that the scientific community seeks and needs better certification of journal procedures to improve the quality of published science. Comprehensive certification of editorial and review procedures could help ensure adequate procedures to detect duplicate and fraudulent submissions.Comment: 25 pages, 12 figures, 6 table

The intellectual influence of economic journals: quality versus quantity

The evaluation of scientific output has a key role in the allocation of research funds and academic positions. Decisions are often based on quality indicators for academic journals, and over the years, a handful of scoring methods have been proposed for this purpose. Discussing the most prominent methods (de facto standards) we show that they do not distinguish quality from quantity at article level. The systematic bias we find is analytically tractable and implies that the methods are manipulable. We introduce modified methods that correct for this bias, and use them to provide rankings of economic journals. Our methodology is transparent; our results are replicable

Bowling Together: Scientific Collaboration Networks of Demographers at European Population Conferences

Studies of collaborative networks of demographers are relatively scarce. Similar studies in other social sciences provide insight into scholarly trends of both the fields and characteristics of their successful scientists. Exploiting a unique database of metadata for papers presented at six European Population Conferences, this report explores factors explaining research collaboration among demographers. We find that (1) collaboration among demographers has increased over the past 10 years, however, among co-authored papers, collaboration across institutions remains relatively unchanged over the period, (2) papers based on core demographic subfields such as fertility, mortality, migration and data and methods are more likely to involve multiple authors and (3) multiple author teams that are all female are less likely to co-author with colleagues in different institutions. Potential explanations for these results are discussed alongside comparisons with similar studies of collaboration networks in other related social sciences

The solution to the Tullock rent-seeking game when R > 2: mixed-strategy equilibria and mean dissipation rates

In Tullock's rent-seeking model, the probability a player wins the game depends on expenditures raised to the power R. We show that a symmetric mixed-strategy Nash equilibrium exists when R>2, and that overdissipation of rents does not arise in any Nash equilibrium. We derive a tight lower bound on the level of rent dissipation that arises in a symmetric equilibrium when the strategy space is discrete, and show that full rent dissipation occurs when the strategy space is continuous. Our results are shown to be consistent with recent experimental evidence on the dissipation of rents. An earlier version of this paper circulated under the title, No, Virginia, There is No Overdissipation of Rents. We are grateful to Dave Furth and Frans van Winden for stimulating conversations, and for comments provided by workshop participants from the CORE-ULB-KUL IUAP project, Purdue University, Pennsylvania State University, Rijksuniversiteit Limburg, and Washington State University. We also thank Max van de Sande Bakhuyzen and Ben Heijdra for useful discussions, and Geert Gielens for computational assistance. An earlier version of the paper was presented at the ESEM 1992 in Brussels and the Mid-West Mathematical Economics Conference in Pittsburgh. All three authors would like to thank CentER for its hospitality during the formative stages of the paper. The second author has also benefited from the financial support of the Katholieke Universitieit Leuven and the Jay N. Ross Young Faculty Scholar Award at Purdue University. The third author benefitted from visiting IGIER where part of the paper was written. The third author also benefitted from grant IUAP 26 of the Belgian Government