65,308 research outputs found
Statistical structures for internet-scale data management
Efficient query processing in traditional database management systems relies on statistics on base data. For centralized systems, there is a rich body of research results on such statistics, from simple aggregates to more elaborate synopses such as sketches and histograms. For Internet-scale distributed systems, on the other hand, statistics management still poses major challenges. With the work in this paper we aim to endow peer-to-peer data management over structured overlays with the power associated with such statistical information, with emphasis on meeting the scalability challenge. To this end, we first contribute efficient, accurate, and decentralized algorithms that can compute key aggregates such as Count, CountDistinct, Sum, and Average. We show how to construct several types of histograms, such as simple Equi-Width, Average-Shifted Equi-Width, and Equi-Depth histograms. We present a full-fledged open-source implementation of these tools for distributed statistical synopses, and report on a comprehensive experimental performance evaluation, evaluating our contributions in terms of efficiency, accuracy, and scalability
A Review of Theory and Practice in Scientometrics
Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the “laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future developments
How journal rankings can suppress interdisciplinary research. A comparison between Innovation Studies and Business & Management
This study provides quantitative evidence on how the use of journal rankings
can disadvantage interdisciplinary research in research evaluations. Using
publication and citation data, it compares the degree of interdisciplinarity
and the research performance of a number of Innovation Studies units with that
of leading Business & Management schools in the UK. On the basis of various
mappings and metrics, this study shows that: (i) Innovation Studies units are
consistently more interdisciplinary in their research than Business &
Management schools; (ii) the top journals in the Association of Business
Schools' rankings span a less diverse set of disciplines than lower-ranked
journals; (iii) this results in a more favourable assessment of the performance
of Business & Management schools, which are more disciplinary-focused. This
citation-based analysis challenges the journal ranking-based assessment. In
short, the investigation illustrates how ostensibly 'excellence-based' journal
rankings exhibit a systematic bias in favour of mono-disciplinary research. The
paper concludes with a discussion of implications of these phenomena, in
particular how the bias is likely to affect negatively the evaluation and
associated financial resourcing of interdisciplinary research organisations,
and may result in researchers becoming more compliant with disciplinary
authority over time.Comment: 41 pages, 10 figure
The metric tide: report of the independent review of the role of metrics in research assessment and management
This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration.
This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture.
The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises
Recommended from our members
Incentives and Gender in a Multi-task Setting: an Experimental Study with Real-Effort Tasks
This paper investigates the behavioural effects of competitive, social-value and social-image incentives on men’s and women’s allocation of effort in a multi-task environment. Specifically, using two real-effort laboratory tasks, we investigate how competitive prizes, social-value generation and public awards affect effort allocation decisions between the tasks. We find that all three types of incentives significantly focus effort allocation towards the task they are applied in, but the effect varies significantly between men and women. The highest effort distortion lies with competitive incentives, which is due to the effort allocation decision of men. Women exert similar amount of effort across the three incentive conditions, with slightly lower effort levels in the social-image incentivized tasks. Our results inform how and why genders differences may persist in competitive workplaces
Scalable Peer-to-Peer Indexing with Constant State
We present a distributed indexing scheme for peer to peer networks. Past work on distributed indexing traded off fast search times with non-constant degree topologies or network-unfriendly behavior such as flooding. In contrast, the scheme we present optimizes all three of these performance measures. That is, we provide logarithmic round searches while maintaining connections to a fixed number of peers and avoiding network flooding. In comparison to the well known scheme Chord, we provide competitive constant factors. Finally, we observe that arbitrary linear speedups are possible and discuss both a general brute force approach and specific economical optimizations
Fault-Tolerant Aggregation: Flow-Updating Meets Mass-Distribution
Flow-Updating (FU) is a fault-tolerant technique that has proved to be
efficient in practice for the distributed computation of aggregate functions in
communication networks where individual processors do not have access to global
information. Previous distributed aggregation protocols, based on repeated
sharing of input values (or mass) among processors, sometimes called
Mass-Distribution (MD) protocols, are not resilient to communication failures
(or message loss) because such failures yield a loss of mass. In this paper, we
present a protocol which we call Mass-Distribution with Flow-Updating (MDFU).
We obtain MDFU by applying FU techniques to classic MD. We analyze the
convergence time of MDFU showing that stochastic message loss produces low
overhead. This is the first convergence proof of an FU-based algorithm. We
evaluate MDFU experimentally, comparing it with previous MD and FU protocols,
and verifying the behavior predicted by the analysis. Finally, given that MDFU
incurs a fixed deviation proportional to the message-loss rate, we adjust the
accuracy of MDFU heuristically in a new protocol called MDFU with Linear
Prediction (MDFU-LP). The evaluation shows that both MDFU and MDFU-LP behave
very well in practice, even under high rates of message loss and even changing
the input values dynamically.Comment: 18 pages, 5 figures, To appear in OPODIS 201
- …