Applied Evaluative Informetrics: Part 1

This manuscript is a preprint version of Part 1 (General Introduction and Synopsis) of the book Applied Evaluative Informetrics, to be published by Springer in the summer of 2017. This book presents an introduction to the field of applied evaluative informetrics, and is written for interested scholars and students from all domains of science and scholarship. It sketches the field's history, recent achievements, and its potential and limits. It explains the notion of multi-dimensional research performance, and discusses the pros and cons of 28 citation-, patent-, reputation- and altmetrics-based indicators. In addition, it presents quantitative research assessment as an evaluation science, and focuses on the role of extra-informetric factors in the development of indicators, and on the policy context of their application. It also discusses the way forward, both for users and for developers of informetric tools.Comment: The posted version is a preprint (author copy) of Part 1 (General Introduction and Synopsis) of a book entitled Applied Evaluative Bibliometrics, to be published by Springer in the summer of 201

General cost analysis for scholarly communication in Germany : results of the "Houghton Report" for Germany

Management Summary: Conducted within the project “Economic Implications of New Models for Information Supply for Science and Research in Germany”, the Houghton Report for Germany provides a general cost and benefit analysis for scientific communication in Germany comparing different scenarios according to their specific costs and explicitly including the German National License Program (NLP). Basing on the scholarly lifecycle process model outlined by Björk (2007), the study compared the following scenarios according to their accounted costs: - Traditional subscription publishing, - Open access publishing (Gold Open Access; refers primarily to journal publishing where access is free of charge to readers, while the authors or funding organisations pay for publication) - Open Access self-archiving (authors deposit their work in online open access institutional or subject-based repositories, making it freely available to anyone with Internet access; further divided into (i) CGreen Open Access’ self-archiving operating in parallel with subscription publishing; and (ii) the ‘overlay services’ model in which self-archiving provides the foundation for overlay services (e.g. peer review, branding and quality control services)) - the NLP. Within all scenarios, five core activity elements (Fund research and research communication; perform research and communicate the results; publish scientific and scholarly works; facilitate dissemination, retrieval and preservation; study publications and apply the knowledge) were modeled and priced with all their including activities. Modelling the impacts of an increase in accessibility and efficiency resulting from more open access on returns to R&D over a 20 year period and then comparing costs and benefits, we find that the benefits of open access publishing models are likely to substantially outweigh the costs and, while smaller, the benefits of the German NLP also exceed the costs. This analysis of the potential benefits of more open access to research findings suggests that different publishing models can make a material difference to the benefits realised, as well as the costs faced. It seems likely that more Open Access would have substantial net benefits in the longer term and, while net benefits may be lower during a transitional period, they are likely to be positive for both ‘author-pays’ Open Access publishing and the ‘over-lay journals’ alternatives (‘Gold Open Access’), and for parallel subscription publishing and self-archiving (‘Green Open Access’). The NLP returns substantial benefits and savings at a modest cost, returning one of the highest benefit/cost ratios available from unilateral national policies during a transitional period (second to that of ‘Green Open Access’ self-archiving). Whether ‘Green Open Access’ self-archiving in parallel with subscriptions is a sustainable model over the longer term is debateable, and what impact the NLP may have on the take up of Open Access alternatives is also an important consideration. So too is the potential for developments in Open Access or other scholarly publishing business models to significantly change the relative cost-benefit of the NLP over time. The results are comparable to those of previous studies from the UK and Netherlands. Green Open Access in parallel with the traditional model yields the best benefits/cost ratio. Beside its benefits/cost ratio, the meaningfulness of the NLP is given by its enforceability. The true costs of toll access publishing (beside the buyback” of information) is the prohibition of access to research and knowledge for society

Internet: turning science communication inside-out?

In the four decades since two university computers were first linked to each other over the prototype internet, scientific researchers have been innovators, early adopters and prolific adapters of internet technologies. Electronic mail, file transfer protocol, telnet, Gopher and the World Wide Web were all developed and applied first in research communities. The Web's development for sharing of information in the high-energy physics community unexpectedly heralded the internet's extension into many aspects of commerce, community, entertainment and governance. But despite the rapid proliferation and diversification of both over the past 15 years, the internet in its various forms has scientific communication indelibly inscribed into its fabric, and internet communication is thoroughly integrated into the practice of science. This chapter reviews some effects of the internet's emergence as a principal means of professional scientific communication, and of public communication of science and technology. It notes several paradoxes that characterise these developments, for example the contradictory trends towards easier collaboration across continents, and towards greater fragmentation. It notes the very significant disturbances caused by electronic publishing in the all-important field of scientific journals. It suggests that these and other developments have made more completely porous than before the boundaries between professional and public communication, facilitating public access to previously private spaces, and thus 'turning science communication inside-out'

Peer review innovations in Humanities: how can scholars in A&H profit of the "wisdom of the crowds"?

Though supported by a large number of scholars in Scientific, Technical, and Medical (STM) disciplines traditional peer review does not live up to the needs of an efficient scholarly communication system and of quality research control. Therefore journals in STM are experimenting different forms of refereeing in combination with more traditional peer review system. Such is the case of PLoSONE, Biology Direct, Atmospheric Chemistry and Physics, Electronic Transactions on Artificial Intelligence, and JIME. However in STM disciplines public peer review is not regarded an alternative to more traditional quality certification forms. It may be the case in the Arts & Humanities. In A&H publishing system peer review is by far a less common practice. Therefore the adoption of a social peer review process could be very useful to foster research in humanities. Scholars in A&H can profit of the interactive evaluation forms of the public peer-review to strengthen the scholarly debate, to foster active international and interdisciplinary discussions, to focus social attention on topics in Humanities, to broaden the borders of the cultural and intellectual discourse among non-scholars (public debate). This paper will provide some examples of how social peer review has been adopted by innovative communities of scholars in humanities to publish new experimental digital book models. In the digital environment the concepts of “document”, of “completeness of a document” and of “evaluation” is fast changing. In a close future in scholarly publishing it might become possible to overcome the rigid distinction between ex-ante and ex-post evaluation as the evaluation process might become an enduring part of the text itsel

A Multi-Relational Network to Support the Scholarly Communication Process

The general pupose of the scholarly communication process is to support the creation and dissemination of ideas within the scientific community. At a finer granularity, there exists multiple stages which, when confronted by a member of the community, have different requirements and therefore different solutions. In order to take a researcher's idea from an initial inspiration to a community resource, the scholarly communication infrastructure may be required to 1) provide a scientist initial seed ideas; 2) form a team of well suited collaborators; 3) located the most appropriate venue to publish the formalized idea; 4) determine the most appropriate peers to review the manuscript; and 5) disseminate the end product to the most interested members of the community. Through the various delinieations of this process, the requirements of each stage are tied soley to the multi-functional resources of the community: its researchers, its journals, and its manuscritps. It is within the collection of these resources and their inherent relationships that the solutions to scholarly communication are to be found. This paper describes an associative network composed of multiple scholarly artifacts that can be used as a medium for supporting the scholarly communication process.Comment: keywords: digital libraries and scholarly communicatio

Overlay journals, repositories and the evolution of scholarly communication

This paper examines the part overlay journals can play in developing new roles for repositories in the scholarly communication process. This requires that we answer some outstanding questions about the overlay journal model: · How are overlay journals distinct from other overlay services and other journals? · What business models are applicable? · What opportunities do overlay journals offer to repositories? And, perhaps most importantly: · What value can an overlay journal bring to the process of scholarly communication? As a result of the answer to the first of these questions, this paper gives a definition of an overlay journal as an entity that performs all the activities of a scholarly journal and relies on structural links with one or more archives or repositories to perform its activities. It finds that the overlay journals that already exist use a variety of business models, which means that repositories can engage with overlay journals in many different ways. Research and practice show that overlay journals offer new possibilities for publishers, repositories, authors and readers alike, and as such have a great deal to offer to scholarly communication

An introduction to overlay journals

An overlay journal performs all the activities of a scholarly journal and relies on structural links with one or more archives or repositories to perform its activities. This paper offers a briefing on the contribution overlay journals can make to scholarly communication. It explains what ‘overlay’ services are, how overlay journals have evolved and what makes their contribution to scholarly communication so valuable

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises