The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises

Bibliometric Studies and Worldwide Research Trends on Global Health

Global health, conceived as a discipline, aims to train, research and respond to problems of a transboundary nature, in order to improve health and health equity at the global level. The current worldwide situation is ruled by globalization, and therefore the concept of global health involves not only health-related issues, but also those related to the environment and climate change. Therefore, in this Special Issue, the problems related to global health have been addressed from a bibliometric approach in four main areas: environmental issues, diseases, health, education and society

Congress UPV Proceedings of the 21ST International Conference on Science and Technology Indicators

This is the book of proceedings of the 21st Science and Technology Indicators Conference that took place in València (Spain) from 14th to 16th of September 2016. The conference theme for this year, ‘Peripheries, frontiers and beyond’ aimed to study the development and use of Science, Technology and Innovation indicators in spaces that have not been the focus of current indicator development, for example, in the Global South, or the Social Sciences and Humanities. The exploration to the margins and beyond proposed by the theme has brought to the STI Conference an interesting array of new contributors from a variety of fields and geographies. This year’s conference had a record 382 registered participants from 40 different countries, including 23 European, 9 American, 4 Asia-Pacific, 4 Africa and Near East. About 26% of participants came from outside of Europe. There were also many participants (17%) from organisations outside academia including governments (8%), businesses (5%), foundations (2%) and international organisations (2%). This is particularly important in a field that is practice-oriented. The chapters of the proceedings attest to the breadth of issues discussed. Infrastructure, benchmarking and use of innovation indicators, societal impact and mission oriented-research, mobility and careers, social sciences and the humanities, participation and culture, gender, and altmetrics, among others. We hope that the diversity of this Conference has fostered productive dialogues and synergistic ideas and made a contribution, small as it may be, to the development and use of indicators that, being more inclusive, will foster a more inclusive and fair world

Entire Issue Volume 2, Number 1

Entire SLIS Connecting Volume 2, Issue

A study assessing the characteristics of big data environments that predict high research impact: application of qualitative and quantitative methods

BACKGROUND: Big data offers new opportunities to enhance healthcare practice. While researchers have shown increasing interest to use them, little is known about what drives research impact. We explored predictors of research impact, across three major sources of healthcare big data derived from the government and the private sector. METHODS: This study was based on a mixed methods approach. Using quantitative analysis, we first clustered peer-reviewed original research that used data from government sources derived through the Veterans Health Administration (VHA), and private sources of data from IBM MarketScan and Optum, using social network analysis. We analyzed a battery of research impact measures as a function of the data sources. Other main predictors were topic clusters and authors’ social influence. Additionally, we conducted key informant interviews (KII) with a purposive sample of high impact researchers who have knowledge of the data. We then compiled findings of KIIs into two case studies to provide a rich understanding of drivers of research impact. RESULTS: Analysis of 1,907 peer-reviewed publications using VHA, IBM MarketScan and Optum found that the overall research enterprise was highly dynamic and growing over time. With less than 4 years of observation, research productivity, use of machine learning (ML), natural language processing (NLP), and the Journal Impact Factor showed substantial growth. Studies that used ML and NLP, however, showed limited visibility. After adjustments, VHA studies had generally higher impact (10% and 27% higher annualized Google citation rates) compared to MarketScan and Optum (p<0.001 for both). Analysis of co-authorship networks showed that no single social actor, either a community of scientists or institutions, was dominating. Other key opportunities to achieve high impact based on KIIs include methodological innovations, under-studied populations and predictive modeling based on rich clinical data. CONCLUSIONS: Big data for purposes of research analytics has grown within the three data sources studied between 2013 and 2016. Despite important challenges, the research community is reacting favorably to the opportunities offered both by big data and advanced analytic methods. Big data may be a logical and cost-efficient choice to emulate research initiatives where RCTs are not possible

Collaboration - changing the global landscape of science: proceedings of 10th International Conference on Webometrics, Informetrics and Scientometrics & 15th COLLNET Meeting 2014, September 3 - 5, 2014, Technische Universität Ilmenau, Germany

The 10th WIS encourages continued investigation into the field of applied scientometrics. The broad focus of the conference is on collaboration and communication in science and technology, science policy, quantitative aspects of science and combination and integration of qualitative and quantitative approaches in study of scientific practices. The conference thus aims to contribute to evidence-based and informed knowledge about scientific research and practices witch in turn may further provide input to institutional, regional, national and international research and innovation policy making

Theories of Informetrics and Scholarly Communication

Scientometrics have become an essential element in the practice and evaluation of science and research, including both the evaluation of individuals and national assessment exercises. Yet, researchers and practitioners in this field have lacked clear theories to guide their work. As early as 1981, then doctoral student Blaise Cronin published The need for a theory of citing - a call to arms for the fledgling scientometric community to produce foundational theories upon which the work of the field could be based. More than three decades later, the time has come to reach out the field again and ask how they have responded to this call. This book compiles the foundational theories that guide informetrics and scholarly communication research. It is a much needed compilation by leading scholars in the field that gathers together the theories that guide our understanding of authorship, citing, and impact

Theories of Informetrics and Scholarly Communication

Scientometrics have become an essential element in the practice and evaluation of science and research, including both the evaluation of individuals and national assessment exercises. Yet, researchers and practitioners in this field have lacked clear theories to guide their work. As early as 1981, then doctoral student Blaise Cronin published "The need for a theory of citing" —a call to arms for the fledgling scientometric community to produce foundational theories upon which the work of the field could be based. More than three decades later, the time has come to reach out the field again and ask how they have responded to this call. This book compiles the foundational theories that guide informetrics and scholarly communication research. It is a much needed compilation by leading scholars in the field that gathers together the theories that guide our understanding of authorship, citing, and impact

Dominant trends in intersectoral research on funding innovation in business companies: A bibliometric analysis approach.

У статті розроблено теоретико-методологічні засади обґрунтування оцінки домінуючих тенденцій у міжгалузевих наукових дослідженнях, пов’язаних з основними інструментами та інструментами фінансування інновацій у бізнес-компаніях. Ми використовуємо аналітичний підхід, а також двоетапний бібліометричний аналіз наукових статей, опублікованих за останні 80 років та проіндексованих у базі даних Scopus для рефератів і цитат, яка була обрана через її чіткість, охоплення та обсяг. Крім того, ми використовуємо результати аналітичного аналізу, проведеного за допомогою інструменту Google Trends. Описаний вище підхід дозволив порівняти пікові періоди зміни пошукових запитів основних понять з даної проблеми з періодами найбільш значущих подій в інноваційній сфері та фінансовій політиці. Крім того, ми застосовуємо програмне забезпечення VOSViewer для виявлення домінуючих тенденцій у міжгалузевих дослідженнях, пов’язаних з фінансуванням інновацій бізнес-компаній, а також для з’ясування того, які інструменти реалізації фінансової політики є більш актуальними для науковців. Наші результати першого етапу показують, що фокус дослідників на фінансуванні інновацій бізнес-компаній та фінансовому регулюванні був спрямований на такі теми, як податкові, монетарні, бюджетні та інвестиційні інструменти. Крім того, наші результати з другого етапу додатково допомогли нам визначити домінуючі тенденції в міжгалузевих дослідженнях, пов’язаних з кожною групою ідентифікованих інструментів. Таким чином, наші висновки сприяли кластеризації теорії фінансування інновацій бізнес-компаній за структурою та основними інструментами. Ці результати можуть бути корисними міждисциплінарним науковцям, підприємцям, інвесторам, новаторам, а також іншим відповідним зацікавленим сторонам і практикам у прийнятті щоденних рішень щодо фінансування інновацій у бізнес-компаніях.Our paper develops theoretical and methodological principles of grounding assessing the dominant trends in intersectoral academic research related linked to the main tools and instruments of funding innovations in business companies. We employ the analytical approach as well as the two-stage bibliometric analysis of sci entific articles published in the past 80 years and indexed in Scopus abstract and citation database which has been selected for its clarity, coverage, as well as its scope. In addition, we employ the outcomes of analytical analysis conducted using with the help of Google Trends tool. This approach described above allowed allows us to compare the peak periods for the changing the search queries of main concepts on this problem with the periods of the most significant events in the innovation sphere and financial policy. Moreover, we apply the VOSViewer software for identifying the dominant trends in intersectoral research related to funding innovation of business companies, as well as for finding out which instruments for the implementation of the financial policy implementation are more relevant for academics and scholars. Our results from the first stage demonstrate that the researchers’ focus on funding innovation of business companies and financial regulation was targeted on such topics as tax, monetary, budget, and investment instruments. Additionally, our results from the second stage additionally helped us to determine the dominant trends in intersectoral research connected to each group of the identified instruments. Thence, our findings contributed to the clustering of the theory of funding innovation of business companies by structure and main instruments. These results can be useful for by the multidisciplinary scientists, entrepreneurs, investors, innovators, as well as other relevant stakeholders and practitioners in making their everyday decisions on funding innovations in business companies