A thousand and one reflections of the publications in the mirrors’ labyrinth of the new metrics

An overview of the metrics available to evaluate scientific production is presented. The classic indicators, based on citations, and the altmetric indicators, based on downloads and the impact on social networks, are compared. In addition, the advantages and disadvantages of the two types of indicators are highlighted. The need for a classification of the indicators, that would help apply them correctly, is discussed

Scientific communication in 2013. Reflections on the informetric community: more networked than ever

The peg of this report is the 14th edition of the International Society for Informetrics and Scientometrics (ISSI) Conference, which was held in Vienna in 2013 and was by far the biggest event in this field. Based on this conference the author discusses the hottest and most controversial topics, which bibliometricians and scientometricians will have to face within the next years and even beyond. These are, amongst others, the adjustment of the peer review process to the challenges of the digital era, the crucial role of research data, the rise and necessity of new metrics, as well as the reconsideration of our proxys. Furthermore it is pointed out to which degree all these novel possibilities in the dissemination and promotion of research output will inevitably influence the role of future scientists. Finally, a short reflection on the scientometric community during this crucial period of change is given

Individual bibliometric assessment at University of Vienna: From numbers to multidimensional profiles

Se muestra cómo se puede implementar la evaluación bibliométrica a nivel individual, lo que se ha hecho con éxito en la Universidad de Viena, a cargo del Departamento Bibliometría y Estrategias de Publicación de la Biblioteca. De acuerdo con la filosofía del Departamento, la bibliometría no es más que un instrumento de evaluación útil con el fin de complementar el sistema de revisión por pares. También se entiende como una brújula para los investigadores en el dilema “publicar o perecer” con el fin de aumentar su visibilidad y para optimizar las estrategias de publicación. La evaluación individual se compone de una entrevista con el investigador evaluado, de la elaboración de un informe bibliométrico de las publicaciones del investigador, de una reunión y de una validación de los resultados obtenidos con el investigador, así como análisis adicionales opcionales. El informe bibliométrico se entrega al investigador, pues así éste conoce los aspectos cuantitativos de los resultados de su investigación. Adicionalmente, también se utiliza como base para discutir con él sus estrategias de publicación. Eventualmente, tal informe sirve para realizar la revisión por pares del investigador con más conocimiento de causa, y por lo tanto se reenvía a Aseguramiento de la calidad, a la oficina del Rector y finalmente a los pares. La característica más importante del informe bibliométrico es su carácter multidimensional e individual. Se fundamenta en varios indicadores básicos y otros parámetros de control con el fin de mejorar la interpretación. Tanto los investigadores, como el personal administrativo y los evaluadores han confirmado la utilidad de este enfoque bibliométrico, por lo que se ha producido una creciente demanda de los mismos. Hasta el momento se han entregado 33. Por otra parte, también se han realizado con gran éxito informes similares para la evaluación bibliométrica de dos facultades. AbstractThis paper shows how bibliometric assessment can be implemented at individual level. This has been successfully done at the University of Vienna carried out by the Department for Bibliometrics and Publication Strategies of the Vienna University Library. According to the department’s philosophy, bibliometrics is not only a helpful evaluation instrument in order to complement the peer review system. It is also meant as a compass for researchers in the ‘publish or perish’ dilemma in order to increase general visibility and to optimize publication strategies. The individual assessment comprises of an interview with the researcher under evaluation, the elaboration of a bibliometric report of the researcher’s publication output, the discussion and validation of the obtained results with the researcher under evaluation as well as further optional analyses. The produced bibliometric reports are provided to the researchers themselves and inform them about the quantitative aspects of their research output. They also serve as a basis for further discussion concerning their publication strategies. These reports are eventually intended for informed peer review practices, and are therefore forwarded to the quality assurance and the Rector’s office and finally sent to the peers. The most important feature of the generated bibliometric report is its multidimensional and individual character. It relies on a variety of basic indicators and further control parameters in order to foster comprehensibility. Researchers, administrative staff and peers alike have confirmed the usefulness of this bibliometric approach. An increasing demand is noticeable. In total, 33 bibliometric reports have been delivered so far. Moreover, similar reports have also been produced for the bibliometric assessment of two faculties with great success

Dilemma Development Partnership – Erfahrungen mit SciVal Spotlight und InCites

The Vienna University Library has been maintaining development partnerships concerning bibliometric products with both Thomson Reuters and Elsevier for several years now. The co-operation with more than one provider should allow a high amount of independence and objectiveness. We describe the activity as Early Adopter respectively Development Partner of SciVal Spotlight and InCites, the best known analytical tools for research Evaluation and planning currently offered in the market. Notwithstanding the considerable amount of work and certain strategic concerns, there are different positive effects for the institution, e.g. active participation in the development, in-depth knowledge about operating methods and specific characteristics of the products, and enhanced communication with their developers

Individual bibliometric assessment at University of Vienna: From numbers to multidimensional profiles

Se muestra cómo se puede implementar la evaluación bibliométrica a nivel individual, lo que se ha hecho con éxito en la Universidad de Viena, a cargo del Departamento Bibliometría y Estrategias de Publicación de la Biblioteca. De acuerdo con la filosofía del Departamento, la bibliometría no es más que un instrumento de evaluación útil con el fin de complementar el sistema de revisión por pares. También se entiende como una brújula para los investigadores en el dilema “publicar o perecer” con el fin de aumentar su visibilidad y para optimizar las estrategias de publicación. La evaluación individual se compone de una entrevista con el investigador evaluado, de la elaboración de un informe bibliométrico de las publicaciones del investigador, de una reunión y de una validación de los resultados obtenidos con el investigador, así como análisis adicionales opcionales. El informe bibliométrico se entrega al investigador, pues así éste conoce los aspectos cuantitativos de los resultados de su investigación. Adicionalmente, también se utiliza como base para discutir con él sus estrategias de publicación. Eventualmente, tal informe sirve para realizar la revisión por pares del investigador con más conocimiento de causa, y por lo tanto se reenvía a Aseguramiento de la calidad, a la oficina del Rector y finalmente a los pares. La característica más importante del informe bibliométrico es su carácter multidimensional e individual. Se fundamenta en varios indicadores básicos y otros parámetros de control con el fin de mejorar la interpretación. Tanto los investigadores, como el personal administrativo y los evaluadores han confirmado la utilidad de este enfoque bibliométrico, por lo que se ha producido una creciente demanda de los mismos. Hasta el momento se han entregado 33. Por otra parte, también se han realizado con gran éxito informes similares para la evaluación bibliométrica de dos facultades. AbstractThis paper shows how bibliometric assessment can be implemented at individual level. This has been successfully done at the University of Vienna carried out by the Department for Bibliometrics and Publication Strategies of the Vienna University Library. According to the department’s philosophy, bibliometrics is not only a helpful evaluation instrument in order to complement the peer review system. It is also meant as a compass for researchers in the ‘publish or perish’ dilemma in order to increase general visibility and to optimize publication strategies. The individual assessment comprises of an interview with the researcher under evaluation, the elaboration of a bibliometric report of the researcher’s publication output, the discussion and validation of the obtained results with the researcher under evaluation as well as further optional analyses. The produced bibliometric reports are provided to the researchers themselves and inform them about the quantitative aspects of their research output. They also serve as a basis for further discussion concerning their publication strategies. These reports are eventually intended for informed peer review practices, and are therefore forwarded to the quality assurance and the Rector’s office and finally sent to the peers. The most important feature of the generated bibliometric report is its multidimensional and individual character. It relies on a variety of basic indicators and further control parameters in order to foster comprehensibility. Researchers, administrative staff and peers alike have confirmed the usefulness of this bibliometric approach. An increasing demand is noticeable. In total, 33 bibliometric reports have been delivered so far. Moreover, similar reports have also been produced for the bibliometric assessment of two faculties with great success

The insoluble problems of books: What does Altmetric.com have to offer?

Purpose: Analyze the capabilities, functionalities and appropriateness of Altmetric.com as a data source for the bibliometric analysis of books in comparison to PlumX. Methodology: We perform an exploratory analysis on the metrics the Altmetric Explorer for Institutions platform offers for books. We use two distinct datasets of books. On the one hand, we analyze the Book Collection included in Altmetric.com. On the other, we use Clarivate’s Master Book List, to analyze Altmetric.com’s capabilities to download and merge data with external databases. Finally, we compare our findings with those obtained in a previous study performed in PlumX. Findings: Altmetric.com combines and orderly tracks a set of data sources combined by DOI identifiers to retrieve metadata from books, being Google Books its main provider. It also retrieves information from commercial publishers and from some Open Access initiatives, including those led by university libraries such as Harvard Library. We find issues with linkages between records and mentions or ISBN discrepancies. Furthermore, we find that automatic bots affect greatly Wikipedia mentions to books. Our comparison with PlumX suggests that none of these tools provide a complete picture of the social attention generated by books and are rather complementary than comparable tools. Practical implications: This study targets different audiences which can benefit from our findings. First, bibliometricians and researchers who seek for alternative sources to develop bibliometric analyses of books, with a special focus on the Social Sciences and Humanities fields. Second, librarians and research managers who are the main clients to which these tools are directed. Third, Altmetric.com itself as well as other altmetric providers who might get a better understanding of the limitations users encounter and improve this promising tool. Originality/value: This is the first study to analyze Altmetric.com’s functionalities and capabilities for providing metric data for books and to compare results from this platform, with those obtained via Plum

Uso de indicadores bibliométricos para el análisis de temas emergentes y su evolución: spin-offs como caso de estudio

[ES] Las spin-offs constituyen una de las áreas de investigación más atractivas, ya que están asociadas con fenómenos como el emprendimiento, la innovación y la transferencia del conocimiento. El presente estudio muestra que la selección y el uso de indicadores bibliométricos permite identificar y caracterizar el desarrollo y la difusión de temas de investigación emergentes como el analizado. Los principales aspectos observados en relación con el desarrollo de la investigación sobre las spin-offs son que se produce un auge en el número de publicaciones después de un largo período de latencia y la marcada naturaleza multidisciplinar del área. El presente enfoque ha analizado la evolución de las publicaciones científicas, los agentes científicos involucrados en las mismas considerando diferentes niveles analíticos, las prácticas cooperativas y las características estructurales de la red de coautorías. Asimismo, se ha analizado la bibliografía citada, la evolución de los indicadores bibliométricos clave, habiéndose efectuado un análisis cualitativo de validación de contenido por parte de un experto en el campo. Se ha determinado el carácter emergente del tema a través de varios indicadores, observando que existe una superposición entre las contribuciones de los autores seminales y el momento en que se convierten en investigadores de referencia en el campo. La antigüedad de la bibliografía citada constituye un destacado indicador para establecer la naturaleza emergente del tema y monitorizar su desarrollo.[EN] Spin-offs are one of the most attractive areas of research; associated with the phenomena of entrepreneurship, innovation, and knowledge transfer. The present study shows that the selection and use of appropriate bibliometric indicators are a highly valuable method for studying emerging topics and analyzing the development and diffusion of the topic under research, including its process of emergence and growth. The primary aspects observed in relation to the development of university spin-off research includes the boom in the number of publications on the topic after a long period of latency and the pronounced multidisciplinary nature of the research. Our approach encompasses the evolution of scientific publication activity in the area, the scientific agents involved with it, and the cooperative practices and structural characteristics of the co-authorship network at different analytical levels. Also, this research explores cited literature, the evolution of key bibliometric indicators, and adds a validating qualitative analysis of content by an expert in the field. Moreover, the emergence of the topic is shown to overlap between seminal authors¿ early research contributions to the topic and the time when they become investigators of reference in the field, with their work featured among the most highly cited documents. Last but not least, the age of the cited bibliography constitutes a prominent indicator for establishing the emerging nature of a topic as well as its stage of development.This work was supported by the Conselleria de Educacion, Investigacion, Cultura y Deporte, Generalitat Valenciana (BEST 2016/153).González-Alcaide, G.; Gorraiz, J.; Hervás Oliver, JL. (2018). On the use of bibliometric indicators for the analysis of emerging topics and their evolution: spin-offs as a case study. El profesional de la información. 27(3):35-52. https://doi.org/10.3145/epi.2018.may.04S355227

Six indicators in search of a meaning

This is a kind of conceptual document, and at the same time a reflection, but also something more, an act of provocation in the form of a piece of theater that shows a reality, something dramatic and comical at the same time, as is the panorama ofour research. It also includes a good portion of humor, but the kind of humor the great writer and thinker Luigi Pirandello defined as „the one that does not provoke the immediate laughter, but only pulls a smile behind which hides the bitter taste of the absurd or the unbearable lightness of our existence.“ This piece of theater, short and concise so as not to bore anyone, narrates the efforts that are required daily, as well as the situations created, to carry out the task –which is perhaps a utopia- of trying to measure something that is incommensurable, as is science on the part of those who are dedicated to its evaluation or to the study of its development. In this comedy, all actors involved, both protagonists and the indicators themselves, have the opportunity to present their hopes and fears, their merits and their complaints, their recommendations and warnings. They narrate the search for a path in a jungle in which the world of information has been converted, that is practicable, augurs a promising future for science, and allows us a more just interpretation