A Proposed Framework for the Evaluation of Academic Librarian Scholarship

The ACRL Impactful Scholarship and Metrics Task Force has created a framework draft that is designed to help librarians and libraries contextualize their impact within academic librarianship. To create this framework, the task force studied existing disciplinary models, institutional guidelines, and surveyed academic librarians. The task force discovered few standard practices regarding impact measurement from disciplinary societies or in institutional documentation, but did find some larger models outlining distinct impact areas. The proposed framework outlines evaluation in two primary impact areas for academic librarians, scholarly and practitioner impact, with suggested metrics for a range of research outputs in each category. It is envisioned that this framework will help initiate conversations at institutions with the aim of reviewing and revising existing documentation, alongside complementary ACRL initiatives that will similarly affect scholarly production and evaluation. The first framework draft was revised based on academic librarian feedback, and could be finalized as an ACRL document in 2020

The academic papers researchers regard as significant are not those that are highly cited

For many years, academia has relied on citation count as the main way to measure the impact or importance of research, informing metrics such as the Impact Factor and the h-index. But how well do these metrics actually align with researchers' subjective evaluation of impact and significance? Rachel Borchardt and Matthew R. Hartings report on a study that compares researchers' perceptions of significance, importance, and what is highly cited with actual citation data. The results reveal a strikingly large discrepancy between perceptions of impact and the metric we currently use to measure it

ACRL Framework for Impactful Scholarship and Metrics

The ACRL Impactful Scholarship and Metrics Task Force was formed primarily to create a framework for the measurement and evaluation of academic librarian scholarship. The framework is designed to address gaps between current scholarly evaluation practices and impactful scholarly activities within academic librarianship, including ways to measure and evaluate the impact of a wide range of research outputs

Resilience trinity: safeguarding ecosystem functioning and services across three different time horizons and decision contexts

Ensuring ecosystem resilience is an intuitive approach to safeguard the functioning of ecosystems and hence the future provisioning of ecosystem services (ES). However, resilience is a multi-faceted concept that is difficult to operationalize. Focusing on resilience mechanisms, such as diversity, network architectures or adaptive capacity, has recently been suggested as means to operationalize resilience. Still, the focus on mechanisms is not specific enough. We suggest a conceptual framework, resilience trinity, to facilitate management based on resilience mechanisms in three distinctive decision contexts and time-horizons: i) reactive, when there is an imminent threat to ES resilience and a high pressure to act, ii) adjustive, when the threat is known in general but there is still time to adapt management, and iii) provident, when time horizons are very long and the nature of the threats is uncertain, leading to a low willingness to act. Resilience has different interpretations and implications at these different time horizons, which also prevail in different disciplines. Social ecology, ecology, and engineering are often implicitly focussing on provident, adjustive, or reactive resilience, respectively, but these different notions and of resilience and their corresponding social, ecological, and economic trade-offs need to be reconciled. Otherwise, we keep risking unintended consequences of reactive actions, or shying away from provident action because of uncertainties that cannot be reduced. The suggested trinity of time horizons and their decision contexts could help ensuring that longer-term management actions are not missed while urgent threats to ES are given priority

New Grads, Meet New Metrics: Why Early Career Librarians Should Care About Altmetrics & Research Impact

In Brief How do academic librarians measure their impact on the field of LIS, particularly in light of eventual career goals related to reappointment, promotion, or tenure? The ambiguity surrounding how to define and measure impact is arguably one of the biggest frustrations that new librarians face, especially if they are interested in producing scholarship […

Meaningful metrics: a 21st century librarian's guide to bibliometrics, altmetrics, and research impact

Foreword A few days ago, we were speaking with an ecologist from Simon Fraser University here in Vancouver about an unsolicited job offer he’d recently received. The offer included an astonishing inducement: Anyone from his to-be-created lab who could wangle a first or corresponding authorship of a Nature paper would receive a bonus of $100,000. Are we seriously this obsessed with a single journal? Who does this benefit? (Not to mention, one imagines the unfortunate middle authors of such a paper, trudging to a rainy bus stop as their endian-authoring colleagues roar by in jewel-encrusted Ferraris.) Although it’s an extreme case, it’s sadly not an isolated one. Across the world, a certain kind of administrator is doubling down on 20th-century, journal-centric metrics like the impact factor. That’s particularly bad timing because our research communication system is just beginning a transition to 21st-century communication tools and norms. We’re increasingly moving beyond the homogeneous, journal- based system that defined 20th-century scholarship

Library Supported Open Access Funds: Criteria, Impact, and Viability

Objective – This study analyzes scholarly publications supported by library open access funds, including author demographics, journal trends, and article impact. It also identifies and summarizes open access fund criteria and viability. The goal is to better understand the sustainability of open access funds, as well as identify potential best practices for institutions with open access funds.Methods – Publication data was solicited from universities with open access (OA) funds, and supplemented with publication and author metrics, including Journal Impact Factor, Altmetric Attention Score, and author h-index. Additionally, data was collected from OA fund websites, including fund criteria and guidelines.Results – Library OA funds tend to support faculty in science and medical fields. Impact varied widely, especially between disciplines, but a limited measurement indicated an overall smaller relative impact of publications funded by library OA funds. Many open access funds operate using similar criteria related to author and publication eligibility, which seem to be largely successful at avoiding the funding of articles published in predatory journals.Conclusions – Libraries have successfully funded many publications using criteria that could constitute best practices in this area. However, institutions with OA funds may need to identify opportunities to increase support for high-impact publications, as well as consider the financial stability of these funds. Alternative models for OA support are discussed in the context of an ever-changing open access landscape

Borchardt Hartings JACSChallenge Data.xlsx

The condensed dataset results of a survey, distributed to chemistry researchers in various chemistry subdisciplines, asking impact-related questions about one issue in JACS