Rethinking Assessment: Information Literacy Instruction and the ACRL Framework

Most information literacy instruction (ILI) done in academic libraries today is based on the ACRL’s Information Literacy Competency Standards for Higher Education, but with the replacement of these standards by the new Framework for Information Literacy for Higher Education, there is a need to re-evaluate current teaching strategies and instructional techniques so that they can better serve the Framework’s goals. This paper explores current trends in ILI instruction and in the area of assessment in particular, since ILI assessment provides an opportunity not only to evaluate teaching effectiveness but also to reinforce the learning goals of the new Framework itself. It proposes several ways that assessment strategies can be aligned with the goals of the Framework by using guided group discussion, online discussion platforms, and social media platforms, and proposes further avenues for research in the evaluation of such strategies

Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data

In this study, we address the question whether (and to what extent, respectively) altmetrics are related to the scientific quality of papers (as measured by peer assessments). Only a few studies have previously investigated the relationship between altmetrics and assessments by peers. In the first step, we analyse the underlying dimensions of measurement for traditional metrics (citation counts) and altmetrics - by using principal component analysis (PCA) and factor analysis (FA). In the second step, we test the relationship between the dimensions and quality of papers (as measured by the post-publication peer-review system of F1000Prime assessments) - using regression analysis. The results of the PCA and FA show that altmetrics operate along different dimensions, whereas Mendeley counts are related to citation counts, and tweets form a separate dimension. The results of the regression analysis indicate that citation-based metrics and readership counts are significantly more related to quality, than tweets. This result on the one hand questions the use of Twitter counts for research evaluation purposes and on the other hand indicates potential use of Mendeley reader counts

Aspects of accountability and assessment in the Netherlands

This article describes aspects of test-based accountability in the Netherlands. It provides a description of the design of the Educational system in the Netherlands, it gives a short introduction to the role of the Dutch Inspectorate of Education in the accountability of schools and describes different assessments that are used as sources of information in the accountability system. For each assessment, the primary function in education and its role in the accountability system are discussed. Finally, the factors that can potentially influence the validity of the accountability indicators and the strong and weak points of the current system are identified and some directions are presented of potential developments of this system

Academic Integrity Resources - links and guides

an online tutorial, a pdf version, a powerpoint presentation, links to regulations

Item-writing Rules: Collective Wisdom

In student assessment, teachers place the greatest weight on tests they have constructed themselves and have an equally great interest in the quality of those tests. To increase the validity of teacher-made tests, many item-writing rules-of-thumb are available in the literature, but few rules have been tested experimentally. In light of the paucity of empirical studies, the validity of any given guideline might best be established by relying on experts. This study analyzed twenty classroom assessment textbooks to identify a consensus list of item-writing rules. Forty rules for which there was agreement among textbook authors are presented. The rules address four different validity concerns-potentially confusing wording or ambiguous requirements, the problem of guessing, test-taking efficiency, and controlling for testwiseness

The metric tide: report of the independent review of the role of metrics in research assessment and management

This report presents the findings and recommendations of the Independent Review of the Role of Metrics in Research Assessment and Management. The review was chaired by Professor James Wilsdon, supported by an independent and multidisciplinary group of experts in scientometrics, research funding, research policy, publishing, university management and administration. This review has gone beyond earlier studies to take a deeper look at potential uses and limitations of research metrics and indicators. It has explored the use of metrics across different disciplines, and assessed their potential contribution to the development of research excellence and impact. It has analysed their role in processes of research assessment, including the next cycle of the Research Excellence Framework (REF). It has considered the changing ways in which universities are using quantitative indicators in their management systems, and the growing power of league tables and rankings. And it has considered the negative or unintended effects of metrics on various aspects of research culture. The report starts by tracing the history of metrics in research management and assessment, in the UK and internationally. It looks at the applicability of metrics within different research cultures, compares the peer review system with metric-based alternatives, and considers what balance might be struck between the two. It charts the development of research management systems within institutions, and examines the effects of the growing use of quantitative indicators on different aspects of research culture, including performance management, equality, diversity, interdisciplinarity, and the ‘gaming’ of assessment systems. The review looks at how different funders are using quantitative indicators, and considers their potential role in research and innovation policy. Finally, it examines the role that metrics played in REF2014, and outlines scenarios for their contribution to future exercises