Users beware: implications of database errors when assessing the individual research records of ecologists and conservation biologists

Abstract

Assessments of scientists’ research records through citations are becoming increasingly important in management and in bibliometric research, but the databases available may contain errors that reduce the reliability of assessments. We investigated this by profiling our personal records in five databases: Scopus, Web of Knowledge, Web of Science, the Cited Reference Search within Web of Science, and the freeware Publish or Perish followed by correction in CleanPoP. We documented disparities between the results and our CVs, noting implications for bibliometric analyses from our perspective as conservation biologists. No database provided a complete, accurate record for anyone. Sometimes publications were out of range or missing, especially if they were books and book chapters. Other errors included mistakes in the order of authors or year of publication, as well as misattribution of publications. The Hirsch index (h) was robust across databases, but other metrics were more volatile. Nevertheless, all metrics except median citations/paper gave high correlations of 0.78 or greater for the rank order of authors across databases. Profiling researchers’ records without knowledge of their CVs will likely result in inaccurate assessments. Reliance on one database compounds the problem if the database does not encompass the researcher’s full output, especially books and book chapters. Coverage may be particularly important for conservation biologists, who sometimes publish material of local relevance in local journals not abstracted in some of the databases. Administrators and researchers seeking citation profiles should query multiple databases to obtain a more complete picture of research output and cross check against a full CV when possible. It may be unjustified to assume that discrepancies between database and CV indicate mistakes made by the researcher — verification from the original publication is necessary. Furthermore, citations are but one of many measures available for assessing the quality, use or impact of research, and their sole use, irrespective of possible errors, may be misleading