Statistical analyses of digital collections: Using a large corpus of systematic reviews to study non-citations

Using statistical methods to analyse digital material for patterns makes it possible to detect patterns in big data that we would otherwise not be able to detect. This paper seeks to exemplify this fact by statistically analysing a large corpus of references in systematic reviews. The aim of the analysis is to study the phenomenon of non-citation: Situations where just one (or some) document(s) are cited from a pool of otherwise equally citable documents. The study is based on more than 120,000 cited studies, and a total number of non-cited studies of more than 1.6 million. The number of cited studies is found to be much smaller than the number of non-cited. Also, the cited and non-cited studies are found to differ in age. Very recent studies tend to be non-cited whereas the cited studies are rarely of recent age (e.g. within the same year). The greatest differences are found within the first 10 years. After 10 years the cited and non-cited studies tend to be more similar in terms of age. Separating the data set into different sub-disciplines reveals that the sub-disciplines vary in terms of age of cited vs. non-cited references. Some fields may be expanding and the number of published studies is thus growing. Consequently, cited and non-cited studies tend to be younger. Other fields may be more slowly progressing fields that use a greater proportion of the older literature within the field. These field differences manifest themselves in the average age of references

Computer Supported Indexing: A History and Evaluation of NASA's MAI System

Computer supported indexing systems may be categorized in several ways. One classification scheme refers to them as statistical, syntactic, semantic or knowledge-based. While a system may emphasize one of these aspects, most systems actually combine two or more of these mechanisms to maximize system efficiency. Statistical systems can be based on counts of words or word stems, statistical association, and correlation techniques that assign weights to word locations or provide lexical disambiguation, calculations regarding the likelihood of word co-occurrences, clustering of word stems and transformations, or any other computational method used to identify pertinent terms. If words are counted, the ones of median frequency become candidate index terms. Syntactical systems stress grammar and identify parts of speech. Concepts found in designated grammatical combinations, such as noun phrases, generate the suggested terms. Semantic systems are concerned with the context sensitivity of words in text. The primary goal of this type of indexing is to identify without regard to syntax the subject matter and the context-bearing words in the text being indexed. Knowledge-based systems provide a conceptual network that goes past thesaurus or equivalent relationships to knowing (e.g., in the National Library of Medicine (NLM) system) that because the tibia is part of the leg, a document relating to injuries to the tibia should he indexed to LEG INJURIES, not the broader MeSH term INJURIES, or knowing that the term FEMALE should automatically be added when the term PREGNANCY is assigned, and also that the indexer should be prompted to add either HUMAN or ANIMAL. Another way of categorizing indexing systems is to identify them as producing either assigned- or derived-term indexes

El análisis de citas en trabajos de investigadores como método para el estudio del uso de información en bibliotecas

[spa] Revisión bibliográfica sobre el análisis de citas como método para el estudio del uso y las necesidades de información por parte de los investigadores como usuarios de bibliotecas. Una vez comparados los estudios locales basados en las publicaciones de los usuarios potenciales de una biblioteca con los estudios de análisis de citas basados en la bibliografía circulante a nivel internacional o nacional, se analizan las diferentes fuentes a partir de las cuales se pueden extraer y estudiar las referencias bibliográficas citadas por los autores estudiados a nivel local. Como un método indirecto para los estudios de usuarios y de uso de información, el análisis de citas es un método eficiente que no interfiere el comportamiento del colectivo estudiado, y que permite obtener un nivel de detalle en la información obtenida difícilmente comparable con el que se consigue por otros métodos. [eng]Bibliographic review about citation analysis of research publications as a method for study information needs and use by research users of libraries. Once compared the local citation analysis studies from the works of potential users of a library against the citation studies from the bibliography in the international or national data bases, the review takes account of the different document sources for the local citation analysis studies. As an indirect method for user and information use studies is an efficient unobtrusive method, that allow to collect information at a level of detail not available through other methods

Peer assessment or promotion by numbers? A comparative study of different measures of researcher performance within the UK Library and Information Science research community

Hirsch’s h-index, Egghe’s g-index, total citation and publication counts, and five proposed new metrics were correlated with one another using Spearman’s Rank Correlation for one hundred randomly selected academics and researchers working in UK Library and Information Science departments. Metrics were compared for individuals of different genders and at institutions awarded different RAE (2001) grades. Individuals’ metrics were rank-correlated against academic ranks and RAE (2001) grades of their employing departments. Metrics calculated using Web of Science and Google Scholar data were compared. Peer- and h-index metric-ranked orders of researchers were rank-correlated. Citation behaviour and attitudes towards peer and citation-based assessment of 263 academics and researchers were investigated by factor analysis of online attitudinal survey responses. h increased curvilinearly with total citation and publication counts, suggesting that h was constrained by the activity in the field preventing individuals producing enough heavily cited publications to increase their h-index scores. Most individuals therefore shared similar h-index scores, making interpersonal comparisons difficult. Total citation counts and Bihui’s a-index scores distinguished between more individuals, though whether they could confidently identify differences between individuals is uncertain. Both databases arbitrarily omitted individuals and publications, systematically biasing citation metrics calculated using them. In contrast to studies of larger fields, no citation metrics correlated with RAE grade, academic rank, or direct peer-assessment, suggesting that citation-based assessment is unsuitable for research fields with relatively little research activity. No gender bias was evident in academic rank, esteem or citedness. At least nine independent factors influence citation behaviour. Mertonian factors dominated. The independence of the factors suggested different individuals have different combinations of non-Mertonian motivations. The overriding meaning of citations was confirmed as signals of relevance and reward. Recommendations for future research include a need to develop simple, robust methods to identify subfields and normalise citations across subfields, to quantify the impact of random bias and to determine whether it varies across subfields, and to study the rate of accumulation of citations and citation distribution changes for individuals (and departments) over time to determine whether career age can be controlled for, in particular

Wissenschaftsindikatoren: Bewertungen in der Wissenschaft

Wissenschaft hat längst den Charme einer liebenswürdig chaotischen Gelehrtenstube verloren und ist zur "big science" geworden. Damit ist sowohl ein neuartiger Reflexionsbedarf entstanden als auch eine Fülle von Steuerungsproblemen. Wissenschaftsindikatoren sind eine Antwort auf die veränderten Rahmenbedingungen. Sie sollen Strukturen, Quantitäten und Qualitäten wissenschaftlicher Produktion meßbar machen. Ihre Entwicklung verlief jedoch in einem sehr pragmatischen Kontext, mit der Folge, daß der Kontakt zur Wissenschaftstheorie verloren ging und die Beurteilung des Nutzens von Indikatoren nicht selten in einen Glaubenskrieg ausartete. Mit dem vorliegenden Band wird nicht nur eine Verbindung zwischen Wissenschaftstheorie, -soziologie und der Indikatorenforschung geschaffen, sondernauch eine detaillierte Darstellung der Leistungsfähigkeit und der Grenzen der Wissenschaftsindikatoren vorgelegt