Intervention au 36e congrès LIBER qui s\u27est tenu à Varsovie du 3 au 7 juillet 2007.
Réflexions et modèles d\u27évaluation pour mesurer l\u27audience des publications à l\u27âge des archives ouvertes

Scholze, Frank

English

Bibliothèque numérique de l'enssib

Bibliothèque numérique de l’enssibCe document est « tous droits réservés ». Il est protégé par le droit d’auteur et le code de la propriété intellectuelle. Il est strictement interdit de le reproduire, dans sa forme ou son contenu, totalement ou partiellement, sans un accord écrit de son auteur.Measuring Impact in an Open Access EnvironmentSCholze, FrankStuttgart University libraryl’ensemble des documents mis en ligne par l’enssib sont accessibles à partir du site : http://www.enssib.fr/bibliotheque-numerique/european integration: conditions and challenges for libraries, 3 au 7 juillet 200736e congrès lIBeRSCholze, Frank. Measuring Impact in an open Access environment. In 36th LIBER Annual General Conference, European integration: conditions and challenges for libraries, Varsovie, du 3 au 7 juillet 2007 [en ligne]. Format PDF.Disponible sur : <http://www.enssib.fr/bibliotheque-numerique/notice-1286>www.uni-stuttgart.deMeasuring Impact  in an Open Access EnvironmentFrank ScholzeStuttgart University LibraryLIBER 36th Annual ConferenceBiblioteka Narodowa, Warsaw, 5.7.2007www.uni-stuttgart.deResearch Impact“Impact is any change resulting from an activity, project, or organization. It includes intended as well as unintended effects, negative as well as positive, and long-term as well as short-term.”From: Susan Wainwright (2002). Measuring Impact - A Guide to Resources. NCVO.¾ Qualitative vs quantitative methods to determine impact¾ Peer-review, voting, “endorsement”¾ Publications are the quantifiable output of the research processwww.uni-stuttgart.deBackground¾ Metrics as a research policy issue ¾ Assessment and evaluation of research Appointment decisions Funding decisions Monitoring trends Prioritize activities and attentionRankings are here to stay, and it is therefore worth the time and effortto get them right.Alan Gilbert (President University of Manchester)From: D Butler (2007) Academics strike back at spurious rankings, Nature 447, pp 514-515www.uni-stuttgart.de¾ Possibility to collect and process quantitative data on electronic publications   Usage Citations¾ Possibility to construct new indicators to measure different aspects of research impact¾ Possibility to enhance and complement existing metricsOpen Access and Metrics become open access as well !www.uni-stuttgart.deCitations and the Journal Impact FactorJournal Impact Factor: mean 2-year citation rate2003 citations to 2001 and 2002 articles in XDivided bynumber of articles published in X in 2001 and 200220032001 2002Journal x All (2003)Widely applied in research evaluation• Fair approximation of journal “status”,…but• Used to rank authors, departments, institutions, regions, nations, etc.• Now common in tenure, promotion and other evaluation procedures!www.uni-stuttgart.deAcademic authors’ views on the value and use of Journal Impact FactorsFrom: Peter T Shepherd (2007) Final Report on the Investigation into the Feasibility of Developing and Implementing Journal Usage Factorswww.uni-stuttgart.dePage RankReciprocal voting of nodes in a networkCan be applied to anycollection of entities withreciprocal referencesRelated approaches in network analysis includeauthority and hub values(HITS algorithm)www.uni-stuttgart.deGoogle’s Page RankTechnoratiWikiosity.comAmazon.comCollaborative filteringFlickr.orgdel.icio.usTag cloudsA Taxonomy of MetricsImpact Factorcitation countsAuthorsUsersFrequentist“counting”Structural“pattern”Network analysisWebometricsBased on: Bollen, Johan and Van de Sompel, Herbert (2005) A framework for assessing the impact of units of scholarly communication based on OAI-PMH harvesting of usage information. OAI4, Genevausagecitationswww.uni-stuttgart.deLink-ResolversCollectAggregateLogfile analysisCollectAggregateCitation analysisParseExtractAggregateData miningMetrics (structural, frequentist)Services (ranking, evaluation, recommender, collectionbuilding and management...)Basic set of documents (Journals, repositories, primarydata etc.)Compare, aggregateMeasuring Publication Impact: The Elements (schematic)choosecollectanalysewww.uni-stuttgart.deDataMiningFilteringMetricsServicesAggregatedlogsLog DBOpenURLContextObjectsLogRepository Link ResolverLogRepository Link ResolverLogRepository Log harvester(Service Provider)COCOCOCOCOCOCOCOCOAggregated Usage DataLog DBWebserver-LogAggregated Usage DataRewritemoduleNormalise (optional) -> Robots, psydonymizationOpenURLContextObjectsor SUSHINormaliseBased on: Bollen, Johan and Van de Sompel, Herbert,  OAI4, GenevaCOUNTERe.g.e.g.Infrastructure for Collecting Usage Datawww.uni-stuttgart.deOngoing Work¾ LANL  bX (with CalState, ExLibris) MESUR …¾ UK University of Southampton (Citebase, IRS, EPStats …) University College London …¾ Germany (DINI / DFG)  Göttingen State and University Library Stuttgart University Library Computer and Media Service Humboldt University Berlin Saarbrücken State and University Librarywww.uni-stuttgart.deExamples: bX - Comparison of Journal Usage PageRank and Journal Impact FactorJournal of Molecular Graphics and ModellingDr. Dobb's Journal Usage > IFwww.uni-stuttgart.deExamples: MESUR – Aspects of ImpactFrom: Bollen, Johan (2007) MESUR: metrics from scholarly usage of resources OAI5, Genevawww.uni-stuttgart.deExamples: DINI - Infrastructure¾ Cluster of proposals to the DFG Network of certified open access repositories 2y 9– National input to EU repository infrastructure project DRIVER Usage statistics demonstrator9 Distributed open access reference citation service demonstrator9¾ Co-operation with German collecting society for copyrightcharges (VG Wort) Statistics based payout to authors (METIS)www.uni-stuttgart.de¾ Scholarly evaluation will evolve as scholarly communication does¾ Infrastructure for collecting and aggregating usage data conceptually available, has to be deployed and implemented in practice on a large scale (DINI/DFG)¾ Investigating metrics for different needs and purposes (MESUR)Conclusion

Measuring Impact in an Open Access Environment

http://www.enssib.fr/bibliotheque-numerique/document-1286

Measuring Impact in an Open Access Environment

Abstract

Similar works

Full text

Available Versions

Bibliothèque numérique de l'enssib