Developing Best Practices in Digital Library Assessment: Year One Update

In the face of limited resources and increasing demand for online access to digital library content, we need to strategically focus our efforts and better understand users, impact, and associated costs. However, methods for assessment of digital libraries are not standardized. In an effort to address this crucial gap, the Digital Library Federation Assessment Interest Group has engaged the community over the past year in the development of best practices and guidelines. With this article, the authors provide an update on progress to date and solicit participation in an evolving effort to develop viable solutions.Librarie

Publishing a Scorecard for Evaluating the Use of Open-Access Journals Using Linked Data Technologies

Open access journals collect, preserve and publish scientific information in digital form, but it is still difficult not only for users but also for digital libraries to evaluate the usage and impact of this kind of publications. This problem can be tackled by introducing Key Performance Indicators (KPIs), allowing us to objectively measure the performance of the journals related to the objectives pursued. In addition, Linked Data technologies constitute an opportunity to enrich the information provided by KPIs, connecting them to relevant datasets across the web. This paper describes a process to develop and publish a scorecard on the semantic web based on the ISO 2789:2013 standard using Linked Data technologies in such a way that it can be linked to related datasets. Furthermore, methodological guidelines are presented with activities. The proposed process was applied to the open journal system of a university, including the definition of the KPIs linked to the institutional strategies, the extraction, cleaning and loading of data from the data sources into a data mart, the transforming of data into RDF (Resource Description Framework), and the publication of data by means of a SPARQL endpoint using the OpenLink Virtuoso application. Additionally, the RDF data cube vocabulary has been used to publish the multidimensional data on the web. The visualization was made using CubeViz a faceted browser to present the KPIs in interactive charts.This work has been partially supported by the Prometeo Project by SENESCYT, Ecuadorian Government

30-year bibliometric analysis of digital library studies

The purpose of this study is to analyse and map the metadata and keywords associated with existing research publications in digital library studies during the last 30 years. This is a quantitative descriptive study that employs bibliometric analysis approaches namely co-authorship analysis, citation analysis, and text data mapping with a help of VOSviewer computer application. The dimensions.ai, an international article indexing service, was used as the source to collect samples in this study. A total of 1,185 documents discussing digital library studies were analysed during 1991 – 2021. The analysis discovered that Edward A. Fox is the most prolific author in the field of digital library studies, the University of Strathclyde is the most productive university, and the United States is the most prolific country. This study also found that an article entitled “Automatic recognition of multi-word terms: the c-value/ nc-value method” is the most cited article within the dataset with 390 citations, and the most productive journal publishing cited articles is d-lib magazine with 201 published documents. In addition, several interesting terms like “academic library”, “university library”, “challenge”, “problem”, “india”, “digital resources”, and “evaluation” appeared to be the most discussed keywords within all publications. Academics and librarians from many countries have spent years researching digital libraries. This work demonstrated the importance of digital libraries for all countries, particularly when it comes to providing students and ordinary readers with access to e-books and other digital materials

An evaluation based on the digital library user: An experience with greenstone software

Digital libraries are a product of digital information aimed at satisfying users’ information needs, through collections of documents to which value added services are provided. To ensure the correct functioning of digital libraries and the services they provide it is essential to adequately include an understanding of the needs of users and their behavior in the design and creation processes. The evaluation of digital libraries has benefited from an effort to establish theoretical frameworks. The model that is proposed is based on techniques centered on the user. The focus is on defining the structure of the analysis, establishing evaluation levels and criteria, and verifying these through an evaluation process, carried out by a group of end users. The user activity identified, and which was the subject of the study, is the process of creating, enriching and configuring a collection for a digital library. A basic evaluation instrument was designed, which combined closed and guided questionnaires with the collection and study of individual opinions of users in the test groups. The results obtained made it possible to confirm that the instrument has produced valid results. The weaknesses and problems inherent in the tool, and the tool's deficiencies which must be corrected, were identified

Use of 5S model digital library functionality assessment

Ovaj rad ima za cilj primijeniti 5S (Stream, Structure, Spaces, Scenario, Society) model za izradu digitalnih knjižnica kao model za evaluaciju digitalnih knjižnica. Temelji se na postojećem teorijskom korpusu donesenom od strane 5S modela da evaluira razne dijelove digitalne knjižnice ili repozitorija uz pomoć aplikacije za evaluaciju koja je napravljena u tu svrhu. Ovaj rad donosi metodu evaluacije uz korištenje aplikacije da evaluira dijelove od svakog od 5S modela te da dade korisniku numeričke, statističke podatke, kontekst i prijedloge kod analiziranog repozitorija ili digitalne knjižnice. Korištenjem ove metode vlasnik digitalne knjižnice može unaprijediti kvalitetu svoje digitalne knjižnice ili repozitorija. Rad se sastoji od teorijskog dijela koji objašnjava evaluacije, postupke i korake kod primjene evaluacije i programskog dijela u kojem je razvijena aplikacija za automatiziranu evaluaciju digitalne knjižnice ili repozitorija u JAVA programskom jeziku te je lokalizirana za primjenu na prostorima Republike Hrvatske. Aplikacija zahtijeva OAI-PMH pristup. Rad donosi metodu evaluacije koja se u potpunosti temelji na 5S modelu, koja je jednostavna za izvedbu, tumačenje i djelomično automatizirana

Use of 5S model digital library functionality assessment

Ovaj rad ima za cilj primijeniti 5S (Stream, Structure, Spaces, Scenario, Society) model za izradu digitalnih knjižnica kao model za evaluaciju digitalnih knjižnica. Temelji se na postojećem teorijskom korpusu donesenom od strane 5S modela da evaluira razne dijelove digitalne knjižnice ili repozitorija uz pomoć aplikacije za evaluaciju koja je napravljena u tu svrhu. Ovaj rad donosi metodu evaluacije uz korištenje aplikacije da evaluira dijelove od svakog od 5S modela te da dade korisniku numeričke, statističke podatke, kontekst i prijedloge kod analiziranog repozitorija ili digitalne knjižnice. Korištenjem ove metode vlasnik digitalne knjižnice može unaprijediti kvalitetu svoje digitalne knjižnice ili repozitorija. Rad se sastoji od teorijskog dijela koji objašnjava evaluacije, postupke i korake kod primjene evaluacije i programskog dijela u kojem je razvijena aplikacija za automatiziranu evaluaciju digitalne knjižnice ili repozitorija u JAVA programskom jeziku te je lokalizirana za primjenu na prostorima Republike Hrvatske. Aplikacija zahtijeva OAI-PMH pristup. Rad donosi metodu evaluacije koja se u potpunosti temelji na 5S modelu, koja je jednostavna za izvedbu, tumačenje i djelomično automatizirana

Learning from Digital Library Evaluations

In this paper we analyse evaluation studies of the Europeana digital library from its launch in 2009 until today. Using Saracevic’s digital library evaluation framework, the studies are categorised by their constructs, contexts, criteria, and methodologies. Concentrating on studies that evaluate Europeana services or single components, we show gaps in the evaluation of certain Europeana aspects. Finally, we derive strategies for building an evaluation archive that serves as memory and supports comparisons.Im vorliegenden Artikel analysieren wir Evaluationsstudien der digitalen Bibliothek Europeana von 2009 bis heute. Unter Berücksichtigung von Saracevic’ Evaluationsframework für digitale Bibliotheken werden die Studien nach ihren Konstrukten, Kontexten, Kriterien und Methodologien kategorisiert. Die Analyse konzentriert sich auf Studien, die Dienstleistungen oder einzelne Komponenten von Europeana evaluieren, und zeigt Lücken in der Evaluation bestimmter Aspekte von Europeana auf. Schließlich werden Strategien diskutiert, um ein Evaluationsarchiv zu entwickeln, welches sowohl der Langzeitarchivierung dient als auch Vergleiche von Evaluationsergebnissen unterstützt.Peer Reviewe

Can digital libraries generate knowledge?

"This paper argues that the information processing paradigm which is still fundamental to digital library engineering is not adequate for the way in which information and informational objects - as stored in Digital Library systems - should be treated. The recent move within information science towards emancipation from the Shannon/ Weaver information theory concept provides some indications of how Digital Libraries could be conceptualized in a way that has not yet been implemented in standardized formal frameworks such as the DELOS or the 5S-Framework. A deeper understanding of information processes beyond the concepts and usage of such systems could help to make better use of the enormous potential and resources in the field of Digital Library engineering. Resulting effects could be the way in which knowledge transfer and acquisition processes are supported by adequate interfaces and, subsequently, by new ways of embedding collections in communities of information practice, such as research teams or learning groups." (author's abstract

Contextualized Digital Library Evaluation: The Perseus Digital Library within Theological Research

As the development of digital libraries increases, the growth has been accompanied by a refinement of evaluation criteria and methods. Evaluators are increasingly aware that the context of digital library usage (the social, cultural, academic, and institutional environments of the users) is an essential consideration of effective assessment. This article evaluates the Perseus Digital Library through the needs and objectives of a specific learning community – those engaged in theological research. After a review of relevant literature, a rationale is given for ranking ten key facets of digital library evaluation by order of importance. These criteria are then applied to the Perseus Digital Library, contextualized through the particular prism of theological research

Implementation and usage of digital Libraries: A case study of Bunda College Greenstone digital Library in Malawi.

The study investigated how the Bunda College Digital Library was implemented and used. 91 undergraduates, 10 postgraduates and 43 academic staff responded to questionnaires on the digital library usage. Facebook instant messages and phone interview were used to obtain data from library staff about managerial and technical activities. The digital library (DL) was created to address the need for digitizing documents on Malawi and/or by Malawians. Staff members were trained in the DL activities, however it lacked funding and largely utilized available resources. All the three categories of respondents agreed that the digital library was useful. Engaging in DL consultancy services, employing one library staff on temporary basis and including activities of the DL in the annual budgets were presented as the major recommendations. The study was limited in the sense that 3rd and 4th undergraduate students were on holiday during the data collection period and many postgraduates were out for their field research. This affected the richness of the study. Bunda College Library staff, Malawian librarians and Bachelor of Library Science students at Mzuzu University would find this research useful. Although similar research initiatives have been carried out, this research is unique as it tackles managerial and technical aspects of DL implementation as well as usage by the end users