Learning from Digital Library Evaluations

In this paper we analyse evaluation studies of the Europeana digital library from its launch in 2009 until today. Using Saracevic’s digital library evaluation framework, the studies are categorised by their constructs, contexts, criteria, and methodologies. Concentrating on studies that evaluate Europeana services or single components, we show gaps in the evaluation of certain Europeana aspects. Finally, we derive strategies for building an evaluation archive that serves as memory and supports comparisons.Im vorliegenden Artikel analysieren wir Evaluationsstudien der digitalen Bibliothek Europeana von 2009 bis heute. Unter Berücksichtigung von Saracevic’ Evaluationsframework für digitale Bibliotheken werden die Studien nach ihren Konstrukten, Kontexten, Kriterien und Methodologien kategorisiert. Die Analyse konzentriert sich auf Studien, die Dienstleistungen oder einzelne Komponenten von Europeana evaluieren, und zeigt Lücken in der Evaluation bestimmter Aspekte von Europeana auf. Schließlich werden Strategien diskutiert, um ein Evaluationsarchiv zu entwickeln, welches sowohl der Langzeitarchivierung dient als auch Vergleiche von Evaluationsergebnissen unterstützt.Peer Reviewe

Contextualized Digital Library Evaluation: The Perseus Digital Library within Theological Research

As the development of digital libraries increases, the growth has been accompanied by a refinement of evaluation criteria and methods. Evaluators are increasingly aware that the context of digital library usage (the social, cultural, academic, and institutional environments of the users) is an essential consideration of effective assessment. This article evaluates the Perseus Digital Library through the needs and objectives of a specific learning community – those engaged in theological research. After a review of relevant literature, a rationale is given for ranking ten key facets of digital library evaluation by order of importance. These criteria are then applied to the Perseus Digital Library, contextualized through the particular prism of theological research

Analysis of navigational interfaces through the use of knowledge representation schemes

(Spa) Actualmente la mayoría de repositorios digitales incluyen esquemas de representación de conocimiento como ontologías y tesauros para ofrecer a creadores y expertos varias alternativas que permitan gestionar, clasificar y organizar objetos de aprendizaje dentro de un repositorio; así como facilitar el uso del repositorio para ayudar a los usuarios a localizar objetos de aprendizaje, desplegar información complementaria, identificar sus relaciones, categorías, áreas de conocimiento, etc. Sin embargo, estos esquemas de clasificación a veces no son utilizados y aprovechados en la medida que sería deseable debido a: i) los usuarios no están familiarizados con los complejos sistemas de representación de conocimiento, que los expertos y creadores de repositorios utilizan para clasificar los recursos, y ii) las interfaces de usuario actuales no siempre facilitan las tareas esenciales de los usuarios que visitan el repositorio: búsqueda y localización de recursos digitales. La motivación de este artículo se centra en el segundo factor, presentando un análisis de las deficiencias de las actividades de búsqueda y navegación mediante el diseño e implementación de interfaces navegacionales a partir del uso de esquemas de representación de conocimiento.(Eng) Currently, the most digital repositories include knowledge representation schemes such as ontologies and thesauri in order to offer to creators and experts several alternatives able to manage, sort and organize learning objects within a repository. Also facilitate the use of the repository to help users locate learning objects, displaying additional information, identify relationships, categories, areas of expertise. However, these classification schemes sometimes have not been used and exploited to the extent desirable due to: i) users are not familiar with the complex systems of knowledge representation, which experts and creators of repositories used to classify resources, and ii) the current user interfaces do not always provide the essential tasks of users who visit the repository: search and locate digital resources. The motivation of this article focuses on the second factor, presenting a gap analysis of the activities of search and navigation through the design and implementation of navigational interfaces from the use of knowledge representation schemes

Digital Collection Evaluation: A Rubric for Holistic Evaluation

Digital collections have been a rising trend in library sciences for over a decade. However, analysis of these collections has still largely been limited to the digital specialists and the digital humanists. This paper summarizes the existing evaluation literature to propose a tool for librarians to use for their own individual collections' evaluations. It also examines the difficulties of evaluation and emphasizes the need for further research into librarian conducted analyses, as their evaluations differ from the evaluations of an expert. It also explains the development of digitization, digital collections and digital evaluation until this point.Master of Science in Library Scienc

Detecting Communities and Analysing Interactions with Learning Objects in Online Learning Repositories

The widespread use of online learning object repositories has raised the need of studies that assess the quality of their contents, and their user’s performance and engagement. The present research addresses two fundamental problems that are central to that need: the need to explore user interaction with these repositories and the detection of emergent communities of users. The current dissertation approaches those directions through investigating and mining the Khan Academy repository as a free, open access, popular online learning repository addressing a wide content scope. It includes large numbers of different learning objects such as instructional videos, articles, and exercises. In addition to a large number of users. Data was collected using the repository’s public application programming interfaces combined with Web scraping techniques to gather data and user interactions. Different research activities were carried out to generate useful insights out of the gathered data. We conducted descriptive analysis to investigate the learning repository and its core features such as growth rate, popularity, and geographical distribution. A number of statistical and quantitative analysis were applied to examine the relation between the users’ interactions and different metrics related to the use of learning objects in a step to assess the users’ behaviour. We also used different Social Network Analysis (SNA) techniques on a network graph built from a large number of user interactions. The resulting network consisted of more than 3 million interactions distributed across more than 300,000 users. The type of those interactions is questions and answers posted on Khan Academy’s instructional videos (more than 10,000 video). In order to analyse this graph and explore the social network structure, we studied two different community detection algorithms to identify the learning interactions communities emerged in Khan Academy then we compared between their effectiveness. After that, we applied different SNA measures including modularity, density, clustering coefficients and different centrality measures in order to assess the users’ behaviour patterns and their presence. Using descriptive analysis, we discovered many characteristics and features of the repository. We found that the number of learning objects in Khan Academy’s repository grows linearly over time, more than 50% of the users do not complete the watched videos, and we found that the average duration for video lessons 5 to 10 minutes which aligns with the recommended duration in literature. By applying community detection techniques and social network analysis, we managed to identify learning communities in Khan Academy’s network. The size distribution of those communities found to follow the power-law distribution which is the case of many real-world networks. Those learning communities are related to more than one domain which means the users are active and interacting across domains. Different centrality measures we applied to focus on the most influential players in those communities. Despite the popularity of online learning repositories and their wide use, the structure of the emerged learning communities and their social networks remain largely unexplored. Our findings could be considered initial insights that may help researchers and educators in better understanding online learning repositories, the learning process inside those repositories, and learner behaviou

Evaluación de la actividad científica en ciencia de la información a partir de indicadores bibliométricos y altmétricos

La presente investigación es un análisis de la producción científica en Ciencia de la Informacion (CI), fundamentada en el contexto epistemológico e histórico de la disciplina, para identificar las tendencias de uso de la información en plataformas de publicación formales e informales. A partir de la implementación de indicadores bibliométricos e indicadores alternativos, se pretende establecer. ¿Como la integración de indicadores altimétricos en la evaluación científica, posibilita la identificación de tendencias en la investigación disciplinar? Y si es valido afirmar, que la altmetría es una herramienta confiable y útil para la evaluación de los dominios científicos. Se toma como referente la producción visible en Web of Science durante el periodo 2012- 2016, para identificar las dinámicas científicas de investigación en la CI, a partir de una muestra de 1224 registros en los cuales se utilizan indicadores bibliometricos de producción, citación o impacto e indicadores altimétricos recuperados de las plataformas ResearchGate (RG) y Plum Analytics (PlumX). Los resultados evidencian que los indicadores alternativos aun están en periodo de desarrollo y necesitan normalización; de lo cual se concluye, que la evaluación científica requiere la complementación de modelos métricos clásicos junto a métricas alternativas que permitan identificar las dinámicas sociales y de comunicación que se generan en la comunidad científica más allá del impacto y la citación.This research is an analysis of the scientific activity in Information Science (CI), based on the epistemological and historical context of the discipline, to identify trends in the use of information in formal and informal publishing platforms. Based on the implementation of bibliometric iand alternative indicators, it is intended to establish: How does the integration of altmetric indicators in scientific evaluation make it possible to identify trends in disciplinary research? And, if it is valid to say that altmetrics is a reliable and useful tool for the scientific evaluation of scientific domains. Visible production in Web of Science during the 2012-2016 period is taken as a reference to identify the scientific dynamics of research in the CI, from a sample of 1224 records in which bibliometric indicators of production, citation or impact and altmetric indicators recovered from the ResearchGate (RG) and Plum Analytics (PlumX) platforms are used. The results show that the alternative indicators are still under development and need to be standardized; from which it is concluded that scientific evaluation requires the complementing of classical metric models with alternative metrics that allow identifying the social and communication dynamics generated in the scientific community beyond the impact and citation.Profesional en Ciencia de la Información - Bibliotecólogo (a)Pregrad