Federated Searching Interface Techniques for Heterogeneous OAI Repositories

Federating repositories by harvesting heterogeneous collections with varying degrees of metadata richness poses a number of challenging issues: (1) how to address the lack of uniform control for various metadata fields in terms of building a rich unified search interface, and (2) how easily new collections and freshly harvested data in existing repositories can be incorporated into the federation supporting a unified interface? This paper focuses on the approaches taken to address these issues in Arc, an Open Archives Initiative compliant federated digital library. At present Arc contains over 1M metadata records from 75 data providers from various subject domains. Analysis of these heterogeneous collections indicates that controlled vocabularies and values are widely used in most repositories. Usage is extremely variable, however. In Arc we solve the problem by implementing an advanced searching interface that allows users to search and select in specific fields with data we construct from the harvested metadata, and also by an interactive search for the subject field. As the metadata records are incrementally harvested we address how to build these services over frequently added new collections and harvested data. The initial result is promising, showing the benefits of immediate feedback to the user in enhancing the search experience as well as in increasing the precision of the user\u27s search

D1.1 Analysis Report on Federated Infrastructure and Application Profile

Kawese, R., Fisichella, M., Deng, F., Friedrich, M., Niemann, K., Börner, D., Holtkamp, P., Hun-Ha, K., Maxwell, K., Parodi, E., Pawlowski, J., Pirkkalainen, H., Rodrigo, C., & Schwertel, U. (2010). D1.1 Analysis Report on Federated Infrastructure and Application Profile. OpenScout project deliverable.The present deliverable aims to report on functionalities of the first step of the described process. In other words, the deliverable describes how the consortium will gather the learning objects metadata, centralize the access to existing learning resources and form a suitable application profile which will contribute to a proper and suitable modeling, retrieval and presentation of the required information (regarding the learning objects) to the interested users. The described approach is the foundation for the federated, skill-based search and learning object retrieval. The deliverable focuses on reporting the analysis of the available repositories and the best infrastructure that can support OpenScout’s initiative. The deliverable explains the motivations behind the chosen infrastructure based on the study of available information and previous research and literature.The work on this publication has been sponsored by the OpenScout (Skill based scouting of open user-generated and community-improved content for management education and training) Targeted Project that is funded by the European Commission’s 7th Framework Programme. Contract ECP-2008-EDU-42801

Lessons Learned with Arc, an OAI-PMH Service Provider

Web-based digital libraries have historically been built in isolation utilizing different technologies, protocols, and metadata. These differences hindered the development of digital library services that enable users to discover information from multiple libraries through a single unified interface. The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a major, international effort to address technical interoperability among distributed repositories. Arc debuted in 2000 as the first end-user OAI-PMH service provider. Since that time, Arc has grown to include nearly 7,000,000 metadata records. Arc has been deployed in a number of environments and has served as the basis for many other OAI-PMH projects, including Archon, Kepler, NCSTRL, and DP9. In this article we review the history of OAI-PMH and Arc, as well as some of the lessons learned while developing Arc and related OAI-PMH services. Reprinted by permission of the publisher

Federating Heterogeneous Digital Libraries by Metadata Harvesting

This dissertation studies the challenges and issues faced in federating heterogeneous digital libraries (DLs) by metadata harvesting. The objective of federation is to provide high-level services (e.g. transparent search across all DLs) on the collective metadata from different digital libraries. There are two main approaches to federate DLs: distributed searching approach and harvesting approach. As the distributed searching approach replies on executing queries to digital libraries in real time, it has problems with scalability. The difficulty of creating a distributed searching service for a large federation is the motivation behind Open Archives Initiatives Protocols for Metadata Harvesting (OAI-PMH). OAI-PMH supports both data providers (repositories, archives) and service providers. Service providers develop value-added services based on the information collected from data providers. Data providers are simply collections of harvestable metadata. This dissertation examines the application of the metadata harvesting approach in DL federations. It addresses the following problems: (1) Whether or not metadata harvesting provides a realistic and scalable solution for DL federation. (2) What is the status of and problems with current data provider implementations, and how to solve these problems. (3) How to synchronize data providers and service providers. (4) How to build different types of federation services over harvested metadata. (5) How to create a scalable and reliable infrastructure to support federation services. The work done in this dissertation is based on OAI-PMH, and the results have influenced the evolution of OAI-PMH. However, the results are not limited to the scope of OAI-PMH. Our approach is to design and build key services for metadata harvesting and to deploy them on the Web. Implementing a publicly available service allows us to demonstrate how these approaches are practical. The problems posed above are evaluated by performing experiments over these services. To summarize the results of this thesis, we conclude that the metadata harvesting approach is a realistic and scalable approach to federate heterogeneous DLs. We present two models of building federation services: a centralized model and a replicated model. Our experiments also demonstrate that the repository synchronization problem can be addressed by push, pull, and hybrid push/pull models; each model has its strengths and weaknesses and fits a specific scenario. Finally, we present a scalable and reliable infrastructure to support the applications of metadata harvesting

A Scalable Architecture for Harvest-Based Digital Libraries

This article discusses the requirements of current and emerging applications based on the Open Archives Initiative (OAI) and emphasizes the need for a common infrastructure to support them. Inspired by HTTP proxy, cache, gateway and web service concepts, a design for a scalable and reliable infrastructure that aims at satisfying these requirements is presented. Moreover, it is shown how various applications can exploit the services included in the proposed infrastructure. The article concludes by discussing the current status of several prototype implementations

A comparative study of the search and retrieval features of OAI harvesting services

Several OAI service providers (Harvesters) are coming up providing cross-search services by harvesting metadata from OAI compliant repositories. OAI facilitates quick discovery of content and free exchange of information among repositories through Service Providers (Harvesters). In order to achieve interoperability in their operations, Service Providers need to incorporate a generalized set of search and browse features in their search interface. Few parameters are drawn to compare the search and retrieval features of harvesters and arrived at a useful checklist for Service Providers to achieve homogeneity and standardization while designing their search interfac

Implementing infrastructures for managing learning objects

Klemke, R., Ternier, S., Kalz, M., & Specht, M. (2010). Implementing infrastructures for managing learning objects. British Journal of Educational Technology, 41(6), 873-882. doi: 10.1111/j.1467-8535.2010.01127.x PrePrint Version. Original available at: http://dx.doi.org/10.1111/j.1467-8535.2010.01127.x Retrieved October 20, 2010.Making learning objects available is critical to reuse learning resources. Making content transparently available and providing added value to different stakeholders is among the goals of the European Commission's eContentPlus programme. This article analyses standards and protocols relevant for making learning objects accessible in distributed data provider networks. Types of metadata associated with learning objects and methods for metadata generation are discussed. Experiences from European projects highlight problems in implementing infrastructures and mapping metadata types into common application profiles. The use of learning contents and its associated metadata in different scenICOPER, Share.TEC, OpenScou

Contexts and Contributions: Building the Distributed Library

This report updates and expands on A Survey of Digital Library Aggregation Services, originally commissioned by the DLF as an internal report in summer 2003, and released to the public later that year. It highlights major developments affecting the ecosystem of scholarly communications and digital libraries since the last survey and provides an analysis of OAI implementation demographics, based on a comparative review of repository registries and cross-archive search services. Secondly, it reviews the state-of-practice for a cohort of digital library aggregation services, grouping them in the context of the problem space to which they most closely adhere. Based in part on responses collected in fall 2005 from an online survey distributed to the original core services, the report investigates the purpose, function and challenges of next-generation aggregation services. On a case-by-case basis, the advances in each service are of interest in isolation from each other, but the report also attempts to situate these services in a larger context and to understand how they fit into a multi-dimensional and interdependent ecosystem supporting the worldwide community of scholars. Finally, the report summarizes the contributions of these services thus far and identifies obstacles requiring further attention to realize the goal of an open, distributed digital library system

Proposal for an IMLS Collection Registry and Metadata Repository

The University of Illinois at Urbana-Champaign proposes to design, implement, and research a collection-level registry and item-level metadata repository service that will aggregate information about digital collections and items of digital content created using funds from Institute of Museum and Library Services (IMLS) National Leadership Grants. This work will be a collaboration by the University Library and the Graduate School of Library and Information Science. All extant digital collections initiated or augmented under IMLS aegis from 1998 through September 30, 2005 will be included in the proposed collection registry. Item-level metadata will be harvested from collections making such content available using the Open Archives Initiative Protocol for Metadata Harvesting (OAI PMH). As part of this work, project personnel, in cooperation with IMLS staff and grantees, will define and document appropriate metadata schemas, help create and maintain collection-level metadata records, assist in implementing OAI compliant metadata provider services for dissemination of item-level metadata records, and research potential benefits and issues associated with these activities. The immediate outcomes of this work will be the practical demonstration of technologies that have the potential to enhance the visibility of IMLS funded online exhibits and digital library collections and improve discoverability of items contained in these resources. Experience gained and research conducted during this project will make clearer both the costs and the potential benefits associated with such services. Metadata provider and harvesting service implementations will be appropriately instrumented (e.g., customized anonymous transaction logs, online questionnaires for targeted user groups, performance monitors). At the conclusion of this project we will submit a final report that discusses tasks performed and lessons learned, presents business plans for sustaining registry and repository services, enumerates and summarizes potential benefits of these services, and makes recommendations regarding future implementations of these and related intermediary and end user interoperability services by IMLS projects.unpublishednot peer reviewe

Learning object retrieval in heterogeneous environments

This paper presents a solution to the problem of the search and retrieval digital tagged content in heterogeneous learning object repositories through architecture for intelligent retrieval of educational content in heterogeneous environments (AIREH) framework. This architecture unifies the search and retrieval of objects, thus facilitating the personalised learning search process by filtering and properly classifying learning objects retrieved for an approach for semantic-aware learning content retrieval based on abstraction layers between the repositories and the search clients. The use of federated databases techniques by using an organisation of agents allows those agents to work in a coordinated manner to solve a common problem, allowing the agents to adapt to the constantly changing environment (users, content repositories, etc.). Combining a complete agent-based architecture that implements the concept of federated search along with IR technologies may help organising and sorting search results in a meaningful way for educational content