Assembling and enriching digital library collections

People who create digital libraries need to gather together the raw material, add metadata as necessary, and design and build new collections. This paper sets out the requirements for these tasks and describes a new tool that supports them interactively, making it easy for users to create their own collections from electronic files of all types. The process involves selecting documents for inclusion, coming up with a suitable metadata set, assigning metadata to each document or group of documents, designing the form of the collection in terms of document formats, searchable indexes, and browsing facilities, building the necessary indexes and data structures, and putting the collection in place for others to use. Moreover, different situations require different workflows, and the system must be flexible enough to cope with these demands. Although the tool is specific to the Greenstone digital library software, the underlying ideas should prove useful in more general contexts

Creating digital library collections with Greenstone

The Greenstone digital library software is a comprehensive system for building and distributing digital library collections. It provides a way of organizing information based on metadata and publishing ti on the Internet. This paper introduces Greenstone and explains how librarians use it to create and customize digital library collections. Through an end-user interface, they add documents and metadata to collections, create new collections whose structure mirrors existing ones, and build collections and put them in place for users to view. More advanced users can design and customize new collection structures

ArchivePress: A Really Simple Solution to Archiving Blog Content

ArchivePress is a new technical solution for collecting and archiving content from blogs. Current solutions are commonly based on typical web archiving activities, whereby a crawler is configured to harvest a copy of the blog and return the copy to a web archive. This approach is perfectly acceptable if the requirement is that the site is presented as an integral whole. However, ArchivePress is based upon the premise that blogs are a distinct class of web-based resource, in which the post, not the page, is atomic, and certain properties, such as layouts and colours, are demonstrably superfluous for many (if not most) users. As a result, an approach that builds on the functionality provided by web feeds to capture only selected aspects of the blog offers more potential. This is particularly the case when institutions wish to develop collections of aggregated blog content from a range of different sources. The presentation will describe our research to develop such an approach, including work to define the significant properties of blogs, details of the technical development, and pilot collections against which the tool has been tested

Lemon: an MPI parallel I/O library for data encapsulation using LIME

We introduce Lemon, an MPI parallel I/O library that is intended to allow for efficient parallel I/O of both binary and metadata on massively parallel architectures. Motivated by the demands of the Lattice Quantum Chromodynamics community, the data is stored in the SciDAC Lattice QCD Interchange Message Encapsulation format. This format allows for storing large blocks of binary data and corresponding metadata in the same file. Even if designed for LQCD needs, this format might be useful for any application with this type of data profile. The design, implementation and application of Lemon are described. We conclude with presenting the excellent scaling properties of Lemon on state of the art high performance computers

A Query Integrator and Manager for the Query Web

We introduce two concepts: the Query Web as a layer of interconnected queries over the document web and the semantic web, and a Query Web Integrator and Manager (QI) that enables the Query Web to evolve. QI permits users to write, save and reuse queries over any web accessible source, including other queries saved in other installations of QI. The saved queries may be in any language (e.g. SPARQL, XQuery); the only condition for interconnection is that the queries return their results in some form of XML. This condition allows queries to chain off each other, and to be written in whatever language is appropriate for the task. We illustrate the potential use of QI for several biomedical use cases, including ontology view generation using a combination of graph-based and logical approaches, value set generation for clinical data management, image annotation using terminology obtained from an ontology web service, ontology-driven brain imaging data integration, small-scale clinical data integration, and wider-scale clinical data integration. Such use cases illustrate the current range of applications of QI and lead us to speculate about the potential evolution from smaller groups of interconnected queries into a larger query network that layers over the document and semantic web. The resulting Query Web could greatly aid researchers and others who now have to manually navigate through multiple information sources in order to answer specific questions