What is usability in the context of the digital library and how can it be measured?

This paper reviews how usability has been defined in the context of the digital library, what methods have been applied and their applicability, and proposes an evaluation model and a suite of instruments for evaluating usability for academic digital libraries. The model examines effectiveness, efficiency, satisfaction, and learnability. It is found that there exists an interlocking relationship among effectiveness, efficiency, and satisfaction. It also examines how learnability interacts with these three attributes

Evaluation of the New Jersey Digital Highway

The aim of this research is to study the usefulness of the New Jersey Digital Highway (NJDH, www.njdigitalhigh way.org) and its portal structure. The NJDH intends to provide an immersive and user-centered portal for New Jersey history and culture. The research recruited 145 participants and used a Web-based questionnaire that contained three sections: for everyone, for educators, and for curators. The feedback on the usefulness of the NJDH was positive and the portal structure was favorable. The research uncovered several reasons why some collections did not want to or could not participate. The findings also suggested priorities for further development. This study is one of the few on the evaluation of cultural heritage digital library

From physical to digital: A case study of computer scientists' behaviour in physical libraries

There has been substantial research on various aspects of people's usage of physical libraries but relatively little on their interaction with individual library artefacts; that is: books, journals, and papers. We have studied people's behaviour when working in physical libraries, focusing particularly on how they interact with these artefacts, how they evaluate them, and how they interact with librarians. This study provides a better understanding of how people interact with paper information, from which we can draw implications for some requirements of the design of digital libraries, while recognising that the term 'library' is a metaphor when applied to electronic document collections. In particular, improved communication with other library users and with librarians could facilitate more rapid access to relevant information and support services, and structuring information presentation so that users can make rapid assessments of its relevance would improve the efficiency of many information searches. © Springer-Verlag 2004

The Importance of Usability in Development of Digital Libraries

Digital libraries are among the most interesting electronic information resources of our time. They are the collection of services and the collection of information objects and services that support users in accessing information objects available on the Internet. They are complex information systems consisting of many components which depend on each other in their operation. To achieve the flawless operation of digital libraries their developers should evaluate their achievements during the phase of development as well as during the phase of use. As a result of their efforts, users should get usable and easy to use information systems which are adapted to their needs and preferences. Evaluation and usability are two important concepts which may help librarians and computer specialists to design better information systems and better services for them. Usability testing may include one (usually the user interface) or many components of an information system for evaluation. Evaluation also includes the research of user community. Information acquired in such a way may also significantly help in information system improvement. Since the goal of evaluation is to achieve quality and improvement, the final result should be a superior information system like digital library and satisfied users

Investigating users’ mental models of traditional and digital libraries

There is much HCI-related literature on mental models and on the usability of digital libraries, however there is no previously published literature on users’ mental models of either traditional or digital libraries. This is surprising, since many digital libraries are difficult to use and it is not immediately clear why. Our study begins to fill this void by examining users’ mental models of traditional and digital libraries through a series of Contextual Inquiry interviews that mix traditional think-aloud observations, which usually demand minimal researcher intervention, and semi-structured interviews, which usually demand significant intervention. The study finds that participants’ mental models of traditional and digital libraries extend beyond surface similarities and differences, such as the hierarchical organisation of items in both types of library and the availability of documents in paper and electronic mediums. These models contain deeper similarities and differences based on the information-seeking goals that can be fulfilled by each type of library, issues concerning the contents and relevance of individual documents and entire libraries, and ‘how searching works’ and how to ‘troubleshoot’ in both types of library. Although the use of concrete analogies to influence users’ understanding or usage of digital libraries was not widespread, participants used their knowledge of Internet search engines to infer how searching might work in digital libraries. Additionally, most participants assumed that even if different at the interface level or at the level of the underlying technology employed, the search components of digital libraries, Internet search engines and other digital entities work in a similar way to bring back search results. The study also finds that a large component of users’ mental models of digital libraries is the notion of access restrictions. The insights gained from the observations relating to the above recurring themes in users’ mental models are discussed with a view of helping to improve digital library usability by reducing access restrictions and notifying users of any such restrictions upfront, by providing dynamic and context-dependent help to users, by carefully introducing analogies into the digital library interface (if and where appropriate) and by making multiple digital libraries searchable under a single front-end to enable them to be accessed, browsed and searched in the same way

Investigating users’ mental models of traditional and digital libraries

There is much HCI-related literature on mental models and on the usability of digital libraries, however there is no previously published literature on users’ mental models of either traditional or digital libraries. This is surprising, since many digital libraries are difficult to use and it is not immediately clear why. Our study begins to fill this void by examining users’ mental models of traditional and digital libraries through a series of Contextual Inquiry interviews that mix traditional think-aloud observations, which usually demand minimal researcher intervention, and semi-structured interviews, which usually demand significant intervention. The study finds that participants’ mental models of traditional and digital libraries extend beyond surface similarities and differences, such as the hierarchical organisation of items in both types of library and the availability of documents in paper and electronic mediums. These models contain deeper similarities and differences based on the information-seeking goals that can be fulfilled by each type of library, issues concerning the contents and relevance of individual documents and entire libraries, and ‘how searching works’ and how to ‘troubleshoot’ in both types of library. Although the use of concrete analogies to influence users’ understanding or usage of digital libraries was not widespread, participants used their knowledge of Internet search engines to infer how searching might work in digital libraries. Additionally, most participants assumed that even if different at the interface level or at the level of the underlying technology employed, the search components of digital libraries, Internet search engines and other digital entities work in a similar way to bring back search results. The study also finds that a large component of users’ mental models of digital libraries is the notion of access restrictions. The insights gained from the observations relating to the above recurring themes in users’ mental models are discussed with a view of helping to improve digital library usability by reducing access restrictions and notifying users of any such restrictions upfront, by providing dynamic and context-dependent help to users, by carefully introducing analogies into the digital library interface (if and where appropriate) and by making multiple digital libraries searchable under a single front-end to enable them to be accessed, browsed and searched in the same way

CONTENTdm(TM) Digital Collection Management Software and End-User Efficacy

Digital library technologies have become increasingly more sophisticated in the effort to provide more and better access to the collections they contain. The evaluation of the usability of these technologies has not kept pace with technological developments, however, and the end-user has in some cases been left behind. This research study evaluates the usability of digital collections created using the CONTENTdm™ Digital Collection Management System. Using usability testing techniques with actual end-users, this study attempts to assess the efficacy of the CONTENTdm™ public interface as well as user attitudes toward it. Ten participants from three user groups performed eleven tasks designed to test the key functions of CONTENTdm™-created collections and then answered a series of questions about their experiences. Results from this study may be used to improve this software system and add to the literature surrounding the usability of digital libraries in general

Using Digital Libraries Non-Visually: Understanding the Help-Seeking Situations of Blind Users

Introduction. This study explores blind users\u27 unique help-seeking situations in interacting with digital libraries. In particular, help-seeking situations were investigated at both the physical and cognitive levels. Method. Fifteen blind participants performed three search tasks, including known-item search, specific information search, and exploratory search, using the selected digital library. Pre-questionnaire, pre- and post-interviews, transaction logs and think-aloud protocols were used to collect data. Analysis. Open coding analysis was used to identify help-seeking situations the physical and cognitive levels. Results. The study identified seventeen help-seeking situations that blind users encountered while using digital libraries, including nine at the physical level and eight at the cognitive level. To be more specific, physical help-seeking situations were categorised into 1) difficulty accessing information, 2) difficulty identifying current status and path, and 3) difficulty evaluating information efficiently. Cognitive help-seeking situations were classified into 1) confusion about multiple programs and structures, 2) difficulty understanding information, 3) difficulty understanding or using digital library features, and 4) avoidance of specific formats or approaches. Conclusion. The identified help-seeking situations reveal a gap between current digital library design practices and special needs of blind users. Practical implications for the design of help features for more blind-friendly digital libraries are suggested based on the findings

Evaluating Library Signage: A Systematic Method for Conducting a Library Signage Inventory

While there is much literature that directs libraries to avoid having too much or insufficient signage, there is no clear guidance on how much signage is “enough” or “too much.” Conducting a signage inventory can be the first step toward determining how many signs a library needs, by establishing how many signs are in the library, of which type, and their condition. This paper proposes a ready-to-use method that any library can use to inventory its signage by adapting the inventory worksheet depending on factors related to the library type. The ultimate goal in developing a standardized method is that it would allow for, comparing results across libraries to attempt development of more specific signage guidelines or a formula that could calculate how many signs are “enough” and “too many” for a library given its type, population, and other criteria

Evaluating the usability and usefulness of a digital library

Purpose - System usability and system usefulness are interdependent properties of system interaction, which in combination, determine system satisfaction and usage. Often approached separately, or in the case of digital libraries, often focused upon usability, there is emerging consensus among the research community for their unified treatment and research attention. However, a key challenge is to identify, both respectively and relatively, what to measure and how, compounded by concerns regarding common understanding of usability measures, and associated calls for more valid and complete measures within integrated and comprehensive models. The purpose of this paper is to address this challenge. Design/methodology/approach - Identified key usability and usefulness attributes and associated measures, compiled an integrated measurement framework, identified a suitable methodological approach for application of the framework, and conducted a pilot study on an interactive search system developed by a Health Service as part of their e-library service. Findings - Effectiveness, efficiency, aesthetic appearance, terminology, navigation, and learnability are key attributes of system usability; and relevance, reliability, and currency key attributes of system usefulness. There are shared aspects to several of these attributes, but each is also sufficiently unique to preserve its respective validity. They can be combined as part of a multi-method approach to system evaluation. Research limitations/implications - Pilot study has demonstrated that usability and usefulness can be readily combined, and that questionnaire and observation are valid multi-method approaches, but further research is called for under a variety of conditions, with further combinations of methods, and larger samples. Originality/value - This paper provides an integrated measurement framework, derived from the goal, question, metric paradigm, which provides a relatively comprehensive and representative set of system usability and system usefulness attributes and associated measures, which could be adapted and further refined on a case-by-case basis