User-Centered Evaluation of a Discovery Layer System with Google Scholar

Discovery layer systems allow library users to obtain search results from multiple library resources and view results in a consistent format. The implementation of a discovery layer is expected to simplify users’ workflow of searching for scholarly information. Previous studies on discovery layer systems focused on functionality and content, but not quality of search results from the user’s perspective. The objective of this study was to obtain users’ assessment of search results of a discovery layer system (Ex Libris Primo®) and compare that with a widely used scholarly search tool (Google Scholar). Results showed that Primo’s search results relevancy is comparable to Google Scholar, but it received significantly lower usability and preference ratings. A number of usability issues of Primo were also identified from the study. Results of the study are used to improve the interface of Primo and adjust relevancy ranking options. The empirical method of search results assessment and feedback collection used in this study can be extended to similar user-centered system implementation and evaluation efforts

What Does it Take to Make Discovery a Success?: A Survey of Discovery Tool Adoption, Instruction, and Evaluation Among Academic Libraries

Discovery tools have been widely adopted by academic libraries, yet little information exists that connects common practices regarding discovery tool implementation, maintenance, assessment, and staffing with conventions for research and instruction. The authors surveyed heads of reference and instruction departments in research and land-grant university libraries. The survey results revealed common practices with discovery tools among academic libraries. This study also draws connections between operational, instructional, and assessment practices and perceptions that participants have of the success of their discovery tool. Participants who indicated successful implementation of their discovery tool hailed from institutions that made significant commitments to the operations, maintenance, and acceptance of their discovery tool. Participants who indicated an unsuccessful implementation, or who were unsure about the success of their implementation, did not make lasting commitments to the technical maintenance, operations, and acceptance of their discovery tool

Collaborative Growth Toward Discovery: Becoming Stronger through Change

The radical act of replacing a traditional OPAC and ILS with a hosted library services platform (LSP) and web-scale discovery (WSD) system creates the impetus for libraries to rethink core workflows and practices. Both of these tools have the potential to greatly improve access to library collections and enhance user experience, but only if the implementation is a collaborative effort between different stakeholders, technical experts and subject librarians, grounded in a thoughtful selection process that emphasizes user needs. Furthermore, because this model removes the traditional OPAC, subject librarians must take on the challenge of understanding the tool and work as partners with backend technical experts as well as the vendor so that they are not left out of providing research assistance as algorithms replace some traditional library skills. Grounded in the literature of collaborative learning and librarian attitudes towards discovery systems, this chapter describe the process of creating a culture of collaboration centered on user needs. Such a model is central to the process of evaluation and implementation of a LSP and a WSD service, particularly in the context of a university with multiple campuses and diverse users. We established a well-structured collaboration model from the systems evaluation phase, and continued to apply the model to the implementation phase. In 2010, the initial solution was to keep a traditional OPAC, and implement WorldCat Local as article discovery tool while doing in-depth user studies and gathering feedback from subject librarians to create a data-driven decision making habit. In 2014, we determined that Alma and Primo should replace the current Voyager ILS and WorldCat Local. The implementation process started in January 2015 and will go live in July 2015. The chapter will share the journey of creating a collaborative model and implementing unified resource management and discovery

EBSCO Discovery Service (EDS) Usage in Israeli Academic Libraries

Awareness to the problem that different search interfaces discourage patrons’ use of library information sources has led academic libraries to implement web-scale discovery services. These services offer the user a “Google-like” search experience of library resources. This study aims to explore library professionals’ satisfaction, patrons’ information behavior, and use of EDS discovery tool service in academic libraries in Israel. Mixed research methods were used in this study: qualitative and quantitative. Qualitative research methods are through content analysis of library directors’ interviews, and quantitative research method is through collected library metrics (from Google analytics) data analysis, regarding usage patterns and search session analysis. The study aims to gain insight regarding library implementation and patrons’ information behavior of the EDS discovery tool, in Israeli higher education institutions

Applying Hierarchical Task Analysis Method to Discovery Layer Evaluation

While usability tests have been helpful in evaluating the success or failure of implementing discovery layers in the library context, the focus of usability tests has remained on the search interface rather than the discovery process for users. The informal site-­‐ and context specific usability tests have offered little to test the rigor of the discovery layers against the user goals, motivations and workflow they have been designed to support. This study proposes hierarchical task analysis (HTA) as an important complementary evaluation method to usability testing of discovery layers. Relevant literature is reviewed for the discovery layers and the HTA method. As no previous application of HTA to the evaluation of discovery layers was found, this paper presents the application of HTA as an expert based and workflow centered (e.g., retrieving a relevant book or a journal article) method to evaluating discovery layers. Purdue University’s Primo by Ex Libris was used to map eleven use cases as HTA charts. Nielsen’s Goal Composition theory was used as an analytical framework to evaluate the goal charts from two perspectives: a) users’ physical interactions (i.e., clicks), and b) user’s cognitive steps (i.e., decision points for what to do next). A brief comparison of HTA and usability test findings is offered as a way of conclusion

Language technologies and the evolution of the semantic web

The availability of huge amounts of semantic markup on the Web promises to enable a quantum leap in the level of support available to Web users for locating, aggregating, sharing, interpreting and customizing information. While we cannot claim that a large scale Semantic Web already exists, a number of applications have been produced, which generate and exploit semantic markup, to provide advanced search and querying functionalities, and to allow the visualization and management of heterogeneous, distributed data. While these tools provide evidence of the feasibility and tremendous potential value of the enterprise, they all suffer from major limitations, to do primarily with the limited degree of scale and heterogeneity of the semantic data they use. Nevertheless, we argue that we are at a key point in the brief history of the Semantic Web and that the very latest demonstrators already give us a glimpse of what future applications will look like. In this paper, we describe the already visible effects of these changes by analyzing the evolution of Semantic Web tools from smart databases towards applications that harness collective intelligence. We also point out that language technology plays an important role in making this evolution sustainable and we highlight the need for improved support, especially in the area of large-scale linguistic resources

Improving Institutional Repositories through User-Centered Design: Indicators from a Focus Group

User experience with intuitive and flexible digital platforms can be enjoyable and satisfying. A strategy to deliver such an experience is to place the users at the center of the design process and analyze their beliefs and perceptions to add appropriate platform features. This study conducted with focus groups as a qualitative method of data collection to investigate users’ preferences and develop a new landing page for institutional repositories with attractive functionalities based on their information-structural rules. The research question was: What are the motivations and experiences of users in an academic community when publishing scientific information in an institutional repository? The focus group technique used in this study had three sessions. Results showed that 50% of the participants did not know the functionalities of the institutional repository nor its benefits. Users’ perceptions of platforms such as ResearchGate or Google Scholar that provide academic production were also identified. The findings showed that motivating an academic community to use an institutional repository requires technological functions, user guidelines that identify what can or cannot be published in open access, and training programs for open access publication practices and institutional repository use. These measures align with global strategies to strengthen the digital identities of scientific communities and thus benefit open science

OhioLINK Librarians and Google Scholar Over Time: A Longitudinal Analysis of Attitudes and Uses

This study provides a longitudinal analysis of the opinions and uses of Google Scholar by members of the OhioLINK consortium of libraries. Using survey data collected in 2007, 2014, and 2019 via the OhioLINK Listserv, this study explores how librarians use and promote Google Scholar within their library instruction sessions and how Google Scholar and its Library Links feature are promoted on library websites. This information is then analyzed across the three date ranges. The results of this research suggested that there were significant changes in the use and opinions of Google Scholar between 2007 and 2014 with more normalization of uses and opinions occurring between 2014 and 2019