Alerting Services inform users of Digital Libraries about new or changed content of their favorite Digital Library collection, or about new documents discussing a topic they are interested in. The users define their interest in form of subscriptions; the alerting service filters new or changed documents against all user subscriptions and notifies the users about matches found. To define their subscriptions, users may, e.g., select a journal from a list, or a topic from a classification. Few services support the definition of IR type queries, or advanced combinations of queries and metadata selections

Hinze, Annika

English

Research Commons@Waikato

Issues in Usability Studies for Alerting in Digital LibrariesAnnika HinzeUniversity of Waikato, New Zealandhinze@cs.waikato.ac.nzAlerting Services inform users of Digital Libraries about new or changed content of their favorite Digital Librarycollection, or about new documents discussing a topic they are interested in. The users define their interest in form ofsubscriptions; the alerting service filters new or changed documents against all user subscriptions and notifies theusers about matches found. To define their subscriptions, users may, e.g., select a journal from a list, or a topic from aclassification. Few services support the definition of IR type queries, or advanced combinations of queries andmetadata selections.In recent years, Alerting Services have been installed for several proprietary libraries such as the ones offered bySpringer and Elsevier or the ACM Digital Library. These offers are restricted to the publisher’s documents; theyprovide only restricted expressiveness and functionality. The main focus of these services has been the delivery ofTOC information and email notification about selected scientific publications. Support for alerting in open DigitalLibrary Projects is sparse. The functionality differs widely, addressing interesting issues such as advancedsubscription definition, interoperability, openness, and information integration from different providers.Our recent user study [1] has shown that users of digital libraries are interested in receiving notifications about avariety of events, such as new documents at a particular server, or music files that adhere to certain specifications ornew software releases. Users are also interested in receiving the notifications via different channels, such as news-feeds (e.g., RSS), email, or a personalized webpage. To the best of our knowledge, no evaluation has been done tostudy the user’s interaction with an alerting system for digital libraries. So far, no study has addressed the questionsof either the(1) successfulness of different alerting methods for long-term user information needs, or(2) usability issues and usefulness of alerting services for digital libraries.In this paper, we study the reasons for this gap, and discuss challenges arising for analysis and study of the twoquestions.When studying users’ needs and usability issues in alerting, a multitude of issues arise. These are especiallypronounced in alerting in the context of digital libraries as a means to fulfill long-term information interests, whereomissions do not become immediately apparent (different to the observation of a power plant).We group the identified issues into the three aspects: (1) users, (2) system design, and (3) longitudinal study.Examples of issues for each of the aspects to be addressed are given below; we will focus particularly on the useraspect.1. Aspect: User issues• Awareness of (long-term) information need: Usability of alerting in digital libraries cannot be studied if theusers are not aware of their long-term information needs. Over the last years, users have become accustomedto search or browse to address short-term information needs. Alerting services can also provide usefulinformation here, especially in the case of highly dynamic information sources. But the value of alerting willmainly become apparent when searching over a longer time period. This needs a different approach toinformation seeking from the users:  an analysis of their interests in a wider context and on a higher level ofabstraction. Users might need to be encouraged to contemplate their information needs and to specify themto the alerting system.• Ability to express (long-term) information need: It is more challenging to define a useful query that will becontinually executed over a long time period than a search query. No immediate results may support thespecification; the user cannot develop the specific query form in a dialogue with the system. Thus, aconceptually successful user subscription for alerting might fail due to wrongly defined query terms. Internalsupport for query definition in the alerting service (such as automatic query generation from previoussearches) might only capture short-term information needs and evaluations would often only analyze thequality of the query-algorithm but not the usefulness of the alerting to the user.• Awareness and notification methods:  When the alerting system discovers information of interest, the userhas to be made aware of it. The notifications sent to the user have to reach them in the right context of theirwork, in the right format to be useful, and in a way that supports a long-term satisfaction of the informationneed. That is, the right information has to reach the user in a way that they feel the information need wasfulfilled and they are able to utilize the information within in their work context. The impact of thenotification methods on the long-term user behavior would have to be measured: short-term satisfactionmight not lead to utilization of the information gained.• Alert management by the user:  Users have different methods to manage alerts that they receive. Forexample, they might store email alerts in a separate folder, which is never to be looked at again; or theymight include and annotate a received bibtex reference into their literature database, so that it is ready to beused. The importance here lies in the incorporation of received alerts into the users’ work processes, whichmay differ greatly.• Psychological impact of the alerts on the users: In the context of a recent study [2], we learned that someusers feel uncomfortable receiving alerts. They perceive a heightened work pressure and feel an expectationto perform better (i.e., read more, research more, and know more). It remains to be analyzed is these usersare less likely to use an alerting service or if it inspires them to act upon the alerts received. When evaluatingthe quality of the alerting service, these aspects have to be taken into account.2. Aspect: System design issues• Subscription interface and interaction design: As known from HCI studies, the design of the user interfaceshas a great impact on the usability of the alerting system. Subscriptions might be created from previoussearch queries, based on browsing results (notify me about new documents similar to ‘this’ document), or byobserving a users work space (notify about documents similar to the ones that are there already). In addition,users may have to be encouraged to enter new fields (similar to recommender systems). Offering interfacesthat support a creative and playful information-need discovery might provide help for some users. Theeffects of the interface design are often only studied in short-term contexts (e.g., can the user find the rightbutton?) but not in the context of long-term user behavior and satisfaction.• Presentation and content for delivered alerts: Similarly to the previous point, the presentation of thenotification is of crucial impact. For example, presentation of document information in a format that is widespread and easily integrated, such as bibtex and endnote. Note that the access to the notifications and storageof the given references does not give indication about the usage of the information given. Easier access forstorage might also lead to less impact on the user (such as, save and forget). Thus the information mighthave to be offered in different formats – to capture the user’s interest and to allow easy storage.3. Aspect: Longitudinal study issues• Studying long-term effects of alerting without interfering: The impact of alerting lies in the quality of theinformation and in the user’s awareness of the given information. How does one study long-term awareness?The user cannot be questioned during the study directly, as this approach would interfere with the workprocess and may change the overall result. Thus, a diary-like approach might not mirror the effects of thealerting service without the study: Naturally, being asked to reflect about one’s awareness of a topicinterferes with ones awareness. A reflective study at the end of a longer period (such as months or years)might not be able to fully capture the work process. Conversely, the fact that a user did not use the givenreferences directly, does not allow conclusions about the user’s awareness or the quality of the references.The users might have been inspired to further searches or different work tasks. The fact that no notificationwas received could be seen by a user as a success (e.g., when testing for competitors).• Cross influences of work methods and alerting system: The usage of an alerting system might positivelyinfluence a user’s work experience. As explained above, the user’s information seeking behavior and theexpectations as to what constitutes a successful usage of alerting and the resulting notifications may differwidely. Cross influences between alerting system and work style might be seen as inspiring by the user (e.g.,discovery of new options and ways of thinking and working) or as a distraction (e.g., users might interact inthe way the system was designed to gain best results).  In all cases, the system design, usage, and personalstyle interact with each other, and an independent study that allows for systematic analysis of the systembecomes harder.The aspects highlighted above give a brief introduction into the issues that one faces when attempting a usabilitystudy of alerting systems that goes beyond a simple short-term interface study. Note that the aspects mentioned arenot independent but are interlinked and interfere with each other during a study.The overall questions remain of how to ascertain the user satisfaction for such a system, i.e., what would beconsidered a successful long-term user/system interaction and how can this be measured.[1] Buchanan, Hinze: A Generic Alerting Service for Digital Libraries, Proceedings of the JCDL, 2005[2] Buchanan, Cunningham, Blandford, Rimmer: Information Seeking by Humanities Scholars,               Proceedings of the ECDL, 2005

Issues in usability studies for alerting in digital libraries

Name not available

http://www.dlib.org/dlib/july05/khoo/05_hinze.pdf

Issues in usability studies for alerting in digital libraries

Abstract

Similar works

Full text

Available Versions

Research Commons@Waikato

Name not available