The goal of this guerilla test was to determine which of four labels (three alternate labels, in addition to the current “available online” label) denoting online availability is preferred by patrons, taking time to ensure that patrons understand the range of situations currently represented by the "available online" label. We also solicited suggestions for alternate labels from participants.Usability GroupUsability Task Forcehttp://deepblue.lib.umich.edu/bitstream/2027.42/106783/1/MirlynGuerillaTest_Feb2011.pd

Desai, Shevon

Fulmer, David

Hill, Rebecca

Koparkar, Saurabh

Moussa, Nancy

Piacentine, Julie

Rothman, Jonathan

Wang, Mei

Deep Blue Documents at the University of Michigan

Deep Blue https://deepblue.lib.umich.edu/documents2011-02-08Mirlyn "Available Online" Label Guerilla TestDesai, Shevon; Piacentine, Julie; Rothman, Jonathan; Fulmer,David; Hill, Rebecca; Koparkar, Saurabh; Moussa, Nancy; Wang, Meihttps://hdl.handle.net/2027.42/106783Downloaded from Deep Blue, University of Michigan's institutional repositoryMirlyn | “Available Online” Label Guerilla Test  This work is available under a Creative Commons Attribution Non-Commercial 3.0 License. http://creativecommons.org/licenses/by-nc/3.0/us/.  © The Regents of the University of Michigan, 2011.  University of Michigan | MLibrary | ul-usability@umich.edu   1  Usability Report Cover Sheet     Project  Overall Project: Mirlyn Project Title: “Available Online” Label Guerilla Test  Committee & Members  Usability Group: Suzanne Chapman (chair), Shevon Desai, Kat Hagedorn, Julie Piacentine, Ken Varnum   Usability Task Force: Shevon Desai (co-convener), Julie Piacentine (co-convener), Jonathan Rothman (stakeholder), David Fulmer, Rebecca Hill, Saurabh Koparkar, Nancy Moussa, Mei Wang  Report Info  Report Author(s): Shevon Desai, Julie Piacentine,  Jonathan Rothman, David Fulmer, Rebecca Hill, Saurabh Koparkar, Nancy Moussa, Mei Wang Contact Information: ul-usability@umich.edu Report Date: February 8, 2011 Online Report Location  www.lib.umich.edu/files/services/usability/MirlynGuerilla_Feb2011.pdf    Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  2 Usability Report  Introduction  "Available online" is a label which appears in Mirlyn and signifies that at least one item attached to a bibliographic record is fully available online. The extent of the online availability varies and a single fully available online item, such as a single issue of a journal, will result in the display of the "available online" label in the Mirlyn record of that. The degree to which there is any online availability can vary wildly. If there is a 20 year run of a journal (with each year bound as a single volume), and even one of those volumes is available in full-text online, the 'available online' label appears. The label also appears in the records of monographs, maps, and audio recordings with online availability.   The "available online" label appears also in Mirlyn as clickable text under the "Availability" search narrowing option in the "Narrow Search" column of search results pages and as the text of the hyperlink to the online version of an item which appears in search results and Mirlyn record web pages.            The earlier heuristic evaluation of Mirlyn conducted by the usability task force, and feedback from both staff and patrons indicate that the meaning of the “available online” label can be confusing. Reference librarians report that patrons are often frustrated when items described as “available online” are not, in fact, available (the particular issue needed might not be available in full-text, or the patron might be looking at something that is “search-only”).   The need to test alternative labels arose as plans to retire the umbrella category “Electronic Resource” developed.  Previously, “Electronic Resource” denoted any item with electronic functionality, including CD-ROMs, data files, and online items, including HathiTrust full-text and search-only items.  As “Electronic Resource” is replaced with more specific sub-categories (i.e., CD-ROM, data files, etc.), stakeholders considered the possibility of replacing the separate but related “Available Online” label.  Because these two labels frequently overlapped, user confusion resulted in distrust of both labels.  It was hypothesized that a label change might help users identify items that may be viewed online.   The goal of this guerilla test was to determine which of four labels (three alternate labels, in addition to the current “available online” label) denoting online availability is preferred by patrons, taking time to ensure that patrons understand the range of situations currently Example of "available online" label within Mirlyn record Example within “Narrow Search” Column Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  3 represented by the "available online" label. We also solicited suggestions for alternate labels from participants.  Test Description & Methods  Four different screenshots of Mirlyn search results were displayed to each participant. Each screenshot depicted the same set of results, the only difference being the label used in place of “available online.”  The different items included in the search results were chosen specifically to illustrate the varying levels of online access reflected by the “available online” label.  Four different labels were tested: • available online	  • some content fully available online	  • electronic resource	  • online content available 	  Within each set of search results, the label in question was highlighted as to draw attention to the difference between the screenshots. The varying levels of availability for different types of items within the set of search results were briefly described to the participants, and participants were asked to choose which label best described all the situations they saw.    Partial image of 1 of the 4 screenshots used for guerilla test, testing the label “Online Content Available” Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  4  The task force group divided into three teams, with each team recruiting 5 participants (undergraduates, graduate students, and faculty) and 1-2 librarians each. Participants were recruited in various locations around campus, and were offered a small incentive of $2 in Munchie Money. Task force members noted the participants’ comments about the label chosen and why, along with suggestions for other labels not included in the test and any other observations. Results   The task force began by conducting pilot tests. Because there were no significant changes made as a result of the pilot testing, we have included the 9 pilot tests with the final results. However, 3 of these 9 pilot tests were conducted testing only 3 of the 4 labels – “electronic resources” was added later, at the stakeholder’s request. Based on the overall results, this difference in the first guerilla tests did not appear to be significant (only 3 of 26 people chose “Electronic Resource”).  Total number of test participants: 29  • Undergraduate students:  16 • Graduate students:  6 • Library Staff:  6 • Faculty Member:  1   Locations where the guerrilla tests were conducted: Partial image of another screenshot used for guerilla test, testing the label “Some Content Fully Available Online” Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  5 • Duderstadt Center, North Campus • Undergraduate Library - atrium, Bert’s Café, and first floor study areas • Graduate Library Reference and Information Center, 2nd floor • Angell Hall building entrance • North Quad • Science Library Conference Room   Question 1-A: Which of these terms - "available online, "some content fully available online" “electronic resource” or "online content available" - do you think best applies to all situations?   Type of User Label Name Number of testers (total=29) Undergraduate Graduate Library Staff Faculty Member Available Online 9 4 2 2 1 Some Content Fully Available Online 9 4 2 3 0 Online Content Available 7 5 1 1 0 Electronic Resource 3 2 1 0 0 • Both “some content fully available online” and “available online” were the most popular labels, the preferred choice of 9 participants each.  Question 1-B: Can you tell us why you chose this label? 1.  Participants who chose ‘Available Online’ (9) Responses indicated that participants who chose “available online” felt that this label was short and meaningful, although it was not necessarily completely truthful in conveying how much content was available online. According to users, this label sounded more clear, familiar, and of the right word length. It should be noted that “available online” is the term currently used in Mirlyn. Selected responses by users who chose ‘available online’: Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  6 • “… it (available online) sounds most clear especially because it says available…” [library staff member]  • Available online “means everything.” The participant said, “After clicking on ‘available online’, I would figure out from there how to get what I need online.” [graduate student]  • “It (available online) tells me immediately that information is available online…” [undergraduate student]   • “It (available online) reads better.” [librarian]  • The participant said that this label “makes most sense” when compared to ‘online content available,’ which is similar in meaning. The participant also said that “some content fully available online” is too long. [undergraduate student]  2.  Participants who chose “Some Content Fully Available Online” (9) These users felt that this label sounded true and meaningful for all situations, but was not short in length. The label was considered most descriptive by users, because it clearly meant that some items, and not all, would be available online. A few users misunderstood that this label referred to HathiTrust full-text and search-only items in the catalog.  Selected responses by users who chose “some content fully available online”: • One participant said that until she learned what HathiTrust was, it was really frustrating for her to think something would be available online and then realize that it was not. It is not clear from the comment whether the participant was referring to records where some volumes are fully available online and others are search-only, or whether she was referring to strictly search-only items.   • This participant was of the opinion that the word “some” in this label is better than all others. The user remarked that “available online” is the same as “online content available,” and that “electronic resource” is not detailed enough (to understand). [graduate student]         • One participant felt that this label was “more inclusive” than others. [undergraduate student]  • One participant mentioned that this label is clearer because it says some and not all content is available online. The label being wordy did not bother this user. [undergraduate student] Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  7  3.  Participants who chose ‘Online Content Available’ (7) Participants’ comments about this label seemed to indicate that while it is short and truthful, it is not adequately meaningful, recognizable, or compelling. Selected responses by users who chose “Online Content Available”: • One participant said that she preferred this label because it included both ‘some and all’ electronic access. [undergraduate student]  • Another participant was of the opinion that this label did not guarantee everything is online but it guarantees electronic access. [librarian]  4.  Participants who chose ‘Electronic Resource’ (3) Very few participants preferred this label.  One comment from a participant indicated that ‘Electronic Resource’ is appropriately ambiguous – users would need to click through to find out specifically what is available online.  In contrast, another participant who chose “Electronic Resource” found the label unambiguous, ruling out search-only items or items that are “not as available.” Selected responses by users who chose “Electronic Resource”: • This participant felt that this label suggests that the resource can be obtained on the internet:  “…it’s digitized, while the word ‘some’ [in the label ‘Some Content Fully Available Online’] means it is not as available.” [graduate student]  • One student explained that if she had to choose one label for all situations then "electronic resource" was her favorite because it indicates that something is available online and "you have to click to find out what's there." [undergraduate student]  Question 2 – Labels suggested by participants A few participants suggested their own labels, listed below: • “Print and electronic” • “Available print and online”  • “Some content online” • “Some content available online” • “Some full text content available online” Comments worth mentioning: Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  8 Some participants had intense reactions after seeing certain label options. This can be illustrated by the following examples: • A graduate student felt that ‘some content fully available online’ and ‘online content available’ are “discouraging.”  • An undergraduate student called the label “some content fully available online” ‘obnoxious.’  • A library staff member said that the label “Online Content Available” “sounds kind of sketchy.”   Recommendations  Continue to use the 'available online' label Test results showed that participants did not strongly prefer any one label over another.  The two labels most preferred by participants were "some content fully available online" and "available online."  We recognize that 32% (9 out of 28) of participants in favor of the current label “available online” is not an overwhelming number, and that the majority of testers preferred some other label. However, we recommend continued use of the current label for a variety of reasons:   • All labels must find an appropriate balance between length, meaningfulness, and truth/accuracy (see image at right, courtesy of Suzanne Chapman). "Some content fully available online" is lengthy, although meaningful and true. “Available online,” while lacking in a detailed level of accuracy, effectively conveys meaning and is appropriate in length.    • The label "some content fully available online" garnered strong negative reactions, described above.  • Since “available online” is one of the two top choices preferred by our test participants, and it is the current label being used, switching to a different label at this point would be unnecessarily disruptive without apparent strong benefit. Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  9  Lessons Learned  What Went Well  The test itself was well-designed. Pilot testing showed us that participants understood the questions and no major changes were necessary.  The four labels presented during the guerilla test were carefully chosen and appeared to be sufficient choices. Most participants preferred one of the labels offered and did not have additional suggestions.  The guerilla test approach suited the question we were asking; the test was not too long, and we had no trouble recruiting participants.   What Did Not Go Well  There were no significant problems with the guerilla testing.  Ideas for Future Testing As with most usability testing, having a larger sample of participants would be helpful, especially since no single label was a “clear winner.” If the test were longer than a guerilla test, we could also include questions that would more accurately determine the user’s level of familiarity with Mirlyn, in order to better gauge their understanding of all the different possibilities needing to be conveyed by the “available online” label.   Mirlyn | “Available Online” Label Guerilla Test  University of Michigan | MLibrary | ul-usability@umich.edu  10 Appendix A  Mirlyn Guerilla Test Script  REMINDERS:      Have one person administer the test and the other write down any comments, questions and answers.  Note your location, and anything else that may influence the person’s answers.  INTRODUCE yourself and ask if willing to help: Hi, I’m ___________  and this is _______________ from the library. Do you have 7-8 minutes to help us answer some questions about the library website? We can give you $2 in Munchie Money for your time.  Any information you give us will be kept anonymous.   First, would you please tell us if you are an undergrad, a graduate student, faculty member, staff member, etc.   SET UP: I'm going to show you screenshots of Mirlyn, the library catalog. Mirlyn is where you would go to search for all the different kinds of resources that the Library has: journals, books, ebooks, musical scores, etc.  Here are four print-outs from Mirlyn all showing the same search results.  There is just one slight difference, which is highlighted. This is a label -- not a link -- which describes access to these items.  Label preference set up: [Point to Dickens] This is a book that is available both in print as well as electronically.  [Pointing to map] This item is a map of Pennsylvania that you can both look at in print as well as view electronically.  [Pointing to British Journal of Health Psych] You can read issues of British Jnl of Health Psychology up through 2004 electronically, but not more recent ones.   QUESTION 1: Which of these terms -- [point to each], "available online, "some content fully available online," “electronic resource” or "online content available" -- do you think best applies to all situations? Can you tell us why? There’s no right answer; we just want to know your preference.       QUESTION 2:  Label suggestion question: Is there a better term you would suggest for this?    One sentence description of other items, in case the person asks about them: [Soldier's tale]: This is a piece of music; you can either check out the CD or listen to it online.  Critical Care Nurse is a journal for which you can read certain issues electronically, but not all issues 

Mirlyn "Available Online" Label Guerilla Test

http://deepblue.lib.umich.edu/bitstream/2027.42/106783/1/MirlynGuerillaTest_Feb2011.pdf

Mirlyn "Available Online" Label Guerilla Test

Abstract

Similar works

Full text

Available Versions

Deep Blue Documents at the University of Michigan