University of Montana, Mansfield Library Professors Kate Zoellner and Sue Samson presented  Mansfield Library Outcomes Assessment

Samson, Sue

Zoellner, Kate

English

University of Montana

Kate Zoellner Associate Professor Assessment Coordinator Sue Samson Professor Library Instruction Coordinator  Montana University System Outcomes Assessment Workshop September 22, 2014 1 LEARNING AND IMPACT Outcome Metrics “The consequences of an individual’s contact with the library.”  Learning Outcomes  “If students learn from their use of library resources and services… in terms of skills, values, and attitudes.”  Impact Outcomes “Concerned with user satisfaction, opinion, and perceptions, including satisfaction with use of the library’s services and programs. They may also include the impact or the effectiveness of library policies and management on services as well as the organizational culture.” Dugan, Hernon, & Nitecki, 2009 2 INFORMATION LITERACY LEARNING OUTCOMES    LIBRARY INSTRUCTION RUBRIC BASED ON THE ACRL INFORMATION LITERACY COMPETENCY STANDARDS FOR HIGHER EDUCATION 1.  The information literate student determines the nature and extent of the information needed. 2.  The information literate student accesses needed information effectively and efficiently. 3.  The information literate student evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system. 4.  The information literate student, individually or as a member of a group, uses information effectively to accomplish a specific purpose. 5.  The information literate student understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.   S C O P E  O F  T H E  L I B R A R Y  I N S T R U C T I O N  P R O G R A M  •  522 Curriculum integrated classes •  10,665 Students •  24,789 In-person reference desk transactions •  1,000 Virtual Reference transactions  3 Trend Data •  Numbers Curriculum Integrated Classes •  Online Student Feedback •  Online Faculty Feedback •  Learning Outcomes •  Peer review of teaching  Reference Desk Transactions •  READ (Reference Effort Assessment Data) •  Chat Transcript Content Analysis In-office Consultations •  READ assessment  4 INSTRUCTIONAL ASSESSMENT MEASURES   5 How might you apply what you learned in this session to your assignment?  In your research, what might you do differently based on what you learned in this session? 6 7 8 LEARNING OUTCOMES ASSESSMENT Example: During spring semester 2014, learning outcomes were analyzed based on the library instruction rubric.  •  Faculty identified the rubrics at each level that are incorporated into their instruction for scheduled classes.  •  Scheduled classes at each level were randomly selected to complete an online set of learning outcomes which were then graded using a 4.0 scale.  •  The Library Instruction Group made recommendations based on these findings to refine and augment the library instruction program. 9 RECOMMENDATIONS FOR INSTRUCTION Emphasize instruction on the following topics: •  Copyright and plagiarism •  Develop a list of examples of why copyright matters to students •  Develop copyright modules •  Free and fee-based information •  Legal/economic impact on access to information Survey departmental faculty to identify key concepts from our rubric they consider most important. Reconsider curriculum-integrated approach to information literacy instruction especially in relationship to new ACRL Framework.  10 IMPACT OUTCOMES TOOLS To capture library users’ satisfaction, opinions, and perceptions:  Research-Based •  Focus Groups •  Interviews •  Open Feedback •  Surveys  Additional •  Advisory Groups •  Liaison Communications 11 Interviews §  Faculty Research Practices §  Content Analysis §  Website Use & Navigation §  Code and theme Open Feedback §  Feedback Form §  Review §  No Book §  Review Surveys §  ClimateQUAL® §  Code and theme §  Compare with norms §  LibQUAL+® §  Content Analysis §  Compare with norms §  Trend across LQ survey years 12 IMPACT OUTCOMES ASSESSMENT LIBQUAL+®  “A suite of services that libraries use to solicit, track, understand, and act upon users' opinions of service quality.” Conducted in spring 2003, fall 2006, and spring 2010; forthcoming spring 2015 Survey includes: •  22 core items that measure users’ perceptions of service quality •  Affect of Service •  Information Control •  Library as Place •  Questions on: •  General Satisfaction •  Information Literacy  •  Library Use •  Demographic Information •  Open-ended Comment Box Association of Research Libraries, 2014 13 Welcome!We are committed to improving your library services. Better understanding your expectations will help us tailor those services to your needs.We are conducting this survey to measure library service quality and identify best practices through the Association of Research Libraries' LibQUAL+® program.Please answer all items. Thank you for your participation!Please rate the following statements (1 is lowest, 9 is highest) by indicating: Minimum -- the number that represents the minimum level of service that you would find acceptable Desired -- the number that represents the level of service that you personally want Perceived -- the number that represents the level of service that you believe our library currently providesFor each item, you must EITHER rate the item in all three columns OR identify the item as "N/A" (not applicable). Selecting "N/A" will override all other answers for that item.When it comes to...My MinimumService Level IsMy DesiredService Level IsPerceived ServicePerformance Is  Low High Low High Low High N/A1) Employees who instill confidence in usersN/A2) Making electronic resources accessible frommy home or officeN/A3) Library space that inspires study and learningN/A4) Giving users individual attentionN/A5) A library Web site enabling me to locateinformation on my ownN/AWhen it comes to...My MinimumService Level IsMy DesiredService Level IsPerceived ServicePerformance Is  Low High Low High Low High N/A6) An environment conducive to learningthrough classes, programs, activities, andmeetingsN/A7) Employees who are consistently courteousN/A8) The printed library materials I need for myworkN/A9) Quiet space for individual activitiesN/A10) Readiness to respond to users' questionsN/AWhen it comes to...My MinimumService Level IsMy DesiredService Level IsPerceived ServicePerformance Is  Low High Low High Low High N/A11) The electronic information resources I needN/A12) Helpful online guides and tutorialsN/A13) Employees who have the knowledge toanswer user questionsN/A14) Library staff providing help that assists infinding information needed now while improvingmy research skillsN/A1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9LibQUAL+® —Survey PreviewThis window will not appear on the live survey.5.5#6#6.5#7#7.5#8#8.5#Employees#who#ins3ll#confidence#in#users#Giving#users#individual#a=en3on#Employees#who#are#consistently#courteous#Readiness#to#respond#to#users'#ques3ons#Employees#who#have#the#knowledge#to#answer#user#ques3ons#Employees#who#deal#with#users#in#a#caring#fashion##Employees#who#understand#the#needs#of#their#users#Willingness#to#help#users#Dependability#in#handling#users'#service#problems#Making#electronic#resources#accessible#from#my#home#or#office#A#library#Web#site#enabling#me#to#locate#informa3on#on#my#own#The#printed#library#materials#I#need#for#my#work#The#electronic#informa3on#resources#I#need#Modern#equipment#that#lets#me#easily#access#needed#informa3on#EasyLtoLuse#access#tools#that#allow#me#to#find#things#on#my#own#Making#informa3on#easily#accessible#for#independent#use#Print#and/or#electronic#journal#collec3ons#Irequire#for#my#work#Library#space#that#inspires#study#and#learning#Quiet#space#for#individual#ac3vi3es#A#comfortable#and#invi3ng#loca3on#A#getaway#for#study,#learning,#or#research#Community#space#for#group#learning#and#group#study#An#environment#conducive#to#learning#through#classes,#programs,#ac3vi3es,#and#mee3ngs#Ready#access#to#computers/Internet/soTware#Space#for#students#to#study#and#work#in#groups#Zones&of&Tolerance:&Affect&of&Service,&Informa7on&Control,&Library&as&Place&===&All&UM&(Except&Lib&Staff)&2010&14 5.67 6.54 6.63 5.67 6.1 6.29 6.91 6.94 6.23 6.53 6.06 6.71 6.93 6.33 6.74 5.89 6.62 6.63 5.26 5.89 6.19 7.29 7.22 5.57 6.55 6.3 7.6 7.31 6.48 6.43 5.95 6.35 6.51 5.14 5.82 6.12 6.67 6.77 5.46 6.01 6.28 6.28 6.44 5.6 6 5.92 7.02 7.14 5.74 6.71 6.09 6.35 6.5 5.68 6.22 6.33 6.3 6.08 5.74 6.08 6.18 6.87 6.91 5.71 6.33 5.83 6.51 6.59 5.41 5.96 6.1 6.9 6.91 6.08 6.58 6.07 6.7 6.79 5.56 6.14 6.14 6.86 6.99 5.84 6.31 6.18 7.07 7.13 6.02 6.43 5 5.25 5.5 5.75 6 6.25 6.5 6.75 7 7.25 7.5 7.75 The library helps me stay abreast of developments in my field(s) of interest. The library aids my advancement in my academic discipline or work. The library enables me to be more efficient in my academic pursuits or work. The library helps me distinguish between trustworthy and untrustworthy information The library provides me with the information skills I need in my work or study. Mean Information Literacy Question Information Literacy 2003, 2006, 2010 by User Group Undergrad 03 Undergrad 06 Undergrad 10 Graduate 03 Graduate 06 Graduate 10 Faculty 03 Faculty 06 Faculty 10 Staff 03 Staff 06 Staff 10 All UM 03 All UM 06 All UM 10 All ARL 03 All ARL 06 All ARL 09    LIBQUAL+® FINDINGS P E R C E P T I O N S  Areas of greatest concern (gap between perceived and desired service levels): •  A library Web site enabling me to locate information on my own •  Print and/or electronic journal collections I require for my work •  Ready access to computers/internet/software (question not on previous surveys) •  Making electronic resources accessible from my home or office  Areas of greatest importance (desired service levels):  •  Print and/or electronic journal collections I require for my work •  Making electronic resources accessible from my home or office •  A library Web site enabling me to locate information on my own  G E N E R A L  S A T I S F A C T I O N  &  A F F E C T  O F  S E R V I C E  General satisfaction increased from 2003 to 2006 to 2010; trajectory parallels that of ARL libraries.  Staff are the outlier group, their satisfaction decreased slightly between 2006 and 2010 in terms of their satisfaction with the way they are treated at the library and the library’s support for their learning, research, and teaching needs.  Overall rank of Affect of Service: 1.  Employees who are consistently courteous 2.  Willingness to help users 3.  Employees who deal with users in a caring fashion  4.  Employees who have the knowledge to answer user questions 5.  Dependability in handling users' service problems 6.  Readiness to respond to users' questions 7.  Employees who understand the needs of their users 8.  Employees who instill confidence in users 9.  Giving users individual attention 15 Outreach §  New employee orientation §  Newsletter §  Non-academic unit liaisons §  Staff Social and workshops  Facilities §  Comfortable furniture §  Quiet study spaces  Instruction §  Connections with liaison librarians §  Workshop series Web Site §  Discovery service §  LibGuides  Collections §  E-journals and Electronic resources §  Digitization projects §  Popular reading materials  Services §  Chat reference §  Exhibits and programming §  Software and equipment §  Supplies 16 CHANGES MADE BASED ON ALL ASSESSMENTS CHALLENGES L E A R N I N G  O U T C O M E S  •  Partnerships with campus faculty •  Curriculum-integrated information literacy outcomes •  Access to student work •  Tracking students across their academic careers I M P A C T  O U T C O M E S  •  Capturing non-users •  Tracking individuals across their academic careers or employment at UM •  No clear benchmarks •  Responding to results 17 BIBLIOGRAPHY Association of College and Research Libraries (2000). Information literacy competency standards for higher education. Chicago, IL: American Library Association. Retrieved from http://www.acrl.org/ala/mgrps/divs/acrl/standards/standards.pdf Association of Research Libraries (2014). About LibQUAL+®. Retrieved from https://www.libqual.org/ Dugan, R. E., Hernon, P., & Nitecki, D. A. (2009). Viewing library metrics from different perspectives: Inputs, outputs, and outcomes. Santa Barbara, CA: Libraries Unlimited.  18 

Track 1: Information Literacy and Library Assessment

https://scholarworks.umt.edu/cgi/viewcontent.cgi?article=1007&amp;context=outcomes-assessment

Track 1: Information Literacy and Library Assessment

Abstract

Similar works

Full text

Available Versions

University of Montana