The UNO campus recently initiated an electronic portfolio system called myMAPP, to capture data to measure programmatic and individual faculty, staff, and student achievement. Developing an online version of the assessment provided an added opportunity to collaborate with the English composition faculty, and utilizing Blackboard was the most efficient, familiar and convenient. Tapping into the existing technologies and the knowledge base of faculty and graduate students on campus proved invaluable. This presentation shows how two librarians developed an effective, outcomes-based assessment instrument in Blackboard with little money and training and how others can do the same

Hillyer, Nora

Maring, Marvel

English

The College at Brockport, State University of New York: Digital Commons @Brockport

Assessment Can Be Easy 
Via Blackboard
Marvel Maring
and
Nora Hillyer
Library 2-LMS,COCID
Brockport, NY
Assessment Can Be Easy 
Via Blackboard
 Why assessment and why now?
 Assessment, and in particular, outcomes-based 
assessment, is a pressing concern in education today. 
 Perform a search in ERIC with the keyword “assessment” 
and you will retrieve a whopping 95,211 articles.
 NCLB has mandated assessment and accountability in 
public education.
 In higher education, we are also experiencing this 
urgency to assess our programs in a meaningful way.
Assessment Can Be Easy 
Via Blackboard
 Why Blackboard?
 Encouraging colleagues and students to participate in an 
information literacy assessment would be dependent on 
making the assessment as easy, convenient and 
integrated into the general course progression as 
possible. 
 Using the campus’ course management system was 
essential for the assessment’s success.
Assessment Can Be Easy 
Via Blackboard
 Today’s presentation
 We’ll provide background the climate of assessment at UNO that 
necessitated the need for an on-line information literacy 
assessment.
 We’ll describe the development of the first on-line information 
literacy assessment via Blackboard at the Criss Library at the 
University of Nebraska at Omaha.
 We’ll share why we selected Blackboard and we’ll demonstrate how 
to use the assessment module of Blackboard.
 We’ll share some of the future developments and upgrades in 
Blackboard.
Assessment Can Be Easy 
Via Blackboard
 Assessment at UNO--impacting every level
 Institutional level—development of an electronic 
portfolio (called myMAPP)to measure campus-wide 
outcomes and to aid in the accreditation process
 myMAPP electronic portfolio is the digital repository 
for all college, program, faculty, staff and 
(eventually) student performance
Assessment Can Be Easy 
Via Blackboard
 What do we know about assessment and 
how do we learn more?  
 ACRL--three-week on-line training course in 
assessment
 We used our information literacy instruction in 
ENGL 116 (Freshmen Composition) as an example 
for designing an assessment
 We gained theoretical knowledge, practice and 
confidence 
Assessment Can Be Easy 
Via Blackboard
 Assessment resources on campus
 Internal grant funding to develop assessment 
projects on campus
 Campus experts in assessment (faculty in 
Psychology, Education and English)
 Graduate students (Computer Science and 
Psychology)
 ITS Support 
Assessment Can Be Easy 
Via Blackboard
 Drafting the Assessment
 What type of assessment?
 How would it be administered?
 When and where would it be given?
Assessment Can Be Easy 
Via Blackboard
 What did the assessment need to measure?
 How effective is information literacy instruction for students 
in our first year English Composition course, ENGL 116?
 Are students learning anything?
 How best do we measure that learning?
Assessment Can Be Easy 
Via Blackboard
 What is the best way to measure the 
effectiveness of our information literacy 
instruction? 
 We wanted to measure students’ information literacy 
skills prior to library instruction.
 We wanted to measure students’ information literacy 
skills after library instruction.
 And the next question was…
Assessment Can Be Easy 
Via Blackboard
 How would we deliver the assessment?
 On-line would be essential
 Criteria we used to make our decision about an on-
line assessment software program included:
 Cost 
 Ease of use
 Convenience
 Accessibility
 Autonomy
 Technical support
 We investigated three different assessment software 
products: 
 Respondus –not responsive enough, and no free trial
 Perception’s Questionmark —compatibility problems with 
Blackboard
 Blackboard—assessment module available and already in place on 
campus
Assessment Can Be Easy Via 
Blackboard
Assessment Can Be Easy Via 
Blackboard
 Throughout the planning and development of the Criss Library 
Assessment Project we were continually challenged to think 
about how using an on-line assessment product would impact:
 Institutional interests
 Program interests
 Librarian interests
 Student interests
Assessment Can Be Easy 
Via Blackboard
 We discovered that this project development was 
making us assess several things at once:
 The questionnaire itself (what type of questions to 
include)
 The on-line assessment software and how it performed 
in our environment
 The students in our ENGL116 library instruction sessions
Assessment Can Be Easy 
Via Blackboard
 How would using Blackboard for our assessment 
intersect, impact or influence:
 Institutional interests
 Program interests
 Librarian/classroom instructor interests
 Student interests
Assessment Can Be Easy Via 
Blackboard
 Institutional Interests
 Compatible with myMAPP
 On-line assessment would be essential for myMAPP
 Reinforces UNO Strategic Plan
 Student Focus
 Strengthens collaboration with the English Department and 
Faculty Consultants
 Build on the success of the current Info. Lit. program
 Utilize experts on campus to learn about assessment 
 Work with English faculty to clarify and prioritize key 
concepts
Assessment Can Be Easy Via 
Blackboard
 Program Interests
 What can we do to improve the library instruction 
curriculum?
 Focus on key concepts
 Return to original curriculum developed several years ago
 Add collaborative and active learning exercises
 Use data gathered from assessment to inform 
discussions about the information literacy program in 
ENGL116
 We now have empirical data to show what students are 
learning
Assessment Can Be Easy Via 
Blackboard
 Librarian Interests
 What can we do to improve our individual 
instruction?
 We focused on content and delivery:
 Librarians reinforced key concepts
 Librarians consistently introduced handouts
 Librarians introduced more collaborative learning exercises
 We now have real data to evaluate instruction
 Librarians can see pre-test scores immediately and adjust from 
instruction session to session
 Aggregated scores can reveal trends in performance per question 
so we can adjust instruction or the way a question is written
Assessment Can Be Easy Via 
Blackboard
 Classroom Instructor interests:
 Returned to curriculum developed collaboratively several 
years before—reinforced solidity of original partnership
 Classroom faculty are “leaders” in Blackboard so they have 
access to student scores immediately
 Faculty can see who has taken the assessment and they can assess 
how well their students  and their class as a whole is doing
 They can track semester by semester assessment scores to see 
improvement
Assessment can be easy
via Blackboard
 Student Interests
 What can we do to improve student learning?
 Provide students with immediate feedback from the 
questionnaire.
 Provide students with an opportunity to compare the pre and 
post questionnaire results. 
 Provide librarians with feedback from the survey so they can 
be responsive to student comments.
Assessment Can Be Easy 
Via Blackboard
 What we did and how we did it…
 We created and administered an on-line pre and post 
questionnaire measuring students’ information literacy skills 
via Blackboard.
 We administered surveys to students and to ENGL 116 faculty 
to assess our effectiveness in delivering library instruction.
 We gathered data from first semester pilot to review 
performance (students’ and Blackboards’).
 We used data to make improvements in the questionnaire and 
the survey for upcoming semesters.
Assessment Can Be Easy 
Via Blackboard
 Now we will show you how Blackboard’s assessment 
module functions…
 Forty-three easy steps to building a test in 
Blackboard
 Build a pool or add question to pool
 Different resources for questions
 Build test from pool
 Add Content Area 
 Add test to content area
 Modify Test
 Modify Test Options
 Test your test
Assessment Can Be Easy Via 
Blackboard
Control Panel
Assessment Can Be Easy 
Via Blackboard
Assessment Can Be Easy 
Via Blackboard
Assessment Can Be Easy 
Via Blackboard
Assessment Can Be Easy 
Via Blackboard
Assessment Can Be Easy 
Via Blackboard
Assessment Can Be Easy 
Via Blackboard
 Challenges of using Blackboard/What 
we learned
 Manual process of adding students
 Building/using groups in the organization
 Limited reporting
 No data analysis
 Reports from original study
Assessment Can Be Easy Via 
Blackboard
Assessment Can Be Easy Via 
Blackboard
Assessment Can Be Easy Via 
Blackboard
Assessment Can Be Easy Via 
Blackboard
 Study Size and Results
 293 Students total
 Reported on 184 (paired sample t-test)
 Pre- and post-questionnaires were 
identical  
 Highest possible score is 15
 Higher test scores following instruction
 Pre-questionnaire 8.23
 Post-questionnaire 9.89
Assessment Can Be Easy Via 
Blackboard
 Study Size and Results
 Impact of instruction and Analysis of 
Variance (ANOVA)
 7.73 pre scores
 10.05 post scores
 Internal Consistency Reliability
 Consistency displayed by questions
 0 = bad 1.00 = good
 .699 = reliable
Assessment Can Be Easy Via 
Blackboard
 Future Developments
 Blackboard Version 8  
 Available May/June 2008
 Gradebook is changing
 Different organization of groups
 Want to see data per group
 Reporting module changing
 Change manual process of adding 
students to an automated process
 Set up visitor accounts to test our tests
Assessment Can Be Easy Via 
Blackboard
 Questions???
 Thank you!
Assessment Can Be Easy Via 
Blackboard


Assessment Can Be Easy Via Blackboard

https://digitalcommons.brockport.edu/cgi/viewcontent.cgi?article=1004&context=lib2lms

Assessment Can Be Easy Via Blackboard

Abstract

Similar works

Full text

Available Versions

The College at Brockport, State University of New York: Digital Commons @Brockport