This study describes the results of a correlation study between iSkills and SAILS, two information literacy assessment instruments

Beile, Penny

Dziuban, Charles

Katz, Irv

Salem, Joe

English

E-LIS

IC3/SAILS Correlation Study      Penny Beile and Chuck Dziuban, Univ of Central FL; S S SMethods ConclusionsIrv Katz, Educational Testing ervice; Joe alem, Kent tate UnivResultsIntroductionProblem StatementInformation literacy has been promoted by librarians as a distinct cognitive model, but it is often cited as being under-theorized.  When working with program faculty to integrate information literacy into the curriculum, librarians should be able to offer some evidence as to what information literacy is and what it is not ForAssessment InstrumentsThe Project SAILS and the ETS iSkills/IC3 assessments were administered to approximately 140 third year Nursing students.  The Project SAILS instrument, which is designed to measure information literacy skill levels and is based on ACRL IL standards, is a multiple-choice item format test Test answers were analyzed in conjunctioniSkills/SAILS correlationAs expected, there was a significant correlation between SAILS and iSkills  r (104) = .56, p < .01.  This befits the two measures as they  purport to assess similar constructs.  [“Actually, this is more than a decent correlation:  In validity work, people get pumped over correlations of .30.”  Irv Katz]Notes about results in general• Selection and administration.  Institutions often don’t know which instrument to use, while others administer both tests to the same population (presumably to measure growth, yet few correlations between instruments have been reported).       .   example, how does information literacy differ from critical thinking?  Is it a technical skill?  A cognitive skill?  How is it bounded?  When information literacy was presented to the faculty at one institution, faculty complained that there are too many literacies – digital, media, numeracy, etc., and asked about the theory behind it.  This prompted the researchers to seek evidence to .       with results from the ETS iSkills assessment (which is now referred to as IC3). iSkills was also inspired by ACRL standards, but includes communications and technology literacy as well.  iSkills assesses ICT literacy through 15 scenario-based tasks, each requiring students to interact with simulated software to solve information-oriented problems. All students in the targeted classes were In a similar study that investigated the amount of overlap between the Project SAILS instrument and the James Madison University’s Information Literacy Test (ILT), researchers found a correlation of .673.  This makes sense as they are such similar tests, and are based exclusively on the ACRL standards.The moderate correlation between SAILS and iSkills should not be surprising, as the tests were both developed based on the ACRL Information Literacy standards, and so rest on a common base.  But does this mean that that the tests are equivalent and interchangeable?  If you consider that a .56 correlation means that only about 30% of the variability in the score of one test can be predicted from the other test it’s clear that each test measuressupport information literacy as a distinct construct.Research ObjectivesThe primary objective of the study was to investigate the concept of information literacy as a distinct construct; as such, this research is theoretical rather than applied.  The approach selected was to administer two standardized assessments that purport to measure similarexpected to complete the tests.  ProtocolIRB approval was granted and negotiations among institutions as to confidentiality and use of the study results were agreed upon.  Students were orally informed of the protocol and purpose of the study and were asked to sign an   ,       mostly unique aspects of information fluency. • Predictive validity. Although overall GPA did not relate to test scores, course grades did align with iSkills, with students earning an A earning significantly higher scores than students earning a C.  This study showed moderate correlations between .56SAT, Course GPA, Cumulative GPA• Scores were reported for verbal (SAT-V) and math      cognitive skills to the same population of students.  It was thought that the amount of overlap, or correlation, between the results of the two administrations could provide some insight into what the instruments were measuring.  Other aims of the study were to aid selection and interpretation decisions regarding using    ,       informed consent document.  The assessments were part of the course requirement, so all students were expected to complete them, but students could choose not to have their results used for the study.  Students completed the assessments in the course classroom; SAILS was administered five weeks into test scores and SAT-V scores.  Earlier research conducted with iSKills revealed the same pattern of correlations with verbal SAT scores higher than with math SAT scores.  This is consistent with the idea that the instruments tap into information literacy skills, which are more closely related to verbal skills (e.g., reading comprehension) than math. C t t lidit d th ti l di I         (SAT-M).  Both instruments correlated with SAT-V around the .40 level; SAILS did not show a significant relationship with SAT-M, but iSkills demonstrated a .35 correlation.  This indicates some differences between the constructs measured.• Neither instrument correlated with overall GPA.• iSkills had a weak correlation with course grade at .24; SAILS was not significant at a .05 level.  standardized assessments for information literacy and to see if information literacy skills correlate to academic success.  Also, when seeking standardized assessment instruments to administer, institutions often don’t know which instrument to use.  Others administer different tests to the same population (presumably to measure growth), yet comparisons are difficult without knowing how the instruments correlatethe course and iSkills followed approximately five weeks after that. Several students were absent each day of the administration and a number of others opted out of the study, which resulted in 105 usable scores.Students’ university-assigned personal identification numbers were used to retrieve SAT scores and c m lati e grade point a erages from• ons ruc  va y an  eore ca  groun ng. n terms of measurement, information fluency is a new construct.  It does not have a long history in the measurement field, and thus there is not a research on how it should be assessed, what types of assessments (multiple-choice, performance-based) assess it best, or the “structure” of the construct in terms of subskills.  The structure of the domain is of particular importance. It is possible that specificFirst Order Correlations [* notes statistical significance]iSkills SAILS GPA Grade SAT-V SAT-MSAILS .56*Overall GPA.02 -.05Course 24* 13 51*     .  Finally, knowing the predictive validity of the instruments, or how information literacy skill levels relate to academic success, is useful for purposes of promoting information literacy.  u u v    v   the Registrars Office.  A  Pearson’s correlation was conducted to compare scores from the assessments to each other.  Scores also were analyzed in conjunction with course GPA, overall GPA, and SAT.        activities identified in the ACRL standards (accessing information, evaluating information, etc.) might not be distinguishable by an assessment.  Instead, people seem to be, overall, either good or not so good  – we do not see people who, for example, are good at Define activities but poor at Evaluate activities.Grade. . .SAT-V .42* .41* .23* .33*SAT-M .35* .16 .24* .20* .50*

iSkills/SAILS Correlation Study

Abstract

Similar works

Full text

Available Versions

E-LIS

E-LIS repository