First Student Poster Presentation. Presenters: Annahita Ball and Hilary Drew - "the 21st Century Assessment: Measuring 21st Century Skills among Middle School Students".The Ohio State University College of Social Wor

Ball, Annahita

Drew, Hilary

KnowledgeBank at OSU

The 21st Century Assessment: Measuring 21st Century Skills among Middle School StudentsAnnahita Ball, Hilary Drew, Dawn Anderson-ButcherThe Ohio State University, College of Social WorkMETHODInstrument DevelopmentThe 21st Century Assessment was developed usingsteps of scale development outlined by DeVellis (2003).Following a thorough review of literature, the researchersinitially developed 29 items to measure 21st Century skillsin the three domain areas identified in the literature. Theseitems also included measures of 21st Century learningenvironments. All items utilized a 5-point Likert-typeresponse scale (1=Not at all true; 5=Really true).Participants and Data CollectionThe 21st Century Assessment was administered tomiddle school students (N = 267) in grades 6 (34.9%), 7(34.9%), and 8 (27.8%). Nearly half (47.9%) ofrespondents reported as male and 46.1% reported asfemale. Half of the students reported their race/ethnicity asAfrican American (50.8%), 25.2% reported as White, and21.1% reported as Multi-Racial.Data AnalysisThis study utilized an exploratory factor analysis(EFA) to examine the psychometric properties of the 21stSkills Assessment, comprised of items measuring both 21stCentury skills and 21st Century learning environments.Principal axis factoring with varimax rotation allowed anexamination of the underlying factor structure of eachcomponent of the 21st Skills Assessment. In addition,estimates of internal consistency and intercorrelations werecalculated among the resulting factors.SUMMARYThe scale construction results of this study indicatethat the 21st Century Assessment offers a brief,psychometrically sound measure of the following criticalskill areas: (1) Self-Directed Learning; (2) WorkingResponsibly with Others; (3) Leadership; and (4) TimeManagement. Additionally, this tool offers a valid andreliable measure of 21st Century Learning Environments.These findings should be interpreted with cautiongiven the exploratory nature of this study. While statisticalanalyses indicated that an EFA was an appropriatetechnique for this study, additional scale developmentprocedures are necessary to further validate thismeasurement tool.Implications for Social Work PracticeNevertheless, this instrument has relevance andimportance to social work practice. Social workers canperform interventions at the micro, mezzo, and macrolevels to foster and advocate for 21st Century skilldevelopment among youth. For instance, social workerscan lead efforts to utilize such tools in school andcommunity programming (i.e., afterschool programs) tofacilitate continuous improvement processes gearedtoward developing these skills in today’s youth.Additionally, social workers can play a key role bycollaborating with teachers and other youth leaders tocultivate learning opportunities that enhance 21st Centuryskill development. Finally, social workers can serve asadvocates for policies that ensure equality in technologyaccess across communities and school districts. Thisaccess is essential in the development of these criticalskills. Ultimately, enhancing 21st Century skills betterprepares youth for the current workforce; thus, these skillsmay prove vital for future efforts to ameliorate poverty.RESULTSTwo separate EFAs were used to determine the factor structures of the two scales comprising the 21st Century Assessment. One EFA was used for the analysis of the 21stCentury Skills scale and another for the analysis of the 21st Century Learning Environment scale. Factor analysis was deemed appropriate given significant (p < 0.01) Bartlett’s (1954) Tests of Sphericity and Kaiser-Meyer-Olkin Measure (KMO; Kaiser, 1974) values of 0.92 and 0.90.Items were eliminated if they did not load greater than .50 (Tabachnik & Fidell, 2007). Results of the EFA indicated that the 21st Century Skills scale resulted in a four-factor solution accounting for 62.75% of the total variance. The 21st Century Learning Environment scale resulted in a one-factor solution accounting for 49.97% of the total variance. The factor loadings are presented in Tables 1 and 2 and discussed further below.21st Century SkillsThe resulting four factors were positively related to each other and demonstrated strong internal consistency. Descriptive Statistics and Correlations for the 21st Century Skills scale are presented in Table 3. The factors are as follows:• Factor 1: Accounted for 37.16% of the variance and comprised five items, labeled Self-Directed Learning.• Factor 2: Accounted for 11.38% of the variance and comprised four items, labeled Working Responsibly with Others.• Factor 3: Accounted for 7.34% of the variance and comprised four items, labeled Leadership.• Factor 4: Accounted for 6.88% of the variance and comprised four items, labeled Time Management. 21st Century Learning EnvironmentThe one-factor solution was comprised of eight items accounting for 49.97% of the total variance. This solution demonstrated strong internal consistency as well (α = .88). The factor loadings are presented in Table 2.Table 1: 21st Century SkillsItems and Factor LoadingsINTRODUCTIONTwenty-first century skills (e.g., teamwork,leadership) represent the knowledge and skills that onemust possess to live in the emerging global marketplace(Silva, 2009). As globalization becomes a reality, the needfor youth to develop such skills has become a pressingpriority. Schools and other youth development settingshave an opportunity to foster such skills. Unfortunately,few instruments specifically assess 21st Century skills(Silva, 2009). A demonstrable need exists forpsychometrically sound instruments to evaluate youth oncritical 21st Century skills such as leadership and timemanagement.Several organizations offer frameworks of 21stCentury skill development (e.g., Partnership for 21stCentury Skills, American Association of SchoolLibrarians). Three key domains of skill are typicallyincluded: Learning and Innovation; Information, Media,and Technology; and Life and Career (Partnership for 21stCentury Skills, 2009). Learning and innovation skillsinclude creativity, critical thinking, problem solving, andcollaboration. Information, Media, and Technologyencompasses several types of literacy related to thegrowing prominence of technology in learning (e.g.,information literacy, media literacy, etc.). Life and careerskills include key skills necessary for youth to work incomplex environments, such as time management,initiative, and self-directed learning.In addition, recent research (Christen, 2009;Pappas, 2009) emphasizes the importance of 21st Centurylearning environments that provide opportunities to apply21st Century skills across content areas. These settingsenable innovative learning methods that integrate the useof technology, inquiry- and problem-based approaches,and higher-order thinking skills. Research highlights theimportance of these settings and notes that an emphasison 21st Century skills in learning environments caninfluence both how and what youth learn (Roschelle et al.,2000).Few measures exist to assess these skills amongyouth. As such, this study aimed to develop apsychometrically sound measure of several core 21stCentury skills, titled the 21st Century Assessment. Severalconcepts were considered in the construction of thismeasure, including those within the three domains of 21stCentury skills. This study also aimed to develop ameasure of learning environment in relation to theseskills.Factor 1. 2. 3. 4.1. Self-Directed Learning (α = .83) -- .584** .464** .518**2. Working Responsibly with Others(α = .78)-- .333** .493**3.  Leadership (α = .76) -- .412**4. Time Management (α = .74) --M 3.85 3.53 2.97 3.22SD 0.91 1.00 0.95 1.07Possible Range 1-5 1-5 1-5 1-5Note. ** p < .01Table 2: 21st Century Learning EnvironmentItems, Factor Loadings, and Descriptive Statistics Item Factor 1 M SD1.  My teachers give me the opportunity to lead groups. .681 3.27 1.352.  I learn about other cultures in my classes. .624 3.76 1.273.  My teachers help me organize my time. .639 3.11 1.424.  My teachers give me time to work with other students. .720 3.67 1.265.  My teachers encourage me to ask questions. .738 3.74 1.296.  My teachers use technology in our classrooms. .555 3.83 1.277.  My teachers encourage me to consider the opinions of others. .764 3.42 1.298.  My teachers help me set goals for myself. .750 3.71 1.33Table 3: Descriptive Statistics and CorrelationsThe authors would like to acknowledge Rebecca Wade-Mdivanian for her insight, guidance, and expertise in relation to the development of the 21st Century Assessment.References are available upon request. For more information, please contact Annahita Ball (ball.1776@osu.edu).Item Factor 1 Factor 2 Factor 3 Factor 41. I set goals for myself in school. .696 .261 .144 .1822.    I find ways to achieve my goals. .745 .176 .164 .2643.    I put all my energy into accomplishing my goals..550 .210 .224 .3224.    I push myself. .539 .229 .253 .0825.    I stand up for what I believe in. .507 .258 .246 .1696.    I respect others when working on a project..363 .572 .088 .2217.    I control my temper when working with others on a project..298 .527 .030 .2038.    I listen to what others say. .161 .743 .037 .0999.    I think about how others see things. .163 .732 .154 .15210.  I motivate others. .175 .073 .615 .07911.  I assign tasks to others. .069 .003 .575 .06012.  I resolve conflicts. .182 .137 .657 .16413.  I lead others to accomplish a goal. .205 .079 .693 .24214.  I organize my time. .199 .210 .130 .57715.  I set priorities. .362 .163 .169 .57016. When I have multiple tasks, I plan ahead of time how to accomplish the tasks..152 .166 .221 .699

The 21st Century Assessment: Measuring 21st Century Skills among Middle School Students

https://kb.osu.edu/bitstream/1811/46549/2/22_1poster_ball_drew.pdf

The 21st Century Assessment: Measuring 21st Century Skills among Middle School Students

Abstract

Similar works

Full text

Available Versions

KnowledgeBank at OSU