Using existing assessments to track longitudinal development of students’ problem solving skills

Abstract

Do we really know how our students develop transferrable skills? Do they? Learning outcomes assessment (LOA) data are most commonly collected via national surveys, alumni surveys / focus groups, and locally developed surveys (MacFarlane & Brumwell, 2016; Williams, 2014). While these methods collect important affective information about students and alumni, they are not well-suited to measure students’ actual achievement and are often disconnected from the classroom where student skills are developed (Campbell & Cabrera, 2011; Gordon, Ludlum, & Hoey, 2008; Porter, 2011, 2013). Detailed LOA data describing students’ actual skills and perceived skills are needed to facilitate targeted improvements to courses and programs and enhance the LOA initiative (Klemenčič & Chirikov, 2015). In this session, we present “The Framework”: a strategy that coordinates three essential perspectives of LOA (intention, achievement, and perception) in the evaluation of a program. Focussing on problem solving for its pilot use, The Framework uses a novel problem solving profile tool to characterize the problem solving challenge of assignments, a program-level developmental rubric for problem solving, and a student survey to collect students’ perceptions of their own problem solving development. To the best of our knowledge, The Framework is original and has not been established elsewhere. Using pilot data from the 2018-2019 academic year, we invite you to explore how The Framework generates both course and program data educators were previously unable to access and how these data can change the questions we ask about our courses/programs and shift our conceptualizations of curriculum mapping. The principles of The Framework are transferrable to any undergraduate context