Center for Student Success Research,

English

ScholarWorks@UMass Amherst

Universal Design in postsecondary learning outcomes assessment

University of Massachusetts Amherst ScholarWorks@UMass Amherst Reports and White Papers Center for Student Success Research 2020 Universal Design in postsecondary learning outcomes assessment Center for Student Success Research Follow this and additional works at: https://scholarworks.umass.edu/cfssr_reports  Part of the Disability and Equity in Education Commons, and the Higher Education Commons Issue Brief: Disability in Higher Education Universal Design in Postsecondary Learning Outcomes Assessment Most colleges and universities do not think systematically about the needs of students with disabilities (Kimball et al., 2016). They may also use assessment instruments that are not inclusive (Peña et al., 2018), which both compromises their overall accuracy and obscures the perceptions of students with disabilities. As part of a broader study examining the college-going of students with disabilities in Massachusetts, we also developed recommendations for thinking carefully about how to include students with disabilities in assessment plans. This brief summarizes key methodological issues and presents a framework for inclusive design.  Methodological Issues & Opportunities The best estimates for the percentage of overall enrollments represented by students with disabilities range from 15-25% (Kimball et al., 2016). As Peña and colleagues (2018) explain, this means that the students with disabilities must be included in any study if it claims to represent the entire student population. Otherwise, the study will be so biased that it cannot meet common methodological norms. However, it can be hard to create inclusive assessment instruments. When instruments require skills or knowledge other than those measured (e.g., skills required to access or deliver a response to test items), it introduces error, and when changes to the assessment intended to promote inclusivity mean that different people may understand or respond to a question in fundamentally different ways, that too can introduce error.  The way we approach learning outcomes assessment on college and university campuses is heavily based on theories of testing and methodological norms within quantitative social science. Nonetheless, we developed this framework to be a broadly useful way of thinking, rather than a specific approach within a single methodological tradition. As a result, the key considerations we describe below can be used in qualitative, mixed methods, or quantitative research with equal utility. The insight they offer is not a specific vocabulary, but rather a structured way of thinking about how disability diagnoses, the way a person with a disability describes their own identity, and varied experiences based on disability status shape learning outcomes for students with disabilities. Confounding Variable(s) of Disability. A confounding variable is one that plays a key role in observed outcomes but which is not included in the assessment design. The problem we described at the outset of this paper, wherein students with disabilities comprise such a large percentage of total enrollment that not intentionally addressing their experiences can produce misleading results, is an example of how disability can function as a confounding variable. To avoid this possibility, we suggest that higher education professionals who are developing plans and instruments for assessment ask: Did we consider disability in our data collection or analysis? How did we collect information about which students have disabilities? What definition of disability does this data collection reflect? Is it possible that a person’s disability status or experience might explain the observed outcomes? Control Variable(s) of Disability. Disability can be an incredibly important part of a person’s identity or it might be something that they only really think about when in an environment that makes it difficult for them to fully participate. In this instance, a person with a disability has a different experience than peers without disabilities. In other words, a person’s disability shapes their experience, which quantitative researchers address using control variables—a set of measurements included in a study to help account for variation in outcomes not linked to the study’s main topic of interest. Higher education professionals developing plans and instruments for assessment can ask the following questions to think about disability as a control variable: Do people with some types of disability have different learning outcomes than peers with or without disabilities? How might disability status interact with other key social identities (e.g., race, class, gender, or sexual orientation). Moderating Variable(s) of Disability. Different people with different disability statuses or identities describe themselves distinctly. For example, an autistic person may answer “No” to a question asking whether they have a disability, “Yes” to a question whether they have an autism spectrum disorder, and in highly variable ways when asked about how (or if) they experience problems related to concentration, problem solving, or social interactions. In social science, moderating variables are used to describe how the ways   that outcomes might vary based on people’s experiences. To address moderation in intentional ways, higher education professionals developing plans and instruments for assessment ask: How might variations in people’s disability status or experience explain variation in outcomes? How might this variation shape the extent to which someone realizes positive or negative outcomes?  Mediating Variable(s) of Disability. Not all people with disabilities share the same experiences, but people with disabilities often have experiences different from their peers. In quantitative research, mediating variables are used to address the possibility that an experience other than the one under investigation is shaping outcomes (for example, a new tutoring center might explain improvements in the quality of writing among junior students rather than a new curriculum). We recommend that higher education professionals developing plans and instruments think about the following questions to incorporate disability as a mediating variable: Do students with disabilities use campus resources differently than peers? Do accommodations shape the experiences of students with disabilities? Do students with disabilities face different time or financial costs than peers?    Summary Colleges and universities can promote inclusive assessment in two main ways: accommodation and universal design. In an accommodation framework, inclusion is achieved by retrofitting inaccessible plans and instruments to allow the participation of people with disabilities. Doing so typically requires modifying either the conditions for or the nature of an assessment. It typically can reduce but not eliminate measurement error. In contrast, universal design tries to create accessible plans and instruments from the outset. The framework for thinking inclusively about variable(s) of disability we described above can aid in this task. By treating disability as part of an assessment plan rather than an unexpected issue to be addressed reactively, it is possible to create more accessible plans and instruments. References Kimball, E., Wells, R., Ostiguy, B., Manly, C., & Lauterbach, A. (2016). Students with disabilities in higher education: A review of the literature and an agenda for future research. In M. Paulsen (Ed.), Higher education: Handbook of theory and research (Vol. 31, pp. 91-156). New York, NY: Springer. Peña, E., Stapleton, L., Brown, K. R., Broido, E., Stygles, K. & Rankin, S. (2018). A universal research design for student affairs scholars and practitioners. College Student Affairs Journal, 36(2), 1-14.  Recommended citation: Center for Student Success Research (2020). Universal Design in postsecondary learning outcomes assessment. (Report No. CSSR 2020-105). Center for Student Success Research. 

https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1005&amp;context=cfssr_reports

Universal Design in postsecondary learning outcomes assessment

Abstract

Similar works

Full text

Available Versions

ScholarWorks@UMass Amherst

ScholarWorks@UMass Amherst