With general education coursework comprising a significant portion of associate and bachelor degree curriculum, conveying the content and importance of student competencies to stakeholder audiences has never been more important. Accreditation pressure and accountability measures are influencing a focus on improving the presentation of competencies aligned to their assessment measures. This study examines the extent to which institutions publicly and proactively disclose relevant information regarding assessment of the general education curriculum. Examined are websites of 71 higher education institutions, representing a range of public settings in all American regions. Results show variability in the alignment of objective to measure and reveal current themes and emphasis regarding how competencies relate to coursework. Provided are recommendations for improving institutional presentation

McGowan, Veronica F

English

SWOSU Digital Commons (Southwestern Oklahoma State University)

Southwestern Oklahoma State UniversitySWOSU Digital CommonsFaculty Articles & Research Center for Excellence in Teaching & Learning2-22-2018Higher Educations’ Institutional Transparency ofGeneral Education Competencies, AssessmentMeasures, and Analysis of Assessment DataVeronica F. McGowanSouthwestern Oklahoma State University, veronica.mcgowan@swosu.eduAbstractWith general education coursework comprising a significant portion of associate and bachelor degreecurriculum, conveying the content and importance of student competencies to stakeholder audienceshas never been more important. Accreditation pressure and accountability measures are influencing afocus on improving the presentation of competencies aligned to their assessment measures. This studyexamines the extent to which institutions publicly and proactively disclose relevant informationregarding assessment of the general... Read MoreFollow this and additional works at: https://dc.swosu.edu/cetl_articlesThis Poster is brought to you for free and open access by the Center for Excellence in Teaching & Learning at SWOSU Digital Commons. It has beenaccepted for inclusion in Faculty Articles & Research by an authorized administrator of SWOSU Digital Commons. An ADA compliant document isavailable upon request. For more information, please contact phillip.fitzsimmons@swosu.edu.Recommended CitationMcGowan, Veronica F., "Higher Educations’ Institutional Transparency of General Education Competencies, Assessment Measures,and Analysis of Assessment Data" (2018). Faculty Articles & Research. 3.https://dc.swosu.edu/cetl_articles/3Higher Educations’ Institutional Transparency of General Education Competencies, Assessment Measures, and Analysis of Assessment Data Dr. Veronica McGowanSouthwestern Oklahoma State UniversityAuthors names and affiliations, can include contact information Abstract MethodsLiterature ReviewConclusion With general education coursework comprising a significant portion of associate and bachelor degree curriculum, conveying the content and importance of student competencies to stakeholder audiences has never been more important. Accreditation pressure and accountability measures are influencing a focus on improving the presentation of competencies aligned to their assessment measures. This study examines the extent to which institutions publicly and proactively disclose relevant information regarding assessment of the general education curriculum. Examined are websites of 71 higher education institutions, representing a range of public settings in all American regions. Results show variability in the alignment of objective to measure and reveal current themes and emphasis regarding how competencies relate to coursework. Provided are recommendations for improving institutional presentation.With nearly one third of an undergraduate degree devoted to general education coursework [1] and evidence that accountability frameworks [2] and general education curriculum requirements [3] continue to expand, the impact of the general education curriculum for preparing students for upper level academic responsibilities and post-college careers is significant. While the development of robust, nation-wide initiatives of voluntary systems of accountability provide a meaningful way for stakeholders to compare metrics, only the institution can convey the philosophy of its general education experience in terms of its development of competency areas and overall integration of subject areas and skill development.Critics deride some general education implementations for: inclusion of a large pool of courses with do not map to stated goals, failure to guide students [4]; alignment efforts of course implementation to guiding principles, faculty knowledge of general education principles, and perpetuation of confusion regarding requirements [5]; and reflecting discipline dominance and internal politics of an institution [6]. Issues with assessing of the general education curriculum have a long history [7, 8] with problems attributed to emerging processes and models [9], collaboration of faculty in different disciplines and departments [10], lack of institutional supports including multi-disciplinary assessment personnel [11], lack of measurable learning outcomes due to time, knowledge and alignment constraints [12], and the need for assessment to occur outside of a course-based framework to reflect the entire general education program [13]. This study surveys seventy-one institutions of higher education with breakout data by institutional type and region in order to explore current practices in institutional presentation of general education competencies and their assessment measures. The following sections provides an overview of general education learning outcomes and assessment followed by a description of the methodology for the study, a relation of findings and discussion of those findings linked to field literature and best practice.80% of AAC&U member institutions having invested in current or recent general education revision [14], often to incorporate accreditation requirements addressing student learning outcomes. Theorists have addressed essential learning outcomes in the general education curriculum, with critical or higher-order thinking, communication skills , and interpersonal relationship development [15], along with engaged citizenship, quantitative reasoning, scientific investigation, humanities and the arts, human culture and social science forwarded as the most important competencies [16]. Current issues:• unstructured or “menu” approach• decrease graduation credit requirements to a 120 credit framework• major knowledge deficits• deficit in student perception of value in the GE curriculum• deficit in empirical studies of general education learning outcomesRegional accreditation is a primary driver of transparency activity [17].General Education AssessmentParticipation in one of the Voluntary System of Accountability (VSA)’s three recommended standardized examinations is at an all-time high [18] although critics are concerned that testing metrics will be used as a sole metric in evaluating institutional quality [19]. While formative, process-oriented assessment measures such as portfolios are difficult to compare and benchmark, they are increasingly advocated as an authentic measure of learning outcomes because they allow students to scaffold and organize communication [20] and assess competencies not easily tested by standardized measures, such as integration of [21]. There is evidence that usage of general education assessment data is changing: 73% of ACT, Inc. survey respondents used the Collegiate Assessment of Academic Proficiency (CAAP) to assess instructional effectiveness, but by 2009 less than half of institutions selected that response with benchmarking, assessing mastery, and complying with accreditation expectations ranking higher [22]. Accreditation pressure is still the primary motivator for assessment change, but internal mechanisms are increasingly important with over 88% of higher education institutions deploying some type of academic continuous improvement model [23]. Increasingly, calls for change are embracing alternative ways to assess higher learning outcomes including data collection on non-traditional student populations such as first-generation students, ESL students, working students, etc. and student milestones such as credential attainment after transfer and time to certificate attainment to value the service that of express degree programs have for serving employment needs [24].Transparency of Institution-Provided Assessment Measures and Analysis While participation in mandated and voluntary accountability systems is at an all-time high in higher education, that participation is less prevalent at the institutional level where stakeholders unfamiliar with national systems are likely to mine data. Literature related to institutional transparency of assessment-related components including learning outcomes, assessment measures, data analysis, and links to institutional change is at deficit levels in higher education, although some field literature has studied assessment implementations, qualified descriptions of successes and promoted models of general education assessment. In preparing for this study, it was noted that expectations of findings regarding analysis of assessment data, known in the field as “closing the loop”, are expected to be low due to significant field literature expressing concerns in this area due to different purposes and uses for effectiveness data [25, 26, 27] or the loss, abandonment, or ignorance of collected data [28].Findings of this literature review had a direct impact on coding instrument development, contributing to overall theme identification and substantiating expectations. • content analysis method • Pilot study to determine an initial set of evaluative criteria, and a possible spectrum of responses • led to a level of abstraction for the inductive categories that will allow the research coders to correctly place each phenomena presentation especially in relation to study parameters. Consensus between coders may result in the revision of categories to best represent viewing perspective and enhance formative and summative reliability. Inter-rater reliability was assured with the deployment of three raters; common training insured that raters used a rubric to determine categorical placement and fit. • searches for findings at logical subsites (assessment office and general education curriculum or program page). Search terms: general education, liberal arts, liberal arts core, core curriculum, institutional core, and general education assessment; the more robust measure, such as standardized exam, coded as a primary finding. Indirect measures were not coded as learning assessments, even if indicated as such by the presenting institution. • Population of 71 institutions, all but one institution presented at least their general education learning outcomes, therefore, sample size is 70• 68.5% deploy the name general education or a close derivative; 15.7% use core curriculum; 5.7% use liberal education; 4.3% are unnamed due to a transitioning process; 2.8% use general education core; and 2.8% use the state system’s transfer curriculum name. • Effective communication was most prevalent competency (96% requiring some demonstration in an oral, written, and\or symbolic medium)1) Compared to literature review study expectations, institutions are providing more explicit statements SLOs.2) institutions are providing a greater amount of evidence in a greater number of assessment areas3) number of competency areas is growing; change from subject area to skill development4) institutions mirroring national average with nearly equal transparency across type despite resource allocations.Implementers seeking to improve institutional transparency might consider NILOA’s Transparency Framework and its recommendations for provision of student learning outcome statements, assessment plans, assessment resources, assessment activities, evidence of learning, and use of evidence. Specific tips include: extrapolation of assessment results normally behind a security wall for public consumption, consolidation of college and departmental results to one convenient location, provision of reports at an eight-grade reading level with restricted use of jargon, and linking statements of student learning outcomes to measure and analysis of results.ReferencesReference list provided separately.National Institute for Learning Outcomes Assessment. (2011). Transparency Framework. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Retrieved from: http://www.learningoutcomesassessment.org/TransparencyFramework.htmIntroductionResults• critical thinking category had largest array of assessment• 27.1% of institutions are offering conflicting information• 57% of learning outcomes were found in a university catalog or disaggregated general education requirements page; 28.5% of learning outcomes were found on a dedicated page• 11.4% of sampled institutions present no information about general education assessment measures, 40% of institutions do not present clear evidence that all learning outcomes have an assessment measure.• 14.3% of sampled institutions provided disaggregated information analyzing general education assessment data for institutional change. • only statistically significant finding on institutional type: non-regional state universities had the lowest participation rate in learning outcome transparency. Results (con’t.) Competency Presentation occurrenceImplied or no measurement Cited measurementEffective communication 96% Implied 48.5%Measure information not provided 10%Field standardized exam 10%Use of mid-level assessment 10%Evaluation rubric or effectiveness study 11.4%Quantitative reasoning\problem solving\mathematics92.8% Implied 52.8%Measure information not provided 12.8%Field standardized exam 8.5%Use of mid-level assessment 8.5%Evaluation rubric or effectiveness study 7.1%Humanities; aesthetics 80% Implied 52.8%Measure information not provided 18.5%Use of mid-level assessment 4.3%Evaluation rubric or effectiveness study 5.7%Social & behavioral science 72.8% Implied 48.5%Measure information not provided 18.5%Evaluation rubric or effectiveness study 5.7%Citizenship; human experience; social beings and institutions; American historical & political68.5% Implied 34.3%Measure information not provided 18.5%Use of mid-level assessment 5.7%Evaluation rubric or effectiveness study 5.7%Critical thinking 57% Implied 20%Measure information not provided 10%Field standardized exam 5.7%Use of mid-level assessment 7.1%Evaluation rubric or effectiveness study 8.5%Completion of GE curriculum indicates critical thinking occurs 5.7%Scientific investigation; acquire and evaluate information; science with lab component48.5% Implied 22.8%Measure information not provided 11.4%Field standardized exam 4.3%Use of mid-level assessment 4.3%Global or international perspective; cultural heritage48.5% Implied 25.7%Measure information not provided 14.3%Use of mid-level assessment 4.2%Evaluation rubric or effectiveness study 2.8%Natural Science 47.1% Implied 28.5%Measure information not provided 14.3%Use of mid-level assessment 4.2%Evaluation rubric or effectiveness study 1.4%Information\digital literacy; computer proficiency\understanding of technological impacts on society44.3% Implied 21.4%Measure information not provided 11.4%Use of mid-level assessment 8.5%Evaluation rubric or effectiveness study 5.7%Fine arts; Creativity 38.5% Implied 32.8%Measure information not provided 2.8%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 1.4%Ethical reasoning; moral decision making 27.1% Implied 14.2%Measure information not provided 7.1%Use of mid-level assessment 2.8%Evaluation rubric or effectiveness study 2.8%Wellness; Develop whole person; Life skills; Physical education22.8% Implied 7.1%Measure information not provided 10%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 4.3%Collaboration with others; Teamwork; Service learning20% Implied 11.4%Measure information not provided 4.3%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 2.8%Research; Data analysis 17.1% Implied 8.5%Measure information not provided 4.3%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 1.4%First year; University studies; Freshman orientation15.7% Implied 4.3%Measure information not provided 8.5%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 1.4%Diversity; Ethnicity and race studies 15.7% Implied 10%Measure information not provided 2.8%Use of mid-level assessment 1.4%Evaluation rubric or effectiveness study 1.4%English; English composition 12.8% Implied 5.7%Measure information not provided 2.8%Use of mid-level assessment 4.3%Historical awareness 12.8% Implied 11.4% Use of mid-level assessment 1.4%Intellectual and professional aptitudes; Disposition11.4% Implied 4.3%Measure information not provided 8.5%Field standardized exam 1.4%Use of mid-level assessment 2.8%Development of responsibility for lifelong learning11.4% Implied 2.8%Measure information not provided 7.1%Use of mid-level assessment 2.8%Sustainability; Environmental awareness 10% Implied 8.6% Use of mid-level assessment 1.4%Values; Personal responsibility 7.1% Implied 4.3%Measure information not provided 1.4%Use of mid-level assessment 1.4%Literature 5.7% Implied 5.7%Interdisciplinary; Integrated 5.7% Implied 4.3% Use of mid-level assessment 1.4%Leadership; Decision-making; Military Science4.3% Measure information not provided 4.3%Foreign language 4.3% Implied 1.4%Measure information not provided 1.4%Evaluation rubric or effectiveness study 1.4%Liberal education 4.3% Implied 1.4%Measure information not provided 1.4%Use of mid-level assessment 1.4%

Higher Educations’ Institutional Transparency of General Education Competencies, Assessment Measures, and Analysis of Assessment Data

https://dc.swosu.edu/cgi/viewcontent.cgi?article=1002&context=cetl_articles

Higher Educations’ Institutional Transparency of General Education Competencies, Assessment Measures, and Analysis of Assessment Data

Abstract

Similar works

Full text

Available Versions

SWOSU Digital Commons (Southwestern Oklahoma State University)