Kramer, Philip I.

English

College of Saint Benedict and Saint John’s University: DigitalCommons@CSB/SJU

College of Saint Benedict and Saint John's University DigitalCommons@CSB/SJU OARCA Staff Publications Office of Academic Review and Curricular Advancement 10-2008 “A Meta-Evaluation of Institutional Outcomes Assessment: The Confluence of Assessment Policy, Procedure, and Practice” or, simply, “Assessing Assessment” Philip I. Kramer College of Saint Benedict/Saint John's University, pkramer@csbsju.edu Follow this and additional works at: https://digitalcommons.csbsju.edu/oarca_pubs  Part of the Educational Assessment, Evaluation, and Research Commons Recommended Citation Kramer, P. I. (2008). “A Meta-evaluation of institutional outcomes assessment: The confluence of assessment policy, procedure, and practice.” Presented at the Indiana University Purdue University Indiana 2008 Assessment Institute Conference, October 26-28, 2008, Indianapolis, IN. This Presentation is brought to you for free and open access by DigitalCommons@CSB/SJU. It has been accepted for inclusion in OARCA Staff Publications by an authorized administrator of DigitalCommons@CSB/SJU. For more information, please contact digitalcommons@csbsju.edu. “A Meta-Evaluation of Institutional Outcomes Assessment: The Confluence of Assessment Policy, Procedure, and Practice”or, simply,“Assessing Assessment”Philip Kramer, College of Saint Benedict and Saint John’s University, Director of Academic Assessment1Why Should We Assess Assessment?Good to periodically evaluate the policy, procedure, and practice of student outcomes assessment. Why? Opportunity to reflect on purposes, structures, and activities of assessment and strategically plan, design, and implement improvements in teaching and learning.2Goals of the WorkshopTo consider one case study of an assessment of assessment and to begin examining  assessment at your institutions;Session designed for all campus stakeholders;To make assessment part of everyday activities and long term, strategic thinking and decision-making.3My Assessment of Assessment (Was I nuts?) I was new to the institutions and wanted to understand the assessment landscape;Recent and significant personnel changes in academic areas affecting assessment;No one had ever assessed assessment.4My Assessment of Assessment (Context)Institutions were researching and writing self-study in preparation for the October 2008 Higher Learning Commission (HLC) accreditation site visit. Ten years ago, the last accreditation report dinged us for concerns related to assessment and program review... 5My Assessment of Assessment(Previous Accreditation Dings)Little faculty understanding of and appreciation for assessment’s potential contribution;Assessment was unevenly designed and implemented across academic departments and general education;Serious deficiency of systemic program evaluation for the purposes of demonstrating program quality.6My Assessment of Assessment(The Report)A ten year descriptive history of our assessment, including assessment origins, stops and starts and stops, faculty governance, faculty resistance, leadership changes, and personnel challenges. 7My Assessment of Assessment(The Report)Evaluative summary of recent assessment developments, including faculty plans to embed assessment; struggles with disciplinary and general ed plans; and failure to articulate administrative and faculty expectations for student learning outcomes.8My Assessment of Assessment(The Report)A glossary of typical assessment terms common to most institutions: e.g., student outcomes assessment; accountability; value-added; methodological, organizational, and implementational issues, pedagogical and curricular improvement.9My Assessment of Assessment(The Report)A glossary of terms and acronyms unique to my institutions: APSAC (Academic Policy, Standards, and Assessment Committee); AAP (Academic Assessment Plan); and ACCCR (annual core/common curriculum report).10My Assessment of Assessment(The Report)Description of our internal assessment stakeholders: faculty; staff; students; administrators; board members; and members of the monastic communities.11My Assessment of Assessment(The Report)Description of our external assessment stakeholders: employers; alumni; parents; general public; accreditors; peer and aspirant institutions.12My Assessment of Assessment(The Report)Budget: a ten year narrative and a spreadsheet fiscal history of monies spent on assessment by majors, minors, and divisions;Fiscal policy analysis of the evolution of assessment budget allocation in Academic Affairs.  13My Assessment of Assessment(The Report)Assessment Personnel: description of the development, transitions, roles and responsibilities of jobs related to assessment.14My Assessment of Assessment(The Report)Extensive evaluative discussion of ten years of assessment policy, procedures, and practice; both then--challenging, irregular, a faculty add-on, vigorously resisted at times, limited resources, seen as decades long imposition by accreditors...15My Assessment of Assessment(The Report)...and now--high level hires, significant organization changes resulted in shifting the purposes, policies, and processes; now, assessment is nearly systemic, mostly accepted, and used increasingly for data-informed decision-making.16My Assessment of Assessment(The Report)Examples of policy, procedure, and practice changes in assessment:Top leadership committed to using assessment to improve teaching and learning;A new general ed curriculum with defined, operationalized, and measurable goals17My Assessment of Assessment(The Report)Examples of policy, procedure, and practice changes in assessment:Faculty committees improved the general ed curriculum and embedded assessment in curricula;Responsibility for assessment has been clearly defined and delegated.18My Assessment of Assessment(The Report)Recommendations made:Clarify purpose of assessment;Increase stakeholder involvement;Improve faculty reward and compensation;Integrate planning...19My Assessment of Assessment(The Report)Recommendations made:...Improve methodologies;Implement systemically;Widely disseminate and use results.20Your Assessment of Assessment(Small Group Discussion)If you’re here with others from your institution, please sit together......If not, that’s okay too--join others from different institutions...21Your Assessment of Assessment(Small Group Discussion)Why do you want to assess assessment?What are the origins of assessment at your institution?Considering the political, cultural, organization, and learning structures and systems at your institution...22Your Assessment of Assessment(Small Group Discussion)List and briefly define the typical assessment descriptions, definitions, (and jargon) common to most institutions that you use.23Your Assessment of Assessment(Small Group Discussion)List and define the assessment terms, including jargon and acronyms, unique to your institution.24Your Assessment of Assessment(Small Group Discussion)Who has responsibility for assessment?Is assessment supported by faculty, staff, students, and the administration? Is it systemic? Is it used? Does it lead to improvements in teaching and learning? Are there rewards? Is there punishment?25Your Assessment of Assessment(Small Group Discussion)Does assessment evolve? Is it evaluated and improved?Is it valued? Is it part of faculty, staff, administrative, and student cultures? Is it part of faculty teaching, scholarship, and service?26Your Assessment of Assessment(Small Group Discussion)List your internal and external assessment stakeholders. How important is assessment to each stakeholder group? How do you communicate assessment policies, procedures, and practice (especially the results) to each group?27Your Assessment of Assessment(Small Group Discussion)Describe the budget for assessment--just approximate for now; Analyze the fiscal policy of assessment budget allocations. Does money influence behavior? Is it enough? Why? Why not?28Your Assessment of Assessment(Small Group Discussion)Describe the development, transitions, challenges, and successes of the roles and responsibilities of assessment personnel.Who does the work? Who leads? Who follows? Who likes it? Who hates it? Who uses it?29Your Assessment of Assessment(Small Group Discussion)Step back and consider... Past, present, and future assessment policies, practices, and procedures and outcomes. Where have you been? Where are you now? Where do you want to go and why? What do you need to get there?30Your Assessment of Assessment(Small Group Discussion)Describe tentative recommendations you will make to improve the policies, procedures, and practices of assessment? Where will you go? Why? How? When? By and with whom? What systems and structures do you want in place? Why?31Your Assessment of Assessment(Small Group Discussion)Group reports: the state of assessment at your institutions--past, present, and future?32Thank you!Thanks for participating!Philip Kramer, College of Saint Benedict and Saint John’s University(320) 363-5289pkramer@csbsju.eduwww.csbsju.edu/assessment33

“A Meta-Evaluation of Institutional Outcomes Assessment: The Confluence of Assessment Policy, Procedure, and Practice” or, simply, “Assessing Assessment”

https://digitalcommons.csbsju.edu/cgi/viewcontent.cgi?article=1001&context=oarca_pubs

“A Meta-Evaluation of Institutional Outcomes Assessment: The Confluence of Assessment Policy, Procedure, and Practice” or, simply, “Assessing Assessment”

Abstract

Similar works

Full text

Available Versions

College of Saint Benedict and Saint John’s University: DigitalCommons@CSB/SJU