Levesque, Vanessa R.

English

UNH Scholars' Repository

University of New HampshireUniversity of New Hampshire Scholars' RepositoryFaculty Publications6-1-2018Measuring student mastery of sustainabilitycompetenciesVanessa R. LevesqueUniversity of New Hampshire, Durham, Vanessa.Levesque@unh.eduFollow this and additional works at: https://scholars.unh.edu/faculty_pubsThis Presentation is brought to you for free and open access by University of New Hampshire Scholars' Repository. It has been accepted for inclusion inFaculty Publications by an authorized administrator of University of New Hampshire Scholars' Repository. For more information, please contactnicole.hentz@unh.edu.Recommended CitationLevesque, V. 2018. Measuring student mastery of sustainability competencies. Association for Environmental Studies and Sciences.Washington, D.C.Measuring student mastery of sustainability competenciesVanessa Levesque, PhDLecturer & Assistant Director, Sustainability Dual MajorUniversity of New HampshireAssociation for Environmental Studies and Sciences June 2018Context: New Sustainability Dual Major at UNHSustainability competencies“Complexes of knowledge, skills, and attitudes that enable successful task performance and problem solving with respect to real-world sustainability problems” (Wiek et al 2011)Sustainability competencies“Complexes of knowledge, skills, and attitudes that enable successful task performance and problem solving with respect to real-world sustainability problems” (Wiek et al 2011)Competence DescriptionSystems Ability to analyze complex systems across multiple domains or scales.Futures (or Temporal)Ability to connect past and future societies & environments and develop potential scenarios.Values (or Ethical)Ability to identify and assess ethical issues, and to apply concepts of justice and fairness.Interpersonal Ability to understand, communicate and collaborate across diverse individuals and organizations.Strategic Ability to understand transitions and apply innovative strategies for transformation.Assessing mastery of sustainability competenciesNo agreed upon method• Course evaluation comments (Brundiers and Wiek 2017)• Student self-assessments (Galt et al. 2013)• Responses to questions about written case studies (Remington-Doucette 2013)• Multiple choice knowledge-based test (Zwickle et al. 2014)Research questions1. To what extent do students graduating with a sustainability dual major demonstrate sustainability competencies? 2. How does participation in an immersive case-study capstone advance student competency?3. What can we learn from different methods of measuring competencies?Sustainability Dual Major (SDM)• Must be paired with primary major• 3 core courses• Introduction• Methods• Capstone• 5 pre-approved electives from across the universitySustainability Dual Major (SDM) • Must be paired with primary major• 3 core courses• Introduction• Methods• Capstone• 5 pre-approved electives from across the universityCapstone Class: Immersive & Participatory case studies• 16 students divided into 4 interdisciplinary groups for entire semester• Campus-community resilience planning• Sustainability tour development• Zero waste in residence halls• Zero waste in academic buildings • Immersive & participatory case study experience• Read materials & meet with mentors to learn about project• Develop project proposal & carry out project as a team• Present at Undergraduate Research Conference• Provide project mentor with data and final reportCapstone Class: Immersive & Participatory case studiesMethods: Quantitative survey & Qualitative analysis of blog postsSurvey• Developed by Redman & Wiek, ASU, being tested in US & Germany• Multiple choice questions for 4 of the 5 competencies we were interested in: Systems, Strategic, Values, Futures (+ Problem solving; -Interpersonal)• 16 Capstone students given survey at start and end of semesterBlogsMethods: Quantitative survey & Qualitative analysis of blog postsBlogsMethods: Quantitative survey & Qualitative analysis of blog postsBlogsMethods: Quantitative survey & Qualitative analysis of blog postsBlogs• We identified all mentions of the 5 competencies and scored each one based on the degree to which the student demonstrates masteryMethods: Quantitative survey & Qualitative analysis of blog postsScore Description0 No mention of competency concepts1 Mentions term/concept with no explanation2 Implies concept in an example3 Describes & applies concepts to an exampleCompetenceSystemsFuturesValuesInterpersonalStrategicExample: Systems thinking, level 1“Through taking Economics of Climate Change, I gained an understanding of just how linked the economy and environment are.”Example: Systems thinking, level 2“…Both corn and soy beans are able to be stored pretty much indefinitely, which makes them good options for providing food to third world countries and to people affected by natural disasters. I think these programs are well intentioned but have some serious consequences that were not intended. For example, providing food to third world countries can actually put their local farmers out of business.”Example: Systems thinking, level 3“…Locals wanted Wagon Hill Farm to be open to all types of recreation, including dog-walking, but there was also a push to conserve the land, as erosion was a problem. These recreational activities directly contributed towards erosion, but to fully condemn them would be very unpopular locally. In sustainability, a major theme deals with balancing human activities with conservation efforts.”ResultsCompetency Avg % correct(Start)Avg % correct (End)Rank(Start)Rank (End) Systems Thinking 61.5% 56.8% 4 4Futures Thinking 42.8% 49.3% 5 5Values Thinking 70.6% 69.2% 2 1Strategic Thinking 71.2% 63.6% 1 2Problem-Solving 68.7% 62.0% 3 3SurveyResultsCompetency Avg % correct(Start)Avg % correct(End)Rank(Start)Rank (End) Systems Thinking 61.5% 56.8% 4 4Futures Thinking 42.8% 49.3% 5 5Values Thinking 70.6% 69.2% 2 1Strategic Thinking 71.2% 63.6% 1 2Problem-Solving 68.7% 62.0% 3 3SurveyResultsCompetency Avg % correct(Start)Avg % correct(End)Rank(Start)Rank (End) Systems Thinking 61.5% 56.8% 4 4Futures Thinking 42.8% 49.3% 5 5Values Thinking 70.6% 69.2% 2 1Strategic Thinking 71.2% 63.6% 1 2Problem-Solving 68.7% 62.0% 3 3SurveyResultsCompetency Avg % correct(Start)Avg % correct(End)Rank(Start)Rank (End) Systems Thinking 61.5% 56.8% 4 4Futures Thinking 42.8% 49.3% 5 5Values Thinking 70.6% 69.2% 2 1Strategic Thinking 71.2% 63.6% 1 2Problem-Solving 68.7% 62.0% 3 3SurveyCompetency Avg score (max =3)RankSystems Thinking 2.4 2Futures Thinking 0.7 5Values Thinking 1.6 4Strategic Thinking 1.5 3Interpersonal 2.6 1BlogsDiscussionQ1: To what extent do students graduating with a sustainability dual major demonstrate sustainability competencies?• It depends on how you test them! • This suggests that competency assessments that rely on one method may not be as validCompetency Avg % correct(Start)Avg % correct (End)Rank(Start)Rank (End) Systems Thinking 61.5% 56.8% 4 4Futures Thinking 42.8% 49.3% 5 5Values Thinking 70.6% 69.2% 2 1Strategic Thinking 71.2% 63.6% 1 2Problem-Solving 68.7% 62.0% 3 3Competency Avg score (max =3)RankSystems Thinking 2.4 2Futures Thinking 0.7 5Values Thinking 1.6 4Strategic Thinking 1.5 3Interpersonal 2.6 1Q2: How does participation in an immersive case-study capstone advance student competency?• Multiple-choice survey suggests no change• But blog posts suggest otherwise…• Multiple-choice survey suggests no change• But blog posts suggest otherwise…I have never worked on a group project with people from all different majors…Our project creating a Sustainability Tour of UNH excels because of the contribution of knowledge from so many areas.This project helped me to truly understand and appreciate the collaborative processes that happen when tackling matters of sustainability… I learned first hand just how important interpersonal, organizational, and various other skills are in these projects.Q2: How does participation in an immersive case-study capstone advance student competency?Q3: What can we learn from different methods of measuring competencies?Neither surveys nor open-ended blog posts seemed to fully assess mastery of competencies• Survey questions rely on knowledge of specific terminology and concepts• Blogs limited by extent to which students can express themselves in writing• Open-ended questions were not designed to map onto the competencies• Yet open-ended questions allowed other possible competencies to emergeDiscussionSuggested new competencies: 1. Basic knowledge of grand challenges (e.g. climate change, water quality & quantity, food systems, waste, etc.)DiscussionSuggested new competencies: 1. Basic knowledge of grand challenges (e.g. climate change, water quality & quantity, food systems, waste, etc.)2. “Sustainability in Practice” – high level understanding of how the concepts within each competency play out in real projectsI personally learn more from direct application... Understanding sustainability is more than a concept, it’s an experience and I believe sustainability students will become more passionate about sustainability if they have more opportunities to do so.DiscussionFinally, are certain pedagogical approaches better suited to teaching certain competencies? • Blog posts scored high on interpersonal competence: Did the immersive case study experience help develop those collaborative skills?Thank you! Questions?Vanessa LevesqueLecturer & Assistant Director, Sustainability Dual MajorUniv. of New HampshireVanessa.levesque@unh.edu

Measuring student mastery of sustainability competencies

https://scholars.unh.edu/cgi/viewcontent.cgi?article=1639&context=faculty_pubs

Measuring student mastery of sustainability competencies

Abstract

Similar works

Full text

Available Versions

UNH Scholars' Repository