Developing Quantitative and Analytic Skills in Undergraduates Using Conservation Examples

This talk was given during the North American Congress for Conservation Biology

Student Interpretation of Conservation Data: Does their Reach exceed their Grasp

This study examined how well undergraduate students can develop data analysis skills relevant to conservation biology over the course of a single semester. Students completed two conservation data analysis exercises, pre and post self-assessments of confidence in data analysis skills, a classroom discussion, and pre/post content assessments. Between the first and second exercises, a data analysis teaching intervention was administered in all classes. Instructional and assessment materials were created and validated by 24 conservation educators led by the Center for Biodiversity and Conservation at AMNH. Results from one semester (100+ students) show that students scored significantly higher on post-content assessments for both exercises. We also found significant increases in student self-assessment of confidence in data analysis skills. However, when evaluated at the level of different skill dimensions, students\u27 ability to represent and interpret data improved between exercises, but ability to complete calculations and draw conclusions was significantly worse on the second exercise. While our study demonstrates that direct instruction in data analysis does improve student performance overall, there is a disconnect between student self-assessment of their data analysis skills and their actual ability. This indicates that some aspects of data analysis may require different teaching intervention approaches

Student Interpretation of Conservation Data: Does Their Reach Exceed their Grasp

Background/Question/Methods: The fast pace of biological data generated nowadays calls for our biology students to be proficient in quantitative skills such as data analysis. This study examined how well undergraduate students can develop data analysis skills relevant to ecology and conservation biology over the course of a single semester. Students completed two data analysis exercises, pre and post self-assessments of confidence in data analysis skills, a classroom discussion, and pre/post content assessments. The two data analysis exercises were adapted from the free online teaching modules on the Network of Conservation Educators and Practitioners website (www.ncep.amnh.org). Between the first exercise (a demography problem involving palm harvests and parrots) and second exercise (calculating diversity indices for spider communities), a data analysis teaching intervention was administered in all classes. Instructional and assessment materials were created and validated by 24 conservation educators led by the Center for Biodiversity and Conservation at AMNH. Results/Conclusions: Results from one semester show that students scored significantly higher on post-content assessments for both conservation exercises (N1 = 207 students; N2 = 199; P \u3c 0.01 for both). We also found significant increases in student self-assessment of confidence in data analysis skills (N = 87). However, when evaluated at the level of different skill dimensions, students’ ability to represent and interpret data improved between exercises (N = 257; P \u3c 0.01), but ability to complete calculations and draw conclusions was significantly worse on the second exercise (P \u3c 0.01). While our study demonstrates that direct instruction in data analysis does improve student performance overall, there is a disconnect between student self-assessment of their data analysis skills and their actual ability. This indicates that some aspects of data analysis may require different teaching intervention approaches

How Much Can Students Gain in Data Analysis and Critical Thinking Skills in One Semester?

Background/Question/Methods: The effective preservation and sustainable use of ecosystems is a complex endeavor that requires proficiency in skills of critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, there is concern that US undergraduate science students do not currently develop these fundamental process skills they will need as professionals. In this study, we investigate how we can best ‘operationalize’ the teaching of process skills and how we can assess their development in undergraduate students. We are implementing a multi-year, multi-institutional research project to: (1) develop a set of instructional materials and assessment tools for critical thinking, oral communication, and data analysis; and (2) pilot these materials in a diversity of classroom settings under two instructional modalities: individual student reflection versus intensive classroom discussion of the skill. Results/Conclusions: Twenty-four conservation biologists have collaborated during the last year to create and validate instructional materials for process skills development, led by the Network of Conservation Educators and Practitioners (ncep.amnh.org). The instructional set for each skill consisted of pre/post student self-assessments, two exercises with rubrics for evaluation of student performance, and pre/post exercise content assessments. In fall 2011, nine professors piloted these materials in biology, ecology, and conservation biology courses. We present preliminary results from a subset of their students, from instructional units on data analysis using an intensive classroom discussion (DA; N=22) and critical thinking using individual student reflection (CT; N=20). For DA, we find significant increases in student self-confidence on data representation and interpretation (P0.05). For both skills, we find gains in content knowledge after the application of exercises (DA: g=0.22±0.082; CT: g=0.4±0.11). Observed gains in the skills vary depending on the aspect analyzed. For DA, students experienced significant gains in data representation and interpretation (V=17,

Developing and Assessing Process Skills in Conservation Biology and Other Integrative Fields

Goals and Intended Outcome: The goals are to: 1) create and validate a set of instructional materials designed to develop and assess process skills important in conservation biology and other integrative fields, and 2) pilot developed teaching and assessment materials in diverse classroom settings. We expect to increase our understanding of how to promote data analysis, critical thinking, and oral communication skills in students. Methods and Strategies: After creation and validation of instructional materials designed to develop and assess critical thinking, data analysis, and oral communication, we are piloting the materials in a diverse group of academic institutions across the US. Each faculty participant implements a set of instructional materials for a particular skill during two separate semesters of the same course. Evaluation Methods and Results: Our evaluation plan includes formative and summative evaluation activities and the application of diverse tools, both quantitative and qualitative. Project personnel are undertaking ongoing project evaluation with input from project advisers. An evaluation specialist will evaluate the project at mid-point and at its conclusion. Dissemination: We are working with 18 faculty members from diverse academic institutions across the US, including Puerto Rico and the Virgin Islands. In the future, we will make instructional units available to a broader audience of faculty members and conservation trainers through the NCEP website (ncep.amnh.org) Impact: Through this project, faculty members participated collaboratively in the development of teaching and assessment materials. By implementing these materials in their courses, faculty participants are not only learning about their students’ development of process skills, they are also learning about their own teaching practices. In terms of students, this project intends to increase students’ proficiency with process skills important in conservation biology. They will do this through a series of exercises and classroom discussion as well as through self-reflection about the targeted skill. Challenges: Participating faculty found our original experimental design of teaching and assessing more than one skill at a time to be logistically challenging. We adjusted the design to implement teaching and assessment materials for only one skill at a time. This modification did not change the overall goal of the project. Also, faculty members’ participation on the development and validation of the teaching materials was less that expected. Project personnel spent more time than planned working on these tasks. Expected Outcomes: In this comparative, multi-year study we will gather new evidence on teaching and assessment approaches that can help develop process skills in undergraduate students. We expect that classroom implementation of the developed teaching and assessment materials will lead to positive gains in students’ proficiency in critical thinking, data analysis and oral communication skills as well as increased discipline-specific content knowledge. We expect the magnitude of student skill gains will be greater in intensive vs. light teaching intervention treatments. Overall, we expect to increase our understanding of how to promote data analysis, critical thinking, and oral communication skills in undergraduate students. Data Impact: The instructional unit for each skill consists of pre/post student self-assessments, two exercises with rubrics for evaluation of student performance, and pre/post exercise content assessments. We use these tools to measure gains in students’ self-confidence on the skill, in the process skill, and in their knowledge of specific content. We use these tools in two classroom settings: one with an intensive classroom discussion and another with an individual student reflection on the skill. Comparisons between these settings will allow us to determine the effects of teaching intervention on students’ gains in the skill. We will also assess gains in faculty through pre/post questionnaires. Collection Methods: We are working with 18 faculty members who are implementing one of the instructional units in their courses. Six professors are using the instructional unit that targets critical thinking skills, seven the one for data analysis skills, and five for oral communication skills. Each faculty member will implement these units in two separate semesters, one with an intensive classroom discussion and another with an individual student reflection. Key Findings: To date we have results from students using instructional units on data analysis with an intensive classroom discussion (DA; N=104), oral communication with individual student reflection (OC; N=84) and critical thinking with both individual student reflection (CT; N=78) and intensive classroom discussion (N=42). For all three skills we find that students improve in skill performance. However, the degree of improvement varied among skills and skill dimensions, suggesting that some dimensions (e.g. drawing conclusions as part of critical thinking) may require interventions of different durations or intensities. In addition, we find changes in students’ confidence in the three skills, but these changes do not consistently mirror changes in skill. Students over- or underestimated their confidence for particular skills and skill dimensions. Finally, we find that students gained content knowledge while using exercises designed to teach topical content as well as develop skills, suggesting that investment in skill development in a class does not affect content learning. Currently, we are completing data collection from institutions that have used both modalities of teaching intervention – intensive classroom discussion and individual student reflection- to determine the effect of intervention level on students’ skill performance, confidence and content knowledge

Cultivating Skills for 21st Century Professionals: Development and Assessment of Process Skills in Ecology and Conservation Biology Students

Background/Question/Methods: Understanding and conserving the biosphere increasingly requires proficiency in skills including critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, a real concern exists that US undergraduate science students do not currently develop these important process skills needed as professionals. First, our work asks “how can we best ‘operationalize’ teaching of process skills?” Second, we must also discover how to evaluate development of specific process skills in our students. Such assessment allows us to determine if we have succeeded in teaching these skills with a gain or loss in content understanding. Here, we present the design and preliminary results of a recently launched experimental study aimed at these two tasks. The study brings together faculty from diverse institutions and professional conservation biologists to create and validate a set of instructional materials for process skills development. Following development, selected faculty will pilot teaching and assessment materials in diverse classroom settings. Project participants, led by investigators from the American Museum of Natural History’s Center for Biodiversity and Conservation, designed a multi-year research experiment in two stages: (1) development of instructional materials and associated assessment tools for three skills (i.e., critical thinking, oral communication, and data analysis), and (2) application of these materials in the classroom under two different instruction modalities, individual reflection versus intensive classroom discussion of the skill. These two modalities investigate student development of the targeted process skill and how intensity of a teaching intervention influences student success. Results/Conclusions: For each selected skill, faculty participants and consultants developed two exercises that focused on ecology and conservation biology topics, either new or based on the modules produced by the Network of Conservation Educators and Practitioners (NCEP; http://ncep.amnh.org). Participants also created the following assessment tools: student self-assessment questionnaires focusing on their confidence for each skill, content gains assessments, and rubrics for assessing skill performance. These will be applied pre and post application of the instructional materials. Our plans include validation and finalization of materials over summer 2011 and pilots in the classrooms occurring in at least 14 US institutions (including Puerto Rico) beginning in fall 2011. While the study focuses on these skills in the context of ecology and conservation biology, our approach and results should be of direct use for other related, integrative fields such as natural resource management, sustainable development studies, and public health. We welcome faculty input and interest in the project