Appendix B. A table presenting observed visitors to Collinsia verna flowers in southwestern Pennsylvania, USA.

A table presenting observed visitors to Collinsia verna flowers in southwestern Pennsylvania, USA

Appendix A. A table presenting characteristics of the pollination environment for Collinsia verna at three populations in Pennsylvania, USA, 1997 and 1999.

A table presenting characteristics of the pollination environment for Collinsia verna at three populations in Pennsylvania, USA, 1997 and 1999

How Much Can Students Gain in Data Analysis and Critical Thinking Skills in One Semester?

Background/Question/Methods: The effective preservation and sustainable use of ecosystems is a complex endeavor that requires proficiency in skills of critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, there is concern that US undergraduate science students do not currently develop these fundamental process skills they will need as professionals. In this study, we investigate how we can best ‘operationalize’ the teaching of process skills and how we can assess their development in undergraduate students. We are implementing a multi-year, multi-institutional research project to: (1) develop a set of instructional materials and assessment tools for critical thinking, oral communication, and data analysis; and (2) pilot these materials in a diversity of classroom settings under two instructional modalities: individual student reflection versus intensive classroom discussion of the skill. Results/Conclusions: Twenty-four conservation biologists have collaborated during the last year to create and validate instructional materials for process skills development, led by the Network of Conservation Educators and Practitioners (ncep.amnh.org). The instructional set for each skill consisted of pre/post student self-assessments, two exercises with rubrics for evaluation of student performance, and pre/post exercise content assessments. In fall 2011, nine professors piloted these materials in biology, ecology, and conservation biology courses. We present preliminary results from a subset of their students, from instructional units on data analysis using an intensive classroom discussion (DA; N=22) and critical thinking using individual student reflection (CT; N=20). For DA, we find significant increases in student self-confidence on data representation and interpretation (P0.05). For both skills, we find gains in content knowledge after the application of exercises (DA: g=0.22±0.082; CT: g=0.4±0.11). Observed gains in the skills vary depending on the aspect analyzed. For DA, students experienced significant gains in data representation and interpretation (V=17,

Developing and Assessing Process Skills in Conservation Biology and Other Integrative Fields

Goals and Intended Outcome: The goals are to: 1) create and validate a set of instructional materials designed to develop and assess process skills important in conservation biology and other integrative fields, and 2) pilot developed teaching and assessment materials in diverse classroom settings. We expect to increase our understanding of how to promote data analysis, critical thinking, and oral communication skills in students. Methods and Strategies: After creation and validation of instructional materials designed to develop and assess critical thinking, data analysis, and oral communication, we are piloting the materials in a diverse group of academic institutions across the US. Each faculty participant implements a set of instructional materials for a particular skill during two separate semesters of the same course. Evaluation Methods and Results: Our evaluation plan includes formative and summative evaluation activities and the application of diverse tools, both quantitative and qualitative. Project personnel are undertaking ongoing project evaluation with input from project advisers. An evaluation specialist will evaluate the project at mid-point and at its conclusion. Dissemination: We are working with 18 faculty members from diverse academic institutions across the US, including Puerto Rico and the Virgin Islands. In the future, we will make instructional units available to a broader audience of faculty members and conservation trainers through the NCEP website (ncep.amnh.org) Impact: Through this project, faculty members participated collaboratively in the development of teaching and assessment materials. By implementing these materials in their courses, faculty participants are not only learning about their students’ development of process skills, they are also learning about their own teaching practices. In terms of students, this project intends to increase students’ proficiency with process skills important in conservation biology. They will do this through a series of exercises and classroom discussion as well as through self-reflection about the targeted skill. Challenges: Participating faculty found our original experimental design of teaching and assessing more than one skill at a time to be logistically challenging. We adjusted the design to implement teaching and assessment materials for only one skill at a time. This modification did not change the overall goal of the project. Also, faculty members’ participation on the development and validation of the teaching materials was less that expected. Project personnel spent more time than planned working on these tasks. Expected Outcomes: In this comparative, multi-year study we will gather new evidence on teaching and assessment approaches that can help develop process skills in undergraduate students. We expect that classroom implementation of the developed teaching and assessment materials will lead to positive gains in students’ proficiency in critical thinking, data analysis and oral communication skills as well as increased discipline-specific content knowledge. We expect the magnitude of student skill gains will be greater in intensive vs. light teaching intervention treatments. Overall, we expect to increase our understanding of how to promote data analysis, critical thinking, and oral communication skills in undergraduate students. Data Impact: The instructional unit for each skill consists of pre/post student self-assessments, two exercises with rubrics for evaluation of student performance, and pre/post exercise content assessments. We use these tools to measure gains in students’ self-confidence on the skill, in the process skill, and in their knowledge of specific content. We use these tools in two classroom settings: one with an intensive classroom discussion and another with an individual student reflection on the skill. Comparisons between these settings will allow us to determine the effects of teaching intervention on students’ gains in the skill. We will also assess gains in faculty through pre/post questionnaires. Collection Methods: We are working with 18 faculty members who are implementing one of the instructional units in their courses. Six professors are using the instructional unit that targets critical thinking skills, seven the one for data analysis skills, and five for oral communication skills. Each faculty member will implement these units in two separate semesters, one with an intensive classroom discussion and another with an individual student reflection. Key Findings: To date we have results from students using instructional units on data analysis with an intensive classroom discussion (DA; N=104), oral communication with individual student reflection (OC; N=84) and critical thinking with both individual student reflection (CT; N=78) and intensive classroom discussion (N=42). For all three skills we find that students improve in skill performance. However, the degree of improvement varied among skills and skill dimensions, suggesting that some dimensions (e.g. drawing conclusions as part of critical thinking) may require interventions of different durations or intensities. In addition, we find changes in students’ confidence in the three skills, but these changes do not consistently mirror changes in skill. Students over- or underestimated their confidence for particular skills and skill dimensions. Finally, we find that students gained content knowledge while using exercises designed to teach topical content as well as develop skills, suggesting that investment in skill development in a class does not affect content learning. Currently, we are completing data collection from institutions that have used both modalities of teaching intervention – intensive classroom discussion and individual student reflection- to determine the effect of intervention level on students’ skill performance, confidence and content knowledge

Cultivating Skills for 21st Century Professionals: Development and Assessment of Process Skills in Ecology and Conservation Biology Students

Background/Question/Methods: Understanding and conserving the biosphere increasingly requires proficiency in skills including critical thinking, data analysis, oral communication, broad synthesis of information and teamwork across diverse groups. However, a real concern exists that US undergraduate science students do not currently develop these important process skills needed as professionals. First, our work asks “how can we best ‘operationalize’ teaching of process skills?” Second, we must also discover how to evaluate development of specific process skills in our students. Such assessment allows us to determine if we have succeeded in teaching these skills with a gain or loss in content understanding. Here, we present the design and preliminary results of a recently launched experimental study aimed at these two tasks. The study brings together faculty from diverse institutions and professional conservation biologists to create and validate a set of instructional materials for process skills development. Following development, selected faculty will pilot teaching and assessment materials in diverse classroom settings. Project participants, led by investigators from the American Museum of Natural History’s Center for Biodiversity and Conservation, designed a multi-year research experiment in two stages: (1) development of instructional materials and associated assessment tools for three skills (i.e., critical thinking, oral communication, and data analysis), and (2) application of these materials in the classroom under two different instruction modalities, individual reflection versus intensive classroom discussion of the skill. These two modalities investigate student development of the targeted process skill and how intensity of a teaching intervention influences student success. Results/Conclusions: For each selected skill, faculty participants and consultants developed two exercises that focused on ecology and conservation biology topics, either new or based on the modules produced by the Network of Conservation Educators and Practitioners (NCEP; http://ncep.amnh.org). Participants also created the following assessment tools: student self-assessment questionnaires focusing on their confidence for each skill, content gains assessments, and rubrics for assessing skill performance. These will be applied pre and post application of the instructional materials. Our plans include validation and finalization of materials over summer 2011 and pilots in the classrooms occurring in at least 14 US institutions (including Puerto Rico) beginning in fall 2011. While the study focuses on these skills in the context of ecology and conservation biology, our approach and results should be of direct use for other related, integrative fields such as natural resource management, sustainable development studies, and public health. We welcome faculty input and interest in the project