Beyond linear regression: A reference for analyzing common data types in discipline based education research

[This paper is part of the Focused Collection on Quantitative Methods in PER: A Critical Examination.] A common goal in discipline-based education research (DBER) is to determine how to improve student outcomes. Linear regression is a common technique used to test hypotheses about the effects of interventions on continuous outcomes (such as exam score) as well as control for student nonequivalence in quasirandom experimental designs. (In quasirandom designs, subjects are not randomly assigned to treatments. For example, when treatment is assigned by classroom, and observations are made on students, the design is quasirandom because treatment is assigned to classroom, not subject (students).) However, many types of outcome data cannot be appropriately analyzed with linear regression. In these instances, researchers must move beyond linear regression and implement alternative regression techniques. For example, student outcomes can be measured on binary scales (e.g., pass or fail), tightly bound scales (e.g., strongly agree to strongly disagree), or nominal scales (i.e., different discrete choices for example multiple tracks within a physics major), each necessitating alternative regression techniques. Here, we review extensions of linear modeling—generalized linear models (glms)—and specifically compare five glms that are useful for analyzing DBER data: logistic, binomial, proportional odds (also called ordinal; including censored regression), multinomial, and Poisson (including negative binomial, hurdle, and zero-inflated) regression. We introduce a diagnostic tool to facilitate a researcher’s identification of the most appropriate glm for their own data. For each model type, we explain when, why, and how to implement the regression approach. When: we provide examples of the types of research questions and outcome data that would motivate this regression approach, including citations to articles in the DBER literature. Why: we name which linear regression assumption is violated by the data type. How: we detail implementation and interpretation of this modeling approach in R, including R syntax and code, and how to discuss the regression output in research papers. Code accompanying each analysis can be found in the online github repository that is associated with this paper (https://github.com/ejtheobald/BeyondLinearRegression). This paper is not an exhaustive review of regression techniques, nor does it review nonregression-based analyses. Rather, it aims to compile and summarize regression techniques useful for the most common types of DBER data and provide examples, citations, and heavily annotated R code so that researchers can easily implement the technique in their work

ASPECT: A Survey to Assess Student Perspective of Engagement in an Active-Learning Classroom

This paper describes the development and validation of a survey to measure students? self-reported engagement during a wide variety of in-class active-learning exercises. The survey provides researchers and instructors alike with a tool to rapidly evaluate different active-learning strategies from the perspective of the learner

A CURE on the Evolution of Antibiotic Resistance in <i>Escherichia coli</i> Improves Student Conceptual Understanding

We developed labs on the evolution of antibiotic resistance to assess the costs and benefits of replacing traditional laboratory exercises in an introductory biology course for majors with a course-based undergraduate research experience (CURE). To assess whether participating in the CURE imposed a cost in terms of exam performance, we implemented a quasi-experiment in which four lab sections in the same term of the same course did the CURE labs, while all other students did traditional labs. To assess whether participating in the CURE impacted other aspects of student learning, we implemented a second quasi-experiment in which all students either did traditional labs over a two-quarter sequence or did CURE labs over a two-quarter sequence. Data from the first experiment showed minimal impact on CURE students' exam scores, while data from the second experiment showed that CURE students demonstrated a better understanding of the culture of scientific research and a more expert-like understanding of evolution by natural selection. We did not find disproportionate costs or benefits for CURE students from groups that are minoritized in science, technology, engineering, and mathematics

Code and data suppporting "Theobald, Breckheimer, and HilleRisLambers. 2017. Climate drives phenological reassembly of a mountain wildflower meadow community. In Press. Ecology

Identical to v1.0, but a new release is required for Zenodo archiving and DOI assignment

Student perception of group dynamics predicts individual performance: Comfort and equity matter.

Active learning in college classes and participation in the workforce frequently hinge on small group work. However, group dynamics vary, ranging from equitable collaboration to dysfunctional groups dominated by one individual. To explore how group dynamics impact student learning, we asked students in a large-enrollment university biology class to self-report their experience during in-class group work. Specifically, we asked students whether there was a friend in their group, whether they were comfortable in their group, and whether someone dominated their group. Surveys were administered after students participated in two different types of intentionally constructed group activities: 1) a loosely-structured activity wherein students worked together for an entire class period (termed the 'single-group' activity), or 2) a highly-structured 'jigsaw' activity wherein students first independently mastered different subtopics, then formed new groups to peer-teach their respective subtopics. We measured content mastery by the change in score on identical pre-/post-tests. We then investigated whether activity type or student demographics predicted the likelihood of reporting working with a dominator, being comfortable in their group, or working with a friend. We found that students who more strongly agreed that they worked with a dominator were 17.8% less likely to answer an additional question correct on the 8-question post-test. Similarly, when students were comfortable in their group, content mastery increased by 27.5%. Working with a friend was the single biggest predictor of student comfort, although working with a friend did not impact performance. Finally, we found that students were 67% less likely to agree that someone dominated their group during the jigsaw activities than during the single group activities. We conclude that group activities that rely on positive interdependence, and include turn-taking and have explicit prompts for students to explain their reasoning, such as our jigsaw, can help reduce the negative impact of inequitable groups

Detecting Montane Flowering Phenology with CubeSat Imagery

Shifts in wildflower phenology in response to climate change are well documented in the scientific literature. The majority of studies have revealed phenological shifts using in-situ observations, some aided by citizen science efforts (e.g., National Phenology Network). Such investigations have been instrumental in quantifying phenological shifts but are challenged by the fact that limited resources often make it difficult to gather observations over large spatial scales and long-time frames. However, recent advances in finer scale satellite imagery may provide new opportunities to detect changes in phenology. These approaches have documented plot level changes in vegetation characteristics and leafing phenology, but it remains unclear whether they can also detect flowering in natural environments. Here, we test whether fine-resolution imagery (<10 m) can detect flowering and whether combining multiple sources of imagery improves the detection process. Examining alpine wildflowers at Mt. Rainier National Park (MORA), we found that high-resolution Random Forest (RF) classification from 3-m resolution PlanetScope (from Planet Labs) imagery was able to delineate the flowering season captured by ground-based phenological surveys with an accuracy of 70% (Cohen's kappa = 0.25). We then combined PlanetScope data with coarser resolution but higher quality imagery from Sentinel and Landsat satellites (10-m Sentinel and 30-m Landsat), resulting in higher accuracy predictions (accuracy = 77%, Cohen's kappa = 0.39). The model was also able to identify the timing of peak flowering in a particularly warm year (2015), despite being calibrated on normal climate years. Our results suggest PlanetScope imagery holds utility in global change ecology where temporal frequency is important. Additionally, we suggest that combining imagery may provide a new approach to cross-calibrate sensors to account for radiometric irregularity inherent in fine resolution PlanetScope imagery. The development of this approach for wildflower phenology predictions provides new possibilities to monitor climate change effects on flowering communities at broader spatiotemporal scales. In protected and tourist areas where the flowering season draws large numbers of visitors, such as Mt. Rainier National Park, the modeling framework presented here could be a useful tool to manage and prioritize park resources. © 2020 by the authors.ISSN:2072-429

Student perception of group dynamics predicts individual performance: Comfort and equity matter

<div><p>Active learning in college classes and participation in the workforce frequently hinge on small group work. However, group dynamics vary, ranging from equitable collaboration to dysfunctional groups dominated by one individual. To explore how group dynamics impact student learning, we asked students in a large-enrollment university biology class to self-report their experience during in-class group work. Specifically, we asked students whether there was a friend in their group, whether they were comfortable in their group, and whether someone dominated their group. Surveys were administered after students participated in two different types of intentionally constructed group activities: 1) a loosely-structured activity wherein students worked together for an entire class period (termed the ‘single-group’ activity), or 2) a highly-structured ‘jigsaw’ activity wherein students first independently mastered different subtopics, then formed new groups to peer-teach their respective subtopics. We measured content mastery by the change in score on identical pre-/post-tests. We then investigated whether activity type or student demographics predicted the likelihood of reporting working with a dominator, being comfortable in their group, or working with a friend. We found that students who more strongly agreed that they worked with a dominator were 17.8% less likely to answer an additional question correct on the 8-question post-test. Similarly, when students were comfortable in their group, content mastery increased by 27.5%. Working with a friend was the single biggest predictor of student comfort, although working with a friend did not impact performance. Finally, we found that students were 67% less likely to agree that someone dominated their group during the jigsaw activities than during the single group activities. We conclude that group activities that rely on positive interdependence, and include turn-taking and have explicit prompts for students to explain their reasoning, such as our jigsaw, can help reduce the negative impact of inequitable groups.</p></div

Final models and associated estimates for predicting students’ performance on the post-test and which students report a dominator, being comfortable, and working with a friend.

<p>Coefficients are presented as odds (transformed from logodds); models were fit as cumulative link mixed models (postscore, dominator, comfort) or logistic regression (friend; see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0181336#sec002" target="_blank">methods</a> for details). Bolded coefficients represent statistical significance to α = 0.05. Grey cells indicate variables that were not included in the full model, empty cells indicate variables that were included in the full model and then were dropped during the model selection process. Superscripted notes indicate starting models and additional notes.</p

Raw means show that a carefully designed jigsaw in-class activity reduced the dominator report rate compared to a single-group activity.

<p>Error bars, which are present but subsumed by the point, indicate standard error. See <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0181336#pone.0181336.t002" target="_blank">Table 2</a> for final models and modeled estimates.</p