Developing the Next Generation of Physics Assessments

Science education at all levels is currently undergoing dramatic changes to its curricula and developing assessments for these new curricula is paramount. We have used the basis of many of these new changes (scientific practices, crosscutting concepts, and core ideas) to develop sets of criteria that can be used to guide assessment development for this new curriculum. We present a case study that uses the criteria we have developed to revise a traditional physics assessment item into an assessment item that is much more aligned with the goals of current transformation efforts. Assessment items developed using this criteria can be used to assess student learning of both the concepts and process of science.Comment: Revised version for PERC 2015 Conference Proceeding

Student and AI responses to physics problems examined through the lenses of sensemaking and mechanistic reasoning

Several reports in education have called for transforming physics learning environments by promoting sensemaking of real-world scenarios in light of curricular ideas. Recent advancements in Generative-Artificial Intelligence has garnered increasing traction in educators' community by virtue of its potential in transforming STEM learning. In this exploratory study, we adopt a mixed-methods approach in comparatively examining student- and AI-generated responses to two different formats of a physics problem through the cognitive lenses of sensemaking and mechanistic reasoning. The student data is derived from think-aloud interviews of introductory students and the AI data comes from ChatGPT's solutions collected using Zero shot approach. The results highlight AI responses to evidence most features of the two processes through well-structured solutions and student responses to effectively leverage representations in their solutions through iterative refinement of arguments. In other words, while AI responses reflect how physics is talked about, the student responses reflect how physics is practiced. Implications of these results in light of development and deployment of AI systems in physics pedagogy are discussed

The Effect of Value-Focused Discussions on Scientists' Ethical Decision Making

Many scientists view science as value-free, despite the fact that both epistemic and non-epistemic values structure scientific inquiry. Current ethics training usually focuses on transmitting knowledge about high-level ethical concepts or rules and is widely regarded as ineffective. We argue that ethics training will be more effective at improving ethical decision making if it focuses on connecting values to science. We pull from philosophy and psychology to define ethical decision making using the Four Component Model. This model states that in order to make an ethical decision someone must consider four components: moral sensitivity, moral reasoning, moral motivation, and moral implementation. We formed a moderated fellowship of fourteen science faculty from different disciplines who met for ten sessions over the course of a year, where they discussed the values embedded in different scientific norms. We then conducted interviews before and after the year-long fellowship that involved guided reflection of scenarios where there was some kind of ethical misconduct where the scientific practice required value judgements (e.g using unpublished data in their own work). We looked at how the fellowship affected the scientists' ability to recognize ethical dimensions regarding the scenarios. We found that this fellowship improved moral sensitivity, but their moral reasoning does not improve. We outlined our approach on how to look at scientists' ethical decision making and made recommendations on how to improve our approach. This work can inform future ethical training to align better with what scientists value and introduce useful concepts from philosophy and psychology to education research in physics

Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol

Citation: Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., . . . Cooper, M. M. (2016). Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol. Plos One, 11(9), 21. doi:10.1371/journal.pone.0162333Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of "three-dimensional learning" is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not