Interpreting Force Concept Inventory Scores: NormalizedGain and SAT Scores

Preinstruction SAT scores and normalized gains (G) on the force concept inventory (FCI) were examined for individual students in interactive engagement (IE) courses in introductory mechanics at one high school (N=335) and one university (N=292), and strong, positive correlations were found for both populations (r=0.57 and r=0.46, respectively). These correlations are likely due to the importance of cognitive skills and abstract reasoning in learning physics. The larger correlation coefficient for the high school population may be a result of the much shorter time interval between taking the SAT and studying mechanics, because the SAT may provide a more current measure of abilities when high school students begin the study of mechanics than it does for college students, who begin mechanics years after the test is taken. In prior research a strong correlation between FCI G and scores on Lawson\u27s Classroom Test of Scientific Reasoning for students from the same two schools was observed. Our results suggest that, when interpreting class average normalized FCI gains and comparing different classes, it is important to take into account the variation of students\u27 cognitive skills, as measured either by the SAT or by Lawson\u27s test. While Lawson\u27s test is not commonly given to students in most introductory mechanics courses, SAT scores provide a readily available alternative means of taking account of students\u27 reasoning abilities. Knowing the students\u27 cognitive level before instruction also allows one to alter instruction or to use an intervention designed to improve students\u27 cognitive level

Diagnostic tests for the physical sciences: A brief review

We present a review of diagnostic testing in the physical sciences. We cover the motivation for using such instruments and their historical development via a case study of probably the most cited and influential test instrument and application: the Force Concept Inventory, developed in the early 1990s by Hestenes and co-workers, and its use to quantify learning gains from different instructional methodologies by Richard Hake. We then present an overview of the process of creation and validation of such instruments, and highlight the results from studies that have made use of some of the many instruments available in the literature. We conclude with a short summary of our own recent work to develop a diagnostic test of data handling skills of physical science undergraduates

Introduction of interactive learning into French university physics classrooms

We report on a project to introduce interactive learning strategies (ILS) to physics classes at the Universit\'e Pierre et Marie Curie (UPMC), one of the leading science universities in France. In Spring 2012, instructors in two large introductory classes, first-year, second-semester mechanics, and second-year introductory E&M, enrolling approximately 500 and 250 students respectively, introduced ILS into some sections of each class. The specific ILS utilized were Think-Pair-Share questions and Peer Instruction in the main lecture classrooms, and UW Tutorials for Introductory Physics in recitation sections. Pre- and post-instruction assessments (FCI and CSEM respectively) were given, along with a series of demographics questions. We were able to compare the results of the FCI and CSEM between interactive and non-interactive classes taught simultaneously with the same curriculum. We also analyzed final exam results, as well as the results of student and instructor attitude surveys between classes. In our analysis, we argue that Multiple Linear Regression modeling is superior to other common analysis tools, including normalized gain. Our results show that ILS are effective at improving student learning by all measures used: research-validated concept inventories and final exam scores, on both conceptual and traditional problem-solving questions. Multiple Linear Regression analysis reveals that interactivity in the classroom is a significant predictor of student learning, showing a similar or stronger relationship with student learning than such ascribed characteristics as parents' education, and achieved characteristics such as GPA and hours studied per week. Analysis of student and instructors attitudes shows that both groups believe that ILS improve student learning in the physics classroom, and increases student engagement and motivation

Ausubel’s principle of prior knowledge in first year mechanics

The Force Concept Inventory, a 30-question multiple choice test, has been used to test the baseline knowledge in mechanics prior to a course of instruction at Hull over the three years corresponding to entry in 2008, 2009 and 2010. Students whose pre-university education occurred outside the UK or who were repeating the year have been excluded from the analysis in order to focus attention on first-time UK students. These constitute the great majority of the entrants and the results essentially characterise the entry-level knowledge of a typical cohort. Two interesting findings have emerged. First, there is a wide range of abilities within each cohort, as judged by the test scores, and secondly, analysis of the scores question by question reveals a remarkable consistency between the different cohorts. This consistency extends even to the distribution of choices within individual questions. Five such questions are analysed in detailed to reveal which aspects of mechanics a typical class finds difficult. Ausubel‟s principle of first finding out what students know in order to teach accordingly can therefore be applied not to the individual students but to the class as a whole and suggestions as to how instruction might be tailored to address the weaknesses revealed by the Force Concept Inventory are discussed

Secondary implementation of interactive engagement teaching techniques: Choices and challenges in a Gulf Arab context

We report on a "Collaborative Workshop Physics" instructional strategy to deliver the first IE calculus-based physics course at Khalifa University, UAE. To these authors' knowledge, this is the first such course on the Arabian Peninsula using PER-based instruction. A brief history of general university and STEM teaching in the UAE is given. We present this secondary implementation (SI) as a case study of a novel context and use it to determine if PER-based instruction can be successfully implemented far from the cultural context of the primary developer and, if so, how might such SIs differ from SIs within the US. With these questions in view, a pre-reform baseline of MPEX, FCI, course exam and English language proficiency data are used to design a hybrid implementation of Cooperative Group Problem Solving. We find that for students with high English proficiency, normalized gain on FCI improves from = 0.16+/-0.10 pre- to = 0.47+/-0.08 post-reform, indicating successful SI. We also find that is strongly modulated by language proficiency and discuss likely causes. Regardless of language skill, problem-solving skill is also improved and course DFW rates drop from 50% to 24%. In particular, we find evidence in post-reform student interviews that prior classroom experiences, and not broader cultural expectations about education, are the more significant cause of expectations at odds with the classroom norms of well-functioning PER-based instruction. This result is evidence that PER-based innovations can be implemented across great changes in cultural context, provided that the method is thoughtfully adapted in anticipation of context and culture-specific student expectations. This case study should be valuable for future reforms at other institutions, both in the Gulf Region and developing world, facing similar challenges involving SI of PER-based instruction outside the US.Comment: v1: 28 pages, 9 figures. v2: 19 pages, 6 figures, includes major reorganization and revisions based on anonymous peer review. v3: 19 pages, 6 figures, minor revisions based on anonymous peer revie

Rasch model based analysis of the Force Concept Inventory

The Force Concept Inventory (FCI) is an important diagnostic instrument which is widely used in the field of physics education research. It is therefore very important to evaluate and monitor its functioning using different tools for statistical analysis. One of such tools is the stochastic Rasch model, which enables construction of linear measures for persons and items from raw test scores and which can provide important insight in the structure and functioning of the test (how item difficulties are distributed within the test, how well the items fit the model, and how well the items work together to define the underlying construct). The data for the Rasch analysis come from the large-scale research conducted in 2006-07, which investigated Croatian high school students’ conceptual understanding of mechanics on a representative sample of 1676 students (age 17–18 years). The instrument used in research was the FCI. The average FCI score for the whole sample was found to be (27.7±0.4)%, indicating that most of the students were still non-Newtonians at the end of high school, despite the fact that physics is a compulsory subject in Croatian schools. The large set of obtained data was analyzed with the Rasch measurement computer software WINSTEPS 3.66. Since the FCI is routinely used as pretest and post-test on two very different types of population (non-Newtonian and predominantly Newtonian), an additional predominantly Newtonian sample (N=141, average FCI score of 64.5%) of first year students enrolled in introductory physics course at University of Zagreb was also analyzed. The Rasch model based analysis suggests that the FCI has succeeded in defining a sufficiently unidimensional construct for each population. The analysis of fit of data to the model found no grossly misfitting items which would degrade measurement. Some items with larger misfit and items with significantly different difficulties in the two samples of students do require further examination. The analysis revealed some problems with item distribution in the FCI and suggested that the FCI may function differently in non-Newtonian and predominantly Newtonian population. Some possible improvements of the test are suggested

Student Misconceptions about Cybersecurity Concepts: Analysis of Think-Aloud Interviews

We conducted an observational study to document student misconceptions about cybersecurity using thematic analysis of 25 think-aloud interviews. By understanding patterns in student misconceptions, we provide a basis for developing rigorous evidence-based recommendations for improving teaching and assessment methods in cybersecurity and inform future research. This study is the first to explore student cognition and reasoning about cybersecurity. We interviewed students from three diverse institutions. During these interviews, students grappled with security scenarios designed to probe their understanding of cybersecurity, especially adversarial thinking. We analyzed student statements using a structured qualitative method, novice-led paired thematic analysis, to document patterns in student misconceptions and problematic reasoning that transcend institutions, scenarios, or demographics. Themes generated from this analysis describe a taxonomy of misconceptions but not their causes or remedies. Four themes emerged: overgeneralizations, conflated concepts, biases, and incorrect assumptions. Together, these themes reveal that students generally failed to grasp the complexity and subtlety of possible vulnerabilities, threats, risks, and mitigations, suggesting a need for instructional methods that engage students in reasoning about complex scenarios with an adversarial mindset. These findings can guide teachers’ attention during instruction and inform the development of cybersecurity assessment tools that enable cross-institutional assessments that measure the effectiveness of pedagogies