Examining the effects of testwiseness in conceptual physics evaluations

Testwiseness is defined as the set of cognitive strategies used by a student that is intended to improve his or her score on a test regardless of the test’s subject matter. Questions with elements that may be affected by testwiseness are common in physics assessments, even in those which have been extensively validated and widely used as evaluation tools in physics education research. The potential effect of several elements of testwiseness were analyzed for questions in the Force Concept Inventory (FCI) and Conceptual Survey on Electricity and Magnetism that contain distractors that are predicted to be influenced by testwiseness. This analysis was performed using data sets collected between fall 2001 and spring 2014 at one midwestern U.S. university (including over 9500 students) and between Spring 2011 and Spring 2015 at a second eastern U.S. university (including over 2500 students). Student avoidance of “none of the above” or “zero” distractors was statistically significant. The effect of the position of a distractor on its likelihood to be selected was also significant. The effects of several potential positive and negative testwiseness effects on student scores were also examined by developing two modified versions of the FCI designed to include additional elements related to testwiseness; testwiseness produced little effect post-instruction in student performance on the modified instruments

Behavioral Self-Regulation in a Physics Class

This study examined the regulation of out-of-class time invested in the academic activities associated with a physics class for 20 consecutive semesters. The academic activities of 1676 students were included in the study. Students reported investing a semester average of 6.5 2.9 h out of class per week. During weeks not containing an examination, a total of 4.3 2.1 h was reported which was divided between 2.5. 1.2 h working homework and 1.8 1.4 h reading. Students reported spending 7.6. 4.8 h preparing for each in-semester examination. Students showed a significant correlation between the change in time invested in examination preparation (r ¼ −0.12, p \u3c 0.0001) and their score on the previous examination. The correlation increased as the data were averaged over semester (r ¼ −0.70, p ¼ 0.0006) and academic year (r ¼ −0.82, p ¼ 0.0039). While significant, the overall correlation indicates a small effect size and implies that an increase of 1 standard deviation of test score (18%) was related to a decrease of 0.12 standard deviations or 0.9 h of study time. Students also modified their time invested in reading as the length of the textbook changed; however, this modification was not proportional to the size of the change in textbook length. Very little regulation of the time invested in homework was detected either in response to test grades or in response to changes in the length of homework assignments. Patterns of regulation were different for higher performing students than for lower performing students with students receiving a course grade of “C” or “D” demonstrating little change in examination preparation time in response to lower examination grades. This study suggests that homework preparation time is a fixed variable while examination preparation time and reading time are weakly mutable variables

New Constraints on Quantum Gravity from X-ray and Gamma-Ray Observations

One aspect of the quantum nature of spacetime is its "foaminess" at very small scales. Many models for spacetime foam are defined by the accumulation power $\alpha$, which parameterizes the rate at which Planck-scale spatial uncertainties (and thephase shifts they produce) may accumulate over large path-lengths. Here $\alpha$ is defined by theexpression for the path-length fluctuations, $\delta \ell$, of a source at distance $\ell$, wherein $\delta \ell \simeq \ell^{1 - \alpha} \ell_P^{\alpha}$, with $\ell_P$ being the Planck length. We reassess previous proposals to use astronomical observations ofdistant quasars and AGN to test models of spacetime foam. We show explicitly how wavefront distortions on small scales cause the image intensity to decay to the point where distant objects become undetectable when the path-length fluctuations become comparable to the wavelength of the radiation. We use X-ray observations from {\em Chandra} to set the constraint $\alpha \gtrsim 0.58$, which rules out the random walk model (with $\alpha = 1/2$). Much firmer constraints canbe set utilizing detections of quasars at GeV energies with {\em Fermi}, and at TeV energies with ground-based Cherenkovtelescopes: $\alpha \gtrsim 0.67$ and $\alpha \gtrsim 0.72$, respectively. These limits on $\alpha$ seem to rule out $\alpha = 2/3$, the model of some physical interest.Comment: 11 pages, 9 figures, ApJ, in pres

Using Machine Learning to Predict Physics Course Outcomes

The use of machine learning and data mining techniques across many disciplines has exploded in recent years with the field of educational data mining growing significantly in the past 15 years. In this study, random forest and logistic regression models were used to construct early warning models of student success in introductory calculus-based mechanics (Physics 1) and electricity and magnetism (Physics 2) courses at a large eastern land-grant university. By combining in-class variables such as homework grades with institutional variables such as cumulative GPA, we can predict if a student will receive less than a “B” in the course with 73% accuracy in Physics 1 and 81% accuracy in Physics 2 with only data available in the first week of class using logistic regression models. The institutional variables were critical for high accuracy in the first four weeks of the semester. In-class variables became more important only after the first in-semester examination was administered. The student’s cumulative college GPA was consistently the most important institutional variable. Homework grade became the most important in-class variable after the first week and consistently increased in importance as the semester progressed; homework grade became more important than cumulative GPA after the first in-semester examination. Demographic variables including gender, race or ethnicity, and first generation status were not important variables for predicting course grade

Multi-Dimensional Item Response Theory and the Force Concept Inventory

Research on the test structure of the Force Concept Inventory (FCI) has largely been performed with exploratory methods such as factor analysis and cluster analysis. Multi-Dimensional Item Response Theory (MIRT) provides an alternative to traditional Exploratory Factor Analysis which allows statistical testing to identify the optimal number of factors. Application of MIRT to a sample of $N=4,716$ FCI post-tests identified a 9-factor solution as optimal. Additional analysis showed that a substantial part of the identified factor structure resulted from the practice of using problem blocks and from pairs of similar questions. Applying MIRT to a reduced set of FCI items removing blocked items and repeated items produced a 6-factor solution; however, the factors had little relation the general structure of Newtonian mechanics. A theoretical model of the FCI was constructed from expert solutions and fit to the FCI by constraining the MIRT parameter matrix to the theoretical model. Variations on the theoretical model were then explored to identify an optimal model. The optimal model supported the differentiation of Newton's 1st and 2nd law; of one-dimensional and three-dimensional kinematics; and of the principle of the addition of forces from Newton's 2nd law. The model suggested by the authors of the FCI was also fit; the optimal MIRT model was statistically superior

Algorithm Development for Column Water Vapor Retrieval Using the SAM Sensor

To understand and model the energetics of the Sun-Earth connection, measurements of specific atmospheric molecules are beneficial. Our objective is to formulate an algorithm to derive temporally varying atmospheric water vapor concentrations as a function of altitude, latitude, and longitude from solar irradiance absorption measurements. The Visidyne SAM instrument is used to study the size distribution of cloud particles. By introducing a spectrometer to the SAM instrument, column water vapor values are produced as an additional data product. A new model algorithm was developed and tested against existing algorithms. Through a least-squares analysis, the new algorithm showed an improvement of a factor of up to 23 over the industry standard. A test was also conducted to ascertain which water absorption bandpass produces the least error. Through these tests an improved model algorithm has been produced

Multiwavelength transit observations of the candidate disintegrating planetesimals orbiting WD 1145+017

We present multiwavelength, ground-based follow-up photometry of the white dwarf WD 1145+017, which has recently been suggested to be orbited by up to six or more short-period, low-mass, disintegrating planetesimals. We detect nine significant dips in flux of between 10% and 30% of the stellar flux in our ~32 hr of photometry, suggesting that WD 1145+017 is indeed being orbited by multiple, short-period objects. Through fits to the asymmetric transits that we observe, we confirm that the transit egress is usually longer than the ingress, and that the transit duration is longer than expected for a solid body at these short periods, all suggesting that these objects have cometary tails streaming behind them. The precise orbital periods of the planetesimals are unclear, but at least one object, and likely more, have orbital periods of ~4.5 hr. We are otherwise unable to confirm the specific periods that have been reported, bringing into question the long-term stability of these periods. Our high-precision photometry also displays low-amplitude variations, suggesting that dusty material is consistently passing in front of the white dwarf, either from discarded material from these disintegrating planetesimals or from the detected dusty debris disk. We compare the transit depths in the V- and R-bands of our multiwavelength photometry, and find no significant difference; therefore, for likely compositions, the radius of single-size particles in the cometary tails streaming behind the planetesimals must be ~0.15 μm or larger, or ~0.06 μm or smaller, with 2σ confidence