Psychometric Analysis of Forensic Examiner Behavior

Forensic science often involves the comparison of crime-scene evidence to a known-source sample to determine if the evidence and the reference sample came from the same source. Even as forensic analysis tools become increasingly objective and automated, final source identifications are often left to individual examiners' interpretation of the evidence. Each source identification relies on judgements about the features and quality of the crime-scene evidence that may vary from one examiner to the next. The current approach to characterizing uncertainty in examiners' decision-making has largely centered around the calculation of error rates aggregated across examiners and identification tasks, without taking into account these variations in behavior. We propose a new approach using IRT and IRT-like models to account for differences among examiners and additionally account for the varying difficulty among source identification tasks. In particular, we survey some recent advances (Luby, 2019a) in the application of Bayesian psychometric models, including simple Rasch models as well as more elaborate decision tree models, to fingerprint examiner behavior

Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the Instructional Quality Assessment (CSE Technical Report 681)

The quality of reading comprehension and mathematics instruction was explored in five urban middle schools using the Instructional Quality Assessment (IQA) toolkit (N = 34 teachers). The IQA is comprised of protocols for rating observed instruction and the quality of teachers' assignments with student work. The purpose of this research was to investigate the reliability and potential validity of the ratings of these data sources. Commensurate with other research on the quality of middle schools, our results indicated that the quality of instruction varied a great deal within schools and was of a "basic" quality overall. Results indicated a moderate to high level of reliability. Four assignments with student work yielded a stable estimate of quality in both content areas, and when teachers complied with the requirements of the research as few as two observations yielded a stable estimate of teaching quality in both content areas as well. The quality of teachers' observations and assignments were significantly associated in mathematics, but not in reading comprehension. Because of the small sample size it was not possible to apply multi-level models. The relation between the IQA and student achievement on the SAT-10 was explored using linear regression techniques. Results indicated that after controlling for students' prior achievement, socio-economic status (SES), ethnicity, language, and IEP status, the IQA assignment measure in reading comprehension predicted student achievement on the Total Reading, Reading Comprehension, and Vocabulary subscores of the SAT-10. The observation measure in reading comprehension predicted student outcomes on the Reading Comprehension subscore of the SAT-10 only. In mathematics, the quality of teachers' assignments predicted students' achievement on the Procedures subscore of the SAT-10. The quality of observed instruction in mathematics predicted students' achievement on the Procedures and Total Math subscores. Without accounting for clustering within classrooms and schools as multilevel models do, our linear regression analyses may lead to results that appear stronger than they actually are. Nevertheless our analyses indicate the direction of trend in these relationships and raise important questions regarding which data sources may be best (classroom assignments or observations) for measuring specific aspects of instruction and student outcomes. Additional research with larger samples of teachers is needed to make definitive conclusions about the validity of the IQA ratings and under what conditions one might choose to either observe in classrooms or collect assignments with student work. The following are appended: (1) IQA Rubrics; and (2) High and Low Quality Examples of Observed Lessons and Assignments. (Contains 21 tables and 9 footnotes.

Numerical Analysis of Time Required for De-stratification in Warehouses

A significant source of energy consumption comes from maintaining desired indoor environment conditions in warehouses and other industrial facilities. To combat the raising energy costs, studies into more efficient heating and cooling strategies has been a topic of consideration for a number of years. One of the areas of investigation is the implications of a thermally stratified environment. In heating, removing the stratification phenomena has been linked to savings in the cost of fuel to heat an environment. Whereas in cooling a highly stratified environment is desired. The primary method of de-stratification is the utilization of ceiling fans. The use of fans reduces the overall savings of de-stratification for heating purposes. A solution to offset the reliance on grid power is the use of solar powered ceiling fans. The challenge with utilizing solar power during heating seasons is a reduction in the time the sun is available to charge and store energy to run the fans. While there are studies on the impact of thermal stratification, with air as a medium in an indoor environment, there is a lack of information on the frequency at which the ceiling fans need to operate to maintain a de-stratified environment. The determination of a fan operation frequency, to maintain a de-stratified environment, informs potential designers on the viability of installing solar powered fans as an alternative to grid powered fans. In the event that solar powered fans were not a viable option, it also provides information on the frequency that a grid powered fan would need to run to maintain de-stratification. To determine a fan operating frequency, a numerical analysis will be performed. This numerical analysis will assess the time required to de-stratify an environment based on inputs such as flow rate and spatial considerations. In order to establish the quality of the numerical analysis, two experiments have been conducted to observe the impact of de-stratification. One experiment is located at a large retail warehouse distribution center, the other is a small classroom. The data collected from these experiments will be compared to the models developed to predict the change in stratification with time

Using the Instructional Quality Assessment toolkit to investigate the quality of reading comprehension assignments and student work (CSE Report 669)

This study presents preliminary findings from research developing an instructional quality assessment (IQA) toolkit that could be used to monitor the influence of reform initiatives on students' learning environments and to guide professional development efforts within a school or district. This report focuses specifically on the portion of the IQA used to evaluate the quality of teachers' reading comprehension assignments and student work. Results are limited due to a very small sample of participating teachers (N = 13, 52 assignments), and indicate a poor to moderate level of inter-rater agreement and a good degree of consistency for the dimensions measuring academic rigor, but not the clarity of teachers' expectations. The rigor of the assignments collected from teachers also was associated with the rigor of observed instruction. Collecting four assignments (two challenging and two recent) from teachers did not yield a stable estimate of quality. Additional analyses looking separately at the two different assignment types indicate, however, that focusing on one assignment type would yield a stable estimate of quality. This suggests that the way in which assignments are collected from teachers should be revised. Implications for professional development are also discussed. The 2003 Draft Observation and Assignment Rubrics for Reading Comprehension is appended. (Contains 6 tables, 4 figures, and 4 footnotes.