Evaluating A Math Instructors Clarity: An Analysis Of Low-To-Moderate Items Versus A High Inference Item

Instructors and students identify communication skills as being essential to successful classroom teaching and student learning. This study investigates the stability of student evaluations with respect to items related to instructor’s clarity. The majority of student evaluations completed at the end of a semester at the college level contain a question directly related to clarity. A pilot study was conducted which utilized four finite mathematics courses being taught by the same instructor. After assessing reliability and validity, a follow-up study was conducted utilizing four sections of introductory statistics employing the same design as the pilot. Ninety-four students completed some component of the researcher-constructed evaluation resulting in an overall 96% response rate. Seventy-one students completed the entire evaluation, allowing for a comparison between the high inference and the low-to-moderate inference items used to measure instructor’s clarity, resulting in a useable response rate of 72%. After a statistical transformation of the high inference item, descriptive statistics were produced for each type of item. Through a t-test, a statistically significant difference was found between the high and low-to-moderate clarity items. The difference between the types of clarity ratings was then analyzed via ANOVA techniques to explore differences by gender and expected course grade. Males tended to rate the instructor inconsistently, assigning lower ratings for the low-to-moderate inference items, while other ratings appear to demonstrate consistency

Exploring The Reliability, Validity, And Utility Of A Higher Education Faculty Review Process

Institutions of learning face significant calls for accountability, leading to an increased interest in measuring faculty performance. The purpose of this research is to describe and analyze the process of faculty evaluation in a college of education at a southeastern university. The process is outlined with attention given to the reliability, validity, and utility through documenting strengths and weaknesses. Included is a detailed review of the instruments utilized in the process.  Suggestions for improvements and modifications are central to the outcomes, including plans for aligning the process with relevant research.  The degree to which the faculty review process is deemed credible, professional, and relevant may serve as an indicator of the link between theory and practice.  Potential ramifications of a weak or bias system, as faculty promotion and tenure are intertwined with this process, are of utmost importance. Legal issues, including challenges, rise to the forefront.  This topic is of importance to students, faculty, and administrators, as all involved and impacted by the review process, either directly or indirectly.  The applicability of this work beyond the example institution is also addressed

Challenging the Validity of Higher Education Course Evaluations

In higher education, course evaluations are given much attention, with results directly impacting such events as merit review and tenure/promotion. The accurate presentation and proper use of the evaluation results is a critical issue. The typical course evaluation process involves distributing a Likert-type survey to a class, compiling the data and reporting means/standard deviations (classical test theory approach, CTT). One alternative analytical technique is the Rasch model. A theoretical review of each model and an empirical example utilizing end of semester course evaluations from an introductory statistics course taught at a Midwest community college is presented to demonstrate the step-by-step process of feedback via each model. A contention is made that the CTT summary is not producing a valid picture of the evaluation data. The survey research community and institutions analyzing similar rating scale data will benefit from the results of this study as it provides a sound methodology for analyzing such data. The education community will also benefit by receiving better-informed results

Determining the Measurement Quality of a Montessori High School Teacher Evaluation Survey

The purpose of this study was to conduct a psychometric validation of a course evaluation instrument, known as a student evaluation of teaching (SET), implemented in a Montessori high school. The authors demonstrate to the Montessori community how to rigorously examine the measurement and assessment quality of instruments used within Montessori schools. The Montessori high school community needs an SET that has been rigorously examined for measurement issues. The examined SET was developed by a Montessori high school, and the sample data were collected from Montessori high school students. Using a Rasch partial credit model, the results of the analysis identified several measurement issues, including multidimensionality, misfit items, and inappropriate item difficulty levels. A revised version of the SET underwent the same analysis procedure, and the results indicated that measurement issues persisted. The authors suggest several ways to improve the overall measurement quality of the instrument while keeping the Montessori foundation. Additional validation studies with a revised version of the SET will be needed before the instrument can be endorsed for full implementation in a Montessori setting

Differential Item Functioning among English Language Learners on a Large-Scale Mathematics Assessment

The current study research showed the nature and potential sources of the gaps in mathematics achievement between English language learners (ELLs) and non-English language learners (non-ELLs). The nature of achievement gap was examined using three DIF methodologies: including Mantel-Haenszel procedure, Rasch model, and Hierarchical Generalized Linear Model (HGLM). These were conducted at the item level in contrast to total test level. Results revealed that the three DIF approaches identified 10 common items. These 10 items demonstrated in favor of non-ELLs. Findings from this study will help educational researchers, administrators, and policymakers understand the nature of the achievement gap in mathematics at item level so that United States can be competitive in middle school mathematics education. This study also suggested that item writers and test developers should construct assessments where language is equally accessible for ELL students

An Investigation Of 'Honesty Check' Items In Higher Education Course Evaluations

The reliability and validity of course evaluations in higher education is often assumed. The typical Likert-type surveys utilized when students' evaluate the course and instructor often overlook measurement issues, or deal with them in an ineffective manner. Given the importance that is placed on higher education course evaluations, with results impacting such events as merit raises and promotion, the proper construction and use of evaluation tools is a critical issue. In an effort to assure 'honesty' in student responses, many institutions include items written positively and negatively, which are intended to measure the same construct. Using 537 course evaluations for a mathematics faculty member at a Midwest college, an item analysis is conducted with attention given to means and standard deviations, frequency counts, nonparametric correlations and tests of significant differences between questions that should, in theory, produce a similar measure or exactly opposite. A contention is made that the way the item is asked does matter, at least in some instances, and it should not be assumed that an item written in the positive and negative should directly correlate. The survey research community and institutions utilizing similar rating scale instruments will benefit from the results of this study, as well as the education community in general

Validating Measures Of Self Control Via Rasch Measurement

Self control has been offered as a fundamental explanation for consumption behavior in a number of marketing settings. Until recently, measurement of self control had been inadequate, with advances being made only in specific domains. Tangney, Baumeister, and Boone (2004) introduced a reflective measure of self control which has gained popularity across social science research. However, the authors did not subject this critical measure to a review of fit and function through a psychometric lens. This study reviews their measure with consideration of fit and function, applying item response theory, and more specifically, Rasch measurement. Findings suggest that moderate levels of the unidimensional construct of self control are captured by the scale but high risk groups may be neglected by the measure in its current form

Evaluation of a Teaching Assistant Program for Third-Year Pharmacy Students

Objectives. To determine if a teaching assistant (TA) program for third-year pharmacy students (PY3s) improves confidence in teaching abilities. Additionally, 3 assessment methods (faculty, student, and TA self-evaluations) were compared for similarities and correlations

Proximal Humerus Fractures in the Elderly: Concomitant Fractures and Management

Introduction. The purpose of this study was to identify additional injuries commonly seen with proximal humerus fractures experienced by patients 65 years or older and to evaluate discrepancies in the management of these patients with regard to provider type. Methods. A retrospective review was conducted of all patients 65 years or older who sustained a proximal humerus fracture. Patient data collected included demographics, injury details, hospital course, and discharge destination. Results. Patients with a concomitant fracture (45.5%, n = 65) had a slightly higher Injury Severity Score (ISS; 8.3 ± 3.0 vs. 6.4 ± 3.0, p < 0.001) and experienced one additional death than those with an isolated fracture (54.5%, n = 78). Slightly more patients were managed by a trauma provider (51.7%, n = 74) than by a non-trauma provider (48.3%, n = 69). Those managed by a trauma provider sustained the most pelvic fractures (12.2% vs. 2.9%, p = 0.038), were more likely to be injured in a motor vehicle collision (8.1% vs. 0%, p = 0.005), had a higher ISS (8.0 ± 3.3 vs. 6.4 ± 2.8, p = 0.003), and had more imaging performed than those treated by a non-trauma provider. There was, however, no difference in operative rates, concomitant injuries, length of stay or discharge disposition regarding provider type.  Conclusions. It is important to recognize proximal humerus fractures as a sign of fragility and to optimize hospital management of these patients