A longitudinal approach to understanding course evaluations

Accessed 16,892 times on https://pareonline.net from April 13, 2005 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Assessing Content Knowledge and Changes in Confidence and Anxiety Related to Economic Literacy in a Professional Development Program for History Teachers

The purpose of this study was to determine the impact of a Teaching American History professional development program on content knowledge, and confidence and anxiety associated with teaching economic literacy. Two content assessments and a confidence and anxiety instrument were administered to teachers prior to and immediately following a two-week Summer Institute. Statistically significant findings included an increase in economics content knowledge and an increase in confidence combined with a decrease in anxiety. The scale and measurement model employed to examine status and subsequent change should be useful for similar professional development initiatives and evaluations

Measuring Teachers’ Assessment for Learning (AfL) Classroom Practices in Elementary Schools

Assessment for Learning (AfL) may be conceptualized as minute-to-minute, day-by-day interactions between learners and teachers with the improvement of learning as the principal focus. This paper traces the development of an AfL measurement instrument (scale) that can be used for research purposes prior to, during and following professional development in the area. Rasch measurement procedures were applied to data drawn from a convenience sample of 594 teachers from 44 elementary schools in Ireland to create a scale consisting of 20 items distributed across four key AfL assessment strategies: learning intentions and success criteria, questioning and classroom discussion, feedback, and peer-and self-assessment.  This scale, the Assessment for Learning Measurement instrument (AfLMi), has good psychometric properties and is interpretable in a way that makes it potentially useful during system wide improvement initiatives focused on AfL

An Enhancement to the Theory and Measurement of Purpose

Development of purpose is an important goal of post-secondary education. This study advances the measurement of purpose by (a) enriching the construct through incorporating the facet of horizon; (b) providing a framework for Rasch/Guttman Scenario score interpretation; and (c) providing evidence of convergent, divergent, and known groups validity

Measuring Purpose in Life in College Students: An Assessment of Invariance Properties by College Year and Undergraduate School

Purpose in life is a key construct in the development of young adults, particularly college students. There are many instruments measuring sense of purpose in life, but few studies have examined their measurement properties among college students. The current study compares the measurement invariance properties of the Purpose in Life (PIL) scale and the Claremont Purpose Scale (CPS) across college year and undergraduate school. Using both a unidimensional and a two-dimensional model, we found that the PIL’s interpretability is limited among college students. Using a three-dimensional model, the CPS was invariant with respect to both grouping variables. The study suggests that the CPS can be used to make meaningful comparisons among college students categorized by school year and undergraduate school. The study also has some implications about the construct of purpose in life; namely, scale structures that work well statistically and theoretically among adults might not generalize to young adults

Teacher Test Accountability.

Given the high stakes of teacher testing, there is no doubt that every teacher test should meet the industry guidelines set forth in the Standards for Educational and Psychological Testing. Unfortunately, however, there is no public or private business or governmental agency that serves to certify or in any other formal way declare that any teacher test does, in fact, meet the psychometric recommendations stipulated in the Standards. Consequently, there are no legislated penalties for faulty products (tests) nor are there opportunities for test takers simply to raise questions about a test and to have their questions taken seriously by an impartial panel. The purpose of this article is to highlight some of the psychometric results reported by National Evaluation Systems (NES) in their 1999 Massachusetts Educator Certification Test (MECT) Technical Report, and more specifically, to identify those technical characteristics of the MECT that are inconsistent with the Standards. A second purpose of this article is to call for the establishment of a standing test auditing organization with investigation and sanctioning power. The significance of the present analysis is twofold: a) psychometric results for the MECT are similar in nature to psychometric results presented as evidence of test development flaws in an Alabama class-action lawsuit dealing with teacher certification (an NES-designed testing system); and b) there was no impartial enforcement agency to whom complaints about the Alabama tests could be brought, other than the court, nor is there any such agency to whom complaints about the Massachusetts tests can be brought. I begin by reviewing NES's role in Allen v. Alabama State Board of Education, 81-697-N. Next I explain the purpose and interpretation of standard item analysis procedures and statistics. Finally, I present results taken directly from the 1999 MECT Technical Report and compare them to procedures, results, and consequences of procedures followed by NES in Alabama

Construct Invariance of the Survey of Knowledge of Internet Risk and Internet Behavior Knowledge Scale

The wide use of the Internet has potential for students to become victims of Internet predators or other students who engage in inappropriate cyberbullying. Educational programming efforts targeted for students, teachers and parents need instrumentation that provides meaningful and reliable data assessing students\u27 knowledge of Internet risk and their actual Internet behaviors. Construct invariance of the Survey of Knowledge of Internet Risk and Internet Behavior Knowledge scale regarding gender and grade level is examined for N=2621 middle school and N=1594 high school students using multi-group confirmatory factor analysis and Rasch rating scale modeling. Implications for score interpretations are discussed

Besieged Institutions and the Massachusetts Teacher Tests

Teacher testing was inaugurated in Massachusetts in 1998 and a 59% failure rate among test-takers led to public shaming of the teacher candidates and their colleges and universities in the media. Within a two-year time period, low-performing teacher education programs in Massachusetts initiated a wide range of test preparatory activities which led to a dramatic increase in their students' pass rates. The authors separate colleges and universities into three categories and examine their differentiated responses to teacher testing. Their finding that institutions of higher education have responded effectively to teacher testing does not preclude critique of teacher testing as currently practiced in Massachusetts

Using Rasch Residuals to Analyze the Demographic Characteristics of Respondents with Unexpected Incorrect Answers: Implications for Construct Validity

This paper illustrates the use of Rasch model residuals to better understand perceived respondent meaning and structure of item content in the context of construct validity. Data were gathered from1366 grade 7-8 students using the Survey of Knowledge of Internet Risk and Internet Behavior. The characteristics of the respondents with unexpected patterns of residuals for persons not fitting the Rasch model are examined for 7 items defining the Knowledge of Internet Risk scale. These analyses contribute to a better understanding of the item content and person scores, and contribute to more meaningful score inferences