This technical report summarizes the results of a study in which we examined the technical adequacy of two potential measures for algebra progress monitoring. Eighty-seven students (11 of whom were receiving special education services) completed two forms of a Translations measure and two forms of a Content Analysis-Multiple Choice measure during each of two data collection sessions. In addition, we gathered data on criterion variables including grades, overall grade point average, teacher ratings of student proficiency, and scores on districtadministered standardized tests, as well as a measure of algebra aptitude. We examined both test-retest and alternate form reliability for both single probe scores and aggregated scores (computed by averaging two individual scores). Criterion validity was examined by computing correlations between students’ single and aggregated scores on the probes with their scores on other indicators of proficiency in algebra. The results of this study suggest that the Translations measure is more promising than the Content Analysis-Multiple Choice measure in terms of both reliability and criterion validity. The strength of the relations obtained in this study were in the low to moderate range and were not as strong as the relations obtained with a different sample in this district using three other algebra measures (see Project AAIMS Technical Report 2 for details of the earlier study). Both measures produced acceptable distributions that were free from floor and ceiling effects. Students had roughly similar means and standard deviations on both measures. Reliability estimates for both measures fell short of expected levels for both single probes and aggregated scores. The Translations measure produced stronger correlations than the Content Analysis-Multiple Choice measure, but did not demonstrate a level of reliability that would be acceptable for instructional decision making. The majority of the criterion validity relations were in the low to moderate range. Aggregated scores produced improvements in the criterion validity estimates for the Translations measure, but not for the Content Analysis-Multiple Choice measure. The strongest relations were identified between the Translations measure and eighth graders’ performance on the district’s math achievement test, as well as between the Translations measure and all students’ performance on the algebra aptitude test. These two relations were in the moderate to strong range; relations between the Translations measure and the remaining criterion variables were in the low range
