Improving Test Score Reporting: Perspectives From the ETS Score Reporting Conference

This volume includes 3 papers based on presentations at a workshop on communicating assessment information to particular audiences, held at Educational Testing Service (ETS) on November 4th, 2010, to explore some issues that influence score reports and new advances that contribute to the effectiveness of these reports. Jessica Hullman, Rebecca Rhodes, Fernando Rodriguez, and Priti Shah present the results of recent research on graph comprehension and data interpretation, especially the role of presentation format, the impact of prior quantitative literacy and domain knowledge, the trade‐off between reducing cognitive load and increasing active processing of data, and the affective influence of graphical displays. Rebecca Zwick and Jeffrey Sklar present the results of the Instructional Tools in Educational Measurement and Statistics for School Personnel (ITEMS) project, funded by the National Science Foundation and conducted at the University of California, Santa Barbara to develop and evaluate 3 web‐based instructional modules intended to help educators interpret test scores. Zwick and Sklar discuss the modules and the procedures used to evaluate their effectiveness. Diego Zapata‐Rivera presents a new framework for designing and evaluating score reports, based on work on designing and evaluating score reports for particular audiences in the context of the CBAL (Cognitively Based Assessment of, for, and as Learning) project (Bennett & Gitomer, 2009), which has been applied in the development and evaluation of reports for various audiences including teachers, administrators and students.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/108325/1/ets202281.pd

50 Years of Test (Un)fairness: Lessons for Machine Learning

Quantitative definitions of what is unfair and what is fair have been introduced in multiple disciplines for well over 50 years, including in education, hiring, and machine learning. We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged. In some cases, earlier definitions of fairness are similar or identical to definitions of fairness in current machine learning research, and foreshadow current formal work. In other cases, insights into what fairness means and how to measure it have largely gone overlooked. We compare past and current notions of fairness along several dimensions, including the fairness criteria, the focus of the criteria (e.g., a test, a model, or its use), the relationship of fairness to individuals, groups, and subgroups, and the mathematical method for measuring fairness (e.g., classification, regression). This work points the way towards future research and measurement of (un)fairness that builds from our modern understanding of fairness while incorporating insights from the past.Comment: FAT* '19: Conference on Fairness, Accountability, and Transparency (FAT* '19), January 29--31, 2019, Atlanta, GA, US

Anderson localisation in spin chains for perfect state transfer

Abstract: Anderson localisation is an important phenomenon arising in many areas of physics, andhere we explore it in the context of quantum information devices. Finite dimensional spinchains have been demonstrated to be important devices for quantum information transport,and in particular can be engineered to allow for “perfect state transfer” (PST). Here wepresent extensive investigations of disordered PST spin chains, demonstrating spatiallocalisation and transport retardation effects, and relate these effects to conventionalAnderson localisation. We provide thresholds for Anderson localisation in these finitequantum information systems for both the spatial and the transport domains. Finally, weconsider the effect of disorder on the eigenstates and energy spectrum of our Hamiltonian,where results support our conclusions on the presence of Anderson localisation. Graphical abstract: [Figure not available: see fulltext.

A Directed Molecular Evolution Approach to Improved Immunogenicity of the HIV-1 Envelope Glycoprotein

A prophylactic vaccine is needed to slow the spread of HIV-1 infection. Optimization of the wild-type envelope glycoproteins to create immunogens that can elicit effective neutralizing antibodies is a high priority. Starting with ten genes encoding subtype B HIV-1 gp120 envelope glycoproteins and using in vitro homologous DNA recombination, we created chimeric gp120 variants that were screened for their ability to bind neutralizing monoclonal antibodies. Hundreds of variants were identified with novel antigenic phenotypes that exhibit considerable sequence diversity. Immunization of rabbits with these gp120 variants demonstrated that the majority can induce neutralizing antibodies to HIV-1. One novel variant, called ST-008, induced significantly improved neutralizing antibody responses when assayed against a large panel of primary HIV-1 isolates. Further study of various deletion constructs of ST-008 showed that the enhanced immunogenicity results from a combination of effective DNA priming, an enhanced V3-based response, and an improved response to the constant backbone sequences

Prime-boost immunization of rabbits with HIV-1 gp120 elicits potent neutralization activity against a primary viral isolate

<div><p>Development of a vaccine for HIV-1 requires a detailed understanding of the neutralizing antibody responses that can be experimentally elicited to difficult-to-neutralize primary isolates. Rabbits were immunized with the gp120 subunit of HIV-1 JR-CSF envelope (Env) using a DNA-prime protein-boost regimen. We analyzed five sera that showed potent autologous neutralizing activity (IC50s at ∼10³ to 10⁴ serum dilution) against pseudoviruses containing Env from the primary isolate JR-CSF but not from the related isolate JR-FL. Pseudoviruses were created by exchanging each variable and constant domain of JR-CSF gp120 with that of JR-FL or with mutations in putative N-glycosylation sites. The sera contained different neutralizing activities dependent on C3 and V5, C3 and V4, or V4 regions located on the glycan-rich outer domain of gp120. All sera showed enhanced neutralizing activity toward an Env variant that lacked a glycosylation site in V4. The JR-CSF gp120 epitopes recognized by the sera are generally distinct from those of several well characterized mAbs (targeting conserved sites on Env) or other type-specific responses (targeting V1, V2, or V3 variable regions). The activity of one serum requires specific glycans that are also important for 2G12 neutralization and this serum blocked the binding of 2G12 to gp120. Our findings show that different fine specificities can achieve potent neutralization of HIV-1, yet this strong activity does not result in improved breadth.</p> </div

The role of admissions test scores, socioeconomic status, and high school grades in predicting college achievement. Revista de Investigación Educacional Latinoamericana

Although test scores are widely used in college admissions in the United States, their use is the subject of ongoing debate, partly because of the association between test performance and socioeconomic status (SES). Although test critics have argued that this association is due to the particular content of admissions tests or to the differential availability of coaching, large socioeconomic effects are also found in assessments that are tied to school achievement and for which coaching is not available, such as the National Assessment of Educational Progress and in other academic measures. Some commentators have argued, however, that high school grade-point average has a smaller correlation with SES than admissions test scores and is therefore a superior admissions criterion. In this paper I examine the association between SES and test scores, as well as the association between SES and high school grades, and discuss the relevance of this complex web of associations to college admissions research. While the perennial finding that socioeconomic inequities manifest themselves as educational inequities is disheartening, the analysis of performance differences can point the way toward possible remedies. Abstrac

The Role of Standardized Tests in College Admissions

This paper summarizes the history of college admissions testing in the United States; how the SAT and ACT are used today in admissions; admissions criteria and their use; the future of admissions tests and alternative approaches to admissions. It also provides thoughts on the definition of "merit" and the value of meritocracy in college admissions

Role of Admissions Test Scores, Socioeconomic Status, and High School Grades in Predicting College Achievement

Although test scores are widely used in college admissions in the United States, their use is the subject of ongoing debate, partly because of the association between test performance and socioeconomic status (SES). Although test critics have argued that this association is due to the particular content of admissions tests or to the differential availability of coaching, large socioeconomic effects are also found in assessments that are tied to school achievement and for which coaching is not available, such as the National Assessment of Educational Progress and in other academic measures. Some commentators have argued, however, that high school grade-point average has a smaller correlation with SES than admissions test scores and is therefore a superior  admissions criterion. In this paper I examine the association between SES and test scores, as well as the association between SES and high school grades, and discuss the relevance of this complex web of associations to college admissions research. While the perennial finding that socioeconomic inequities manifest themselves as educational inequities is disheartening, the analysis of performance differences can point the way toward possible remedies

A multivariate perspective on the analysis of categorical data

Psychological research often involves analysis of an I x J contingency table consisting of the responses of J groups of individuals on a criterion variable with I nominal categories. The conventional statistical approach for comparing responses across groups is the Pearson chi-square test. Alternatively, this analysis can be viewed as a multivariate analysis of variance with binary dependent variables, a canonical correlation analysis with two sets of binary variables, or a form of correspondence analysis. Although these analysis approaches stem from different traditions, they produce equivalent results when applied to an I x J table