The Use of Loglinear Models for Assessing Differential Item Functioning Across Manifest and Latent Examinee Groups

Loglinear latent class models are used to detect differential item functioning (DIF). These models are formulated in such a manner that the attribute to be assessed may be continuous, as in a Rasch model, or categorical, as in Latent Class Mastery models. Further, an item may exhibit DIF with respect to a manifest grouping variable, a latent grouping variable, or both. Likelihood-ratio tests for assessing the presence of various types of DIF are described, and these methods are illustrated through the analysis of a "real world" data set

Modeling peer assessment as a personalized predictor of teacher's grades: The case of OpenAnswer

Questions with open answers are rarely used as e-learning assessment tools because of the resulting high workload for the teacher/tutor that should grade them. This can be mitigated by having students grade each other's answers, but the uncertainty on the quality of the resulting grades could be high. In our OpenAnswer system we have modeled peer-assessment as a Bayesian network connecting a set of sub-networks (each representing a participating student) to the corresponding answers of her graded peers. The model has shown good ability to predict (without further info from the teacher) the exact teacher mark and a very good ability to predict it within 1 mark from the right one (ground truth). From the available datasets we noticed that different teachers sometimes disagree in their assessment of the same answer. For this reason in this paper we explore how the model can be tailored to the specific teacher to improve its prediction ability. To this aim, we parametrically define the CPTs (Conditional Probability Tables) describing the probabilistic dependence of a Bayesian variable from others in the modeled network, and we optimize the parameters generating the CPTs to obtain the smallest average difference between the predicted grades and the teacher's marks (ground truth). The optimization is carried out separately with respect to each teacher available in our datasets, or respect to the whole datasets. The paper discusses the results and shows that the prediction performance of our model, when optimized separately for each teacher, improves against the case in which our model is globally optimized respect to the whole dataset, which in turn improves against the predictions of the raw peer-assessment. The improved prediction would allow us to use OpenAnswer, without teacher intervention, as a class monitoring and diagnostic tool

Teacher Stability and Turnover in Los Angeles: The Influence of Teacher and School Characteristics

Analyzes how teacher and school characteristics - including demographics, quality and qualification, specialty, school type (public, magnet, charter) and size, academic climate, and teacher-student racial match - influence teacher turnover

Family Background, Family Income, Cognitive Tests Scores, Behavioural Scales and their Relationship with Post-secondary Education Participation: Evidence from the NLSCY

This paper exploits the panel feature of the Canadian National Longitudinal Survey of Children and Youth (NLSCY) and the large diversity of measures collected on the children ad their families over 6 cycles (1994-1995 to 2004-2005) to explain high school graduation and postsecondary education (PSE) choices of Canadian youth aged 18 to 21 observed in the most recent wave of the survey. In estimating how family background, family income, cognitive abilities, non-cognitive abilities and behavioural scores influence schooling choices they can be used as markers for identifying children at risk of not pursuing PSE. We focus on the impact of measures that are specific to the NLSCY which contains a host of scores on several dimensions such as the cognitive achievement of children (reading and math test scores); behavioural scores that measure the levels of hyperactivity, aggression, and pro-sociality; scores that measure self-esteem and self-control (non-cognitive abilities); and, family scores that measure the quality of parenting, family dysfunction, of neighbourhoods and schools quality. The math and reading scores are particularly interesting because they are computed from objective tests and are not based on any type of recall, as compared, for example, with the Youth in Transition Survey (YITS) data set. Despite the fact that income, as measured as the mean income ($2002) of the family during cycles 1 to 4, does not seem to be a key player for PSE attendance or high school graduation, the sign of its effect is generally positive and non-linear, increases for children in very low income will have a large effect that those with higher levels. More importantly, several variables that are characteristics of low-income families play a key role for schooling attainment. For example, being from a single-parent/guardian home with a poorly educated PMK and with less than (perceived) excellent/very good health or with high levels of hyperactivity for males or high levels of aggression for young teenage females will almost negate any chance of attaining the level of PSE.High school graduation, postsecondary education, schooling transition, gender, youth, longitudinal data

Comparing Economic Mobility with Heterogeneity Indices: an Application to Education in Peru

The long literature on intergenerational transmission of well-being has largerly been driven by concerns for inequality of opportunity and the persistence of low levels of wellbeing among certain social groups.A comparative strand of this literature seeks to compare indicators of these transmission mechanisms, i.e. mobility regimes, across societies, regions or time. In this paper I contribute to this literature by suggesting an additional way of comparing mobility regimes with indices of heterogeneity across distributions based on a traditional homogeneity test of multinomial distributions, which is helpful to compare discrete-time transition matrices. The indices measure the degree of dissimilarity between two or more transition matrices controlling for population size and the dimensions of the matrix. The indices provide a good alternative to between-group comparisons based on linear parametric models (chiefly OLS) in which either slope coefficients are compared directly or group dummy variables are interacted with parameters from the models. They also provide complementary information to comparisons based on summary indicators of transition matrices. An application to educational mobility in Peru shows that the transition matrices of males and females are more similar among the youngest cohorts of adults.

50 Years of Test (Un)fairness: Lessons for Machine Learning

Quantitative definitions of what is unfair and what is fair have been introduced in multiple disciplines for well over 50 years, including in education, hiring, and machine learning. We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged. In some cases, earlier definitions of fairness are similar or identical to definitions of fairness in current machine learning research, and foreshadow current formal work. In other cases, insights into what fairness means and how to measure it have largely gone overlooked. We compare past and current notions of fairness along several dimensions, including the fairness criteria, the focus of the criteria (e.g., a test, a model, or its use), the relationship of fairness to individuals, groups, and subgroups, and the mathematical method for measuring fairness (e.g., classification, regression). This work points the way towards future research and measurement of (un)fairness that builds from our modern understanding of fairness while incorporating insights from the past.Comment: FAT* '19: Conference on Fairness, Accountability, and Transparency (FAT* '19), January 29--31, 2019, Atlanta, GA, US