Auditing Search Engines for Differential Satisfaction Across Demographics

Many online services, such as search engines, social media platforms, and digital marketplaces, are advertised as being available to any user, regardless of their age, gender, or other demographic factors. However, there are growing concerns that these services may systematically underserve some groups of users. In this paper, we present a framework for internally auditing such services for differences in user satisfaction across demographic groups, using search engines as a case study. We first explain the pitfalls of na\"ively comparing the behavioral metrics that are commonly used to evaluate search engines. We then propose three methods for measuring latent differences in user satisfaction from observed differences in evaluation metrics. To develop these methods, we drew on ideas from the causal inference literature and the multilevel modeling literature. Our framework is broadly applicable to other online services, and provides general insight into interpreting their evaluation metrics.Comment: 8 pages Accepted at WWW 201

Depression, anxiety, and within-person variability in adults aged 18 to 85 years

Mild depression and anxiety were investigated in relation to measures of within-person (WP) variability and mean RT from psychomotor, executive function, visual search, and word recognition tasks in a continuous age range (18 to 85 years, M = 50.33, SD = 20.37) of 300 community-dwelling adults. Structural equation modeling identified a significant Age x Depression interaction in relation to visual search for measures of WP variability but not for mean RT. Older more depressed adults exhibited greater variability. WP variability in executive function and other cognitive constructs covaried, and the significant Age x Depression interaction with visual search was accounted for by WP variability in executive control. The findings suggest that age-, and depression-related reductions in attentional resources may contribute to increased variability in visual search, and that variability in executive control may be the mechanism underlying these effects.This article may not exactly replicate the final version published in the APA journal. It is not the copy of record.This paper has not (yet) been published

Effect of vicarious fear learning on children's heart rate responses and attentional bias for novel animals

Research with children has shown that vicarious learning can result in changes to 2 of Lang's (1968) 3 anxiety response systems: subjective report and behavioral avoidance. The current study extended this research by exploring the effect of vicarious learning on physiological responses (Lang's final response system) and attentional bias. The study used Askew and Field's (2007) vicarious learning procedure and demonstrated fear-related increases in children's cognitive, behavioral, and physiological responses. Cognitive and behavioral changes were retested 1 week and 1 month later, and remained elevated. In addition, a visual search task demonstrated that fear-related vicarious learning creates an attentional bias for novel animals, which is moderated by increases in fear beliefs during learning. The findings demonstrate that vicarious learning leads to lasting changes in all 3 of Lang's anxiety response systems and is sufficient to create attentional bias to threat in children

Structure and correlates of cognitive aging in a narrow age cohort

Aging-related changes occur for multiple domains of cognitive functioning. An accumulating body of research indicates that, rather than representing statistically independent phenomena, aging-related cognitive changes are moderately to strongly correlated across domains. However, previous studies have typically been conducted in age-heterogeneous samples over longitudinal time lags of 6 or more years, and have failed to consider whether results are robust to a comprehensive set of controls. Capitalizing on 3-year longitudinal data from the Lothian Birth Cohort of 1936, we took a longitudinal narrow age cohort approach to examine cross-domain cognitive change interrelations from ages 70 to 73 years. We fit multivariate latent difference score models to factors representing visuospatial ability, processing speed, memory, and crystallized ability. Changes were moderately interrelated, with a general factor of change accounting for 47% of the variance in changes across domains. Change interrelations persisted at close to full strength after controlling for a comprehensive set of demographic, physical, and medical factors including educational attainment, childhood intelligence, physical function, APOE genotype, smoking status, diagnosis of hypertension, diagnosis of cardiovascular disease, and diagnosis of diabetes. Thus, the positive manifold of aging-related cognitive changes is highly robust in that it can be detected in a narrow age cohort followed over a relatively brief longitudinal period, and persists even after controlling for many potential confounders

Effects of Selection Systems on Job Search Decisions

On the basis of Gilliland\u27s (1993) model of selection system fairness, the present study investigated the relationships between selection procedures, perceived selection system fairness, and job search decisions in both hypothetical and actual organizations. We conducted two studies to test the model. In Study 1, we used an experimental method to examine job seekers\u27 perceptions of, and reactions to, five widely used selection procedures. Results suggested that applicants viewed employment interviews and cognitive ability tests as more job related than biographical inventories (biodata), personality tests, and drug tests, and that job relatedness significantly affected fairness perceptions, which in turn affected job search decisions. Study 2 examined the hypothesized relationships between the selection systems and job seekers\u27 pursuit of actual, relevant organizations. Results from both studies offer support for the hypothesized model, suggesting that selection tests have differential effects on perceived selection system validity and fairness, which affect subsequent job search decisions

Fertility and its Meaning: Evidence from Search Behavior

Fertility choices are linked to the different preferences and constraints of individuals and couples, and vary importantly by socio-economic status, as well by cultural and institutional context. The meaning of childbearing and child-rearing, therefore, differs between individuals and across groups. In this paper, we combine data from Google Correlate and Google Trends for the U.S. with ground truth data from the American Community Survey to derive new insights into fertility and its meaning. First, we show that Google Correlate can be used to illustrate socio-economic differences on the circumstances around pregnancy and birth: e.g., searches for "flying while pregnant" are linked to high income fertility, and "paternity test" are linked to non-marital fertility. Second, we combine several search queries to build predictive models of regional variation in fertility, explaining about 75% of the variance. Third, we explore if aggregated web search data can also be used to model fertility trends.Comment: This is a preprint of a short paper accepted at ICWSM'17. Please cite that version instea

Mind the gap! A multilevel analysis of factors related to variation in published cost-effectiveness estimates within and between countries

Background. Published cost-effectiveness estimates can vary considerably, both within and between countries. Despite extensive discussion, little is known empirically about factors relating to these variations. Objectives. To use multilevel statistical modeling to integrate cost-effectiveness estimates from published economic evaluations to investigate potential causes of variation. Methods. Cost-effectiveness studies of statins for cardiovascular disease prevention were identified by systematic review. Estimates of incremental costs and effects were extracted from reported base case, sensitivity, and subgroup analyses, with estimates grouped in studies and in countries. Three bivariate models were developed: a cross-classified model to accommodate data from multinational studies, a hierarchical model with multinational data allocated to a single category at country level, and a hierarchical model excluding multinational data. Covariates at different levels were drawn from a long list of factors suggested in the literature. Results. We found 67 studies reporting 2094 cost-effectiveness estimates relating to 23 countries (6 studies reporting for more than 1 country). Data and study-level covariates included patient characteristics, intervention and comparator cost, and some study methods (e.g., discount rates and time horizon). After adjusting for these factors, the proportion of variation attributable to countries was negligible in the cross-classified model but moderate in the hierarchical models (14%−19% of total variance). Country-level variables that improved the fit of the hierarchical models included measures of income and health care finance, health care resources, and population risks. Conclusions. Our analysis suggested that variability in published cost-effectiveness estimates is related more to differences in study methods than to differences in national context. Multinational studies were associated with much lower country-level variation than single-country studies. These findings are for a single clinical question and may be atypical

Domain-specific queries and Web search personalization: some investigations

Major search engines deploy personalized Web results to enhance users' experience, by showing them data supposed to be relevant to their interests. Even if this process may bring benefits to users while browsing, it also raises concerns on the selection of the search results. In particular, users may be unknowingly trapped by search engines in protective information bubbles, called "filter bubbles", which can have the undesired effect of separating users from information that does not fit their preferences. This paper moves from early results on quantification of personalization over Google search query results. Inspired by previous works, we have carried out some experiments consisting of search queries performed by a battery of Google accounts with differently prepared profiles. Matching query results, we quantify the level of personalization, according to topics of the queries and the profile of the accounts. This work reports initial results and it is a first step a for more extensive investigation to measure Web search personalization.Comment: In Proceedings WWV 2015, arXiv:1508.0338