Foresight for ethical AI

There is growing expectation that artificial intelligence (AI) developers foresee and mitigate harms that might result from their creations; however, this is exceptionally difficult given the prevalence of emergent behaviors that occur when integrating AI into complex sociotechnical systems. We argue that Naturalistic Decision Making (NDM) principles, models, and tools are well-suited to tackling this challenge. Already applied in high-consequence domains, NDM tools such as the premortem, and others, have been shown to uncover a reasonable set of risks of underlying factors that would lead to ethical harms. Such NDM tools have already been used to develop AI that is more trustworthy and resilient, and can help avoid unintended consequences of AI built with noble intentions. We present predictive policing algorithms as a use case, highlighting various factors that led to ethical harms and how NDM tools could help foresee and mitigate such harms

A Test-Retest Reliability Generalization Meta-Analysis of Judgments Via the Policy-Capturing Technique

Policy capturing is a widely used technique, but the temporal stability of policy-capturing judgments has long been a cause for concern. This article emphasizes the importance of reporting reliability, and in particular test-retest reliability, estimates in policy-capturing studies. We found that only 164 of 955 policy-capturing studies (i.e., 17.17%) reported a test-retest reliability estimate. We then conducted a reliability generalization meta-analysis on policy-capturing studies that did report test-retest reliability estimates—and we obtained an average reliability estimate of .78. We additionally examined 16 potential methodological and substantive antecedents to test-retest reliability (equivalent to moderators in validity generalization studies). We found that test-retest reliability was robust to variation in 14 of the 16 factors examined but that reliability was higher in paper-and-pencil studies than in web-based studies and was higher for behavioral intention judgments than for other (e.g., attitudinal and perceptual) judgments. We provide an agenda for future research. Finally, we provide several best-practice recommendations for researchers (and journal reviewers) with regard to (a) reporting test-retest reliability, (b) designing policy-capturing studies for appropriate reportage, and (c) properly interpreting test-retest reliability in policy-capturing studies

Lockheed Martin Research Team

Please read Wiki

USC Research Team

Please read Wiki

Notre Dame Research Team

Please read Wiki

Individual Differences in Judgment and Decision-Making: Novel Predictors of Counterproductive Work Behavior

Supplemental materials for Dr. Balca Alaybek's "Individual Differences in Judgment and Decision-Making: Novel Predictors of Counterproductive Work Behavior

Within-person job performance variability over short timeframes: Theory, empirical research, and practice

We begin by charting the evolution of the dominant perspective on job performance from one that viewed performance as static to one that viewed it as dynamic over long timeframes (e.g., months, years, decades) to one that views it as dynamic over not just long but also short timeframes (e.g., minutes, hours, days, weeks)-and that accordingly emphasizes the within-person level of analysis. The remainder of the article is devoted to the newer, short-timeframe research on within-person variability in job performance. We emphasize personality states and affective states as motivational antecedents. We provide accessible reviews of relevant theories and highlight the convergence of theorizing across the personality and affect antecedent domains. We then focus on several major avenues for future research. Finally, we discuss the implications of these perspectives for personnel selection and performance management in organizations as well as for employees aiming to optimize their job performance

Thinking Styles and Task Performance Meta-Analysis

This project provides the data and the references of the primary studies used in the thinking style–task performance meta-analysis. The results of the meta-analysis are reported in the manuscript entitled "The relations of reflective and intuitive thinking styles with task performance: A meta-analysis," which has been accepted for publication at Personnel Psychology