4,343 research outputs found

    A Case Study on Designing Evaluations of ML Explanations with Simulated User Studies

    Full text link
    When conducting user studies to ascertain the usefulness of model explanations in aiding human decision-making, it is important to use real-world use cases, data, and users. However, this process can be resource-intensive, allowing only a limited number of explanation methods to be evaluated. Simulated user evaluations (SimEvals), which use machine learning models as a proxy for human users, have been proposed as an intermediate step to select promising explanation methods. In this work, we conduct the first SimEvals on a real-world use case to evaluate whether explanations can better support ML-assisted decision-making in e-commerce fraud detection. We study whether SimEvals can corroborate findings from a user study conducted in this fraud detection context. In particular, we find that SimEvals suggest that all considered explainers are equally performant, and none beat a baseline without explanations -- this matches the conclusions of the original user study. Such correspondences between our results and the original user study provide initial evidence in favor of using SimEvals before running user studies. We also explore the use of SimEvals as a cheap proxy to explore an alternative user study set-up. We hope that this work motivates further study of when and how SimEvals should be used to aid in the design of real-world evaluations.Comment: 9 pages, 2 figure

    Perspectives on Incorporating Expert Feedback into Model Updates

    Full text link
    Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey

    Meaningful Gameplay Design and the Effect on Eudaimonic and Hedonic Gaming Experience

    Get PDF
    Past research has looked into meaningfulness in video game narratives, as well as meaningful play. However, more empirical studies are needed to understand the mechanisms of meaningful play. This paper first investigates how the design of game dialogues may enable meaningfulness in video games (eudaimonic gaming experience) and how it relates to game enjoyment (hedonic gaming experience). Second, it also examines how meaningful play influences prosocial attitudes toward culturally different outgroups. A pre-posttest experiment of a management game was conducted with 174 adult participants. Participants were randomly assigned to either a game with moral scenarios or one without. Results showed that participants who played the game with moral decision-making perceived a higher level of meaningfulness and game enjoyment. No significant relationship was found between perceived meaningfulness and intergroup perceptions

    UBP12 and UBP13 negatively regulate the activity of the ubiquitin-dependent peptidases DA1, DAR1 and DAR2

    Get PDF
    Protein ubiquitination is a very diverse post-translational modification leading to protein degradation or delocalization, or altering protein activity. In Arabidopsis thaliana, two E3 ligases, BIG BROTHER (BB) and DA2, activate the latent peptidases DA1, DAR1 and DAR2 by mono-ubiquitination at multiple sites. Subsequently, these activated peptidases destabilize various positive regulators of growth. Here, we show that two ubiquitin-specific proteases, UBP12 and UBP13, deubiquitinate DA1, DAR1 and DAR2, hence reducing their peptidase activity. Overexpression of UBP12 or UBP13 strongly decreased leaf size and cell area, and resulted in lower ploidy levels. Mutants in which UBP12 and UBP13 were downregulated produced smaller leaves that contained fewer and smaller cells. Remarkably, neither UBP12 nor UBP13 were found to be cleavage substrates of the activated DA1. Our results therefore suggest that UBP12 and UBP13 work upstream of DA1, DAR1 and DAR2 to restrict their protease activity and hence fine-tune plant growth and development
    corecore