Let's go to the Alien Zoo: Introducing an experimental framework to study usability of counterfactual explanations for machine learning

IntroductionTo foster usefulness and accountability of machine learning (ML), it is essential to explain a model's decisions in addition to evaluating its performance. Accordingly, the field of explainable artificial intelligence (XAI) has resurfaced as a topic of active research, offering approaches to address the “how” and “why” of automated decision-making. Within this domain, counterfactual explanations (CFEs) have gained considerable traction as a psychologically grounded approach to generate post-hoc explanations. To do so, CFEs highlight what changes to a model's input would have changed its prediction in a particular way. However, despite the introduction of numerous CFE approaches, their usability has yet to be thoroughly validated at the human level.MethodsTo advance the field of XAI, we introduce the Alien Zoo, an engaging, web-based and game-inspired experimental framework. The Alien Zoo provides the means to evaluate usability of CFEs for gaining new knowledge from an automated system, targeting novice users in a domain-general context. As a proof of concept, we demonstrate the practical efficacy and feasibility of this approach in a user study.ResultsOur results suggest the efficacy of the Alien Zoo framework for empirically investigating aspects of counterfactual explanations in a game-type scenario and a low-knowledge domain. The proof of concept study reveals that users benefit from receiving CFEs compared to no explanation, both in terms of objective performance in the proposed iterative learning task, and subjective usability.DiscussionWith this work, we aim to equip research groups and practitioners with the means to easily run controlled and well-powered user studies to complement their otherwise often more technology-oriented work. Thus, in the interest of reproducible research, we provide the entire code, together with the underlying models and user data: https://github.com/ukuhl/IntroAlienZoo

For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI

Counterfactual explanations (CFEs) are a popular approach in explainable artificial intelligence (xAI), highlighting changes to input data necessary for altering a model's output. A CFE can either describe a scenario that is better than the factual state (upward CFE), or a scenario that is worse than the factual state (downward CFE). However, potential benefits and drawbacks of the directionality of CFEs for user behavior in xAI remain unclear. The current user study (N=161) compares the impact of CFE directionality on behavior and experience of participants tasked to extract new knowledge from an automated system based on model predictions and CFEs. Results suggest that upward CFEs provide a significant performance advantage over other forms of counterfactual feedback. Moreover, the study highlights potential benefits of mixed CFEs improving user performance compared to downward CFEs or no explanations. In line with the performance results, users' explicit knowledge of the system is statistically higher after receiving upward CFEs compared to downward comparisons. These findings imply that the alignment between explanation and task at hand, the so-called regulatory fit, may play a crucial role in determining the effectiveness of model explanations, informing future research directions in xAI. To ensure reproducible research, the entire code, underlying models and user data of this study is openly available: https://github.com/ukuhl/DirectionalAlienZooComment: 22 pages, 3 figures This work has been accepted for presentation at the 1st World Conference on eXplainable Artificial Intelligence (xAI 2023), July 26-28, 2023 - Lisbon, Portuga

Toward enriched Cognitive Learning with XAI

As computational systems supported by artificial intelligence (AI) techniques continue to play an increasingly pivotal role in making high-stakes recommendations and decisions across various domains, the demand for explainable AI (XAI) has grown significantly, extending its impact into cognitive learning research. Providing explanations for novel concepts is recognised as a fundamental aid in the learning process, particularly when addressing challenges stemming from knowledge deficiencies and skill application. Addressing these difficulties involves timely explanations and guidance throughout the learning process, prompting the interest of AI experts in developing explainer models. In this paper, we introduce an intelligent system (CL-XAI) for Cognitive Learning which is supported by XAI, focusing on two key research objectives: exploring how human learners comprehend the internal mechanisms of AI models using XAI tools and evaluating the effectiveness of such tools through human feedback. The use of CL-XAI is illustrated with a game-inspired virtual use case where learners tackle combinatorial problems to enhance problem-solving skills and deepen their understanding of complex concepts, highlighting the potential for transformative advances in cognitive learning and co-learning.Comment: 10 pages, 2 figure

Professionalization of Inclusive School Development Processes

Die Umsetzung schulischer Inklusion (UN-BRK 2008) stellt Schulen und Lehrkräfte vor große Herausforderungen, auf die Lehrkräfte im Rahmen ihres Studiums oft nicht hinreichend vorbereitet wurden (Werning 2017). Einem entsprechend großen Fortbildungsbedarf steht jedoch oft kein passendes Fortbildungsangebot gegenüber (Richter, Kuhl und Pant 2012). Die Fortbildungsreihe »Jede(r) ist besonders« (JIB) der Deutschen Schulakademie (DSA) intendiert, zur Professionalisierung der schulischen Arbeit von Schulleitungen und Lehrkräften beizutragen und gute inklusive Praxis zu verbreiten. Die begleitende wissenschaftliche Evaluation, die JIB-Evaluation, soll es mithilfe einer Triangulation verschiedener methodischer Zugänge ermöglichen, evidenzbasiert Aussagen über die Wirksamkeit von JIB zu treffen. Im vorliegenden Beitrag werden erste Befunde der Analysen zu Schulentwicklungszielen, die sich die Schulen im Rahmen von JIB gesetzt haben, berichtet und diskutiert, um Transferund Implementationsprozesse im Kontext sich verändernder Schulen nachzuzeichnen.The implementation of inclusive schooling (UN-BRK 2008) poses major challenges for schools and teachers. Through their studies teachers often feel unprepared for working in inclusive schools and express a need for further training (Werning 2017). This need, however, is often not matched by suitable training possibilities (Richter, Kuhl, and Pant 2012). The teacher training series »Everybody is Unique« (Jede[r] ist besonders – JIB), developed by the German School Academy (Deutsche Schulakademie – DSA) intends to contribute to the professionalization of principals and teachers and to disseminate good inclusive practice. The accompanying scientific evaluation of JIB aims to provide evidence on the effectiveness of JIB by triangulating various methodological approaches. In this paper, we present first findings on the school development goals JIB participants reported for their schools. Moreover, we discuss these school development goals as means of developmental processes in the context of changing schools

Cooperation at inclusive schools: About collaboration within an inclusive teacher training programme and at schools in the view of teachers

Kooperation gilt als ein Merkmal guter Schulen und erfolgreicher Schulentwicklung. Insbesondere an Schulen, die erfolgreich inklusiv arbeiten, wird Kooperation systematisch und auf einem hohen Niveau umgesetzt. Für ausgeprägte Formen der Kooperation sind allerdings langfristige und komplexe Interventionsformen bedeutsam, welche es in Form von Aus- und Weiterbildung von Lehrpersonen – insbesondere im Bereich der inklusiven Schulentwicklung noch nicht in ausreichendem Maße gibt. Ein Schulentwicklungsprogramm, welches zur Professionalisierung von Lehrkräften im Umgang mit Heterogenität beitragen soll, ist Jede/r ist besonders (JIB) der Deutschen Schulakademie (DSA). Die zweite, zweijährige Zirkelrunde von JIB mit 24 Schulen in fünf Bundesländern wurde von 2017 bis 2020 wissenschaftlich mithilfe eines Mixed-Method-Ansatzes begleitet. Auf Grundlage des Modells zum Fortbildungserfolg nach Kirkpatrick wurde Kooperation innerhalb der Fortbildung und an den teilnehmenden Schulen untersucht. Ergebnisse aus drei Teilstudien weisen darauf hin, dass innerhalb der Fortbildung JIB erfolgreich kooperiert wird, dass Kooperation bedeutsam ist für die Umsetzung schulischer Entwicklungsziele und dass im Laufe der zwei Jahre von JIB das gemeinsame pädagogische Verständnis innerhalb der Kollegien und die außerschulische Zusammenarbeit an den JIB-Schulen weiterentwickelt werden konnten. (DIPF/Orig.)Cooperation is a feature of high quality schools and successful school development. Especially schools that work successfully inclusive, cooperate systematically and at high levels. For elaborated forms of cooperation, long-term and extensive teacher training programmes are needed. A programme of school development that is set to qualify teachers to cope with heterogeneity, is JIB (Everyone is special) by the German School Academy. From 2017 to 2020 the second, two year term of JIB with 24 schools in five federal states in Germany was scientifically evaluated by a mixed-method study. Using the model of Kirkpatrick addressing success of training programmes, cooperation was analysed within the training programme JIB and in schools. Findings of three study parts suggest a successful way of cooperating within the teacher training programme, a significant role of cooperation in the realization of school development goals and an improvement of the collective pedagogical comprehension among teacher councils and of cooperation between parents and schools

Virulence-associated genes, resistance genes and adhesion and probiotic activity tested by a new screening method

We established an automated screening method to characterize adhesion of Escherichia coli to intestinal porcine epithelial cells (IPEC-J2) and their probiotic activity against infection by enteropathogenic E. coli (EPEC). 104 intestinal E. coli isolates from domestic pigs were tested by PCR for the occurrence of virulence-associated genes, genes coding for resistances to antimicrobial agents and metals, and for phylogenetic origin by PCR. Adhesion rates and probiotic activity were examined for correlation with the presence of these genes. Finally, data were compared with those from 93 E. coli isolates from wild boars. Isolates from domestic pigs carried a broad variety of all tested genes and showed great diversity in gene patterns. Adhesions varied with a maximum of 18.3 or 24.2 mean bacteria adherence per epithelial cell after 2 or 6 hours respectively. Most isolates from domestic pigs and wild boars showed low adherence, with no correlation between adhesion/probiotic activity and E. coli genes or gene clusters. The gene sfa/foc, encoding for a subunit of F1C fimbriae did show a positive correlative association with adherence and probiotic activity; however E. coli isolates from wild boars with the sfa/foc gene showed less adhesion and probiotic activity than E. coli with the sfa/foc gene isolated from domestic pigs after 6 hour incubation. In conclusion, screening porcine E. coli for virulence associated genes genes, adhesion to intestinal epithelial cells, and probiotic activity revealed a single important adhesion factor, several probiotic candidates, and showed important differences between E. coli of domestic pigs and wild boars

Porcine E. coli: Virulence-Associated Genes, Resistance Genes and Adhesion and Probiotic Activity Tested by a New Screening Method

We established an automated screening method to characterize adhesion of Escherichia colito intestinal porcine epithelial cells (IPEC-J2) and their probiotic activity against infection by enteropathogenic E. coli (EPEC). 104 intestinal E. coli isolates from domestic pigs were tested by PCR for the occurrence of virulence-associated genes, genes coding for resistances to antimicrobial agents and metals, and for phylogenetic origin by PCR. Adhesion rates and probiotic activity were examined for correlation with the presence of these genes. Finally, data were compared with those from 93 E. coli isolates from wild boars. Isolates from domestic pigs carried a broad variety of all tested genes and showed great diversity in gene patterns. Adhesions varied with a maximum of 18.3 or 24.2 mean bacteria adherence per epithelial cell after 2 or 6 hours respectively. Most isolates from domestic pigs and wild boars showed low adherence, with no correlation between adhesion/probiotic activity and E. coli genes or gene clusters. The gene sfa/foc, encoding for a subunit of F1C fimbriae did show a positive correlative association with adherence and probiotic activity; however E. coliisolates from wild boars with the sfa/foc gene showed less adhesion and probiotic activity thanE. coli with the sfa/foc gene isolated from domestic pigs after 6 hour incubation. In conclusion, screening porcine E. coli for virulence associated genes genes, adhesion to intestinal epithelial cells, and probiotic activity revealed a single important adhesion factor, several probiotic candidates, and showed important differences between E. coli of domestic pigs and wild boars

Early cortical surface plasticity relates to basic mathematical learning

Children lay the foundation for later academic achievement by acquiring core mathematical abilities in the first school years. Neural reorganization processes associated with individual differences in early mathematical learning, however, are still poorly understood. To fill this research gap, we followed a sample of 5-6-year-old children longitudinally to the end of second grade in school (age 7–8 years) combining magnetic resonance imaging (MRI) with comprehensive behavioral assessments. We report significant links between the rate of neuroplastic change of cortical surface anatomy, and children's early mathematical skills. In particular, most of the behavioral variance (about 73%) of children's visuospatial abilities was explained by the change in cortical thickness in the right superior parietal cortex. Moreover, half of the behavioral variance (about 55%) of children's arithmetic abilities was explained by the change in cortical folding in the right intraparietal sulcus. Additional associations for arithmetic abilities were found for cortical thickness change of the right temporal lobe, and the left middle occipital gyrus. Visuospatial abilities were related to right precentral and supramarginal thickness, as well as right medial frontal gyrus folding plasticity. These effects were independent of other individual differences in IQ, literacy and maternal education. Our findings highlight the critical role of cortical plasticity during the acquisition of fundamental mathematical abilities

Shared heritability and functional enrichment across six solid cancers

Correction: Nature Communications 10 (2019): art. 4386 DOI: 10.1038/s41467-019-12095-8Quantifying the genetic correlation between cancers can provide important insights into the mechanisms driving cancer etiology. Using genome-wide association study summary statistics across six cancer types based on a total of 296,215 cases and 301,319 controls of European ancestry, here we estimate the pair-wise genetic correlations between breast, colorectal, head/neck, lung, ovary and prostate cancer, and between cancers and 38 other diseases. We observed statistically significant genetic correlations between lung and head/neck cancer (r(g) = 0.57, p = 4.6 x 10(-8)), breast and ovarian cancer (r(g) = 0.24, p = 7 x 10(-5)), breast and lung cancer (r(g) = 0.18, p = 1.5 x 10(-6)) and breast and colorectal cancer (r(g) = 0.15, p = 1.1 x 10(-4)). We also found that multiple cancers are genetically correlated with non-cancer traits including smoking, psychiatric diseases and metabolic characteristics. Functional enrichment analysis revealed a significant excess contribution of conserved and regulatory regions to cancer heritability. Our comprehensive analysis of cross-cancer heritability suggests that solid tumors arising across tissues share in part a common germline genetic basis.Peer reviewe