64 research outputs found
Leveraging Explanations in Interactive Machine Learning: An Overview
Explanations have gained an increasing level of interest in the AI and
Machine Learning (ML) communities in order to improve model transparency and
allow users to form a mental model of a trained ML model. However, explanations
can go beyond this one way communication as a mechanism to elicit user control,
because once users understand, they can then provide feedback. The goal of this
paper is to present an overview of research where explanations are combined
with interactive capabilities as a mean to learn new models from scratch and to
edit and debug existing ones. To this end, we draw a conceptual map of the
state-of-the-art, grouping relevant approaches based on their intended purpose
and on how they structure the interaction, highlighting similarities and
differences between them. We also discuss open research issues and outline
possible directions forward, with the hope of spurring further research on this
blooming research topic
Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning
Contains fulltext :
228326pre.pdf (preprint version ) (Open Access)
Contains fulltext :
228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202
Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization
We tackle the problem of graph out-of-distribution (OOD) generalization.
Existing graph OOD algorithms either rely on restricted assumptions or fail to
exploit environment information in training data. In this work, we propose to
simultaneously incorporate label and environment causal independence (LECI) to
fully make use of label and environment information, thereby addressing the
challenges faced by prior methods on identifying causal and invariant
subgraphs. We further develop an adversarial training strategy to jointly
optimize these two properties for casual subgraph discovery with theoretical
guarantees. Extensive experiments and analysis show that LECI significantly
outperforms prior methods on both synthetic and real-world datasets,
establishing LECI as a practical and effective solution for graph OOD
generalization
Establishing Data Provenance for Responsible Artificial Intelligence Systems
Data provenance, a record that describes the origins and processing of data, offers new promises in the increasingly important role of artificial intelligence (AI)-based systems in guiding human decision making. To avoid disastrous outcomes that can result from bias-laden AI systems, responsible AI builds on four important characteristics: fairness, accountability, transparency, and explainability. To stimulate further research on data provenance that enables responsible AI, this study outlines existing biases and discusses possible implementations of data provenance to mitigate them. We first review biases stemming from the data’s origins and pre-processing. We then discuss the current state of practice, the challenges it presents, and corresponding recommendations to address them. We present a summary highlighting how our recommendations can help establish data provenance and thereby mitigate biases stemming from the data’s origins and pre-processing to realize responsible AI-based systems. We conclude with a research agenda suggesting further research avenues
Large Language Model Alignment: A Survey
Recent years have witnessed remarkable progress made in large language models
(LLMs). Such advancements, while garnering significant attention, have
concurrently elicited various concerns. The potential of these models is
undeniably vast; however, they may yield texts that are imprecise, misleading,
or even detrimental. Consequently, it becomes paramount to employ alignment
techniques to ensure these models to exhibit behaviors consistent with human
values.
This survey endeavors to furnish an extensive exploration of alignment
methodologies designed for LLMs, in conjunction with the extant capability
research in this domain. Adopting the lens of AI alignment, we categorize the
prevailing methods and emergent proposals for the alignment of LLMs into outer
and inner alignment. We also probe into salient issues including the models'
interpretability, and potential vulnerabilities to adversarial attacks. To
assess LLM alignment, we present a wide variety of benchmarks and evaluation
methodologies. After discussing the state of alignment research for LLMs, we
finally cast a vision toward the future, contemplating the promising avenues of
research that lie ahead.
Our aspiration for this survey extends beyond merely spurring research
interests in this realm. We also envision bridging the gap between the AI
alignment research community and the researchers engrossed in the capability
exploration of LLMs for both capable and safe LLMs.Comment: 76 page
Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation
We investigate whether three types of post hoc model explanations--feature
attribution, concept activation, and training point ranking--are effective for
detecting a model's reliance on spurious signals in the training data.
Specifically, we consider the scenario where the spurious signal to be detected
is unknown, at test-time, to the user of the explanation method. We design an
empirical methodology that uses semi-synthetic datasets along with
pre-specified spurious artifacts to obtain models that verifiably rely on these
spurious training signals. We then provide a suite of metrics that assess an
explanation method's reliability for spurious signal detection under various
conditions. We find that the post hoc explanation methods tested are
ineffective when the spurious artifact is unknown at test-time especially for
non-visible artifacts like a background blur. Further, we find that feature
attribution methods are susceptible to erroneously indicating dependence on
spurious signals even when the model being explained does not rely on spurious
artifacts. This finding casts doubt on the utility of these approaches, in the
hands of a practitioner, for detecting a model's reliance on spurious signals
Seeking information about assistive technology: Exploring current practices, challenges, and the need for smarter systems
Ninety percent of the 1.2 billion people who need assistive technology (AT) do not have access. Information seeking practices directly impact the ability of AT producers, procurers, and providers (AT professionals) to match a user's needs with appropriate AT, yet the AT marketplace is interdisciplinary and fragmented, complicating information seeking. We explored common limitations experienced by AT professionals when searching information to develop solutions for a diversity of users with multi-faceted needs. Through Template Analysis of 22 expert interviews, we find current search engines do not yield the necessary information, or appropriately tailor search results, impacting individuals’ awareness of products and subsequently their availability and the overall effectiveness of AT provision. We present value-based design implications to improve functionality of future AT-information seeking platforms, through incorporating smarter systems to support decision-making and need-matching whilst ensuring ethical standards for disability fairness remain
Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero
Artificial Intelligence (AI) systems have made remarkable progress, attaining
super-human performance across various domains. This presents us with an
opportunity to further human knowledge and improve human expert performance by
leveraging the hidden knowledge encoded within these highly performant AI
systems. Yet, this knowledge is often hard to extract, and may be hard to
understand or learn from. Here, we show that this is possible by proposing a
new method that allows us to extract new chess concepts in AlphaZero, an AI
system that mastered the game of chess via self-play without human supervision.
Our analysis indicates that AlphaZero may encode knowledge that extends beyond
the existing human knowledge, but knowledge that is ultimately not beyond human
grasp, and can be successfully learned from. In a human study, we show that
these concepts are learnable by top human experts, as four top chess
grandmasters show improvements in solving the presented concept prototype
positions. This marks an important first milestone in advancing the frontier of
human knowledge by leveraging AI; a development that could bear profound
implications and help us shape how we interact with AI systems across many AI
applications.Comment: 61 pages, 29 figure
- …