173 research outputs found
One Explanation Does Not Fit XIL
Current machine learning models produce outstanding results in many areas
but, at the same time, suffer from shortcut learning and spurious correlations.
To address such flaws, the explanatory interactive machine learning (XIL)
framework has been proposed to revise a model by employing user feedback on a
model's explanation. This work sheds light on the explanations used within this
framework. In particular, we investigate simultaneous model revision through
multiple explanation methods. To this end, we identified that \textit{one
explanation does not fit XIL} and propose considering multiple ones when
revising models via XIL
Revision Transformers: Instructing Language Models to Change their Values
Current transformer language models (LM) are large-scale models with billions
of parameters. They have been shown to provide high performances on a variety
of tasks but are also prone to shortcut learning and bias. Addressing such
incorrect model behavior via parameter adjustments is very costly. This is
particularly problematic for updating dynamic concepts, such as moral values,
which vary culturally or interpersonally. In this work, we question the current
common practice of storing all information in the model parameters and propose
the Revision Transformer (RiT) to facilitate easy model updating. The specific
combination of a large-scale pre-trained LM that inherently but also diffusely
encodes world knowledge with a clear-structured revision engine makes it
possible to update the model's knowledge with little effort and the help of
user interaction. We exemplify RiT on a moral dataset and simulate user
feedback demonstrating strong performance in model revision even with small
data. This way, users can easily design a model regarding their preferences,
paving the way for more transparent AI models
Mitigating Inappropriateness in Image Generation: Can there be Value in Reflecting the World's Ugliness?
Text-conditioned image generation models have recently achieved astonishing
results in image quality and text alignment and are consequently employed in a
fast-growing number of applications. Since they are highly data-driven, relying
on billion-sized datasets randomly scraped from the web, they also reproduce
inappropriate human behavior. Specifically, we demonstrate inappropriate
degeneration on a large-scale for various generative text-to-image models, thus
motivating the need for monitoring and moderating them at deployment. To this
end, we evaluate mitigation strategies at inference to suppress the generation
of inappropriate content. Our findings show that we can use models'
representations of the world's ugliness to align them with human preferences
Does CLIP Know My Face?
With the rise of deep learning in various applications, privacy concerns
around the protection of training data has become a critical area of research.
Whereas prior studies have focused on privacy risks in single-modal models, we
introduce a novel method to assess privacy for multi-modal models, specifically
vision-language models like CLIP. The proposed Identity Inference Attack (IDIA)
reveals whether an individual was included in the training data by querying the
model with images of the same person. Letting the model choose from a wide
variety of possible text labels, the model reveals whether it recognizes the
person and, therefore, was used for training. Our large-scale experiments on
CLIP demonstrate that individuals used for training can be identified with very
high accuracy. We confirm that the model has learned to associate names with
depicted individuals, implying the existence of sensitive information that can
be extracted by adversaries. Our results highlight the need for stronger
privacy protection in large-scale models and suggest that IDIAs can be used to
prove the unauthorized use of data for training and to enforce privacy laws.Comment: 15 pages, 6 figure
LEDITS++: Limitless Image Editing using Text-to-Image Models
Text-to-image diffusion models have recently received increasing interest for
their astonishing ability to produce high-fidelity images from solely text
inputs. Subsequent research efforts aim to exploit and apply their capabilities
to real image editing. However, existing image-to-image methods are often
inefficient, imprecise, and of limited versatility. They either require
time-consuming fine-tuning, deviate unnecessarily strongly from the input
image, and/or lack support for multiple, simultaneous edits. To address these
issues, we introduce LEDITS++, an efficient yet versatile and precise textual
image manipulation technique. LEDITS++'s novel inversion approach requires no
tuning nor optimization and produces high-fidelity results with a few diffusion
steps. Second, our methodology supports multiple simultaneous edits and is
architecture-agnostic. Third, we use a novel implicit masking technique that
limits changes to relevant image regions. We propose the novel TEdBench++
benchmark as part of our exhaustive evaluation. Our results demonstrate the
capabilities of LEDITS++ and its improvements over previous methods. The
project page is available at https://leditsplusplus-project.static.hf.space
Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness
Generative AI models have recently achieved astonishing results in quality
and are consequently employed in a fast-growing number of applications.
However, since they are highly data-driven, relying on billion-sized datasets
randomly scraped from the internet, they also suffer from degenerated and
biased human behavior, as we demonstrate. In fact, they may even reinforce such
biases. To not only uncover but also combat these undesired effects, we present
a novel strategy, called Fair Diffusion, to attenuate biases after the
deployment of generative text-to-image models. Specifically, we demonstrate
shifting a bias, based on human instructions, in any direction yielding
arbitrarily new proportions for, e.g., identity groups. As our empirical
evaluation demonstrates, this introduced control enables instructing generative
image models on fairness, with no data filtering and additional training
required
Electrolysis in reduced gravitational environments: current research perspectives and future applications
Electrochemical energy conversion technologies play a crucial role in space missions, for example, in the Environmental Control and Life Support System (ECLSS) on the International Space Station (ISS). They are also vitally important for future long-term space travel for oxygen, fuel and chemical production, where a re-supply of resources from Earth is not possible. Here, we provide an overview of currently existing electrolytic energy conversion technologies for space applications such as proton exchange membrane (PEM) and alkaline electrolyzer systems. We discuss the governing interfacial processes in these devices influenced by reduced gravitation and provide an outlook on future applications of electrolysis systems in, e.g., in-situ resource utilization (ISRU) technologies. A perspective of computational modelling to predict the impact of the reduced gravitational environment on governing electrochemical processes is also discussed and experimental suggestions to better understand efficiency-impacting processes such as gas bubble formation and detachment in reduced gravitational environments are outlined
- …