10,437 research outputs found
On information captured by neural networks: connections with memorization and generalization
Despite the popularity and success of deep learning, there is limited
understanding of when, how, and why neural networks generalize to unseen
examples. Since learning can be seen as extracting information from data, we
formally study information captured by neural networks during training.
Specifically, we start with viewing learning in presence of noisy labels from
an information-theoretic perspective and derive a learning algorithm that
limits label noise information in weights. We then define a notion of unique
information that an individual sample provides to the training of a deep
network, shedding some light on the behavior of neural networks on examples
that are atypical, ambiguous, or belong to underrepresented subpopulations. We
relate example informativeness to generalization by deriving nonvacuous
generalization gap bounds. Finally, by studying knowledge distillation, we
highlight the important role of data and label complexity in generalization.
Overall, our findings contribute to a deeper understanding of the mechanisms
underlying neural network generalization.Comment: PhD thesi
Requirements for Explainability and Acceptance of Artificial Intelligence in Collaborative Work
The increasing prevalence of Artificial Intelligence (AI) in safety-critical
contexts such as air-traffic control leads to systems that are practical and
efficient, and to some extent explainable to humans to be trusted and accepted.
The present structured literature analysis examines n = 236 articles on the
requirements for the explainability and acceptance of AI. Results include a
comprehensive review of n = 48 articles on information people need to perceive
an AI as explainable, the information needed to accept an AI, and
representation and interaction methods promoting trust in an AI. Results
indicate that the two main groups of users are developers who require
information about the internal operations of the model and end users who
require information about AI results or behavior. Users' information needs vary
in specificity, complexity, and urgency and must consider context, domain
knowledge, and the user's cognitive resources. The acceptance of AI systems
depends on information about the system's functions and performance, privacy
and ethical considerations, as well as goal-supporting information tailored to
individual preferences and information to establish trust in the system.
Information about the system's limitations and potential failures can increase
acceptance and trust. Trusted interaction methods are human-like, including
natural language, speech, text, and visual representations such as graphs,
charts, and animations. Our results have significant implications for future
human-centric AI systems being developed. Thus, they are suitable as input for
further application-specific investigations of user needs
Quantum Solutions to the Privacy vs. Utility Tradeoff
In this work, we propose a novel architecture (and several variants thereof)
based on quantum cryptographic primitives with provable privacy and security
guarantees regarding membership inference attacks on generative models. Our
architecture can be used on top of any existing classical or quantum generative
models. We argue that the use of quantum gates associated with unitary
operators provides inherent advantages compared to standard Differential
Privacy based techniques for establishing guaranteed security from all
polynomial-time adversaries
Adversarial Learning in Real-World Fraud Detection: Challenges and Perspectives
Data economy relies on data-driven systems and complex machine learning
applications are fueled by them. Unfortunately, however, machine learning
models are exposed to fraudulent activities and adversarial attacks, which
threaten their security and trustworthiness. In the last decade or so, the
research interest on adversarial machine learning has grown significantly,
revealing how learning applications could be severely impacted by effective
attacks. Although early results of adversarial machine learning indicate the
huge potential of the approach to specific domains such as image processing,
still there is a gap in both the research literature and practice regarding how
to generalize adversarial techniques in other domains and applications. Fraud
detection is a critical defense mechanism for data economy, as it is for other
applications as well, which poses several challenges for machine learning. In
this work, we describe how attacks against fraud detection systems differ from
other applications of adversarial machine learning, and propose a number of
interesting directions to bridge this gap
Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations
The local explanation provides heatmaps on images to explain how
Convolutional Neural Networks (CNNs) derive their output. Due to its visual
straightforwardness, the method has been one of the most popular explainable AI
(XAI) methods for diagnosing CNNs. Through our formative study (S1), however,
we captured ML engineers' ambivalent perspective about the local explanation as
a valuable and indispensable envision in building CNNs versus the process that
exhausts them due to the heuristic nature of detecting vulnerability. Moreover,
steering the CNNs based on the vulnerability learned from the diagnosis seemed
highly challenging. To mitigate the gap, we designed DeepFuse, the first
interactive design that realizes the direct feedback loop between a user and
CNNs in diagnosing and revising CNN's vulnerability using local explanations.
DeepFuse helps CNN engineers to systemically search "unreasonable" local
explanations and annotate the new boundaries for those identified as
unreasonable in a labor-efficient manner. Next, it steers the model based on
the given annotation such that the model doesn't introduce similar mistakes. We
conducted a two-day study (S2) with 12 experienced CNN engineers. Using
DeepFuse, participants made a more accurate and "reasonable" model than the
current state-of-the-art. Also, participants found the way DeepFuse guides
case-based reasoning can practically improve their current practice. We provide
implications for design that explain how future HCI-driven design can move our
practice forward to make XAI-driven insights more actionable.Comment: 32 pages, 6 figures, 5 tables. Accepted for publication in the
Proceedings of the ACM on Human-Computer Interaction (PACM HCI), CSCW 202
HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness
Convolutional Neural Networks (CNN) are known to exhibit poor generalization
performance under distribution shifts. Their generalization have been studied
extensively, and one line of work approaches the problem from a
frequency-centric perspective. These studies highlight the fact that humans and
CNNs might focus on different frequency components of an image. First, inspired
by these observations, we propose a simple yet effective data augmentation
method HybridAugment that reduces the reliance of CNNs on high-frequency
components, and thus improves their robustness while keeping their clean
accuracy high. Second, we propose HybridAugment++, which is a hierarchical
augmentation method that attempts to unify various frequency-spectrum
augmentations. HybridAugment++ builds on HybridAugment, and also reduces the
reliance of CNNs on the amplitude component of images, and promotes phase
information instead. This unification results in competitive to or better than
state-of-the-art results on clean accuracy (CIFAR-10/100 and ImageNet),
corruption benchmarks (ImageNet-C, CIFAR-10-C and CIFAR-100-C), adversarial
robustness on CIFAR-10 and out-of-distribution detection on various datasets.
HybridAugment and HybridAugment++ are implemented in a few lines of code, does
not require extra data, ensemble models or additional networks.Comment: Accepted to ICCV 202
Reinforcement learning in large state action spaces
Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios.
This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory).
In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications
Using machine learning to predict pathogenicity of genomic variants throughout the human genome
Geschätzt mehr als 6.000 Erkrankungen werden durch Veränderungen im Genom verursacht. Ursachen gibt es viele: Eine genomische Variante kann die Translation eines Proteins stoppen, die Genregulation stören oder das Spleißen der mRNA in eine andere Isoform begünstigen. All diese Prozesse müssen überprüft werden, um die zum beschriebenen Phänotyp passende Variante zu ermitteln. Eine Automatisierung dieses Prozesses sind Varianteneffektmodelle. Mittels maschinellem Lernen und Annotationen aus verschiedenen Quellen bewerten diese Modelle genomische Varianten hinsichtlich ihrer Pathogenität.
Die Entwicklung eines Varianteneffektmodells erfordert eine Reihe von Schritten: Annotation der Trainingsdaten, Auswahl von Features, Training verschiedener Modelle und Selektion eines Modells. Hier präsentiere ich ein allgemeines Workflow dieses Prozesses. Dieses ermöglicht es den Prozess zu konfigurieren, Modellmerkmale zu bearbeiten, und verschiedene Annotationen zu testen. Der Workflow umfasst außerdem die Optimierung von Hyperparametern, Validierung und letztlich die Anwendung des Modells durch genomweites Berechnen von Varianten-Scores.
Der Workflow wird in der Entwicklung von Combined Annotation Dependent Depletion (CADD), einem Varianteneffektmodell zur genomweiten Bewertung von SNVs und InDels, verwendet. Durch Etablierung des ersten Varianteneffektmodells für das humane Referenzgenome GRCh38 demonstriere ich die gewonnenen Möglichkeiten Annotationen aufzugreifen und neue Modelle zu trainieren. Außerdem zeige ich, wie Deep-Learning-Scores als Feature in einem CADD-Modell die Vorhersage von RNA-Spleißing verbessern. Außerdem werden Varianteneffektmodelle aufgrund eines neuen, auf Allelhäufigkeit basierten, Trainingsdatensatz entwickelt.
Diese Ergebnisse zeigen, dass der entwickelte Workflow eine skalierbare und flexible Möglichkeit ist, um Varianteneffektmodelle zu entwickeln. Alle entstandenen Scores sind unter cadd.gs.washington.edu und cadd.bihealth.org frei verfügbar.More than 6,000 diseases are estimated to be caused by genomic variants. This can happen in many possible ways: a variant may stop the translation of a protein, interfere with gene regulation, or alter splicing of the transcribed mRNA into an unwanted isoform. It is necessary to investigate all of these processes in order to evaluate which variant may be causal for the deleterious phenotype. A great help in this regard are variant effect scores. Implemented as machine learning classifiers, they integrate annotations from different resources to rank genomic variants in terms of pathogenicity.
Developing a variant effect score requires multiple steps: annotation of the training data, feature selection, model training, benchmarking, and finally deployment for the model's application. Here, I present a generalized workflow of this process. It makes it simple to configure how information is converted into model features, enabling the rapid exploration of different annotations. The workflow further implements hyperparameter optimization, model validation and ultimately deployment of a selected model via genome-wide scoring of genomic variants.
The workflow is applied to train Combined Annotation Dependent Depletion (CADD), a variant effect model that is scoring SNVs and InDels genome-wide. I show that the workflow can be quickly adapted to novel annotations by porting CADD to the genome reference GRCh38. Further, I demonstrate the integration of deep-neural network scores as features into a new CADD model, improving the annotation of RNA splicing events. Finally, I apply the workflow to train multiple variant effect models from training data that is based on variants selected by allele frequency.
In conclusion, the developed workflow presents a flexible and scalable method to train variant effect scores. All software and developed scores are freely available from cadd.gs.washington.edu and cadd.bihealth.org
Introducing Foundation Models as Surrogate Models: Advancing Towards More Practical Adversarial Attacks
Recently, the no-box adversarial attack, in which the attacker lacks access
to the model's architecture, weights, and training data, become the most
practical and challenging attack setup. However, there is an unawareness of the
potential and flexibility inherent in the surrogate model selection process on
no-box setting. Inspired by the burgeoning interest in utilizing foundational
models to address downstream tasks, this paper adopts an innovative idea that
1) recasting adversarial attack as a downstream task. Specifically, image noise
generation to meet the emerging trend and 2) introducing foundational models as
surrogate models. Harnessing the concept of non-robust features, we elaborate
on two guiding principles for surrogate model selection to explain why the
foundational model is an optimal choice for this role. However, paradoxically,
we observe that these foundational models underperform. Analyzing this
unexpected behavior within the feature space, we attribute the lackluster
performance of foundational models (e.g., CLIP) to their significant
representational capacity and, conversely, their lack of discriminative
prowess. To mitigate this issue, we propose the use of a margin-based loss
strategy for the fine-tuning of foundational models on target images. The
experimental results verify that our approach, which employs the basic Fast
Gradient Sign Method (FGSM) attack algorithm, outstrips the performance of
other, more convoluted algorithms. We conclude by advocating for the research
community to consider surrogate models as crucial determinants in the
effectiveness of adversarial attacks in no-box settings. The implications of
our work bear relevance for improving the efficacy of such adversarial attacks
and the overall robustness of AI systems
Improving Adversarial Transferability by Intermediate-level Perturbation Decay
Intermediate-level attacks that attempt to perturb feature representations
following an adversarial direction drastically have shown favorable performance
in crafting transferable adversarial examples. Existing methods in this
category are normally formulated with two separate stages, where a directional
guide is required to be determined at first and the scalar projection of the
intermediate-level perturbation onto the directional guide is enlarged
thereafter. The obtained perturbation deviates from the guide inevitably in the
feature space, and it is revealed in this paper that such a deviation may lead
to sub-optimal attack. To address this issue, we develop a novel
intermediate-level method that crafts adversarial examples within a single
stage of optimization. In particular, the proposed method, named
intermediate-level perturbation decay (ILPD), encourages the intermediate-level
perturbation to be in an effective adversarial direction and to possess a great
magnitude simultaneously. In-depth discussion verifies the effectiveness of our
method. Experimental results show that it outperforms state-of-the-arts by
large margins in attacking various victim models on ImageNet (+10.07% on
average) and CIFAR-10 (+3.88% on average). Our code is at
https://github.com/qizhangli/ILPD-attack.Comment: Revision of ICML '23 submission for better clarit
- …