7 research outputs found
Information-Theoretic Analysis of Unsupervised Domain Adaptation
This paper uses information-theoretic tools to analyze the generalization
error in unsupervised domain adaptation (UDA). We present novel upper bounds
for two notions of generalization errors. The first notion measures the gap
between the population risk in the target domain and that in the source domain,
and the second measures the gap between the population risk in the target
domain and the empirical risk in the source domain. While our bounds for the
first kind of error are in line with the traditional analysis and give similar
insights, our bounds on the second kind of error are algorithm-dependent, which
also provide insights into algorithm designs. Specifically, we present two
simple techniques for improving generalization in UDA and validate them
experimentally
MLN-net: A multi-source medical image segmentation method for clustered microcalcifications using multiple layer normalization
Accurate segmentation of clustered microcalcifications in mammography is
crucial for the diagnosis and treatment of breast cancer. Despite exhibiting
expert-level accuracy, recent deep learning advancements in medical image
segmentation provide insufficient contribution to practical applications, due
to the domain shift resulting from differences in patient postures, individual
gland density, and imaging modalities of mammography etc. In this paper, a
novel framework named MLN-net, which can accurately segment multi-source images
using only single source images, is proposed for clustered microcalcification
segmentation. We first propose a source domain image augmentation method to
generate multi-source images, leading to improved generalization. And a
structure of multiple layer normalization (LN) layers is used to construct the
segmentation network, which can be found efficient for clustered
microcalcification segmentation in different domains. Additionally, a branch
selection strategy is designed for measuring the similarity of the source
domain data and the target domain data. To validate the proposed MLN-net,
extensive analyses including ablation experiments are performed, comparison of
12 baseline methods. Extensive experiments validate the effectiveness of
MLN-net in segmenting clustered microcalcifications from different domains and
the its segmentation accuracy surpasses state-of-the-art methods. Code will be
available at https://github.com/yezanting/MLN-NET-VERSON1.Comment: 17 pages, 9 figures, 3 table
Generalization Bounds: Perspectives from Information Theory and PAC-Bayes
A fundamental question in theoretical machine learning is generalization.
Over the past decades, the PAC-Bayesian approach has been established as a
flexible framework to address the generalization capabilities of machine
learning algorithms, and design new ones. Recently, it has garnered increased
interest due to its potential applicability for a variety of learning
algorithms, including deep neural networks. In parallel, an
information-theoretic view of generalization has developed, wherein the
relation between generalization and various information measures has been
established. This framework is intimately connected to the PAC-Bayesian
approach, and a number of results have been independently discovered in both
strands. In this monograph, we highlight this strong connection and present a
unified treatment of generalization. We present techniques and results that the
two perspectives have in common, and discuss the approaches and interpretations
that differ. In particular, we demonstrate how many proofs in the area share a
modular structure, through which the underlying ideas can be intuited. We pay
special attention to the conditional mutual information (CMI) framework;
analytical studies of the information complexity of learning algorithms; and
the application of the proposed methods to deep learning. This monograph is
intended to provide a comprehensive introduction to information-theoretic
generalization bounds and their connection to PAC-Bayes, serving as a
foundation from which the most recent developments are accessible. It is aimed
broadly towards researchers with an interest in generalization and theoretical
machine learning.Comment: 222 page