161 research outputs found
Model-Based Deep Learning
Signal processing, communications, and control have traditionally relied on
classical statistical modeling techniques. Such model-based methods utilize
mathematical formulations that represent the underlying physics, prior
information and additional domain knowledge. Simple classical models are useful
but sensitive to inaccuracies and may lead to poor performance when real
systems display complex or dynamic behavior. On the other hand, purely
data-driven approaches that are model-agnostic are becoming increasingly
popular as datasets become abundant and the power of modern deep learning
pipelines increases. Deep neural networks (DNNs) use generic architectures
which learn to operate from data, and demonstrate excellent performance,
especially for supervised problems. However, DNNs typically require massive
amounts of data and immense computational resources, limiting their
applicability for some signal processing scenarios. We are interested in hybrid
techniques that combine principled mathematical models with data-driven systems
to benefit from the advantages of both approaches. Such model-based deep
learning methods exploit both partial domain knowledge, via mathematical
structures designed for specific problems, as well as learning from limited
data. In this article we survey the leading approaches for studying and
designing model-based deep learning systems. We divide hybrid
model-based/data-driven systems into categories based on their inference
mechanism. We provide a comprehensive review of the leading approaches for
combining model-based algorithms with deep learning in a systematic manner,
along with concrete guidelines and detailed signal processing oriented examples
from recent literature. Our aim is to facilitate the design and study of future
systems on the intersection of signal processing and machine learning that
incorporate the advantages of both domains
DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds
We present a new deep unfolding network for analysis-sparsity-based
Compressed Sensing. The proposed network coined Decoding Network (DECONET)
jointly learns a decoder that reconstructs vectors from their incomplete, noisy
measurements and a redundant sparsifying analysis operator, which is shared
across the layers of DECONET. Moreover, we formulate the hypothesis class of
DECONET and estimate its associated Rademacher complexity. Then, we use this
estimate to deliver meaningful upper bounds for the generalization error of
DECONET. Finally, the validity of our theoretical results is assessed and
comparisons to state-of-the-art unfolding networks are made, on both synthetic
and real-world datasets. Experimental results indicate that our proposed
network outperforms the baselines, consistently for all datasets, and its
behaviour complies with our theoretical findings.Comment: Accepted in IEEE Transactions on Signal Processin
Deep learning methods for solving linear inverse problems: Research directions and paradigms
The linear inverse problem is fundamental to the development of various scientific areas. Innumerable attempts have been carried out to solve different variants of the linear inverse problem in different applications. Nowadays, the rapid development of deep learning provides a fresh perspective for solving the linear inverse problem, which has various well-designed network architectures results in state-of-the-art performance in many applications. In this paper, we present a comprehensive survey of the recent progress in the development of deep learning for solving various linear inverse problems. We review how deep learning methods are used in solving different linear inverse problems, and explore the structured neural network architectures that incorporate knowledge used in traditional methods. Furthermore, we identify open challenges and potential future directions along this research line
Generalization analysis of an unfolding network for analysis-based Compressed Sensing
Unfolding networks have shown promising results in the Compressed Sensing
(CS) field. Yet, the investigation of their generalization ability is still in
its infancy. In this paper, we perform generalization analysis of a
state-of-the-art ADMM-based unfolding network, which jointly learns a decoder
for CS and a sparsifying redundant analysis operator. To this end, we first
impose a structural constraint on the learnable sparsifier, which parametrizes
the network's hypothesis class. For the latter, we estimate its Rademacher
complexity. With this estimate in hand, we deliver generalization error bounds
for the examined network. Finally, the validity of our theory is assessed and
numerical comparisons to a state-of-the-art unfolding network are made, on
synthetic and real-world datasets. Our experimental results demonstrate that
our proposed framework complies with our theoretical findings and outperforms
the baseline, consistently for all datasets
- …