167 research outputs found
Deep Convolutional Neural Networks Based on Semi-Discrete Frames
Deep convolutional neural networks have led to breakthrough results in
practical feature extraction applications. The mathematical analysis of these
networks was pioneered by Mallat, 2012. Specifically, Mallat considered
so-called scattering networks based on identical semi-discrete wavelet frames
in each network layer, and proved translation-invariance as well as deformation
stability of the resulting feature extractor. The purpose of this paper is to
develop Mallat's theory further by allowing for different and, most
importantly, general semi-discrete frames (such as, e.g., Gabor frames,
wavelets, curvelets, shearlets, ridgelets) in distinct network layers. This
allows to extract wider classes of features than point singularities resolved
by the wavelet transform. Our generalized feature extractor is proven to be
translation-invariant, and we develop deformation stability results for a
larger class of deformations than those considered by Mallat. For Mallat's
wavelet-based feature extractor, we get rid of a number of technical
conditions. The mathematical engine behind our results is continuous frame
theory, which allows us to completely detach the invariance and deformation
stability proofs from the particular algebraic structure of the underlying
frames.Comment: Proc. of IEEE International Symposium on Information Theory (ISIT),
Hong Kong, China, June 2015, to appea
On the Inductive Bias of Neural Tangent Kernels
State-of-the-art neural networks are heavily over-parameterized, making the
optimization algorithm a crucial ingredient for learning predictive models with
good generalization properties. A recent line of work has shown that in a
certain over-parameterized regime, the learning dynamics of gradient descent
are governed by a certain kernel obtained at initialization, called the neural
tangent kernel. We study the inductive bias of learning in such a regime by
analyzing this kernel and the corresponding function space (RKHS). In
particular, we study smoothness, approximation, and stability properties of
functions with finite norm, including stability to image deformations in the
case of convolutional networks, and compare to other known kernels for similar
architectures.Comment: NeurIPS 201
Neural Networks for Constitutive Modeling -- From Universal Function Approximators to Advanced Models and the Integration of Physics
Analyzing and modeling the constitutive behavior of materials is a core area
in materials sciences and a prerequisite for conducting numerical simulations
in which the material behavior plays a central role. Constitutive models have
been developed since the beginning of the 19th century and are still under
constant development. Besides physics-motivated and phenomenological models,
during the last decades, the field of constitutive modeling was enriched by the
development of machine learning-based constitutive models, especially by using
neural networks. The latter is the focus of the present review, which aims to
give an overview of neural networks-based constitutive models from a methodical
perspective. The review summarizes and compares numerous conceptually different
neural networks-based approaches for constitutive modeling including neural
networks used as universal function approximators, advanced neural network
models and neural network approaches with integrated physical knowledge. The
upcoming of these methods is in-turn closely related to advances in the area of
computer sciences, what further adds a chronological aspect to this review. We
conclude this review paper with important challenges in the field of learning
constitutive relations that need to be tackled in the near future
- …