150 research outputs found
A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networks
We develop a mathematically rigorous framework for multilayer neural networks
in the mean field regime. As the network's width increases, the network's
learning trajectory is shown to be well captured by a meaningful and
dynamically nonlinear limit (the \textit{mean field} limit), which is
characterized by a system of ODEs. Our framework applies to a broad range of
network architectures, learning dynamics and network initializations. Central
to the framework is the new idea of a \textit{neuronal embedding}, which
comprises of a non-evolving probability space that allows to embed neural
networks of arbitrary widths.
We demonstrate two applications of our framework. Firstly the framework gives
a principled way to study the simplifying effects that independent and
identically distributed initializations have on the mean field limit. Secondly
we prove a global convergence guarantee for two-layer and three-layer networks.
Unlike previous works that rely on convexity, our result requires a certain
universal approximation property, which is a distinctive feature of
infinite-width neural networks. To the best of our knowledge, this is the first
time global convergence is established for neural networks of more than two
layers in the mean field regime
The adhesive contact of viscoelastic spheres
International audienceWe have formulated the restricted self-consistent model for the adhesive contact of linear viscoelastic spheres. This model is a generalization of both the Ting (J. Appl. Mech. 33 (1966) 845) approach to the viscoelastic contact of adhesionless spheres and the restricted self-consistent model for adhesive axisymmetric bodies. We also show how the model can be used in practice by giving a few examples of numerical solutions
Efficient Raman converter in the yellow range with high spatial and spectral brightness
International audienceWe present a Raman converter emitting at 583 nm on the second Stokes order of a line of propan-2-ol pumped by a microlaser at 532 nm in the sub-nanosecond regime. We used a mixture of liquids to adapt the transmission band of a photonic bandgap fiber. The internal conversion efficiency is 67% in photon numbers, and the output power is 1.06 mW, corresponding to a maximum peak power of 338 W. The beam delivered by the converter presents a Gaussian spatial structure and a high spectral brightness, typically more than five times higher than supercontinuum sources in this spectral range
A Deep Learning Architecture with Spatio-Temporal Focusing for Detecting Respiratory Anomalies
This paper presents a deep learning system applied for detecting anomalies
from respiratory sound recordings. Our system initially performs audio feature
extraction using Continuous Wavelet transformation. This transformation
converts the respiratory sound input into a two-dimensional spectrogram where
both spectral and temporal features are presented. Then, our proposed deep
learning architecture inspired by the Inception-residual-based backbone
performs the spatial-temporal focusing and multi-head attention mechanism to
classify respiratory anomalies. In this work, we evaluate our proposed models
on the benchmark SPRSound (The Open-Source SJTU Paediatric Respiratory Sound)
database proposed by the IEEE BioCAS 2023 challenge. As regards the Score
computed by an average between the average score and harmonic score, our robust
system has achieved Top-1 performance with Scores of 0.810, 0.667, 0.744, and
0.608 in Tasks 1-1, 1-2, 2-1, and 2-2, respectively.Comment: arXiv admin note: text overlap with arXiv:2303.0410
Passive Temperature-Compensating Technique for Microstructured Fiber Bragg Gratings
The thermal drift of the characteristic wavelength of fiber Bragg gratings
(FBGs) photowritten in the core of microstructured fibers (MOFs) is
significantly reduced by inserting a liquid of suitable refractive index into
their holes. For instance, the spectral range of variations is divided by a
factor of 4 over a temperature range larger than 20\degree C in a six-hole MOF,
and the maximum sensitivity is reduced. Such passive FBG temperature
compensation technique is of great interest for applications involving accurate
sensing free of thermal effects
An Inception-Residual-Based Architecture with Multi-Objective Loss for Detecting Respiratory Anomalies
This paper presents a deep learning system applied for detecting anomalies
from respiratory sound recordings. Initially, our system begins with audio
feature extraction using Gammatone and Continuous Wavelet transformation. This
step aims to transform the respiratory sound input into a two-dimensional
spectrogram where both spectral and temporal features are presented. Then, our
proposed system integrates Inception-residual-based backbone models combined
with multi-head attention and multi-objective loss to classify respiratory
anomalies. Instead of applying a simple concatenation approach by combining
results from various spectrograms, we propose a Linear combination, which has
the ability to regulate equally the contribution of each individual spectrogram
throughout the training process. To evaluate the performance, we conducted
experiments over the benchmark dataset of SPRSound (The Open-Source SJTU
Paediatric Respiratory Sound) proposed by the IEEE BioCAS 2022 challenge. As
regards the Score computed by an average between the average score and harmonic
score, our proposed system gained significant improvements of 9.7%, 15.8%,
17.8%, and 16.1% in Task 1-1, Task 1-2, Task 2-1, and Task 2-2, respectively,
compared to the challenge baseline system. Notably, we achieved the Top-1
performance in Task 2-1 and Task 2-2 with the highest Score of 74.5% and 53.9%,
respectively
M^2UNet: MetaFormer Multi-scale Upsampling Network for Polyp Segmentation
Polyp segmentation has recently garnered significant attention, and multiple
methods have been formulated to achieve commendable outcomes. However, these
techniques often confront difficulty when working with the complex polyp
foreground and their surrounding regions because of the nature of convolution
operation. Besides, most existing methods forget to exploit the potential
information from multiple decoder stages. To address this challenge, we suggest
combining MetaFormer, introduced as a baseline for integrating CNN and
Transformer, with UNet framework and incorporating our Multi-scale Upsampling
block (MU). This simple module makes it possible to combine multi-level
information by exploring multiple receptive field paths of the shallow decoder
stage and then adding with the higher stage to aggregate better feature
representation, which is essential in medical image segmentation. Taken all
together, we propose MetaFormer Multi-scale Upsampling Network (MUNet) for
the polyp segmentation task. Extensive experiments on five benchmark datasets
demonstrate that our method achieved competitive performance compared with
several previous methods
- …