Achieve the Minimum Width of Neural Networks for Universal Approximation

The universal approximation property (UAP) of neural networks is fundamental for deep learning, and it is well known that wide neural networks are universal approximators of continuous functions within both the $L^p$ norm and the continuous/uniform norm. However, the exact minimum width, $w_{\min}$, for the UAP has not been studied thoroughly. Recently, using a decoder-memorizer-encoder scheme, \citet{Park2021Minimum} found that $w_{\min} = \max(d_x+1,d_y)$ for both the $L^p$-UAP of ReLU networks and the $C$-UAP of ReLU+STEP networks, where $d_x,d_y$ are the input and output dimensions, respectively. In this paper, we consider neural networks with an arbitrary set of activation functions. We prove that both $C$-UAP and $L^p$-UAP for functions on compact domains share a universal lower bound of the minimal width; that is, $w^*_{\min} = \max(d_x,d_y)$. In particular, the critical width, $w^*_{\min}$, for $L^p$-UAP can be achieved by leaky-ReLU networks, provided that the input or output dimension is larger than one. Our construction is based on the approximation power of neural ordinary differential equations and the ability to approximate flow maps by neural networks. The nonmonotone or discontinuous activation functions case and the one-dimensional case are also discussed

Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation

The study of universal approximation properties (UAP) for neural networks (NN) has a long history. When the network width is unlimited, only a single hidden layer is sufficient for UAP. In contrast, when the depth is unlimited, the width for UAP needs to be not less than the critical width $w^*_{\min}=\max(d_x,d_y)$, where $d_x$ and $d_y$ are the dimensions of the input and output, respectively. Recently, \cite{cai2022achieve} shows that a leaky-ReLU NN with this critical width can achieve UAP for $L^p$ functions on a compact domain $K$, \emph{i.e.,} the UAP for $L^p(K,\mathbb{R}^{d_y})$. This paper examines a uniform UAP for the function class $C(K,\mathbb{R}^{d_y})$ and gives the exact minimum width of the leaky-ReLU NN as $w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1}$, which involves the effects of the output dimensions. To obtain this result, we propose a novel lift-flow-discretization approach that shows that the uniform UAP has a deep connection with topological theory.Comment: ICML2023 camera read

MFMAN-YOLO: A Method for Detecting Pole-like Obstacles in Complex Environment

In real-world traffic, there are various uncertainties and complexities in road and weather conditions. To solve the problem that the feature information of pole-like obstacles in complex environments is easily lost, resulting in low detection accuracy and low real-time performance, a multi-scale hybrid attention mechanism detection algorithm is proposed in this paper. First, the optimal transport function Monge-Kantorovich (MK) is incorporated not only to solve the problem of overlapping multiple prediction frames with optimal matching but also the MK function can be regularized to prevent model over-fitting; then, the features at different scales are up-sampled separately according to the optimized efficient multi-scale feature pyramid. Finally, the extraction of multi-scale feature space channel information is enhanced in complex environments based on the hybrid attention mechanism, which suppresses the irrelevant complex environment background information and focuses the feature information of pole-like obstacles. Meanwhile, this paper conducts real road test experiments in a variety of complex environments. The experimental results show that the detection precision, recall, and average precision of the method are 94.7%, 93.1%, and 97.4%, respectively, and the detection frame rate is 400 f/s. This research method can detect pole-like obstacles in a complex road environment in real time and accurately, which further promotes innovation and progress in the field of automatic driving.Comment: 11 page

A Robotic Training System for Studies of Post-SCI Stand Rehabilitation

This paper describes a robotic training system designed to rehabilitate the standing ability of mice after spinal cord injury(SCI). The system is composed of a 6 Degree-of-Freedom (DOF) parallel mechanism, an active weight support system, and other measuring equipments which can monitor the response of the mouse. Preliminary experiments showed that the mouse could generate a certain degree of weight-support stand response during the training

Many-Body Chiral Edge Currents and Sliding Phases of Atomic Spin Waves in Momentum-Space Lattice

Collective excitations (spinwaves) of long-lived atomic hyperfine states can be synthesized into a Bose-Hubbard model in momentum space. We explore many-body ground states and dynamics of a two-leg momentum-space lattice formed by two coupled hyperfine states. Essential ingredients of this setting are a staggered artificial magnetic field engineered by lasers that couple the spinwave states, and a state-dependent long-range interaction, which is induced by laser-dressing a hyperfine state to a Rydberg state. The Rydberg dressed two-body interaction gives rise to a state-dependent blockade in momentum space, and can amplify staggered flux induced anti-chiral edge currents in the many-body ground state in the presence of magnetic flux. When the Rydberg dressing is applied to both hyperfine states, exotic sliding insulating and superfluid/supersolid phases emerge. Due to the Rydberg dressed long-range interaction, spinwaves slide along a leg of the momentum-space lattice without costing energy. Our study paves a route to the quantum simulation of topological phases and exotic dynamics with interacting spinwaves of atomic hyperfine states in momentum-space lattice. Introduction-Chiral edge states have played an important role in understanding quantum Hall effects [1-3] in solid state materials [4-6]. Ultracold atoms exposed to artificial gauge fields provide an ideal platform to simulate chiral edge currents in and out of equilibrium. This is driven by the ability to precisely control and in-situ monitor [7, 8] internal and external degrees of freedom, and atom-atom interactions [9]. Chiral dynamics [10-13] has been examined in the continuum space [14, 15], ladders [16-20], and optical lattices [21-28]. However, chiral states realized in the coordinate space require extremely low temperatures (typical in the order of a few kilo Hz) to protect the topological states from being destroyed by motional fluctuations [13]. Up to now, experimental observations of chiral phenomena in ultracold gases are largely at a single-particle level, due to unavoidable dis-sipations (e.g. spontaneous emission and heating) [9, 29-33], while the realization of many-body chiral edge currents in ultracold atoms is still elusive

Isolation and Characterization of 89K Pathogenicity Island-Positive ST-7 Strains of Streptococcus suis Serotype 2 from Healthy Pigs, Northeast China

Streptococcus suis is a swine pathogen which can also cause severe infection, such as meningitis, and streptococcal-like toxic shock syndrome (STSS), in humans. In China, most of the S. suis infections in humans were reported in the southern areas with warm and humid climates, but little attention had been paid to the northern areas. Data presented here showed that the virulent serotypes 1, 2, 7, and 9 of S. suis could be steadily isolated from the healthy pigs in the pig farms in all the three provinces of Northeast China. Notably, a majority of the serotype 2 isolates belonged to the 89K pathogenicity island-positive ST-7 clone that had historically caused the human STSS outbreaks in the Sichuan and Jiangsu provinces of China, although the human STSS case caused by S. suis had never been reported in northern areas of China. Data presented here indicated that the survey of S. suis should be expanded to or reinforced in the northern areas of China

DynaComm: Accelerating Distributed CNN Training between Edges and Clouds through Dynamic Communication Scheduling

To reduce uploading bandwidth and address privacy concerns, deep learning at the network edge has been an emerging topic. Typically, edge devices collaboratively train a shared model using real-time generated data through the Parameter Server framework. Although all the edge devices can share the computing workloads, the distributed training processes over edge networks are still time-consuming due to the parameters and gradients transmission procedures between parameter servers and edge devices. Focusing on accelerating distributed Convolutional Neural Networks (CNNs) training at the network edge, we present DynaComm, a novel scheduler that dynamically decomposes each transmission procedure into several segments to achieve optimal layer-wise communications and computations overlapping during run-time. Through experiments, we verify that DynaComm manages to achieve optimal layer-wise scheduling for all cases compared to competing strategies while the model accuracy remains untouched.Comment: 16 pages, 12 figures. IEEE Journal on Selected Areas in Communication

Effects of Assist-as-needed Robotic Training Paradigms on the Locomotor Recovery of Adult Spinal Mice

This paper introduces a new “assist-as needed” (AAN) training paradigm for rehabilitation of spinal cord injuries via robotic training devices. In the pilot study reported in this paper, nine female adult Swiss-Webster mice were divided into three groups, each experiencing a different robotic training control strategy: a fixed training trajectory (Fixed Group, A), an AAN training method without inter-limb coordination (Band Group, B), and an AAN training method with bilateral hindlimb coordination (Window Group, C). Fourteen days after complete transection at the mid-thoracic level, the mice were robotically trained to step in the presence of an acutely administered serotonin agonist, quipazine, for a period of six weeks. The mice that received AAN training (Groups B and C) show higher levels of recovery than Group A mice, as measured by the number, consistency, and periodicity of steps realized during testing sessions. Group C displays a higher incidence of alternating stepping than Group B. These results indicate that this training approach may be more effective than fixed trajectory paradigms in promoting robust post-injury stepping behavior. Furthermore, the constraint of inter-limb coordination appears to be an important contribution to successful training. Presented in this paper are also some preliminary results from a recent full-scale study that complements the conclusions from this pilot study