Improve Generalization Ability of Deep Wide Residual Network with A Suitable Scaling Factor

Deep Residual Neural Networks (ResNets) have demonstrated remarkable success across a wide range of real-world applications. In this paper, we identify a suitable scaling factor (denoted by $\alpha$) on the residual branch of deep wide ResNets to achieve good generalization ability. We show that if $\alpha$ is a constant, the class of functions induced by Residual Neural Tangent Kernel (RNTK) is asymptotically not learnable, as the depth goes to infinity. We also highlight a surprising phenomenon: even if we allow $\alpha$ to decrease with increasing depth $L$, the degeneration phenomenon may still occur. However, when $\alpha$ decreases rapidly with $L$, the kernel regression with deep RNTK with early stopping can achieve the minimax rate provided that the target regression function falls in the reproducing kernel Hilbert space associated with the infinite-depth RNTK. Our simulation studies on synthetic data and real classification tasks such as MNIST, CIFAR10 and CIFAR100 support our theoretical criteria for choosing $\alpha$

Generalization Ability of Wide Residual Networks

In this paper, we study the generalization ability of the wide residual network on $\mathbb{S}^{d-1}$ with the ReLU activation function. We first show that as the width $m\rightarrow\infty$, the residual network kernel (RNK) uniformly converges to the residual neural tangent kernel (RNTK). This uniform convergence further guarantees that the generalization error of the residual network converges to that of the kernel regression with respect to the RNTK. As direct corollaries, we then show $i)$ the wide residual network with the early stopping strategy can achieve the minimax rate provided that the target regression function falls in the reproducing kernel Hilbert space (RKHS) associated with the RNTK; $ii)$ the wide residual network can not generalize well if it is trained till overfitting the data. We finally illustrate some experiments to reconcile the contradiction between our theoretical result and the widely observed ``benign overfitting phenomenon''Comment: 28 pages, 3 figure

Statistical Optimality of Deep Wide Neural Networks

In this paper, we consider the generalization ability of deep wide feedforward ReLU neural networks defined on a bounded domain $\mathcal X \subset \mathbb R^{d}$. We first demonstrate that the generalization ability of the neural network can be fully characterized by that of the corresponding deep neural tangent kernel (NTK) regression. We then investigate on the spectral properties of the deep NTK and show that the deep NTK is positive definite on $\mathcal{X}$ and its eigenvalue decay rate is $(d+1)/d$. Thanks to the well established theories in kernel regression, we then conclude that multilayer wide neural networks trained by gradient descent with proper early stopping achieve the minimax rate, provided that the regression function lies in the reproducing kernel Hilbert space (RKHS) associated with the corresponding NTK. Finally, we illustrate that the overfitted multilayer wide neural networks can not generalize well on $\mathbb S^{d}$. We believe our technical contributions in determining the eigenvalue decay rate of NTK on $\mathbb R^{d}$ might be of independent interests

Functional Slicing-free Inverse Regression via Martingale Difference Divergence Operator

Functional sliced inverse regression (FSIR) is one of the most popular algorithms for functional sufficient dimension reduction (FSDR). However, the choice of slice scheme in FSIR is critical but challenging. In this paper, we propose a new method called functional slicing-free inverse regression (FSFIR) to estimate the central subspace in FSDR. FSFIR is based on the martingale difference divergence operator, which is a novel metric introduced to characterize the conditional mean independence of a functional predictor on a multivariate response. We also provide a specific convergence rate for the FSFIR estimator. Compared with existing functional sliced inverse regression methods, FSFIR does not require the selection of a slice number. Simulations demonstrate the efficiency and convenience of FSFIR

Demonstration of Performance Improvement in Multi-User NOMA VLC System Using Joint Transceiver Optimization

The bit error ratio (BER) performance of a non-orthogonal multiple access (NOMA) visible light communication (VLC) system is poor due to the unequal distances between adjacent points in the superposition constellation (SC). In this paper, we propose a novel scheme to improve the BER performance by adjusting parameters to change the shape of SC at the transmitter and by adjusting the parameters of successive interference cancellation (SIC) decoding at the receiver simultaneously, which is called a SC and SIC adjustment (SC-SIC-A) scheme. For multi-user NOMA VLC system, we derive the closed-form BER expression for each user, where the modulation format is four-quadrature amplitude modulation. According to the derived BER expressions, we formulate an optimization problem that minimizes the average BER for all users by adjusting the obtained parameters of SC and SIC decoding via differential evolution algorithm. The improvement of capacity performance is investigated consequently. In order to verify the feasibility and effectiveness of the proposed SC-SIC-A scheme, we carried out theoretical analysis, Monte Carlo simulation and experiments of two-user and three-user NOMA VLC systems. Results show that the SC-SIC-A scheme outperforms the existing schemes in NOMA VLC system, where the signal-to-noise ratio (SNR) reductions to achieve BER of 10−3 are 1.3 dB and 0.8 dB for both users in the two-user NOMA VLC system, respectively, and the SNR reductions to achieve BER of 10−3 are 5.7 dB, 4.3 dB and 4.6 dB for all users in the three-user NOMA VLC system, respectively

Demonstration of Performance Improvement in Multi-User NOMA VLC System Using Joint Transceiver Optimization

The bit error ratio (BER) performance of a non-orthogonal multiple access (NOMA) visible light communication (VLC) system is poor due to the unequal distances between adjacent points in the superposition constellation (SC). In this paper, we propose a novel scheme to improve the BER performance by adjusting parameters to change the shape of SC at the transmitter and by adjusting the parameters of successive interference cancellation (SIC) decoding at the receiver simultaneously, which is called a SC and SIC adjustment (SC-SIC-A) scheme. For multi-user NOMA VLC system, we derive the closed-form BER expression for each user, where the modulation format is four-quadrature amplitude modulation. According to the derived BER expressions, we formulate an optimization problem that minimizes the average BER for all users by adjusting the obtained parameters of SC and SIC decoding via differential evolution algorithm. The improvement of capacity performance is investigated consequently. In order to verify the feasibility and effectiveness of the proposed SC-SIC-A scheme, we carried out theoretical analysis, Monte Carlo simulation and experiments of two-user and three-user NOMA VLC systems. Results show that the SC-SIC-A scheme outperforms the existing schemes in NOMA VLC system, where the signal-to-noise ratio (SNR) reductions to achieve BER of 10−3 are 1.3 dB and 0.8 dB for both users in the two-user NOMA VLC system, respectively, and the SNR reductions to achieve BER of 10−3 are 5.7 dB, 4.3 dB and 4.6 dB for all users in the three-user NOMA VLC system, respectively

Segmental Ureterectomy is Acceptable for High-risk Ureteral Carcinoma Comparing to Radical Nephroureterectomy

Purpose: To determine if segmental ureterectomy (SU) could be chosen for wider oncological indications than low-risk ureteral carcinoma, given the difficulties in accurate preoperative risk stratification determination and kidney-sparing needs for successive therapy. Methods: Data from ureteral carcinoma patients who underwent open SU or laparoscopic radical nephroureterectomy (RNU) between 2011 and 2016 were retrospectively reviewed. Kaplan–Meier survival analysis and Cox regression model with patients' baseline characteristics (age, bladder cancer history, hydronephrosis), procedure type, and tumor characteristics (site, size, pathological features) as covariates were used to evaluate oncological outcomes. Life quality parameters including preoperative renal function, Karnofsky performance status, pain score, and surgical complications were set as second endpoints. Results: Sixty-three patients (24 in SU group, 39 in RNU group) who had at least one high-risk factor were enrolled. In the mean follow-up time of 24.67 months, no significant difference was found in recurrence-free survival (66.7% and 69.2%, p = 0.798), overall survival (79.2% and 84.6%, p = 0.453), and cancer-specific survival (83.3% and 89.7%, p = 0.405) between SU and RNU groups. The Cox regression demonstrated that procedure type was not associated with oncological outcomes. Patients in SU group experienced significant mean estimated glomerular filtration rate (eGFR) increase by 4.60 ml/(min·1.73 m2) (p < 0.001). Proportion of patients having poor eGFR also decreased postoperatively in SU group. Mere tendency in physical performance status improvement and serious complications reduction was detected in SU group. Conclusion: SU is acceptable for high-risk ureteral carcinoma comparing to RNU with satisfying tumor control efficacy and advantage in renal function preservation

Bismuthene Arrays Harvesting Reversible Plating‐Alloying Electrochemistry Toward Robust Lithium Metal Batteries

3D lithiophilic skeletons have attracted enormous attention in homogenizing local current distribution and optimizing metal deposition in the pursuit of robust Li metal anodes. Nonetheless, their practicability is markedly plagued by the cumbersome production routes and mediocre Coulombic efficiency (CE) of Li plating/stripping. Herein, scalable in situ growth of uniform bismuthene arrays over commercial Cu foam via spontaneous galvanic replacement reaction is demonstrated. Exhaustive structural/electrochemical measurements in combination with theoretical calculations collectively disclose the reversible plating‐alloying mechanism, wherein the formed Li3Bi alloy interphase aids to lower the Li nucleation overpotential and elevate the CE performance. The thus‐designed Li metal electrode sustains a stable cyclic operation at 1 mA cm−2/1 mAh cm−2 for 1600 h. When paired with LiFePO4 and sulfur cathodes, the Li metal batteries enable gratifying rate capability and cycling durability. This straightforward maneuver opens a new frontier in the scalable manufacturing of pragmatic current collectors in an economic fashion

Achieving high energy storage properties in perovskite oxide via high-entropy design

In recent years, “high-entropy” materials have attracted great attention in various fields due to their unique design concepts and crystal structures. The definition of high-entropy is also more diverse, gradually expanding from a single phase with an equal molar ratio to a multi-phase with a non-equimolar ratio. This study selected (Na0.2Bi0.2Ba0.2Sr0.2Ca0.2)TiO3 high entropy ceramics with excellent relaxation behavior. The A-site elements are divided into (x = Na, Bi, Ba) and ((1-3x)/2 = Sr, Ca) according to their inherent properties. A novel ABO3 structural energy storage ceramics (NaBaBi)x(SrCa)(1-3x)/2TiO3 (x = 0.19, 0.195, 0.2, 0.205 and 0.21) was successfully fabricated using the high entropy design concept. The ferroelectric and dielectric properties of non-equimolar ratio high-entropy ceramics were studied in detail. It was found that the dielectric constant of ∼4920 and the recoverable energy storage density of 3.86 J/cm3 (at 335 kV/cm) can be achieved simultaneously at x = 0.205. The results indicate that the design concept of high-entropy materials with a non-equal molar ratio is an effective means to achieve distinguished energy storage performance in lead-free ceramics.</p