The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We systematically design various experiments to verify the benefits of the anisotropic noise, compared with full gradient descent plus isotropic diffusion (i.e. Langevin dynamics).Comment: ICML 2019 camera read

Open and increasing paths on N-ary trees with different fitness values

Consider a rooted N-ary tree. For every vertex of this tree, we atttach an i.i.d. Bernoulli random variable. A path is called open if all the random variables that are assigned on the path are 1. We consider limiting behaviors for the number of open paths from the root to leaves and the longest open path. In addition, when all fitness values are i.i.d. continuous random variables, some asymptotic properties of the longest increasing path are proved

Combined effects of tenofovir and interferon α1b on viral load and levels of peripheral regulatory T cells in chronic hepatitis B subjects

Purpose: To study the combined effects of tenofovir and interferon α1b on viral load and peripheral blood regulatory T cell concentrations of chronic hepatitis B (CHB) subjects. Methods: Patients with chronic hepatitis B (86 cases) were randomly assigned to two groups: control group and study group. In control subjects, tenofovir was given orally (300 mg/kg bwt/day). In addition to tenofovir, the study group received interferon α1b injection intramuscularly at a dose of 50 μg/kg thrice a week. Liver function, serum hepatitis B viral (HBV) load, and serum levels of peripheral blood regulatory T-lymphocytes were determined. Clinical effectiveness and adverse reactions in both groups were also assessed. Results: After treatment, total effectiveness was higher in the study group (86.04 %) than in control patients (62.79 %) (p < 0.05). Serum aspartate transaminase (AST), alanine aminotransferase (ALT) and total bilirubin (TBIL) significantly decreased in the study group, relative to control, but HBV DNAnegative, HbeAg-negative and HbsAg-negative cells were markedly higher in patients in the study group (p < 0.05). Moreover, there were higher CD4+ T and CD8+ T counts, and CD4+ T/CD8+ T ratio in study subjects than in control subjects (p < 0.05). Conclusion: The combination of tenofovir with interferon α1b effectively improves liver functions in patients with CHB, reduces viral load, and exerts anti-HBV effect by regulating the levels of peripheral blood T-lymphocytes

Multi-sensor Image Data Fusion based on Pixel-Level Weights of Wavelet and the PCA Transform

Abstract -The goal of image fusion is to create new images that are more suitable for the purposes of human visual perception, object detection and target recognition. For Automatic Target Recognition (ATR), we can use multi-sensor data including visible and infrared images to increase the recognition rate. In this paper, we propose a new multiresolution data fusion scheme based on the principal component analysis (PCA) transform and the pixel-level weights wavelet transform including thermal weights and visual weights. In order to get a more ideal fusion result, a linear local mapping which based on the PCA is used to create a new "origin" image of the image fusion. We use multiresolution decompositions to represent the input images at different scales, present a multiresolution/ multimodal segmentation to partition the image domain at these scales. The crucial idea is to use this segmentation to guide the fusion process. Physical thermal weights and perceptive visual weights are used as segmentation multimodals. Daubechies Wavelet is choosen as the Wavelet Basis. Experimental results confirm that the proposed algorithm is the best image sharpening method and can best maintain the spectral information of the original infrared image. Also, the proposed technique performs better than the other ones in the literature, more robust and effective, from both subjective visual effects and objective statistical analysis results

Analysis of small RNAs revealed differential expressions during pollen and embryo sac development in autotetraploid rice

Protein-protein interaction of meiosis-related genes with the targets predicted by the DEM associated with meiosis. Table S18. 21 nt-phasiRNAs triggered by the miR2118. Table S19. 24 nt-phasiRNAs triggered by the miR2275. Table S20. Overview of 24 nt TEs-siRNAs during pollen and embryo sac development of 02428-4x and 02428-2x. Table S21. Distribution of 24 nt TEs-siRNAs in autotetraploid and diploid rice. Table S22. Differentially expressed 24 nt TEs-siRNAs during pollen and embryo sac development of autotetraploid rice. Table S23. Anther length during pollen development stages in autotetraploid and diploid rice. Table S24. The stem–loop RT primers used in the present study. (XLSX 1012 kb

Laser Ablation of Electrode to Extend RSW Electrode Life and Save Production Cost

This invention presents an RSW electrode cost saving solution for resistance spot welding of aluminums, including apparatus with automation, ablation process and method. It is achieved by cleaning with a laser ablation process that minimizes the metal removal amount each time compared to a current mechanical electrode dressing method (Laser ablation 0.02mm vs electrode dressing 0.2mm). To make it effective and reproductive, the laser ablation process is preferably performed on the welding electrode prior to serious electrode sticking. RSW on various aluminum grades and joint stacks have been tested and validated in a lab scale, it has shown that the electrode life with laser ablation lasts at least 10 times longer in total than that of current electrode dressing method

Effect of maternal serum albumin level on birthweight and gestational age: an analysis of 39200 singleton newborns

BackgroundSerum albumin plays a pivotal role in regulating plasma oncotic pressure and modulating fluid distribution among various body compartments. Previous research examining the association between maternal serum albumin levels and fetal growth yielded limited and inconclusive findings. Therefore, the specific influence of serum albumin on fetal growth remains poorly understood and warrants further investigation.MethodsA retrospective study involved 39200 women who had a singleton live birth at a tertiary-care academic medical center during the period from January 2017 to December 2020. Women were categorized into four groups according to the quartile of albumin concentration during early pregnancy: Q1 group, ≤41.0 g/L; Q2 group, 41.1-42.6 g/L; Q3 group, 42.7-44.3 g/L and Q4 group, >44.3 g/L. The main outcome measures were mid-term estimated fetal weight, birthweight and gestational age. Multivariate linear and logistic regression analysis were performed to detect the independent effect of maternal serum albumin level on fetal growth after adjusting for important confounding variables.ResultsIn the crude analysis, a significant inverse correlation was found between early pregnancy maternal serum albumin levels and fetal growth status, including mid-term ultrasound measurements, mid-term estimated fetal weight, birthweight, and gestational age. After adjustment for a number of confounding factors, mid-term estimated fetal weight, birthweight, and birth height decreased significantly with increasing albumin levels. Compared to the Q2 group, the Q4 group had higher rates of preterm birth (aOR, 1.16; 95% CI, 1.01–1.34), small-for-gestational-age (aOR, 1.27; 95% CI, 1.11–1.45) and low birthweight (aOR, 1.41; 95% CI, 1.18–1.69), and lower rate of large-for-gestational-age (aOR, 0.85; 95% CI, 0.78–0.94). Moreover, to achieve the optimal neonatal outcome, women with higher early pregnancy albumin levels required a greater reduction in albumin levels in later pregnancy stages.ConclusionsA higher maternal serum albumin level during early pregnancy was associated with poor fetal growth, with the detrimental effects becoming apparent as early as the mid-gestation period. These findings provided vital information for clinicians to predict fetal growth status and identify cases with a high risk of adverse neonatal outcomes early on

Radio Jet Feedback and Star Formation in Heavily Obscured Quasars at Redshifts ~0.3-3, I: ALMA Observations

We present ALMA 870 micron (345 GHz) data for 49 high redshift (0.47<z<2.85), luminous (11.7 < log L(bol) (Lsun) < 14.2) radio-powerful AGN, obtained to constrain cool dust emission from starbursts concurrent with highly obscured radiative-mode black hole (BH) accretion in massive galaxies which possess a small radio jet. The sample was selected from WISE with extremely steep (red) mid-infrared (MIR) colors and with compact radio emission from NVSS/FIRST. Twenty-six sources are detected at 870 microns, and we find that the sample has large mid- to far-infrared luminosity ratios consistent with a dominant and highly obscured quasar. The rest-frame 3 GHz radio powers are 24.7 < log P3.0 GHz (W/Hz) < 27.3, and all sources are radio-intermediate or radio-loud. BH mass estimates are 7.7 < log M(BH) (Msun) < 10.2. The rest frame 1-5 um SEDs are very similar to the "Hot DOGs" (Hot Dust Obscured Galaxies), and steeper (redder) than almost any other known extragalactic sources. ISM masses estimated for the ALMA detected sources are 9.9 < log M(ISM) (Msun) < 11.75 assuming a dust temperature of 30K. The cool dust emission is consistent with star formation rates (SFRs) reaching several thousand Msun/yr, depending on the assumed dust temperature, however we cannot rule out the alternative that the AGN powers all the emission in some cases. Our best constrained source has radiative transfer solutions with ~ equal contributions from an obscured AGN and a young (10-15 Myr) compact starburst.Comment: 29 pages, 8 figures. To appear in Astrophysical Journal. Update on Sept 14 to correct the ALMA proposal id. to ADS/JAO.ALMA#2011.0.00397.S and to add a missing acknowledgemen

Bubble-Wall Plot: A New Tool for Data Visualization

This research aimed to design a new tool for data visualization with performed features - named Bubble-Wall Plot and assumed that it could be an effective tool for developing data visualization systems. This research reviewed seven data visualization approaches for identifying the outliers, including Line Charts, Parallel Coordinates Plot, Scatter Plots, TreeMap, Glyphs, Pixel-based techniques, and Redial visualizations. The challenges for current data visualization approaches were also summarized. Two principles were addressed to design the new tool- keep it simple strategy with the smallest strategy. As a result, the newly designed Bubble-Wall Plot has successfully been adopted to develop a warning system for identifying the outliers in a Case Study company, which was deployed for user acceptance testing in May 2021. The main contribution is that this newly designed tool with the simplest style was well-designed and proven to effectively develop a warning visualization system