Deep Learning with S-shaped Rectified Linear Activation Units

Rectified linear activation units are important components for state-of-the-art deep convolutional networks. In this paper, we propose a novel S-shaped rectified linear activation unit (SReLU) to learn both convex and non-convex functions, imitating the multiple function forms given by the two fundamental laws, namely the Webner-Fechner law and the Stevens law, in psychophysics and neural sciences. Specifically, SReLU consists of three piecewise linear functions, which are formulated by four learnable parameters. The SReLU is learned jointly with the training of the whole deep network through back propagation. During the training phase, to initialize SReLU in different layers, we propose a "freezing" method to degenerate SReLU into a predefined leaky rectified linear unit in the initial several training epochs and then adaptively learn the good initial values. SReLU can be universally used in the existing deep networks with negligible additional parameters and computation cost. Experiments with two popular CNN architectures, Network in Network and GoogLeNet on scale-various benchmarks including CIFAR10, CIFAR100, MNIST and ImageNet demonstrate that SReLU achieves remarkable improvement compared to other activation functions.Comment: Accepted by AAAI-1

High-coverage whole-genome analysis of 1220 cancers reveals hundreds of genes deregulated by rearrangement-mediated cis-regulatory alterations.

The impact of somatic structural variants (SVs) on gene expression in cancer is largely unknown. Here, as part of the ICGC/TCGA Pan-Cancer Analysis of Whole Genomes (PCAWG) Consortium, which aggregated whole-genome sequencing data and RNA sequencing from a common set of 1220 cancer cases, we report hundreds of genes for which the presence within 100 kb of an SV breakpoint associates with altered expression. For the majority of these genes, expression increases rather than decreases with corresponding breakpoint events. Up-regulated cancer-associated genes impacted by this phenomenon include TERT, MDM2, CDK4, ERBB2, CD274, PDCD1LG2, and IGF2. TERT-associated breakpoints involve ~3% of cases, most frequently in liver biliary, melanoma, sarcoma, stomach, and kidney cancers. SVs associated with up-regulation of PD1 and PDL1 genes involve ~1% of non-amplified cases. For many genes, SVs are significantly associated with increased numbers or greater proximity of enhancer regulatory elements near the gene. DNA methylation near the promoter is often increased with nearby SV breakpoint, which may involve inactivation of repressor elements

Retrospective evaluation of whole exome and genome mutation calls in 746 cancer samples

Funder: NCI U24CA211006Abstract: The Cancer Genome Atlas (TCGA) and International Cancer Genome Consortium (ICGC) curated consensus somatic mutation calls using whole exome sequencing (WES) and whole genome sequencing (WGS), respectively. Here, as part of the ICGC/TCGA Pan-Cancer Analysis of Whole Genomes (PCAWG) Consortium, which aggregated whole genome sequencing data from 2,658 cancers across 38 tumour types, we compare WES and WGS side-by-side from 746 TCGA samples, finding that ~80% of mutations overlap in covered exonic regions. We estimate that low variant allele fraction (VAF < 15%) and clonal heterogeneity contribute up to 68% of private WGS mutations and 71% of private WES mutations. We observe that ~30% of private WGS mutations trace to mutations identified by a single variant caller in WES consensus efforts. WGS captures both ~50% more variation in exonic regions and un-observed mutations in loci with variable GC-content. Together, our analysis highlights technological divergences between two reproducible somatic variant detection efforts

ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation

Data augmentation (DA) is a widely used technique for enhancing the training of deep neural networks. Recent DA techniques which achieve state-of-the-art performance always meet the need for diversity in augmented training samples. However, an augmentation strategy that has a high diversity usually introduces out-of-distribution (OOD) augmented samples and these samples consequently impair the performance. To alleviate this issue, we propose ReSmooth, a framework that firstly detects OOD samples in augmented samples and then leverages them. To be specific, we first use a Gaussian mixture model to fit the loss distribution of both the original and augmented samples and accordingly split these samples into in-distribution (ID) samples and OOD samples. Then we start a new training where ID and OOD samples are incorporated with different smooth labels. By treating ID samples and OOD samples unequally, we can make better use of the diverse augmented data. Further, we incorporate our ReSmooth framework with negative data augmentation strategies. By properly handling their intentionally created ODD samples, the classification performance of negative data augmentations is largely ameliorated. Experiments on several classification benchmarks show that ReSmooth can be easily extended to existing augmentation strategies (such as RandAugment, rotate, and jigsaw) and improve on them

Segmentation over detection via optimal sparse reconstructions

This paper addresses the problem of semantic segmentation, where the possible class labels are from a predefined set. We exploit top-down guidance, i.e., the coarse localization of the objects and their class labels provided by object detectors. For each detected bounding box, figure-ground segmentation is performed and the final result is achieved by merging the figure-ground segmentations. The main idea of the proposed approach, which is presented in our preliminary work, is to reformulate the figure-ground segmentation problem as sparse reconstruction pursuing the object mask in a nonparametric manner. The latent segmentation mask should be coherent subject to sparse error caused by intra-category diversity; thus, the object mask is inferred by making use of sparse representations over the training set. To handle local spatial deformations, local patch-level masks are also considered and inferred by sparse representations over the spatially nearby patches. The sparse reconstruction coefficients and the latent mask are alternately optimized by applying the Lasso algorithm and the accelerated proximal gradient method. The proposed formulation results in a convex optimization problem; thus, the global optimal solution is achieved. In this paper, we provide theoretical analysis of the convergence and optimality. We also give an extended numerical analysis of the proposed algorithm and a comprehensive comparison with the related semantic segmentation methods on the challenging PASCAL visual object class object segmentation datasets and the Weizmann horse dataset. The experimental results demonstrate that the proposed algorithm achieves a competitive performance when compared with the state of the arts

Tropospheric Delay Model Based on VMF and ERA5 Reanalysis Data

The global tropospheric zenith delay grid products of VMF1 and VMF3 (Vienna mapping functions) with different resolutions are used to calculate the tropospheric zenith delay of eight IGS (International GNSS Service) stations in China, and the accuracy of the two products under different interpolation methods is analyzed. As a result, the accuracy of utilizing different interpolation methods shows no obvious differences. The interpolation accuracy of the VMF3 grid model is slightly higher than that of the VMF1, and the interpolation accuracy of tropospheric delay is related to the elevation difference of grid points. In addition, according to ERA5 (the fifth generation of the Global Climate Information Analysis data set), the atmospheric stratification tropospheric delay is obtained, and a ZTD (the zenith tropospheric delay) height change grid model is constructed using the least squares exponential fitting method. The accuracy of the model is verified using the tropospheric delay product provided by the IGS. Finally, the constructed ZTD height change grid model is used as ZTD height reduction to solve the problem of large tropospheric delay errors in the VMF interpolation when the height change is large. The model accuracy of URUM station improve from 96.47 mm.to 32.97 mm (65.82%)

Performance Evaluation of Closed-Loop Spatial Multiplexing Codebook Based on Indoor MIMO Channel Measurement

Closed-loop MIMO technique standardized in LTE can support different layer transmissions through precoding operation to match the channel multiplexing capability. However, the performance of the limited size codebook still needs to be evaluated in real channel environment for further insights. Based on the wideband MIMO channel measurement in a typical indoor scenario, capacity loss (CL) of the limited size codebook relative to perfect precoding is studied first in two extreme channel conditions. The results show that current codebook design for single layer transmission is nearly capacity lossless, and the CL will increase with the number of transmitted layers. Furthermore, the capacity improvement of better codebook selection criterions is very limited compared to CL. Then we define the maximum capacity boost achieved by frequency domain layer adaption (FDLA) and investigate its sensitivity to SNR and channel condition. To survey the effect of frequency domain channel variation on MIMO-OFDM system, we define a function to measure the fluctuation levels of the key channel metrics within a subband and reveal the inherent relationship between them. Finally, a capacity floor resulted as the feedback interval increases in frequency domain

Structure-Texture Dual Preserving for Remote Sensing Image Super Resolution

Most of the existing remote sensing image super-resolution (SR) methods based on deep learning tend to learn the mapping from low-resolution (LR) images to high-resolution (HR) images directly. But they ignore the potential structure and texture consistency of LR and HR spaces, which cause the loss of high-frequency information and produce artifacts. A structure-texture dual preserving method is proposed to solve this problem and generate pleasing details. Specifically, we propose a novel edge prior enhancement strategy that uses the edges of LR images and the proposed interactive supervised attention module (ISAM) to guide SR reconstruction. First, we introduce the LR edge map as a prior structural expression for SR reconstruction, which further enhances the SR process with edge preservation capability. In addition, to obtain finer texture edge information, we propose a novel ISAM in order to correct the initial LR edge map with high-frequency information. By introducing LR edges and ISAM-corrected HR edges, we build LR&#x2013;HR edge mapping to preserve the consistency of LR and HR edge structure and texture, which provides supervised information for SR reconstruction. Finally, we explore the salient features of the image and its edges in the ascending space, and restored the difference between LR and HR images by residual and dense learning. A large number of experimental results on Draper and NWPU-RESISC45 datasets show that our model is superior to several advanced SR algorithms in both objective and subjective image quality