Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, researchers have been investigating approaches to boost accuracy through either deeper or wider network structures, which brings with them the exponential increment of the computational and storage cost, delaying the responding time. In this paper, we propose a general training framework named self distillation, which notably enhances the performance (accuracy) of convolutional neural networks through shrinking the size of the network rather than aggrandizing it. Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself. The networks are firstly divided into several sections. Then the knowledge in the deeper portion of the networks is squeezed into the shallow ones. Experiments further prove the generalization of the proposed self distillation framework: enhancement of accuracy at average level is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 as maximum. In addition, it can also provide flexibility of depth-wise scalable inference on resource-limited edge devices.Our codes will be released on github soon.Comment: 10page

CAP-VSTNet: Content Affinity Preserved Versatile Style Transfer

Content affinity loss including feature and pixel affinity is a main problem which leads to artifacts in photorealistic and video style transfer. This paper proposes a new framework named CAP-VSTNet, which consists of a new reversible residual network and an unbiased linear transform module, for versatile style transfer. This reversible residual network can not only preserve content affinity but not introduce redundant information as traditional reversible networks, and hence facilitate better stylization. Empowered by Matting Laplacian training loss which can address the pixel affinity loss problem led by the linear transform, the proposed framework is applicable and effective on versatile style transfer. Extensive experiments show that CAP-VSTNet can produce better qualitative and quantitative results in comparison with the state-of-the-art methods.Comment: CVPR 202

Structure and regulation of ZCCHC4 in m6A-methylation of 28S rRNA.

N6-methyladenosine (m6A) modification provides an important epitranscriptomic mechanism that critically regulates RNA metabolism and function. However, how m6A writers attain substrate specificities remains unclear. We report the 3.1 Å-resolution crystal structure of human CCHC zinc finger-containing protein ZCCHC4, a 28S rRNA-specific m6A methyltransferase, bound to S-adenosyl-L-homocysteine. The methyltransferase (MTase) domain of ZCCHC4 is packed against N-terminal GRF-type and C2H2 zinc finger domains and a C-terminal CCHC domain, creating an integrated RNA-binding surface. Strikingly, the MTase domain adopts an autoinhibitory conformation, with a self-occluded catalytic site and a fully-closed cofactor pocket. Mutational and enzymatic analyses further substantiate the molecular basis for ZCCHC4-RNA recognition and a role of the stem-loop structure within substrate in governing the substrate specificity. Overall, this study unveils unique structural and enzymatic characteristics of ZCCHC4, distinctive from what was seen with the METTL family of m6A writers, providing the mechanistic basis for ZCCHC4 modulation of m6A RNA methylation

Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+

Recent developments in deep learning have made remarkable progress in speeding up the prediction of quantum chemical (QC) properties by removing the need for expensive electronic structure calculations like density functional theory. However, previous methods learned from 1D SMILES sequences or 2D molecular graphs failed to achieve high accuracy as QC properties primarily depend on the 3D equilibrium conformations optimized by electronic structure methods, far different from the sequence-type and graph-type data. In this paper, we propose a novel approach called Uni-Mol+ to tackle this challenge. Uni-Mol+ first generates a raw 3D molecule conformation from inexpensive methods such as RDKit. Then, the raw conformation is iteratively updated to its target DFT equilibrium conformation using neural networks, and the learned conformation will be used to predict the QC properties. To effectively learn this update process towards the equilibrium conformation, we introduce a two-track Transformer model backbone and train it with the QC property prediction task. We also design a novel approach to guide the model's training process. Our extensive benchmarking results demonstrate that the proposed Uni-Mol+ significantly improves the accuracy of QC property prediction in various datasets. We have made the code and model publicly available at \url{https://github.com/dptech-corp/Uni-Mol}

Effect of temperature on the hydrolysis of actinide elements in solution

Recent experimental data on the hydrolysis of U(VI), Pu(VI), Np(V), and Th(IV) at variable temperatures are summarized in this review. Data indicate that the hydrolysis reactions of U(VI), Pu(VI), Np(V), and Th (IV) are all enhanced when temperature is increased from 283 to 358 K. In general, the tendency of actinide elements in different oxidation states toward hydrolysis follows the order: An(IV) > An(VI) > An(V), which can be well described by the electrostatic model. The enhancement of hydrolysis at higher temperatures can be attributed to the increase of ionization of water with the increase of temperature. A few theoretical thermodynamic approaches for predicting the effect of temperature, including the constant enthalpy approach, the constant heat capacity approach, the DQUANT equation, and the Ryzhenko-Bryzgalin model, are tested with the experimental data

Uni-QSAR: an Auto-ML Tool for Molecular Property Prediction

Recently deep learning based quantitative structure-activity relationship (QSAR) models has shown surpassing performance than traditional methods for property prediction tasks in drug discovery. However, most DL based QSAR models are restricted to limited labeled data to achieve better performance, and also are sensitive to model scale and hyper-parameters. In this paper, we propose Uni-QSAR, a powerful Auto-ML tool for molecule property prediction tasks. Uni-QSAR combines molecular representation learning (MRL) of 1D sequential tokens, 2D topology graphs, and 3D conformers with pretraining models to leverage rich representation from large-scale unlabeled data. Without any manual fine-tuning or model selection, Uni-QSAR outperforms SOTA in 21/22 tasks of the Therapeutic Data Commons (TDC) benchmark under designed parallel workflow, with an average performance improvement of 6.09\%. Furthermore, we demonstrate the practical usefulness of Uni-QSAR in drug discovery domains