NORM: Knowledge Distillation via N-to-One Representation Matching

Existing feature distillation methods commonly adopt the One-to-one Representation Matching between any pre-selected teacher-student layer pair. In this paper, we present N-to-One Representation (NORM), a new two-stage knowledge distillation method, which relies on a simple Feature Transform (FT) module consisting of two linear layers. In view of preserving the intact information learnt by the teacher network, during training, our FT module is merely inserted after the last convolutional layer of the student network. The first linear layer projects the student representation to a feature space having N times feature channels than the teacher representation from the last convolutional layer, and the second linear layer contracts the expanded output back to the original feature space. By sequentially splitting the expanded student representation into N non-overlapping feature segments having the same number of feature channels as the teacher's, they can be readily forced to approximate the intact teacher representation simultaneously, formulating a novel many-to-one representation matching mechanism conditioned on a single teacher-student layer pair. After training, such an FT module will be naturally merged into the subsequent fully connected layer thanks to its linear property, introducing no extra parameters or architectural modifications to the student network at inference. Extensive experiments on different visual recognition benchmarks demonstrate the leading performance of our method. For instance, the ResNet18|MobileNet|ResNet50-1/4 model trained by NORM reaches 72.14%|74.26%|68.03% top-1 accuracy on the ImageNet dataset when using a pre-trained ResNet34|ResNet50|ResNet50 model as the teacher, achieving an absolute improvement of 2.01%|4.63%|3.03% against the individually trained counterpart. Code is available at https://github.com/OSVAI/NORMComment: The paper of NORM is published at ICLR 2023. Code and models are available at https://github.com/OSVAI/NOR

Facile synthesis of superhydrophobic surface of ZnO nanoflakes: chemical coating and UV-induced wettability conversion

This work reports an oriented growth process of two-dimensional (2D) ZnO nanoflakes on aluminum substrate through a low temperature hydrothermal technique and proposes the preliminary growth mechanism. A bionic superhydrophobic surface with excellent corrosion protection over a wide pH range in both acidic and alkaline solutions was constructed by a chemical coating treatment with stearic acid (SA) molecules on ZnO nanoflakes. It is found that the superhydrophobic surface of ZnO nanoflake arrays shows a maximum water contact angle (CA) of 157° and a low sliding angle of 8°, and it can be reversibly switched to its initial superhydrophilic state under ultraviolet (UV) irradiation, which is due to the UV-induced decomposition of the coated SA molecules. This study is significant for simple and inexpensive building of large-scale 2D ZnO nanoflake arrays with special wettability which can extend the applications of ZnO films to many other important fields

Morphology-dependent field emission properties and wetting behavior of ZnO nanowire arrays

The fabrication of three kinds of ZnO nanowire arrays with different structural parameters over Au-coated silicon (100) by facile thermal evaporation of ZnS precursor is reported, and the growth mechanism are proposed based on structural analysis. Field emission (FE) properties and wetting behavior were revealed to be strongly morphology dependent. The nanowire arrays in small diameter and high aspect ratio exhibited the best FE performance showing a low turn-on field (4.1 V/μm) and a high field-enhancement factor (1745.8). The result also confirmed that keeping large air within the films was an effective way to obtain super water-repellent properties. This study indicates that the preparation of ZnO nanowire arrays in an optimum structural model is crucial to FE efficiency and wetting behavior

Fabrication and ultraviolet photoresponse characteristics of ordered SnOx (x ≈ 0.87, 1.45, 2) nanopore films

Based on the porous anodic aluminum oxide templates, ordered SnOx nanopore films (approximately 150 nm thickness) with different x (x ≈ 0.87, 1.45, 2) have been successfully fabricated by direct current magnetron sputtering and oxidizing annealing. Due to the high specific surface area, this ordered nanopore films exhibit a great improvement in recovery time compared to thin films for ultraviolet (UV) detection. Especially, the ordered SnOx nanopore films with lower x reveal higher UV light sensitivity and shorter current recovery time, which was explained by the higher concentration of the oxygen vacancies in this SnOx films. This work presents a potential candidate material for UV light detector

Morphology-dependent field emission properties and wetting behavior of ZnO nanowire arrays

<p>Abstract</p> <p>The fabrication of three kinds of ZnO nanowire arrays with different structural parameters over Au-coated silicon (100) by facile thermal evaporation of ZnS precursor is reported, and the growth mechanism are proposed based on structural analysis. Field emission (FE) properties and wetting behavior were revealed to be strongly morphology dependent. The nanowire arrays in small diameter and high aspect ratio exhibited the best FE performance showing a low turn-on field (4.1 V/&#956;m) and a high field-enhancement factor (1745.8). The result also confirmed that keeping large air within the films was an effective way to obtain super water-repellent properties. This study indicates that the preparation of ZnO nanowire arrays in an optimum structural model is crucial to FE efficiency and wetting behavior.</p

Strength Damage and Acoustic Emission Characteristics of Water-Bearing Coal Pillar Dam Samples from Shangwan Mine, China

Long-term erosion and repeated scouring of water significantly affect the technical properties of coals, which are the essential elements that must be considered in evaluating an underground reservoir coal column dam’s standing sustainability. In the paper, the coal pillar dam body of the 22 layers of coal in the Shangwan Coal Mine is studied (22 represents No. 2 coal seam), and the water content of this coal pillar dam body is simplified into two types of different water content and dry–wet cycle. Through acoustic emission detection technology and energy dissipation analysis method, the internal failure mechanism of coal water action is analyzed. This study revealed three findings. (1) The crest pressure, strain, and resilient modulus in the coal sample were inversely related to the water content along with the dry–wet cycle number, while the drying–wetting cycle process had a certain time effect on the failure to the sample. (2) As the moisture content and the dry–wet cycle times incremented, three features were shown: first, the breakage pattern is the mainly stretching fracture for the coal specimen; second, the number and absolute value of acoustic emission count peaks decrease; third, the RA-AF probability density plot (RA is the ratio of AE Risetime and Amplitude, and AF is the ratio of AE Count and Duration) corresponds more closely to the large-scale destruction characteristics for the coal samples. (3) A higher quantity of wet and dry cycles results in a smoother energy dissipation curve in the compacted and flexible phases of the crack, indicating that this energy is released earlier. The research results can be applied to the long-term sustainability assessment of the dams of coal columns for underground reservoirs and can also serve as valuable content to the excogitation of water-bearing coal column dams under similar engineering conditions