DisWOT: Student Architecture Search for Distillation WithOut Training

Knowledge distillation (KD) is an effective training strategy to improve the lightweight student models under the guidance of cumbersome teachers. However, the large architecture difference across the teacher-student pairs limits the distillation gains. In contrast to previous adaptive distillation methods to reduce the teacher-student gap, we explore a novel training-free framework to search for the best student architectures for a given teacher. Our work first empirically show that the optimal model under vanilla training cannot be the winner in distillation. Secondly, we find that the similarity of feature semantics and sample relations between random-initialized teacher-student networks have good correlations with final distillation performances. Thus, we efficiently measure similarity matrixs conditioned on the semantic activation maps to select the optimal student via an evolutionary algorithm without any training. In this way, our student architecture search for Distillation WithOut Training (DisWOT) significantly improves the performance of the model in the distillation stage with at least 180$\times$ training acceleration. Additionally, we extend similarity metrics in DisWOT as new distillers and KD-based zero-proxies. Our experiments on CIFAR, ImageNet and NAS-Bench-201 demonstrate that our technique achieves state-of-the-art results on different search spaces. Our project and code are available at https://lilujunai.github.io/DisWOT-CVPR2023/.Comment: Accepted by CVPR202

On the Mathematics of RNA Velocity II: Algorithmic Aspects

In a previous paper [CSIAM Trans. Appl. Math. 2 (2021), 1-55], the authors proposed a theoretical framework for the analysis of RNA velocity, which is a promising concept in scRNA-seq data analysis to reveal the cell state-transition dynamical processes underlying snapshot data. The current paper is devoted to the algorithmic study of some key components in RNA velocity workflow. Four important points are addressed in this paper: (1) We construct a rational time-scale fixation method which can determine the global gene-shared latent time for cells. (2) We present an uncertainty quantification strategy for the inferred parameters obtained through the EM algorithm. (3) We establish the optimal criterion for the choice of velocity kernel bandwidth with respect to the sample size in the downstream analysis and discuss its implications. (4) We propose a temporal distance estimation approach between two cell clusters along the cellular development path. Some illustrative numerical tests are also carried out to verify our analysis. These results are intended to provide tools and insights in further development of RNA velocity type methods in the future.Comment: 32 pages, 5 figure

Optimized Live 4K Video Multicast

4K videos are becoming increasingly popular. However, despite advances in wireless technology, streaming 4K videos over mmWave to multiple users is facing significant challenges arising from directional communication, unpredictable channel fluctuation and high bandwidth requirements. This paper develops a novel 4K layered video multicast system. We (i) develop a video quality model for layered video coding, (ii) optimize resource allocation, scheduling, and beamforming based on the channel conditions of different users, and (iii) put forward a streaming strategy that uses fountain code to avoid redundancy across multicast groups and a Leaky-Bucket-based congestion control. We realize an end-to-end system on commodity-off-the-shelf (COTS) WiGig devices. We demonstrate the effectiveness of our system with extensive testbed experiments and emulation

Pre-Peak Deformation and Damage Features of Sandstone under Cyclic Loading

In this paper, several sandstone specimens are prepared and subjected to uniaxial compression and cyclic loading. For each specimen, the loading segment of the stress-strain curve was fitted, and the peak slope of this segment was taken as the elastic modulus of the specimen in that cycle. It is learned that, under cyclic loading, the elastic modulus of each specimen increased with the growing number of load cycles, and tended to be stable; meanwhile, strain hardening was observed on all specimens. Moreover, the specimens are similar in corresponding stress, although varied in corresponding strain. In the same cycle, the tangent modulus of the loading phase was smaller than that of the unloading phase under the same stress. Finally, the damage variables of sandstone specimens under cyclic loading were defined from the angle of energy, revealing that the damage variables had logarithmic growth with the load cycles in the later stage

Hysteresis Characteristics of Brittle Rock Deformation under Constant Load Cyclic Loading and Unloading

This paper mainly explores the deformation characteristics of limestone specimens under constant load cyclic loading. For limestone specimens under uniaxial compression, the stress-strain curve can be divided into three stages: compaction stage, elastic stage and sudden failure stage. Under cyclic loading, the hysteresis loop on the stress-strain curve is long and thin, taking the shape of "toothpicks". The axial strain and radial strain both change with the stress amplitude and cycle number, but in different variation patterns. There is a stress amplitude "threshold" for radial deformation, indicating that the radial deformation is more sensitive to stress amplitude than the axial deformation. It is calculated that the incremental deformation between peaks includes both plastic deformation and the deformation recoverable after unloading, and the recoverable deformation is positively correlated with the load amplitude of the cyclic loading