3 research outputs found
Lightweight Neural Network with Knowledge Distillation for CSI Feedback
Deep learning (DL) has shown promise in enhancing channel state information
(CSI) feedback. However, many studies indicate that better feedback performance
often accompanies higher computational complexity. Pursuing better
performance-complexity tradeoffs is crucial to facilitate practical deployment,
especially on computation-limited devices, which may have to use lightweight
autoencoder with unfavorable performance. To achieve this goal, this paper
introduces knowledge distillation (KD) to achieve better tradeoffs, where
knowledge from a complicated teacher autoencoder is transferred to a
lightweight student autoencoder for performance improvement. Specifically, two
methods are proposed for implementation. Firstly, an autoencoder KD-based
method is introduced by training a student autoencoder to mimic the
reconstructed CSI of a pretrained teacher autoencoder. Secondly, an encoder
KD-based method is proposed to reduce training overhead by performing KD only
on the student encoder. Additionally, a variant of encoder KD is introduced to
protect user equipment and base station vendor intellectual property. Numerical
simulations demonstrate that the proposed KD methods can significantly improve
the student autoencoder's performance, while reducing the number of floating
point operations and inference time to 3.05%-5.28% and 13.80%-14.76% of the
teacher network, respectively. Furthermore, the variant encoder KD method
effectively enhances the student autoencoder's generalization capability across
different scenarios, environments, and bandwidths.Comment: 28 pages, 4 figure
SPOC learner's final grade prediction based on a novel sampling batch normalization embedded neural network method
Recent years have witnessed the rapid growth of Small Private Online Courses
(SPOC) which is able to highly customized and personalized to adapt variable
educational requests, in which machine learning techniques are explored to
summarize and predict the learner's performance, mostly focus on the final
grade. However, the problem is that the final grade of learners on SPOC is
generally seriously imbalance which handicaps the training of prediction model.
To solve this problem, a sampling batch normalization embedded deep neural
network (SBNEDNN) method is developed in this paper. First, a combined
indicator is defined to measure the distribution of the data, then a rule is
established to guide the sampling process. Second, the batch normalization (BN)
modified layers are embedded into full connected neural network to solve the
data imbalanced problem. Experimental results with other three deep learning
methods demonstrates the superiority of the proposed method.Comment: 11 pages, 5 figures, ICAIS 202