Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon

How to develop slim and accurate deep neural networks has become crucial for real- world applications, especially for those employed in embedded systems. Though previous work along this research line has shown some promising results, most existing methods either fail to significantly compress a well-trained deep network or require a heavy retraining process for the pruned deep network to re-boost its prediction performance. In this paper, we propose a new layer-wise pruning method for deep neural networks. In our proposed method, parameters of each individual layer are pruned independently based on second order derivatives of a layer-wise error function with respect to the corresponding parameters. We prove that the final prediction performance drop after pruning is bounded by a linear combination of the reconstructed errors caused at each layer. Therefore, there is a guarantee that one only needs to perform a light retraining process on the pruned network to resume its original prediction performance. We conduct extensive experiments on benchmark datasets to demonstrate the effectiveness of our pruning method compared with several state-of-the-art baseline methods

Hidden structure in amorphous solids

Recent theoretical studies of amorphous silicon [Y. Pan et al. Phys. Rev. Lett. 100 206403 (2008)] have revealed subtle but significant structural correlations in network topology: the tendency for short (long) bonds to be spatially correlated with other short (long) bonds). These structures were linked to the electronic band tails in the optical gap. In this paper, we further examine these issues for amorphous silicon, and demonstrate that analogous correlations exist in amorphous SiO2, and in the organic molecule, b-carotene. We conclude with a discussion of the origin of the effects and its possible generality

The Laplacian Eigenvalues and Invariants of Graphs

In this paper, we investigate some relations between the invariants (including vertex and edge connectivity and forwarding indices) of a graph and its Laplacian eigenvalues. In addition, we present a sufficient condition for the existence of Hamiltonicity in a graph involving its Laplacian eigenvalues.Comment: 10 pages,Filomat, 201