1 research outputs found
ThinResNet: A New Baseline for Structured Convolutional Networks Pruning
Pruning is a compression method which aims to improve the efficiency of
neural networks by reducing their number of parameters while maintaining a good
performance, thus enhancing the performance-to-cost ratio in nontrivial ways.
Of particular interest are structured pruning techniques, in which whole
portions of parameters are removed altogether, resulting in easier to leverage
shrunk architectures. Since its growth in popularity in the recent years,
pruning gave birth to countless papers and contributions, resulting first in
critical inconsistencies in the way results are compared, and then to a
collective effort to establish standardized benchmarks. However, said
benchmarks are based on training practices that date from several years ago and
do not align with current practices. In this work, we verify how results in the
recent literature of pruning hold up against networks that underwent both
state-of-the-art training methods and trivial model scaling. We find that the
latter clearly and utterly outperform all the literature we compared to,
proving that updating standard pruning benchmarks and re-evaluating classical
methods in their light is an absolute necessity. We thus introduce a new
challenging baseline to compare structured pruning to: ThinResNet.Comment: 11 pages, 2 figures, 3 table