Designing a high-efficiency and high-quality expressive network architecture
has always been the most important research topic in the field of deep
learning. Most of today's network design strategies focus on how to integrate
features extracted from different layers, and how to design computing units to
effectively extract these features, thereby enhancing the expressiveness of the
network. This paper proposes a new network design strategy, i.e., to design the
network architecture based on gradient path analysis. On the whole, most of
today's mainstream network design strategies are based on feed forward path,
that is, the network architecture is designed based on the data path. In this
paper, we hope to enhance the expressive ability of the trained model by
improving the network learning ability. Due to the mechanism driving the
network parameter learning is the backward propagation algorithm, we design
network design strategies based on back propagation path. We propose the
gradient path design strategies for the layer-level, the stage-level, and the
network-level, and the design strategies are proved to be superior and feasible
from theoretical analysis and experiments.Comment: 12 pages, 9 figure