Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, that encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two challenging large scale datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture. Our models will be available to the research community later.Comment: Technical report for our submissions to the ILSVRC 2015 Scene Classification Challenge, where we won the first plac

A systematic analysis of equivalence in multistage networks

Many approaches to switching in optoelectronic and optical networks decompose the switching function across multiple stages or hops. This paper addresses the problem of determining whether two multistage or multihop networks are functionally equivalent. Various ad-hoc methods have been used in the past to establish such equivalences. A systematic method for determining equivalence is presented based on properties of the link permutations used to interconnect stages of the network. This method is useful in laying out multistage networks, in determining optimal channel assignments for multihop networks, and in establishing the routing required in such networks. A purely graphical variant of the method, requiring no mathematics or calculations, is also described