Domain Adaptation and Transfer Learning in StochasticNets

Transfer learning is a recent field of machine learning research thataims to resolve the challenge of dealing with insufficient trainingdata in the domain of interest. This is a particular issue with traditionaldeep neural networks where a large amount of trainingdata is needed. Recently, StochasticNets was proposed to takeadvantage of sparse connectivity in order to decrease the numberof parameters that needs to be learned, which in turn may relaxtraining data size requirements. In this paper, we study the efficacyof transfer learning on StochasticNet frameworks. Experimental resultsshow 7% improvement on StochasticNet performance whenthe transfer learning is applied in training step