Deep neural networks (DNNs) have found applications in diverse signal
processing (SP) problems. Most efforts either directly adopt the DNN as a
black-box approach to perform certain SP tasks without taking into account of
any known properties of the signal models, or insert a pre-defined SP operator
into a DNN as an add-on data processing stage. This paper presents a novel
hybrid-NN framework in which one or more SP layers are inserted into the DNN
architecture in a coherent manner to enhance the network capability and
efficiency in feature extraction. These SP layers are properly designed to make
good use of the available models and properties of the data. The network
training algorithm of hybrid-NN is designed to actively involve the SP layers
in the learning goal, by simultaneously optimizing both the weights of the DNN
and the unknown tuning parameters of the SP operators. The proposed hybrid-NN
is tested on a radar automatic target recognition (ATR) problem. It achieves
high validation accuracy of 96\% with 5,000 training images in radar ATR.
Compared with ordinary DNN, hybrid-NN can markedly reduce the required amount
of training data and improve the learning performance