This paper proposes a deep-neural-network-based semi-supervised method for polarimetric synthetic aperture radar (PolSAR) data classification. The proposed method focuses on achieving a well-trained deep neural network (DNN) when the amount of the labeled samples is limited. In the proposed method, the probability vectors, where each entry indicates the probability of a sample associated with a category, are first evaluated for the unlabeled samples, leading to an augmented training set. With this augmented training set, the parameters in the DNN are learned by solving the optimization problem, where the log-likelihood cost function and the class probability vectors are used. To alleviate the “salt-and-pepper” appearance in the classification results of PolSAR images, the spatial interdependencies are incorporated by introducing a Markov random field (MRF) prior in the prediction step. The experimental results on two realistic PolSAR images demonstrate that the proposed method effectively incorporates the spatial interdependencies and achieves the good classification accuracy with a limited number of labeled samples

Huang, Shaoguang

Li, Heng-Chao

Liao, Wenzhi

Liu, Chi

Philips, Wilfried

English

Ghent University Academic Bibliography

A Deep-Neural-Network-Based Hybrid Method forSemi-Supervised Classification of Polarimetric SARDataChi Liu1,2, Wenzhi Liao1, Heng-Chao Li2, Shaoguang Huang1, and Wilfried Philips11 TELIN-IPI, IMEC-Ghent University, Ghent, Belgium2 Sichuan Provincial Key Laboratory of Information Coding and TransmissionSouthwest Jiaotong University, Chengdu, ChinaAbstract—This paper proposes a deep-neural-network-basedsemi-supervised method for polarimetric synthetic aperture radar(PolSAR) data classification. The proposed method focuses onachieving a well-trained deep neural network (DNN) when theamount of the labeled samples is limited. In the proposed method,the probability vectors, where each entry indicates the probabilityof a sample associated with a category, are first evaluated forthe unlabeled samples, leading to an augmented training set.With this augmented training set, the parameters in the DNNare learned by solving the optimization problem, where the log-likelihood cost function and the class probability vectors are used.To alleviate the “salt-and-pepper” appearance in the classificationresults of PolSAR images, the spatial interdependencies areincorporated by introducing a Markov random field (MRF) priorin the prediction step. The experimental results on two realisticPolSAR images demonstrate that the proposed method effectivelyincorporates the spatial interdependencies and achieves the goodclassification accuracy with a limited number of labeled samples.Index Terms—Polarimetric synthetic aperture radar (PolSAR),semi-supervised classification, deep neural networks, remotesensing.I. INTRODUCTIONTerrain classification is an important application of polari-metric synthetic aperture radar (PolSAR). PolSAR acquiresmulti-channel data by different combinations of horizontal andvertical polarization, which can provide the useful informationabout the physical scattering characteristics to accurately iden-tify terrain types [1]. Furthermore, in view of the ability ofradar waves to penetrate through clouds, PolSAR can alwaysreceive backscattered signals from the ground, facilitating acomplete interpretation of areas of interest.Recently, deep neural networks (DNNs) have become pop-ular in classification tasks of PolSAR images. Many effectivemethods have been developed based on multilayer autoen-coders [2], [3], stacked sparse autoencoders [4], Wishart-autoencoders, Wishart-convolutional-autoencoders [5], anddeep convolutional neural networks [6]. These methods lever-age the power of DNNs in extracting discriminative featuresThis work was supported in part by the China Scholarship Council, bythe FWO project G037115N: Data Fusion for Image Analysis in RemoteSensing, and by the National Natural Science Foundation of China under Grant61871335. Wenzhi Liao is a postdoctoral fellow of the Research FoundationFlanders (FWO-Vlaanderen, Belgium) and acknowledges its support.for subsequent supervised classification, leading to a signifi-cant improvement on the classification performance over con-ventional methods. However, in view of both the complicatedstructures of DNNs and the nature of supervised classificationin these DNN-based methods, their performance could beheavily dependent on the amount of training samples.For the supervised classification of a large-scale PolSARimage, manually labeling a large number of samples fortraining costs a lot of resources. The use of the limited numberof labeled samples presents a challenge to achieve well-trainedDNNs, producing unreliable classification results. To addressthis limitation, a DNN can be learned based on both the labeledsamples and the unlabeled samples, leading to semi-supervisedclassification methods for PolSAR images [7]–[10].In view of the active imaging mechanism, PolSAR imagespresent a granular noise pattern, which always results inthe “salt-and-pepper” appearance in classification results andsignificantly degrades the classification performance. To alle-viate the “salt-and-pepper” effect, the abovementioned semi-supervised methods incorporate the spatial interdependencies.Specifically, the semi-supervised method for feature extractionof PolSAR data constructs the spatial groups by combiningneighboring pixels [7]. In addition, the average of the featurevectors (including polarimetric features and the elements fromPolSAR data as the entries) for the local pixels is exploitedto suppress the “salt-and-pepper” effect [8], [10]. Althoughthe scheme of averaging neighboring pixels incorporates thespatial interdependencies, the resulting features fail to preservethe complete information, leading to the loss of details in theclassification results.In this paper, we propose a semi-supervised DNN-and-MRF-based method (SSDNN-MRF) for PolSAR image classi-fication. Different from [10], which only selects the unlabeledsamples with high confidence to enlarge the training set for theDNN, our proposed method exploits all the unlabeled samples.Thus, the operation to select credible samples is eliminated,simplifying the procedure of the semi-supervised classificationmethods. To this end, the class probability vector associatedwith each unlabeled sample is first evaluated. In view ofthe label uncertainty for the augmented data, the training forthe DNN is performed based on a log-likelihood function,which can directly deal with the class probability vectors. Toincorporate the spatial interdependencies, the average featurevector associated with a local region is fed to the DNN inthe method of Geng et al. [10], and is used to representthe spatial group in the graph-based methods [7], [8]. Thesemethods incorporate the spatial interdependencies through theinput features. In contrast, the proposed method considers thespatial interdependencies on the labels and leaves the inputfeatures untouched, which preserves the complete informationof the input features.II. PRELIMINARIESA. PolSAR Data and Polarimetric FeaturesPolSAR acquires multi-channel data by different combi-nations of transmitting and receiving polarization (e.g., hor-izontal and vertical polarization). The PolSAR data vector isgiven by S = [Shh, Shv, Svh, Svv]T , where the superscriptT is the transpose operator, and the subscripts h and vindicate the horizontal polarization and the vertical polariza-tion. Shv represents the backscattered signal with horizontaltransmitting polarization and vertical receiving polarization.For a reciprocal media, the PolSAR data vector reduces toS = [Shh,√2Shv, Svv]T in the Lexicographic bases andtransforms to k = 1/√2[Shh + Svv, Shh − Svv, 2Shv]T inthe Pauli bases [1]. The multilook PolSAR covariance andcoherency matrix are the second-order statistics, which arederived for speckle reduction by averaging n neighboringpixels, i.e., C = 1n∑ni=1 SiSHi and T =1n∑ni=1 kikHi . Here,H is the Hermitian transpose operator.By factorizing the PolSAR data of these types into thecombination of bases associated with physical scattering mod-els, target decomposition (TD) theorems were proposed toextract additional information [1], [11], [12]. The Krogagerdecomposition analyzes the contribution of the sphere, diplane,and helix targets to the vector S [13]. Based on the multilookpolarimetric matrices, the Freeman-Durden decomposition ex-tracts the polarimetric parameters for the volume, double-bounce, and surface scattering [14]. The polarimetric featuresfrom TD theorems provide insights into the physical scatteringmechanisms over illuminating areas, and facilitate the physicalinterpretation of PolSAR images.B. Deep Neural Networks and BackpropagationA DNN has an architecture of multiple hidden layers, whereeach hidden layer adopts the outputs from the previous layer asits inputs and yields its outputs by passing a weighted sum ofits inputs through a non-linear function [15]. For classificationtasks, a DNN aims to map input features to target labels.To obtain this mapping, a L-layer DNN can be discrimina-tively trained by solving min{Wl},{bl} C(hL,Y) [15], wherehL and Y respectively indicate the output of the DNN andthe underlying discrete labels, and C(·, ·) is a cost function toevaluate the discrepancy between them. {Wl} and {bl} arethe weight matrices and the bias vectors used for the feed-forward procedure. Specifically, the outputs of the l-th hiddenlayer hl (l = 1, 2, · · · , L) are evaluated by hl = fl(Zl), whereZl = hl−1Wl + bl and the equation of fl(·) is a non-linearactivation function.The weights {Wl} and the biases {bl} are commonlylearned based on stochastic gradient descent (SGD) and back-propagation [15]. The backpropagation procedure computesthe derivatives of the cost function with respect to the weightsand the biases. By backpropagating the error derivatives fromthe output layer to the input layer, the gradients of the outputsover the weights and the biases [see (2) and (3)] are obtainedusing the chain rule, and SGD can be applied to update {Wl}and {bl}. With N samples involved for the evaluation of thegradients, the backpropagation equations are given by [16]∂C∂hl= (∂C∂hl+1◦ f ′l (Zl+1))(Wl+1)T , (1)∂C∂Wl=1N· (hl−1)T ( ∂C∂hl◦ f ′l (Zl)), (2)∂C∂bl=1N· sum( ∂C∂hl◦ f ′l (Zl)), (3)where ◦ and the superscript T perform the element-wiseproduct and the transpose operation. f ′l (Zl) =∂f(Zl)∂Zl, andsum(·) is the summation over the N training samples.III. METHODOLOGYIn the proposed method for semi-supervised classificationof PolSAR images, polarimetric features from TD theoremsare collected as the input for a DNN. The input vectorsare formulated by stacking 48 features from 10 different TDapproaches [17]. This adopted feature set is shown to provideuseful information for the effective interpretation of PolSARimages [17].The proposed method (i.e., SSDNN-MRF) can be imple-mented by alternatively performing the prediction and thebackpropagation procedure. In the prediction step, the aug-mented training set is achieved and the spatial interdepen-dencies are incorporated. In the subsequent backpropagationprocedure, the parameters in the DNN are learned.A. Augmenting Training Set and Incorporating Spatial Inter-dependenciesTraining set is enlarged by adding all the unlabeled samples,each of which is associated with a class probability vector. Toachieve these probability vectors, the L-layer DNN exploitsa softmax layer in the rear to perform the classification task.This softmax layer for M categories provides the output hnL =[hn1L , hn2L , · · · , hnML ], wherehniL =exp(∑j hnjL−1WjiL + biL)∑Mk=1 exp(∑j hnjL−1WjkL + bkL). (4)Here, hnjL−1 is obtained for an unlabeled sample by the feed-forward procedure in the DNN. hniL can be interpreted as theprobability of a sample belonging to the i-th category.To incorporate the spatial interdependencies, a MRF priorp(Ŷ n = i; Ŷ ∂n , β) is imposed on hnL, leading to the probabil-ity vector pn = [pn1, · · · , pnM ] aspni = hniL · p(Ŷ n = i|Ŷ ∂n ;β), (5)where hniL can be found in (4). Ŷn is the estimated label ofpixel n, and the estimated labels of its neighbors are denotedby Ŷ ∂n . The MRF prior takes the form of [18]p(Ŷ n = i|Ŷ ∂n ;β) = exp(β∑m∈∂n δ(Ŷm, i))∑Mj=1 exp(β∑m∈∂n δ(Ŷm, j)). (6)Here, δ(Ŷ m, i) = 1 if and only if Ŷ m = i; otherwise, 0.The discrete label for each unlabeled sample is deter-mined by identifying the largest element in pn, i.e., Ŷ n =argmaxi pni. Compared with the discrete labels by thiscriterion, the probability vectors by (5) better preserve the labeluncertainty of the augmented data.B. Learning the DNNTo learn the DNN, the log-likelihood cost function isused in the optimization problem, which allows for the labeluncertainty of the augmented data. Given an index set L forlabeled samples and an index set U for augmented data, theoptimization problem takes the form ofmin{Wl},{bl}1C0(− λLJ∑j=1M∑m=1δ(Y j ,m) lnhjmL− λUK∑n=1M∑i=1pni lnhniL +2‖ {Wl} ‖2F),(7)where C0 = JλL + KλU. λL and λU are the balanceparameters for data sets L and U. hniL and pni can be foundfrom (4) and (5).  is a regularization parameter. ‖ • ‖Fperforms the Frobenius norm.The backpropagation equations for this optimization prob-lem are given in element-wise form by∂C∂ZniL={λL(hniL − Y ni), if n ∈ LλU(hniL − pni), if n ∈ U , (8)∂C∂hnil=∑k∂C∂Znkl+1·W ikl+1, (9)∂C∂Znil=∂C∂hnil· f ′l (Znil ), (10)where l = 1, 2, · · · , L− 1. We have the partial derivatives as∂C∂bil=1C0∑n∈{L,U}∂C∂Znil, (11)∂C∂W ikl=1C0( ∑n∈{L,U}hnil−1∂C∂Znkl+ W ikl). (12)The biases and the weights can be updated by (bil)(t) =(bil)(t−1)−η ∂C∂bil, and (W ikl )(t) = (W ikl )(t−1)−η ∂C∂W ikl, wherethe superscripts t− 1 and t respectively indicate the previousand current iteration. η is the learning rate.IV. EXPERIMENTAL RESULTS AND DISCUSSIONThe propose method is tested based on two realistic PolSARimages, referred to as “Foulum” and “Flevoland”. The super-vised DNN (SDNN), the SSDNN, and the proposed methodadopt the same DNN architecture as the SRDNN method, i.e.,two hidden layers with 150 and 40 units.For data set “Foulum”, 1000 samples per category are usedfor training. The classification maps are presented in Fig. 1.The average values for producer’s accuracy, overall accuracy(OA), and kappa coefficient (κ) over 10 independent tests arereported in Table I. The experimental results demonstrate theadvantage of the proposed method over the multi-step method(i.e., “SSDNN+MF”). Moreover, the higher OA and κ of theproposed method over the SDNN and the SSDNN reveal thatthe proposed method benefits from exploiting the unlabeleddata and incorporating the contextual information.For data set “Flevoland”, all the compared methods use1% of the underlying labels (a total of 1564 samples for 15categories). The experimental results are presented in Fig. 2and Table II. Since we cannot find the codes for the SNC andthe SRDNN, their classification maps and accuracy values arefrom references [7] and [10]. Compared with the SNC and theSRDNN method, the improved performance has been achievedby the proposed method. The proposed method augmentsthe training set with all the unlabeled data as well as theirclass probability vectors (instead of discrete labels) so asto consider the label uncertainty. In addition, the proposedmethod eliminates the procedure of selecting samples, whichsimplifies the procedure of the semi-supervised method.TABLE ICLASSIFICATION ACCURACY WITH THE FOULUM DATA SET.V. CONCLUSIONA DNN-based semi-supervised method has been proposedfor PolSAR image classification. All the unlabeled samplesas well as the labeled samples were exploited to train theDNN, which overcame the limitation of a small number oflabeled samples. The proposed method incorporated the spatialinterdependencies through the MRF prior to alleviate the “salt-and-pepper” effect in the classification results. The improvedresults over the SDNN and SSDNN method have verified theeffectiveness of the proposed method in exploiting the unla-beled data and in incorporating the spatial interdependencies.The good performance of the proposed method implies thepotential of the DNN in the semi-supervised classification ofthe PolSAR image.(a) (b) (c) (d) (e) (f) (g)Fig. 1. (a) Pauli RGB image of “Foulum”. (b) Ground truth. (c) Legend. (d) Supervised DNN method. (e) SSDNN method. (f) “SSDNN+MF”. The SSDNNfollowed by a post-processing step using the mode filter (MF). (g) The proposed method.Fig. 2. (a) Pauli RGB image of “Flevoland”. (b) Ground truth. (c) Legend. (d) SDNN method. (e) SSDNN method. (f) SNC method [7]. (g) SRDNN method[10]. (h) The proposed method.TABLE IICLASSIFICATION ACCURACY WITH THE FLEVOLAND DATA SET.REFERENCES[1] J.-S. Lee and E. Pottier, Polarimetric Imaging: From Basics to Appli-cations, CRC Press, Boca Raton, FL, 2009.[2] B. Hou, H. Kou, and L. Jiao, “Classification of polarimetric SAR imagesusing multilayer autoencoders and superpixels,” IEEE J. Sel. TopicsAppl. Earth Observ. Remote Sens., vol. 9, no. 7, pp. 3072–3081, July2016.[3] H. Liu, S. Yang, S. Gou, D. Zhu, R. Wang, and L. Jiao, “PolarimetricSAR feature extraction with neighborhood preservation-based deeplearning,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol.10, no. 4, pp. 1456–1466, Apr. 2017.[4] L. Zhang, W. Ma, and D. Zhang, “Stacked sparse autoencoder in PolSARdata classification using local spatial information,” IEEE Geosci. RemoteSens. Lett., vol. 13, no. 9, pp. 1359–1363, Sep. 2016.[5] W. Xie, L. Jiao, B. Hou, W. Ma, J. Zhao, S. Zhang, and F. Liu,“POLSAR image classification via Wishart-AE model or Wishart-CAEmodel,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 10,no. 8, pp. 3604–3615, Aug. 2017.[6] S. W. Chen and C. S. Tao, “PolSAR image classification usingpolarimetric-feature-driven deep convolutional neural network,” IEEEGeosci. Remote Sens. Lett., vol. 15, no. 4, pp. 627–631, Apr. 2018.[7] H. Liu, D. Zhu, S. Yang, B. Hou, S. Gou, T. Xiong, and L. Jiao,“Semisupervised feature extraction with neighborhood constraints forpolarimetric SAR classification,” IEEE J. Sel. Topics Appl. Earth Observ.Remote Sens., vol. 9, no. 7, pp. 3001–3015, July 2016.[8] H. Liu, S. Yang, S. Gou, P. Chen, Y. Wang, and L. Jiao, “Fastclassification for large polarimetric SAR data based on refined spatial-anchor graph,” IEEE Geosci. Remote Sens. Lett., vol. 14, no. 9, pp.1589–1593, Sep. 2017.[9] B. Hou, Q. Wu, Z. Wen, and L. Jiao, “Robust semisupervised classifica-tion for PolSAR image with noisy labels,” IEEE Trans. Geosci. RemoteSens., vol. 55, no. 11, pp. 6440–6455, Nov. 2017.[10] J. Geng, X. Ma, J. Fan, and H. Wang, “Semisupervised classification ofpolarimetric SAR image via superpixel restrained deep neural network,”IEEE Geosci. Remote Sens. Lett., vol. 15, no. 1, pp. 122–126, Jan. 2018.[11] S. R. Cloude and E. Pottier, “A review of target decomposition theoremsin radar polarimetry,” IEEE Trans. Geosci. Remote Sens., vol. 34, no.2, pp. 498–518, Mar. 1996.[12] R. Touzi, “Target scattering decomposition in terms of roll-invarianttarget parameters,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 1,pp. 73–84, Jan. 2007.[13] E. Krogager, “New decomposition of the radar target scattering matrix,”Electron. Lett., vol. 26, no. 18, pp. 1525–1527, Aug 1990.[14] A. Freeman and S. L. Durden, “A three-component scattering modelfor polarimetric SAR data,” IEEE Trans. Geosci. Remote Sens., vol. 36,no. 3, pp. 963–973, May 1998.[15] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep learning,”nature, vol. 521, pp. 436–444, 2015.[16] H. Wang and D. Yeung, “Towards Bayesian deep learning: A frameworkand some existing methods,” IEEE Trans. Knowl. Data Eng., vol. 28,no. 12, pp. 3395–3408, Dec. 2016.[17] M. Tao, F. Zhou, Y. Liu, and Z. Zhang, “Tensorial independentcomponent analysis-based feature extraction for polarimetric SAR dataclassification,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 5, pp.2481–2495, May 2015.[18] S. P. Chatzis and G. Tsechpenakis, “The infinite hidden Markov randomfield model,” IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 1004–1014,June 2010.

A deep-neural-network-based hybrid method for semi-supervised classification of polarimetric SAR data

Crossref

A Deep-Neural-Network-Based Hybrid Method for Semi-Supervised Classification of Polarimetric SAR Data

https://biblio.ugent.be/publication/8642044/file/8642152.pdf

A deep-neural-network-based hybrid method for semi-supervised classification of polarimetric SAR data

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography

Crossref