Joint Target Tracking and Recognition Using Shape-based Generative Model

Recently a generative model that combines both of identity and view manifolds was proposed for multi-view shape modeling that was originally used for pose estimation and recognition of civilian vehicles from image sequences. In this thesis, we extend this model to both civilian and military vehicles, and examine its effectiveness for real-world automated target tracking and recognition (ATR) applications in both infrared and visible image sequences. A particle filter-based ATR algorithm is introduced where the generative model is used for shape interpolation along both the view and identity manifolds. The ATR algorithm is tested on the newly released SENSIAC (Military Sensing Information Analysis Center) infrared database along with some visible-band image sequences. Overall tracking and recognition performance is evaluated in terms of the accuracy of 3D position/pose estimation and target classification.</School of Electrical & Computer Engineerin

Hybrid machine learning approaches for scene understanding: From segmentation and recognition to image parsing

We alleviate the problem of semantic scene understanding by studies on object segmentation/recognition and scene labeling methods respectively. We propose new techniques for joint recognition, segmentation and pose estimation of infrared (IR) targets. The problem is formulated in a probabilistic level set framework where a shape constrained generative model is used to provide a multi-class and multi-view shape prior and where the shape model involves a couplet of view and identity manifolds (CVIM). A level set energy function is then iteratively optimized under the shape constraints provided by the CVIM. Since both the view and identity variables are expressed explicitly in the objective function, this approach naturally accomplishes recognition, segmentation and pose estimation as joint products of the optimization process. For realistic target chips, we solve the resulting multi-modal optimization problem by adopting a particle swarm optimization (PSO) algorithm and then improve the computational efficiency by implementing a gradient-boosted PSO (GB-PSO). Evaluation was performed using the Military Sensing Information Analysis Center (SENSIAC) ATR database, and experimental results show that both of the PSO algorithms reduce the cost of shape matching during CVIM-based shape inference. Particularly, GB-PSO outperforms other recent ATR algorithms, which require intensive shape matching, either explicitly (with pre-segmentation) or implicitly (without pre-segmentation). On the other hand, under situations when target boundaries are not obviously observed and object shapes are not preferably detected, we explored some sparse representation classification (SRC) methods on ATR applications, and developed a fusion technique that combines the traditional SRC and a group constrained SRC algorithm regulated by a sparsity concentration index for improved classification accuracy on the Comanche dataset. Moreover, we present a compact rare class-oriented scene labeling framework (RCSL) with a global scene assisted rare class retrieval process, where the retrieved subset was expanded by choosing scene regulated rare class patches. A complementary rare class balanced CNN is learned to alleviate imbalanced data distribution problem at lower cost. A superpixels-based re-segmentation was implemented to produce more perceptually meaningful object boundaries. Quantitative results demonstrate the promising performances of proposed framework on both pixel and class accuracy for scene labeling on the SIFTflow dataset, especially for rare class objects

Concave-convex local binary features for automatic target recognition in infrared imagery

This paper presents a novel feature extraction algorithm based on the local binary features for automatic target recognition (ATR) in infrared imagery. Since the inception of the local binary pattern (LBP) and local ternary pattern (LTP) features, many extensions have been proposed to improve their robustness and performance in a variety of applications. However, most attentions were paid to improve local feature extraction with little consideration on the incorporation of global or regional information. In this work, we propose a new concave-convex partition (CCP) strategy to improve LBP and LTP by dividing local features into two distinct groups, i.e., concave and convex, according to the contrast between local and global intensities. Then two separate histograms built from the two categories are concatenated together to form a new LBP/LTP code that is expected to better reflect both global and local information. Experimental results on standard texture images demonstrate the improved discriminability of the proposed features and those on infrared imagery further show that the proposed features can achieve competitive ATR results compared with state-of-the-art methods.Peer reviewedElectrical and Computer Engineerin

Joint Infrared Target Recognition and Segmentation Using a Shape Manifold-Aware Level Set

We propose new techniques for joint recognition, segmentation and pose estimation of infrared (IR) targets. The problem is formulated in a probabilistic level set framework where a shape constrained generative model is used to provide a multi-class and multi-view shape prior and where the shape model involves a couplet of view and identity manifolds (CVIM). A level set energy function is then iteratively optimized under the shape constraints provided by the CVIM. Since both the view and identity variables are expressed explicitly in the objective function, this approach naturally accomplishes recognition, segmentation and pose estimation as joint products of the optimization process. For realistic target chips, we solve the resulting multi-modal optimization problem by adopting a particle swarm optimization (PSO) algorithm and then improve the computational efficiency by implementing a gradient-boosted PSO (GB-PSO). Evaluation was performed using the Military Sensing Information Analysis Center (SENSIAC) ATR database, and experimental results show that both of the PSO algorithms reduce the cost of shape matching during CVIM-based shape inference. Particularly, GB-PSO outperforms other recent ATR algorithms, which require intensive shape matching, either explicitly (with pre-segmentation) or implicitly (without pre-segmentation)

Joint Infrared Target Recognition and Segmentation Using a Shape Manifold-Aware Level Set

We propose new techniques for joint recognition, segmentation and pose estimation of infrared (IR) targets. The problem is formulated in a probabilistic level set framework where a shape constrained generative model is used to provide a multi-class and multi-view shape prior and where the shape model involves a couplet of view and identity manifolds (CVIM). A level set energy function is then iteratively optimized under the shape constraints provided by the CVIM. Since both the view and identity variables are expressed explicitly in the objective function, this approach naturally accomplishes recognition, segmentation and pose estimation as joint products of the optimization process. For realistic target chips, we solve the resulting multi-modal optimization problem by adopting a particle swarm optimization (PSO) algorithm and then improve the computational efficiency by implementing a gradient-boosted PSO (GB-PSO). Evaluation was performed using the Military Sensing Information Analysis Center (SENSIAC) ATR database, and experimental results show that both of the PSO algorithms reduce the cost of shape matching during CVIM-based shape inference. Particularly, GB-PSO outperforms other recent ATR algorithms, which require intensive shape matching, either explicitly (with pre-segmentation) or implicitly (without pre-segmentation)

The Performance of Airborne C-Band PolInSAR Data on Forest Growth Stage Types Classification

In this paper, we propose a classification scheme for forest growth stage types and other cover types using a support vector machine (SVM) based on the Polarimetric SAR Interferometric (PolInSAR) data acquired by Chinese Multidimensional Space Joint-observation SAR (MSJosSAR) system. Firstly, polarimetric, texture, and coherence features were calculated from the PolInSAR data. Secondly, the capabilities of the polarimetric, texture, and coherence features in land use/cover classification were quantified independently through histograms. Following this, the polarimetric features were used for the classification of land use/cover types, followed by a combination of texture and coherence features. Finally, the three classification results were validated against test samples using the confusion matrix. It was shown that, with the integration of texture and coherence features, the producer’s accuracy for afforested land, young forest land, medium forest land, and near-mature forest land improved by 6%, 31%, 11%, and 6%, respectively, compared with the former experiment using solely polarimetric features. Our study indicates that the forest and non-forest lands can be discriminated by the polarimetric features, which also play an important role in the separation between afforested land and other forest types as well as medium forest land and near-mature forest land. The texture features further discriminate afforested land and other forest types, while the coherence features obviously improved the separation of young forest land and medium forest land. This paper provides an effective way of identifying various land use/cover types, especially for distinguishing forest growth stages with SAR data. It would be of great interest in regions with frequent cloud coverage and limited optical data for the monitoring of land use/cover types

A Novel Jamming Method against SAR Using Nonlinear Frequency Modulation Waveform with Very High Sidelobes

Synthetic aperture radar (SAR) systems have the capacity for day-and-night and all-weather surveillance, which has become increasingly indispensable for military surveillance and global comprehensive environmental monitoring. With the development of the high-resolution SAR imaging technique, studies on SAR jamming have also received much interest. Traditional jamming methods are based on linear-frequency-modulated (LFM) signals, and this method can achieve high main-lobe jamming gain. However, its sidelobe jamming energy is very low. To solve this issue, a novel nonlinear frequency modulation (NLFM) waveform design method for SAR jamming is proposed in this paper. Compared with LFM waveforms, the designed waveforms have the same main-lobe jamming gain and very high sidelobes, which can significantly improve jamming performance. Moreover, detailed simulation experiments were carried out to verify the effectiveness of the newly proposed jamming scheme

Joint Target Tracking, Recognition and Segmentation for Infrared Imagery Using a Shape Manifold-Based Level Set

We propose a new integrated target tracking, recognition and segmentation algorithm, called ATR-Seg, for infrared imagery. ATR-Seg is formulated in a probabilistic shape-aware level set framework that incorporates a joint view-identity manifold (JVIM) for target shape modeling. As a shape generative model, JVIM features a unified manifold structure in the latent space that is embedded with one view-independent identity manifold and infinite identity-dependent view manifolds. In the ATR-Seg algorithm, the ATR problem formulated as a sequential level-set optimization process over the latent space of JVIM, so that tracking and recognition can be jointly optimized via implicit shape matching where target segmentation is achieved as a by-product without any pre-processing or feature extraction. Experimental results on the recently released SENSIAC ATR database demonstrate the advantages and effectiveness of ATR-Seg over two recent ATR algorithms that involve explicit shape matching

A Novel Jamming Method against SAR Using Nonlinear Frequency Modulation Waveform with Very High Sidelobes

Synthetic aperture radar (SAR) systems have the capacity for day-and-night and all-weather surveillance, which has become increasingly indispensable for military surveillance and global comprehensive environmental monitoring. With the development of the high-resolution SAR imaging technique, studies on SAR jamming have also received much interest. Traditional jamming methods are based on linear-frequency-modulated (LFM) signals, and this method can achieve high main-lobe jamming gain. However, its sidelobe jamming energy is very low. To solve this issue, a novel nonlinear frequency modulation (NLFM) waveform design method for SAR jamming is proposed in this paper. Compared with LFM waveforms, the designed waveforms have the same main-lobe jamming gain and very high sidelobes, which can significantly improve jamming performance. Moreover, detailed simulation experiments were carried out to verify the effectiveness of the newly proposed jamming scheme