We present an automated mechanism that can detect and characterize the building changes by analyzing airborne or satellite imagery.
The proposed framework can be categorized into three stages: building detection, boundary extraction and change identification. To detect the buildings, we utilize local phase and local amplitude from monogenic signal to extract building features for addressing issues of varying illumination. Then a support vector machine with Radial basis kernel is used for classification. In the boundary extraction stage, a level-set function with self-organizing map based segmentation method is used to find the building boundary and compute physical area of the building segments. In the last stage, the change of the detected building is identified by computing the area differences of the same building that captured at different times.
The experiments are conducted on a set of real-life aerial imagery to show the effectiveness of the proposed method

Albalooshi, Fatema

Asari, Vijayan K.

Essa, Almabrok

Santhaseelan, Varun

Sidike, Paheding

English

University of Dayton

University of DaytoneCommonsElectrical and Computer Engineering FacultyPublicationsDepartment of Electrical and ComputerEngineering6-2015Automatic Building Change Detection in WideArea SurveillancePaheding SidikeUniversity of DaytonAlmabrok EssaUniversity of DaytonFatema AlbalooshiUniversity of DaytonVijayan K. AsariUniversity of Dayton, vasari1@udayton.eduVarun SanthaseelanAuviz SystemsFollow this and additional works at: http://ecommons.udayton.edu/ece_fac_pubPart of the Electrical and Computer Engineering Commons, Structures and Materials Commons,and the Systems Engineering and Multidisciplinary Design Optimization CommonsThis Conference Paper is brought to you for free and open access by the Department of Electrical and Computer Engineering at eCommons. It hasbeen accepted for inclusion in Electrical and Computer Engineering Faculty Publications by an authorized administrator of eCommons. For moreinformation, please contact frice1@udayton.edu, mschlangen1@udayton.edu.eCommons CitationSidike, Paheding; Essa, Almabrok; Albalooshi, Fatema; Asari, Vijayan K.; and Santhaseelan, Varun, "Automatic Building ChangeDetection in Wide Area Surveillance" (2015). Electrical and Computer Engineering Faculty Publications. Paper 394.http://ecommons.udayton.edu/ece_fac_pub/394Automatic Building Change Detection in  Wide Area Surveillance 1Paheding Sidike, 1Almabrok Essa, 1Fatema Albalooshi, 1Vijayan Asari and 2Varun Santhaseelan 1Department of Electrical and Computer Engineering, University of Dayton, Dayton, USA 2Auviz Systems, Campbell, CA, USA    Abstract - We present an automated mechanism that can detect and characterize the building changes by analyzing airborne or satellite imagery. The proposed framework can be categorized into three stages: building detection, boundary extraction and change identification. To detect the buildings, we utilize local phase and local amplitude from monogenic signal to extract building features for addressing issues of varying illumination. Then a support vector machine with Radial basis kernel is used for classification. In the boundary extraction stage, a level-set function with self-organizing map based segmentation method is used to find the building boundary and compute physical area of the building segments. In the last stage, the change of the detected building is identified by computing the area differences of the same building that captured at different times. The experiments are conducted on a set of real-life aerial imagery to show the effectiveness of the proposed method.  Keywords - Building detection; change detection, level set function; self-organizing map; active contour models;   I. INTRODUCTION  Automatic building change detection has been an attractive research field over decades. Accurately detecting buildings and identifying changes have potential applications in wide area surveillance applications such as territorial planning and pipeline infrastructure monitoring. To identify changes in the buildings from an aerial or a satellite imagery is a challenging task due to low spatial resolution, different illumination conditions and most importantly buildings that may have large intra-class variation because of their diverse and complicated shapes and structure. They can also be occluded by other objects such as overhanging vegetation. Therefore, a more sophisticated method is required to tackle this challenging problem.  For building change detection, building detection is an important first step. Many building detection algorithms have been proposed in the literatures. Huertas et al. [1] utilized the shapes of the structures with cast shadows to identify buildings. They assume that the visible building surface contains smooth regions and sides of the building consist vertical structures. In [2-3], the relationship between shadow and building was analyzed to aid in extracting building structures. Kim et al. [4] proposed a four stage building detection algorithm based on a graphical model. Akcay and Aksoy [5] introduced a minimum spanning tree based method to detect buildings in complex appearances and shapes. In [6], a morphological filtering with clustering-based approach was presented for detecting building from Quickbird VHR-image.  Recently, a color invariant scheme with entropy filtering process is introduced in [7]. Neural network based building detection algorithms are also gaining a lot of popularity. In [8], support vector machine (SVM) was employed for rooftop detection, whereas Senaras et al. [9] proposed to use a two-layer hierarchical classification method to detect buildings in satellite images.    In this paper, we present a new method for building detection and change identification from aerial or satellite imagery. The proposed framework consists the following three stages: 1) building detection, 2) building boundary extraction, and 3) change identification. In the first stage of the proposed scheme, we utilize monogenic signal [10] with machine learning technique for the robust building detection and classification. This method is able to address the issues of varying illumination, varying shapes, sizes, orientation and occlusion in the buildings of interest. The generation of local characteristic features is important to distinguish the buildings from the background images. To detect the regions that contain buildings in a scene, we define a histogram of illumination invariant features based on monogenic signal analysis to represent the segmented region that belongs to the building as one class and the other regions as another class. A SVM with Radial basis kernel was used for training segments from both classes. In the second stage of the algorithm, we employ a level-set prior-based segmentation approach. One of the merits Amplitude  Local phase  Fig. 1. Feature extraction.Final DescriptorInput imagein this method is that small seed patches representing the objects of interest and other small seed patches representing the background region from one single reference image is enough to complete the training process. The outcome of the algorithm includes the detection of the objects of interest and extraction of the boundaries. Finally, for the building change identification, we compute the area (number of pixels) differences of the same building that captured at different times. The actual ground area of the buildings can be calculated based on the GSD (Ground Sample Distance) information from the sensors.   II. TECHNICAL APPROACH A. Building Detection To detect regions that contain buildings in a scene, we define a histogram of illumination invariant features to represent segments that belong to buildings as one class and other regions as another class. A SVM was trained using training segments from both classes. A test image was divided into various segments and passed through the trained model to detect whether it constitutes a building or not. The feature extraction is based on the monogenic signal representation. Computing monogenic signal enables us to split the local phase information from the local amplitude thereby achieving desired local information which is invariant to illumination. This feature extraction step is depicted in Fig. 1. The detection of construction equipment in the pipeline right of way using the local phase from the monogenic signal has been shown in our previous works [11-13]. A sample building detection result is shown in Fig. 2.  Initial detection from the aforementioned method may produces false positives. To achieve a better result, morphological operations such as image erosion followed by dilation are employed. The result of this step is illustrated in Fig. 3. B. Boundary Extraction The purpose of the boundary extraction is to compute the area of the detected buildings and identify changes. To achieve this, we propose to use a neural network based segmentation algorithm as described below.  Classical segmentation techniques usually perform the segmentation process to cluster and classify pixels in the whole image. In applications that require particular object segmentation, prior information of the objects of interest has to    be incorporated in the segmentation algorithm. Our primary objective is to obtain an accurate region segmentation and boundary extraction of objects in cluttered environments. The active contour model (ACM) [14] method has achieved success in image segmentation due to its capability to extract boundaries of objects with complex shapes and it can also handle topological changes. One limitation of this approach is that it may stop at a local minima which causes over-segmentation and leads to poor boundary output. Motivated by the specific ability of Self-organizing Maps (SOMs) [15] to learn information about the objects of interest, SOM based ACMs have been proposed with the aim of modeling and controlling the evolution of the active contour [16-18]. 1)  The Self-organizing Maps: SOM or self-organizing feature map (SOFM) is an artificial neural network (ANN) which provides unsupervised representations of high dimensional feature spaces, in significantly lower dimensional output grid of nodes known as best matching units (BMUs). The BMU is determined by calculating the minimum Euclidean distance between each node’s weight vector and the current input vector as                 ܤܯܷ ൌ  min௜,   ௝ ሼܦ݅ݏݐ௜,௝൫ݑ௞, ݓ௜௝൯ሽ                           (1)                    where  ݑ௞  is the current input pattern and ݓ௜௝  is the node’s weight vector. Every node within the BMU’s neighborhood (including BMU) has its weight vector adjusted as follows:       ݓ௜௝ሺݐ ൅ 1ሻ ൌ  ݓ௜௝ሺݐሻ ൅  Θሺݐሻܮሺݐሻฮݑ௞ െ ݓ௜௝ฮ             (2)                   where ݐ is the time step, ܮሺݐሻ is the learning rate, and Θሺݐሻ is the amount of influence that a node’s distance has on its learning.  2)  SOM based ACM: Learning based ACM is based on a modified level-set prior-based segmentation approach that integrates SOM with the level-set active contour models for boundary extraction of objects in cluttered environment. The first step in our method includes clustering the intensity information of both the building (obtained from our aforementioned building detection algorithm) and its background (randomly selected from the regions that are not Fig. 3. Refining building detection. (a) Original image, (b) initial detected building area (white regions), (c) refined mask using morphological operations, and (d) final detection output.  (a)(b) (c)(d) Fig. 2. A sample building detection result. (a) Original image, and (b) detection output (yellow bounding box showing locations of detected buildings) (a) (b) the building) through the unsupervised SOM technique to produce two SOM maps; one SOM network to represent the building and another SOM network representing the background region. The trained networks are employed into the second stage of our segmentation process to map the intensity levels of an input testing image. The mapped testing neurons are then utilized into the evolving curve energy functional of a level set protocol in the third stage of our proposed approach. A flow diagram of this process is shown in Fig. 4.    Once the building boundary is extracted, we can easily compute the actual number of the pixels within the boundary. Then according to the GSD information of the imagery, the actual area of a segmented building can be computed. A sample result of building segmentation and area (number of pixels) computation is shown in Fig. 5. III. EXPERIMENTS One of the efficient ways to identify the changes of the building in 2D imagery is to compute the building area from different times. This can be achieved through our abovementioned building area computation method. The only condition here is to have same image scene taken in different time. To show the effectiveness of our algorithm, we use two real-life imagery taken in different time obtained from Google Earth as shown in Fig. 6. Then we compute the area of each building in the images and compare their differences to find changes. The estimated building area is shown in Figs. 6(c) and 6(d).  Comparing Figs. 6(a) to 6(d), it is clear that many changes happened for the same set of buildings between 2012 and 2014. And one of the noticeable changes happened for the building 16 where there is area extension about 700 pixels as indicated in Figs. 6(b) and 6(d).  IV. CONCLUSION In this paper, we presented a building change detection scheme, where it integrates monogenic signal based feature extraction and SOM-based ACM for building change detection. Experimental results show that the proposed method is robust to image background complexity and building variations. However, we observed that when textural information of the building and background (non-building) are similar, the false positive rate goes higher. One of the reasons is that texture information extracted from local phase or local amplitude produces ambiguity for separating a building from the background, thus it also affects the segmentation results. In future, fusing with other advanced features will be investigated for improving the detection accuracy.    ACKNOWLEDGEMENTS The data set for the initial experiments was provided by the Pipeline Research Council International (PRCI).   REFERENCES [1] A. Huertas and R. Nevatia, “Detecting buildings in aerial images.” Computer Vision, Graphics, and Image Processing, vol. 41, no. 2, pp. 131-152, 1988.  [2] R. B. Irvin, D. M. McKeown, “Methods for exploiting the relationship between buildings and their shadows in aerial imagery,” IEEE Transactions on Systems, Man and Cybernetics, vol.19, no.6, pp.1564-1575, Nov/Dec 1989.  [3] Y.-T. Liow, T. Pavlidis, “Use of shadows for extracting buildings in aerial images,” Computer Vision, Graphics, and Image Processing, vol. 49, issue 2, pp. 242-277, February 1990. [4] T. Kim, J.-P. Muller, “Development of a graph-based approach for building detection,” Image and Vision Computing, vol. 17, issue 1, pp. 3-14, January 1999. (a) (b) Fig. 5. Building area computation: (a) Building boundary, (b) Computed area of each building in terms of total number of pixels. ܺ and ܻ indicate the pixel location of the building centroid (‘*’) in the image.  Fig. 4. A flow diagram for the building boundary extraction. [5] H. G. Akcay, S. Aksoy, “Building detection using directional spatial constraints,”  IEEE International Geoscience and Remote Sensing Symposium, pp. 1932-1935, July 2010. [6] S. Lefevre, J. Weber, D. Sheeren, “Automatic Building Extraction in VHR Images Using Advanced Morphological Operators,” Urban Remote Sensing Joint Event, pp. 1-5, April 2007. [7] N. Shorter and T. Kasparis, “Automatic Vegetation Identification and Building Detection from a Single Nadir Aerial Image”, Remote Sensing, vol. 1, pp. 731-757, 2009. [8] H. Baluyan, B. Joshi, A. A. Hinai, and W. L. Woon, “Novel Approach for Rooftop Detection Using Support Vector Machine,” ISRN Machine Vision, vol. 2013, pp. 1-11, 2013.  [9] C. Senaras, B. Yuksel, M. Ozay, F. Y-Vural, “Automatic building detection with feature space fusion using ensemble learning,”  IEEE International Geoscience and Remote Sensing Symposium, pp. 6713-6716, July 2012.  [10] M. Felsberg and G. Sommer, “The monogenic signal,” IEEE Transactions on Signal Processing, vol .49, pp. 3136-3144, 2001. [11] Binu Nair, Varun Santhaseelan, Chen Cui, and Vijayan K. Asari, “Intrusion detection on oil pipeline right of way (ROW) using monogenic signal representation,” SPIE Conference on Defense, Security, and Sensing: Signal Processing, Sensor Fusion, and Target Recognition XXII, 8745, 2013. [12] Vijayan K. Asari, Paheding Sidike, Chen Cui, and Varun Santhaseelan, “Recent progress in wide-area surveillance: Protecting our pipeline infrastructure,” IS&T/SPIE International Conference on Electronic Imaging: Imaging and Multimedia Analytics in a Web and Mobile World, 940802, March 2015. [13] Vijayan K. Asari, Paheding Sidike, Chen Cui, and Varun Santhaseelan, “New wide-area surveillance techniques for protection of infrastructure,” SPIE Newsroom, January 2015. [14] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” International Journal of Computer Vision, vol. 1, no. 4, pp. 321–331, 1988.  [15] T. Kohonen, “The self-organizing map,” Proceedings of the IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 1990. [16] M. Abdelsamea, G. Gnecco, and M. M. Gaber, “A concurrent SOM-based Chan-Vese model for image segmentation,” Advances in Self-Organizing Maps and Learning Vector Quantization. Springer, pp. 199-208, 2014. [17] F. Albalooshi, P. Sidike, and V. Asari, “Efficient hyperspectral image segmentation using geometric active contour formulation,” SPIE Remote Sensing Conference: Image and Signal Processing for Remote Sensing, pp. 924406, 2014. [18] F. Albalooshi, S. Smith, Y. Diskin, P. Sidike, and V. Asari, “Automatic segmentation of carcinoma in radiographs,” IEEE International Workshop on Applied Imagery and Pattern Recognition, pp. 14-16, Oct. 2014.     Fig. 6. The same scene from Google Earth imagery taken in different time: (a) May 2012, (b) June 2014, (c) segmentation result for (a), and (d) result for (b). (a) (b) (c) (d) 

Automatic Building Change Detection in Wide Area Surveillance

https://ecommons.udayton.edu/cgi/viewcontent.cgi?article=1371&amp;context=ece_fac_pub

Automatic Building Change Detection in Wide Area Surveillance

Abstract

Similar works

Full text

Available Versions

University of Dayton