Remote Sensing methods for power line corridor surveys

AbstractTo secure uninterrupted distribution of electricity, effective monitoring and maintenance of power lines are needed. This literature review article aims to give a wide overview of the possibilities provided by modern remote sensing sensors in power line corridor surveys and to discuss the potential and limitations of different approaches. Monitoring of both power line components and vegetation around them is included. Remotely sensed data sources discussed in the review include synthetic aperture radar (SAR) images, optical satellite and aerial images, thermal images, airborne laser scanner (ALS) data, land-based mobile mapping data, and unmanned aerial vehicle (UAV) data. The review shows that most previous studies have concentrated on the mapping and analysis of network components. In particular, automated extraction of power line conductors has achieved much attention, and promising results have been reported. For example, accuracy levels above 90% have been presented for the extraction of conductors from ALS data or aerial images. However, in many studies datasets have been small and numerical quality analyses have been omitted. Mapping of vegetation near power lines has been a less common research topic than mapping of the components, but several studies have also been carried out in this field, especially using optical aerial and satellite images. Based on the review we conclude that in future research more attention should be given to an integrated use of various data sources to benefit from the various techniques in an optimal way. Knowledge in related fields, such as vegetation monitoring from ALS, SAR and optical image data should be better exploited to develop useful monitoring approaches. Special attention should be given to rapidly developing remote sensing techniques such as UAVs and laser scanning from airborne and land-based platforms. To demonstrate and verify the capabilities of automated monitoring approaches, large tests in various environments and practical monitoring conditions are needed. These should include careful quality analyses and comparisons between different data sources, methods and individual algorithms

Bounding Box-Free Instance Segmentation Using Semi-Supervised Learning for Generating a City-Scale Vehicle Dataset

Vehicle classification is a hot computer vision topic, with studies ranging from ground-view up to top-view imagery. In remote sensing, the usage of top-view images allows for understanding city patterns, vehicle concentration, traffic management, and others. However, there are some difficulties when aiming for pixel-wise classification: (a) most vehicle classification studies use object detection methods, and most publicly available datasets are designed for this task, (b) creating instance segmentation datasets is laborious, and (c) traditional instance segmentation methods underperform on this task since the objects are small. Thus, the present research objectives are: (1) propose a novel semi-supervised iterative learning approach using GIS software, (2) propose a box-free instance segmentation approach, and (3) provide a city-scale vehicle dataset. The iterative learning procedure considered: (1) label a small number of vehicles, (2) train on those samples, (3) use the model to classify the entire image, (4) convert the image prediction into a polygon shapefile, (5) correct some areas with errors and include them in the training data, and (6) repeat until results are satisfactory. To separate instances, we considered vehicle interior and vehicle borders, and the DL model was the U-net with the Efficient-net-B7 backbone. When removing the borders, the vehicle interior becomes isolated, allowing for unique object identification. To recover the deleted 1-pixel borders, we proposed a simple method to expand each prediction. The results show better pixel-wise metrics when compared to the Mask-RCNN (82% against 67% in IoU). On per-object analysis, the overall accuracy, precision, and recall were greater than 90%. This pipeline applies to any remote sensing target, being very efficient for segmentation and generating datasets.Comment: 38 pages, 10 figures, submitted to journa

Container Number Recognition Method Based on SSD_MobileNet and SVM

Aiming at how to realize the recognition of the container number on the container surface at the entrance and exit of the port, a method based on image affine transformation and SVM classifier is proposed. The main process includes truck target detection, box number area detection, text correction stage, image preprocessing stage and segmentation detection and recognition stage. Firstly, a kind of container truck detection program based on frame difference method and decreasing sequence of connected domain is proposed; secondly, a method of container number area detection based on SSD_MobileNet is proposed; in the case number recognition stage, a text correction method based on image affine transformation is proposed, and different processing methods are proposed for vertical sequence box number and horizontal sequence box number in image preprocessing stage In the stage of segmentation detection and recognition, a character segmentation algorithm based on connected domain segmentation and a segmentation detection and recognition algorithm based on SVM classifier are proposed. Through the detection and recognition of container images in the field monitoring video, the accuracy rate of regional detection can reach 97%, and the accuracy rate of character recognition can reach 95%, and it can achieve good real-time performance

Automatic vehicle detection and tracking in aerial video

This thesis is concerned with the challenging tasks of automatic and real-time vehicle detection and tracking from aerial video. The aim of this thesis is to build an automatic system that can accurately localise any vehicles that appear in aerial video frames and track the target vehicles with trackers. Vehicle detection and tracking have many applications and this has been an active area of research during recent years; however, it is still a challenge to deal with certain realistic environments. This thesis develops vehicle detection and tracking algorithms which enhance the robustness of detection and tracking beyond the existing approaches. The basis of the vehicle detection system proposed in this thesis has different object categorisation approaches, with colour and texture features in both point and area template forms. The thesis also proposes a novel Self-Learning Tracking and Detection approach, which is an extension to the existing Tracking Learning Detection (TLD) algorithm. There are a number of challenges in vehicle detection and tracking. The most difficult challenge of detection is distinguishing and clustering the target vehicle from the background objects and noises. Under certain conditions, the images captured from Unmanned Aerial Vehicles (UAVs) are also blurred; for example, turbulence may make the vehicle shake during flight. This thesis tackles these challenges by applying integrated multiple feature descriptors for real-time processing. In this thesis, three vehicle detection approaches are proposed: the HSV-GLCM feature approach, the ISM-SIFT feature approach and the FAST-HoG approach. The general vehicle detection approaches used have highly flexible implicit shape representations. They are based on training samples in both positive and negative sets and use updated classifiers to distinguish the targets. It has been found that the detection results attained by using HSV-GLCM texture features can be affected by blurring problems; the proposed detection algorithms can further segment the edges of the vehicles from the background. Using the point descriptor feature can solve the blurring problem, however, the large amount of information contained in point descriptors can lead to processing times that are too long for real-time applications. So the FAST-HoG approach combining the point feature and the shape feature is proposed. This new approach is able to speed up the process that attains the real-time performance. Finally, a detection approach using HoG with the FAST feature is also proposed. The HoG approach is widely used in object recognition, as it has a strong ability to represent the shape vector of the object. However, the original HoG feature is sensitive to the orientation of the target; this method improves the algorithm by inserting the direction vectors of the targets. For the tracking process, a novel tracking approach was proposed, an extension of the TLD algorithm, in order to track multiple targets. The extended approach upgrades the original system, which can only track a single target, which must be selected before the detection and tracking process. The greatest challenge to vehicle tracking is long-term tracking. The target object can change its appearance during the process and illumination and scale changes can also occur. The original TLD feature assumed that tracking can make errors during the tracking process, and the accumulation of these errors could cause tracking failure, so the original TLD proposed using a learning approach in between the tracking and the detection by adding a pair of inspectors (positive and negative) to constantly estimate errors. This thesis extends the TLD approach with a new detection method in order to achieve multiple-target tracking. A Forward and Backward Tracking approach has been proposed to eliminate tracking errors and other problems such as occlusion. The main purpose of the proposed tracking system is to learn the features of the targets during tracking and re-train the detection classifier for further processes. This thesis puts particular emphasis on vehicle detection and tracking in different extreme scenarios such as crowed highway vehicle detection, blurred images and changes in the appearance of the targets. Compared with currently existing detection and tracking approaches, the proposed approaches demonstrate a robust increase in accuracy in each scenario

A geometrical-based approach to recognise structure of complex interiors

3D modelling of building interiors has gained a lot of interest recently, specifically since the rise of Building Information Modeling (BIM). A number of methods have been developed in the past, however most of them are limited to modelling non-complex interiors. 3D laser scanners are the preferred sensor to collect the 3D data, however the cost of state-of-the-art laser scanners are prohibitive to many. Other types of sensors could also be used to generate the 3D data but they have limitations especially when dealing with clutter and occlusions. This research has developed a platform to produce 3D modelling of building interiors while adapting a low-cost, low-level laser scanner to generate the 3D interior data. The PreSuRe algorithm developed here, which introduces a new pipeline in modelling building interiors, combines both novel methods and adapts existing approaches to produce the 3D modelling of various interiors, from sparse room to complex interiors with non-ideal geometrical structure, highly cluttered and occluded. This approach has successfully reconstructed the structure of interiors, with above 96% accuracy, even with high amount of noise data and clutter. The time taken to produce the resulting model is almost real-time, compared to existing techniques which may take hours to generate the reconstruction. The produced model is also equipped with semantic information which differentiates the model from a regular 3D CAD drawing and can be use to assist professionals and experts in related fields

Integrazione di Misurazioni Laser e Visione Artificiale nella Ricostruzione di Ambienti 3D per la Navigazione Autonoma di Veicoli in Ambienti Sconosciuti

Utilizzando uno scanner laser e un dispositivo di visione artificiale sono discussi algoritmi per il riconoscimento di segmenti e superfici in ambienti 3D. Alle superfici riconosciute sono applicate textures prese dalla camera. Il laser e la visione sono infine utilizzati in modo integrato per permettere ad un veicolo su cui sono installati di poter navigare autonomamente nell'ambiente inizialmente sconosciuto e contemporaneamente mapparlo

A comparison of line extraction algorithms using 2D range data for indoor mobile robotics

This paper presents an experimental evaluation of different line extraction algorithms applied to 2D laser scans for indoor environments. Six popular algorithms in mobile robotics and computer vision are selected and tested. Real scan data collected from two office environments by using different platforms are used in the experiments in order to evaluate the algorithms. Several comparison criteria are proposed and discussed to highlight the advantages and drawbacks of each algorithm, including speed, complexity, correctness and precision. The results of the algorithms are compared with ground truth using standard statistical methods. An extended case study is performed to further evaluate the algorithms in a SLAM applicatio

Power line mapping technique using all-terrain mobile laser scanning

Power line mapping using remote sensing can automate the traditionally labor-intensive power line corridor inspection. Land-based mobile laser scanning (MLS) can be a good choice for the power line mapping if an aerial inspection is impossible, too costly or slow, unsafe, prohibited by regulations, or if more detailed information on the power line corridor is needed. The mapping of the power lines using MLS was studied in a rural environment outside the road network for the first time. An automatic power line extraction algorithm was developed. The algorithm first found power line candidate points based on the shape and orientation of the local neighborhood of a point using principal component analysis. Power lines were retrieved from the candidates using random sample consensus (Ransac) and a new power line labeling method, which takes into account the three-dimensional shape of the power lines. The new labeling method was able to find the power lines and remove false detections, which were found, for example, from the forest. The algorithm was tested in forested and open field (arable land) areas, outside the road environment using two different platforms of MLS, namely, personal backpack and all-terrain vehicle. The recall and precision of the power line extraction were 93.3% and 93.6%, respectively, using 10 cm as a distance criterion for a successful detection. Drifting of the positioning solution of the scanner was the largest error source, being the (contributory) cause for 60–70% of the errors. The platform did not have a significant effect on the power line extraction accuracy. The accuracy was higher in the open field compared to the forest, because the one-dimensional point density along the power line was inhomogeneous and GNSS (global navigation satellite system) signal was weak in the forest. The results suggest that the power lines can be mapped accurately enough for inspection purposes using MLS in a rural environment outside the road network.</p

3D Classification of Power Line Scene Using Airborne Lidar Data

Failure to adequately maintain vegetation within a power line corridor has been identified as a main cause of the August 14, 2003 electric power blackout. Such that, timely and accurate corridor mapping and monitoring are indispensible to mitigate such disaster. Moreover, airborne LiDAR (Light Detection And Ranging) has been recently introduced and widely utilized in industries and academies thanks to its potential to automate the data processing for scene analysis including power line corridor mapping. However, today’s corridor mapping practice using LiDAR in industries still remains an expensive manual process that is not suitable for the large-scale, rapid commercial compilation of corridor maps. Additionally, in academies only few studies have developed algorithms capable of recognizing corridor objects in the power line scene, which are mostly based on 2-dimensional classification. Thus, the objective of this dissertation is to develop a 3-dimensional classification system which is able to automatically identify key objects in the power line corridor from large-scale LiDAR data. This dissertation introduces new features for power structures, especially for the electric pylon, and existing features which are derived through diverse piecewise (i.e., point, line and plane) feature extraction, and then constructs a classification model pool by building individual models according to the piecewise feature sets and diverse voltage training samples using Random Forests. Finally, this dissertation proposes a Multiple Classifier System (MCS) which provides an optimal committee of models from the model pool for classification of new incoming power line scene. The proposed MCS has been tested on a power line corridor where medium voltage transmission lines (115 kV and 230 kV) pass. The classification results based on the MCS applied by optimally selecting the pre-built classification models according to the voltage type of the test corridor demonstrate a good accuracy (89.07%) and computationally effective time cost (approximately 4 hours/km) without additional training fees