Deep Learning Approach for UAV Visual Electrical Assets Inspection

The growth in the electrical demand by most countries around the world requires bigger and more complex energy systems, which leads to the requirement of having even more monitoring, inspection and maintenance of those systems. To respond to this need, inspection methods based on Unmanned Aerial Vehicles (UAV) have emerged which, when equipped with the appropriate sensors, allow a greater reduction of costs and risks and an increase in efficiency and effectiveness compared to traditional methods, such as inspection with foot patrols or helicopter-assisted. To make the inspection process more autonomous and reliable, most of the methods apply visual detection methods that use highly complex Deep Learning based algorithms and that require a very large computational power. This dissertation intends to present a system for inspection of electrical assets, able to be integrated onboard the UAV, based on Deep Learning, which allows to collect visual samples grouped and aggregated for each electrical asset detected. To this end, a perception system capable of detecting electrical insulators or structures, such as poles or transmission towers, was developed, using the Movidius Neural Compute Stick portable platform that is capable of processing lightweight object detection Convolutional Neural Networks, allowing a modular, low-cost system that meets real-time processing requirements. In addition to this perception system, an electrical asset monitoring system has been implemented that allows tracking and mapping each asset throughout the inspection process, based on the previous system’s detections and a UAV navigation system. Finally, an autonomous inspection system is proposed, which consists of a set of trajectories that allow an efficient application of the monitoring system along a power line, through the mapping of structures and the gathering of insulator samples around that structure.O grande crescimento da exigência elétrica pela maioria dos países por todo o mundo, requer que os sistemas de energia sejam maiores e mais complexos, o que conduz a uma maior necessidade de monitorização, inspeção e manutenção desses sistemas. Para responder a esta necessidade, surgiram métodos de inspeção baseados em Veículos Aéreos Não Tripulados (VANT) que, quando equipados com os sensores apropriados, permitem uma maior redução de custos e riscos e um grande aumento de eficiência e eficácia em comparação com os métodos tradicionais, como a inspeção com patrulhas pedonais ou assistida por helicóptero. Para tornar processo de inspeção mais autónomo e confiável, a maioria dos métodos realiza método de deteção visuais que utilizam algoritmos baseados em Deep Learning de elevada complexidade e que requerem um poder computacional muito grande. Nesta dissertação pretende-se apresentar um sistema de inspeção de ativos elétricos, para integração em VANTs, baseado em Apredizagem Profunda, que permite recolher amostras visuais agrupadas e agregadas por cada ativo elétrico detetado. Para tal foi desenvolvido um sistema de perceção capaz de detetar isoladores elétricos ou estruturas, como postes ou torres de transmissão, com recurso `a plataforma portátil Movidius Neural Compute Stick que ´e capaz de processar Redes Neuronais Convolucionais leves de deteção de objetos, permitindo assim um sistema modular, de baixo custo e que cumpre requisitos de processamento em tempo real. Para além deste sistema de perceção, foi implementado um sistema de monitorização de ativos elétricos que permite seguir e mapear cada ativo ao longo do processo de inspeção, com base nas deteções do sistema anterior e no sistema de navegação do VANT. Por fim, ´e proposto um sistema de inspeção autónomo que consiste num conjunto de trajetórias que permitem aplicar o sistema de monitorização de ativos elétricos ao longo de uma linha elétrica, através do mapeamento de estruturas e na recolha de amostras de isoladores em torno dessa estrutura

Arbitrary-Oriented Detection of Insulators in Thermal Imagery via Rotation Region Network

10.13039/501100004607-Natural Science Foundation of Guangxi Province (Grant Number: 2018JJB160056); Guangxi Science and Technology Base and Talent Project (Grant Number: 2020AC19010); 10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 51907034)

3D Reconstruction of Building Rooftop and Power Line Models in Right-of-Ways Using Airborne LiDAR Data

The research objectives aimed to achieve thorough the thesis are to develop methods for reconstructing models of building and PL objects of interest in the power line (PL) corridor area from airborne LiDAR data. For this, it is mainly concerned with the model selection problem for which model is more optimal in representing the given data set. This means that the parametric relations and geometry of object shapes are unknowns and optimally determined by the verification of hypothetical models. Therefore, the proposed method achieves high adaptability to the complex geometric forms of building and PL objects. For the building modeling, the method of implicit geometric regularization is proposed to rectify noisy building outline vectors which are due to noisy data. A cost function for the regularization process is designed based on Minimum Description Length (MDL) theory, which favours smaller deviation between a model and observation as well as orthogonal and parallel properties between polylines. Next, a new approach, called Piecewise Model Growing (PMG), is proposed for 3D PL model reconstruction using a catenary curve model. It piece-wisely grows to capture all PL points of interest and thus produces a full PL 3D model. However, the proposed method is limited to the PL scene complexity, which causes PL modeling errors such as partial, under- and over-modeling errors. To correct the incompletion of PL models, the inner and across span analysis are carried out, which leads to replace erroneous PL segments by precise PL models. The inner span analysis is performed based on the MDL theory to correct under- and over-modeling errors. The across span analysis is subsequently carried out to correct partial-modeling errors by finding start and end positions of PLs which denotes Point Of Attachment (POA). As a result, this thesis addresses not only geometrically describing building and PL objects but also dealing with noisy data which causes the incompletion of models. In the practical aspects, the results of building and PL modeling should be essential to effectively analyze a PL scene and quickly alleviate the potentially hazardous scenarios jeopardizing the PL system

Image-Based Localization of Ultraviolet Corona

Systems and methods for identifying a fault in an electrical distribution system may include capturing an image of a corona event associated with a component of the electrical distribution system and processing the image of the corona event to identify the center and boundary of the corona event. An overlay for display with the image of the corona event may be generated, where the overlay identifying at least a center of the corona event and may be displayed to a user via a user interface device. In some cases, an unmanned aerial vehicle may comprise one or more image sensors capable of capturing the image of the corona event and communicate the images to a remote device for processing and display to the user. The image captured by the image sensors may include an ultraviolet image, a visible spectrum image, an infrared image, or a combination of image types

Remote Sensing methods for power line corridor surveys

AbstractTo secure uninterrupted distribution of electricity, effective monitoring and maintenance of power lines are needed. This literature review article aims to give a wide overview of the possibilities provided by modern remote sensing sensors in power line corridor surveys and to discuss the potential and limitations of different approaches. Monitoring of both power line components and vegetation around them is included. Remotely sensed data sources discussed in the review include synthetic aperture radar (SAR) images, optical satellite and aerial images, thermal images, airborne laser scanner (ALS) data, land-based mobile mapping data, and unmanned aerial vehicle (UAV) data. The review shows that most previous studies have concentrated on the mapping and analysis of network components. In particular, automated extraction of power line conductors has achieved much attention, and promising results have been reported. For example, accuracy levels above 90% have been presented for the extraction of conductors from ALS data or aerial images. However, in many studies datasets have been small and numerical quality analyses have been omitted. Mapping of vegetation near power lines has been a less common research topic than mapping of the components, but several studies have also been carried out in this field, especially using optical aerial and satellite images. Based on the review we conclude that in future research more attention should be given to an integrated use of various data sources to benefit from the various techniques in an optimal way. Knowledge in related fields, such as vegetation monitoring from ALS, SAR and optical image data should be better exploited to develop useful monitoring approaches. Special attention should be given to rapidly developing remote sensing techniques such as UAVs and laser scanning from airborne and land-based platforms. To demonstrate and verify the capabilities of automated monitoring approaches, large tests in various environments and practical monitoring conditions are needed. These should include careful quality analyses and comparisons between different data sources, methods and individual algorithms

Visual localisation of electricity pylons for power line inspection

Inspection of power infrastructure is a regular maintenance event. To date the inspection process has mostly been done manually, but there is growing interest in automating the process. The automation of the inspection process will require an accurate means for the localisation of the power infrastructure components. In this research, we studied the visual localisation of a pylon. The pylon is the most prominent component of the power infrastructure and can provide a context for the inspection of the other components. Point-based descriptors tend to perform poorly on texture less objects such as pylons, therefore we explored the localisation using convolutional neural networks and geometric constraints. The crossings of the pylon, or vertices, are salient points on the pylon. These vertices aid with recognition and pose estimation of the pylon. We were successfully able to use a convolutional neural network for the detection of the vertices. A model-based technique, geometric hashing, was used to establish the correspondence between the stored pylon model and the scene object. We showed the effectiveness of the method as a voting technique to determine the pose estimation from a single image. In a localisation framework, the method serves as the initialization of the tracking process. We were able to incorporate an extended Kalman filter for subsequent incremental tracking of the camera relative to the pylon. Also, we demonstrated an alternative tracking using heatmap details from the vertex detection. We successfully demonstrated the proposed algorithms and evaluated their effectiveness using a model pylon we built in the laboratory. Furthermore, we revalidated the results on a real-world outdoor electricity pylon. Our experiments illustrate that model-based techniques can be deployed as part of the navigation aspect of a robot

Vegetation Detection and Classification for Power Line Monitoring

Electrical network maintenance inspections must be regularly executed, to provide a continuous distribution of electricity. In forested countries, the electrical network is mostly located within the forest. For this reason, during these inspections, it is also necessary to assure that vegetation growing close to the power line does not potentially endanger it, provoking forest fires or power outages. Several remote sensing techniques have been studied in the last years to replace the labor-intensive and costly traditional approaches, be it field based or airborne surveillance. Besides the previously mentioned disadvantages, these approaches are also prone to error, since they are dependent of a human operator’s interpretation. In recent years, Unmanned Aerial Vehicle (UAV) platform applicability for this purpose has been under debate, due to its flexibility and potential for customisation, as well as the fact it can fly close to the power lines. The present study proposes a vegetation management and power line monitoring method, using a UAV platform. This method starts with the collection of point cloud data in a forest environment composed of power line structures and vegetation growing close to it. Following this process, multiple steps are taken, including: detection of objects in the working environment; classification of said objects into their respective class labels using a feature-based classifier, either vegetation or power line structures; optimisation of the classification results using point cloud filtering or segmentation algorithms. The method is tested using both synthetic and real data of forested areas containing power line structures. The Overall Accuracy of the classification process is about 87% and 97-99% for synthetic and real data, respectively. After the optimisation process, these values were refined to 92% for synthetic data and nearly 100% for real data. A detailed comparison and discussion of results is presented, providing the most important evaluation metrics and a visual representations of the attained results.Manutenções regulares da rede elétrica devem ser realizadas de forma a assegurar uma distribuição contínua de eletricidade. Em países com elevada densidade florestal, a rede elétrica encontra-se localizada maioritariamente no interior das florestas. Por isso, durante estas inspeções, é necessário assegurar também que a vegetação próxima da rede elétrica não a coloca em risco, provocando incêndios ou falhas elétricas. Diversas técnicas de deteção remota foram estudadas nos últimos anos para substituir as tradicionais abordagens dispendiosas com mão-de-obra intensiva, sejam elas através de vigilância terrestre ou aérea. Além das desvantagens mencionadas anteriormente, estas abordagens estão também sujeitas a erros, pois estão dependentes da interpretação de um operador humano. Recentemente, a aplicabilidade de plataformas com Unmanned Aerial Vehicles (UAV) tem sido debatida, devido à sua flexibilidade e potencial personalização, assim como o facto de conseguirem voar mais próximas das linhas elétricas. O presente estudo propõe um método para a gestão da vegetação e monitorização da rede elétrica, utilizando uma plataforma UAV. Este método começa pela recolha de dados point cloud num ambiente florestal composto por estruturas da rede elétrica e vegetação em crescimento próximo da mesma. Em seguida,múltiplos passos são seguidos, incluindo: deteção de objetos no ambiente; classificação destes objetos com as respetivas etiquetas de classe através de um classificador baseado em features, vegetação ou estruturas da rede elétrica; otimização dos resultados da classificação utilizando algoritmos de filtragem ou segmentação de point cloud. Este método é testado usando dados sintéticos e reais de áreas florestais com estruturas elétricas. A exatidão do processo de classificação é cerca de 87% e 97-99% para os dados sintéticos e reais, respetivamente. Após o processo de otimização, estes valores aumentam para 92% para os dados sintéticos e cerca de 100% para os dados reais. Uma comparação e discussão de resultados é apresentada, fornecendo as métricas de avaliação mais importantes e uma representação visual dos resultados obtidos