Autonomous UAV System for Cleaning Insulators in Power Line Inspection and Maintenance

The inspection and maintenance tasks of electrical installations are very demanding. Nowadays, insulator cleaning is carried out manually by operators using scaffolds, ropes, or even helicopters. However, these operations involve potential risks for humans and the electrical structure. The use of Unmanned Aerial Vehicles (UAV) to reduce the risk of these tasks is rising. This paper presents an UAV to autonomously clean insulators on power lines. First, an insulator detection and tracking algorithm has been implemented to control the UAV in operation. Second, a cleaning tool has been designed consisting of a pump, a tank, and an arm to direct the flow of cleaning liquid. Third, a vision system has been developed that is capable of detecting soiled areas using a semantic segmentation neuronal network, calculating the trajectory for cleaning in the image plane, and generating arm trajectories to efficiently clean the insulator. Fourth, an autonomous system has been developed to land on a charging pad to charge the batteries and potentially fill the tank with cleaning liquid. Finally, the autonomous system has been validated in a controlled outdoor environment.Ministerio de Ciencia e Innovación (CDTI) AERIAL-CORE H2020 ICT-10-2019-2020FEDER INTERCONECT

Assessing thermal imagery integration into object detection methods on ground-based and air-based collection platforms

Object detection models commonly deployed on uncrewed aerial systems (UAS) focus on identifying objects in the visible spectrum using Red-Green-Blue (RGB) imagery. However, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) images to increase the performance of object detection machine learning (ML) models. Currently LWIR ML models have received less research attention, especially for both ground- and air-based platforms, leading to a lack of baseline performance metrics evaluating LWIR, RGB and LWIR-RGB fused object detection models. Therefore, this research contributes such quantitative metrics to the literature. The results found that the ground-based blended RGB-LWIR model exhibited superior performance compared to the RGB or LWIR approaches, achieving a mAP of 98.4%. Additionally, the blended RGB-LWIR model was also the only object detection model to work in both day and night conditions, providing superior operational capabilities. This research additionally contributes a novel labelled training dataset of 12,600 images for RGB, LWIR, and RGB-LWIR fused imagery, collected from ground-based and air-based platforms, enabling further multispectral machine-driven object detection research.Comment: 18 pages, 12 figures, 2 table

Identification and three-dimensional positioning of urban energy lines from optical images to aid a teleoperated pruning robot

Orientador : Prof. Dr. Leandro dos Santos CoelhoDissertação (mestrado) - Universidade Federal do Paraná, Setor de Tecnologia, Programa de Pós-Graduação em Engenharia Elétrica. Defesa: Curitiba, 30/08/2016Inclui referências : f. 138-144Área de concentraçãoResumo: Diversos fatores podem impactar a qualidade da distribuição de energia elétrica, entre eles, um dos mais impactantes é o contato de vegetação com linhas aéreas energizadas. Assim sendo, é de suma importância a poda de vegetação próxima à linhas energizadas. Visando-se aprimorar esse processo, pode-se empregar um robô teleoperado de poda, de forma que a poda possa ser realizada de maneira remota e segura. As câmeras instaladas no braço robótico permitem que o operador tenha visão da área de corte mesmo quando a visada direta do solo estiver obstruída. Um dos problemas de se visualizar a região de corte por meio de um monitor é a perda de noção de profundidade, o que pode dificultar a operação. Dessa forma, seria relevante uma técnica de visão computacional capaz de detectar as linhas de energia e seu posicionamento tridimensional (3D) a fim de auxiliar o operador. Revisando-se a literatura, avaliou-se que, no geral, os trabalhos já propostos para detecção de linhas em imagens operam em situações com fundo limpo, não urbanizado e com vista superior das linhas de energia. Assim sendo, nesse trabalho é proposta uma técnica para detecção de linhas energizadas em imagens de regiões urbanas e a obtenção de seu posicionamento 3D, fator ainda não explorado na literatura recente. Para se alcançar esse objetivo é proposta a utilização de câmeras de espectro visível posicionadas em paralelo. Assim, regiões com potencial para serem linhas de energia são selecionadas utilizando-se detecção de bordas seguidas por filtragens geométricas aplicando-se técnicas inspiradas em algoritmos de grafos e ajuste de pontos selecionados a uma curva. Após a seleção de regiões candidatas a linha de energia, o posicionamento 3D é obtido utilizando-se de visão estéreo. Para tal, a correspondência entre pontos visíveis em ambas as câmeras é encontrada e com triangulação o posicionamento 3D da linha de energia é recuperado. Com a informação 3D disponível falsos candidatos são reduzidos por um fator de aproximadamente sete vezes e finalmente as linhas são detectadas. Para avaliação do método foi criada uma base de dados contendo imagens estéreo obtidas de um cenário montado com dois postes, três linhas de energia e uma árvore entre essas, na qual foi possível atingir níveis de precisão de 98% ao término do processo de detecção, contando-se com 91% de taxa de verdadeiro positivos. As causas dos falsos negativos são evidenciadas para que trabalhos futuros possam encontrar alternativas às dificuldades apresentadas. O algoritmo aqui proposto fornece como saída um mapa de cor sobre as linhas de energia para identificação da profundidade em 2D e uma nuvem de pontos para visualização em 3D. Palavras-chave: Visão Computacional. Reconhecimento de objetos. Visão estéreo. Linhas de energia. Robô de poda.Abstract: Different factors may affect energy distribution quality, among them, one of the main causes is when vegetation gets into contact with overhead energy lines. Therefore, it is of main importance to prune vegetation close to energy lines. To improve this process it is possible to use a teleoperated robot, what allows the pruning activity to be accomplished in a remote and safe way. Cameras installed in the robot arm provide images from the pruning region to the operator even when direct sight is not an option. One of the main problems viewing the prunning region using a display is the lost of depth perception, what could make the operator unintentialy colide the robot with energy lines. Therefore, it would be of great aid a computer vision method capable of detecting energy lines and their three-dimensional (3D) positioning to aid the operator. During the state of the art review of energy line detection in images, it was perceived that, in general, the already proposed works operate in regions where the images present a clear background, not urbanized, and with the energy lines seen from above. Therefore, in this work, it is proposed a technique to detect energy lines and their 3D positioning in images taken in urban settings, factor yet unexplored in the recent literature. To reach this objective it is proposed the use of two visible spectrum cameras installed in parallel. In this way, regions with potential to be energy line are selected using edge detection followed by the geometric filtering designed using techniques inspired in graphs algorithms and curve fitting. After the regions with potential to be energy lines are found, their 3D position is obtained with stereo vision. To do so, the matching among points visible by both cameras is found and with triangulation, it is possible to recover the energy line 3D position. With the 3D information available, false positives are reduced by a factor of about seven and finally the energy lines are detected. A dataset containing stereo images of a scenario built with two power poles, three energy lines, and a tree between them was created in order to evaluate the presented method. In the commented dataset it was possible to reach accuracy of 98% at the end of the detection process, with 91% true positive rate. The causes of the false negatives cases are put in evidence in order to allow them to be overcame by future works. The algorithm proposed here outputs a colormap projected over the energy lines to inform the depth of each one in 2D and a point cloud to visualize each line in 3D. Key words: Computer vision. Object Recognition. Stereo vision. Overhead Energy Lines. Pruning Robo