Landmark-based Localization using Stereo Vision and Deep Learning in GPS-Denied Battlefield Environment

Localization in a battlefield environment is increasingly challenging as GPS connectivity is often denied or unreliable, and physical deployment of anchor nodes across wireless networks for localization can be difficult in hostile battlefield terrain. Existing range-free localization methods rely on radio-based anchors and their average hop distance which suffers from accuracy and stability in dynamic and sparse wireless network topology. Vision-based methods like SLAM and Visual Odometry use expensive sensor fusion techniques for map generation and pose estimation. This paper proposes a novel framework for localization in non-GPS battlefield environments using only the passive camera sensors and considering naturally existing or artificial landmarks as anchors. The proposed method utilizes a customcalibrated stereo vision camera for distance estimation and the YOLOv8s model, which is trained and fine-tuned with our real-world dataset for landmark recognition. The depth images are generated using an efficient stereomatching algorithm, and distances to landmarks are determined by extracting the landmark depth feature utilizing a bounding box predicted by the landmark recognition model. The position of the unknown node is then obtained using the efficient least square algorithm and then optimized using the L-BFGS-B (limited-memory quasi-Newton code for bound-constrained optimization) method. Experimental results demonstrate that our proposed framework performs better than existing anchorbased DV-Hop algorithms and competes with the most efficient vision-based algorithms in terms of localization error (RMSE).Comment: arXiv admin note: text overlap with arXiv:2402.1232

Navigation, localization and stabilization of formations of unmanned aerial and ground vehicles

A leader-follower formation driving algorithm developed for control of heterogeneous groups of unmanned micro aerial and ground vehicles stabilized under a top-view relative localization is presented in this paper. The core of the proposed method lies in a novel avoidance function, in which the entire 3D formation is represented by a convex hull projected along a desired path to be followed by the group. Such a representation of the formation provides non-collision trajectories of the robots and respects requirements of the direct visibility between the team members in environment with static as well as dynamic obstacles, which is crucial for the top-view localization. The algorithm is suited for utilization of a simple yet stable visual based navigation of the group (referred to as GeNav), which together with the on-board relative localization enables deployment of large teams of micro-scale robots in environments without any available global localization system. We formulate a novel Model Predictive Control (MPC) based concept that enables to respond to the changing environment and that provides a robust solution with team members' failure tolerance included. The performance of the proposed method is verified by numerical and hardware experiments inspired by reconnaissance and surveillance missions

Sistemas de posicionamento baseados em comunicação por luz para ambientes interiores

The demand for highly precise indoor positioning systems (IPSs) is growing rapidly due to its potential in the increasingly popular techniques of the Internet of Things, smart mobile devices, and artificial intelligence. IPS becomes a promising research domain that is getting wide attention due to its benefits in several working scenarios, such as, industries, indoor public locations, and autonomous navigation. Moreover, IPS has a prominent contribution in day-to-day activities in organizations such as health care centers, airports, shopping malls, manufacturing, underground locations, etc., for safe operating environments. In indoor environments, both radio frequency (RF) and optical wireless communication (OWC) based technologies could be adopted for localization. Although the RF-based global positioning system, such as, Global positioning system offers higher penetration rates with reduced accuracy (i.e., in the range of a few meters), it does not work well in indoor environments (and not at all in certain cases such as tunnels, mines, etc.) due to the very weak signal and no direct access to the satellites. On the other hand, the light-based system known as a visible light positioning (VLP) system, as part of the OWC systems, uses the pre-existing light-emitting diodes (LEDs)-based lighting infrastructure, could be used at low cost and high accuracy compared with the RF-based systems. VLP is an emerging technology promising high accuracy, high security, low deployment cost, shorter time response, and low relative complexity when compared with RFbased positioning. However, in indoor VLP systems, there are some concerns such as, multipath reflection, transmitter tilting, transmitter’s position, and orientation uncertainty, human shadowing/blocking, and noise causing the increase in the positioning error, thereby reducing the positioning accuracy of the system. Therefore, it is imperative to capture the characteristics of different VLP channel and properly model them for the dual purpose of illumination and localization. In this thesis, firstly, the impact of transmitter tilting angles and multipath reflections are studied and for the first time, it is demonstrated that tilting the transmitter can be beneficial in VLP systems considering both line of sight (LOS) and non-line of sight transmission paths. With the transmitters oriented towards the center of the receiving plane, the received power level is maximized due to the LOS components. It is also shown that the proposed scheme offers a significant accuracy improvement of up to ~66% compared with a typical non-tilted transmitter VLP. The effect of tilting the transmitter on the lighting uniformity is also investigated and results proved that the uniformity achieved complies with the European Standard EN 12464-1. After that, the impact of transmitter position and orientation uncertainty on the accuracy of the VLP system based on the received signal strength (RSS) is investigated. Simulation results show that the transmitter uncertainties have a severe impact on the positioning error, which can be leveraged through the usage of more transmitters. Concerning a smaller transmitter’s position epochs, and the size of the training set. It is shown that, the ANN with Bayesian regularization outperforms the traditional RSS technique using the non-linear least square estimation for all values of signal to noise ratio. Furthermore, a novel indoor VLP system is proposed based on support vector machines and polynomial regression considering two different multipath environments of an empty room and a furnished room. The results show that, in an empty room, the positioning accuracy improvement for the positioning error of 2.5 cm are 36.1, 58.3, and 72.2 % for three different scenarios according to the regions’ distribution in the room. For the furnished room, a positioning relative accuracy improvement of 214, 170, and 100 % is observed for positioning error of 0.1, 0.2, and 0.3 m, respectively. Ultimately, an indoor VLP system based on convolutional neural networks (CNN) is proposed and demonstrated experimentally in which LEDs are used as transmitters and a rolling shutter camera is used as receiver. A detection algorithm named single shot detector (SSD) is used which relies on CNN (i.e., MobileNet or ResNet) for classification as well as position estimation of each LED in the image. The system is validated using a real-world size test setup containing eight LED luminaries. The obtained results show that the maximum average root mean square positioning error achieved is 4.67 and 5.27 cm with SSD MobileNet and SSD ResNet models, respectively. The validation results show that the system can process 67 images per second, allowing real-time positioning.A procura por sistemas de posicionamento interior (IPSs) de alta precisão tem crescido rapidamente devido ao seu interesse nas técnicas cada vez mais populares da Internet das Coisas, dispositivos móveis inteligentes e inteligência artificial. O IPS tornou-se um domínio de pesquisa promissor que tem atraído grande atenção devido aos seus benefícios em vários cenários de trabalho, como indústrias, locais públicos e navegação autónoma. Além disso, o IPS tem uma contribuição destacada no dia a dia de organizações, como, centros de saúde, aeroportos, supermercados, fábricas, locais subterrâneos, etc. As tecnologias baseadas em radiofrequência (RF) e comunicação óptica sem fio (OWC) podem ser adotadas para localização em ambientes interiores. Embora o sistema de posicionamento global (GPS) baseado em RF ofereça taxas de penetração mais altas com precisão reduzida (ou seja, na faixa de alguns metros), não funciona bem em ambientes interiores (e não funciona bem em certos casos como túneis, minas, etc.) devido ao sinal muito fraco e falta de acesso direto aos satélites. Por outro lado, o sistema baseado em luz conhecido como sistema de posicionamento de luz visível (VLP), como parte dos sistemas OWC, usa a infraestrutura de iluminação baseada em díodos emissores de luz (LEDs) pré-existentes, é um sistemas de baixo custo e alta precisão quando comprado com os sistemas baseados em RF. O VLP é uma tecnologia emergente que promete alta precisão, alta segurança, baixo custo de implantação, menor tempo de resposta e baixa complexidade relativa quando comparado ao posicionamento baseado em RF. No entanto, os sistemas VLP interiores, exibem algumas limitações, como, a reflexão multicaminho, inclinação do transmissor, posição do transmissor e incerteza de orientação, sombra/bloqueio humano e ruído, que têm como consequência o aumento do erro de posicionamento, e consequente redução da precisão do sistema. Portanto, é imperativo estudar as características dos diferentes canais VLP e modelá-los adequadamente para o duplo propósito de iluminação e localização. Esta tesa aborda, primeiramente, o impacto dos ângulos de inclinação do transmissor e reflexões multipercurso no desempenho do sistema de posicionamento. Demonstra-se que a inclinação do transmissor pode ser benéfica em sistemas VLP considerando tanto a linha de vista (LOS) como as reflexões. Com os transmissores orientados para o centro do plano recetor, o nível de potência recebido é maximizado devido aos componentes LOS. Também é mostrado que o esquema proposto oferece uma melhoria significativa de precisão de até ~66% em comparação com um sistema VLP de transmissor não inclinado típico. O efeito da inclinação do transmissor na uniformidade da iluminação também é investigado e os resultados comprovam que a uniformidade alcançada está de acordo com a Norma Europeia EN 12464-1. O impacto da posição do transmissor e incerteza de orientação na precisão do sistema VLP com base na intensidade do sinal recebido (RSS) foi também investigado. Os resultados da simulação mostram que as incertezas do transmissor têm um impacto severo no erro de posicionamento, que pode ser atenuado com o uso de mais transmissores. Para incertezas de posicionamento dos transmissores menores que 5 cm, os erros médios de posicionamento são 23.3, 15.1 e 13.2 cm para conjuntos de 4, 9 e 16 transmissores, respetivamente. Enquanto que, para a incerteza de orientação de um transmissor menor de 5°, os erros médios de posicionamento são 31.9, 20.6 e 17 cm para conjuntos de 4, 9 e 16 transmissores, respetivamente. O trabalho da tese abordou a investigação dos aspetos de projeto de um sistema VLP indoor no qual uma rede neuronal artificial (ANN) é utilizada para estimativa de posicionamento considerando um canal multipercurso. O estudo considerou a influência do ruído como indicador de desempenho para a comparação entre diferentes abordagens de projeto. Três algoritmos de treino de ANNs diferentes foram considerados, a saber, Levenberg-Marquardt, regularização Bayesiana e algoritmos de gradiente conjugado escalonado, para minimizar o erro de posicionamento no sistema VLP. O projeto da ANN foi otimizado com base no número de neurónios nas camadas ocultas, no número de épocas de treino e no tamanho do conjunto de treino. Mostrou-se que, a ANN com regularização Bayesiana superou a técnica RSS tradicional usando a estimação não linear dos mínimos quadrados para todos os valores da relação sinal-ruído. Foi proposto um novo sistema VLP indoor baseado em máquinas de vetores de suporte (SVM) e regressão polinomial considerando dois ambientes interiores diferentes: uma sala vazia e uma sala mobiliada. Os resultados mostraram que, numa sala vazia, a melhoria da precisão de posicionamento para o erro de posicionamento de 2.5 cm são 36.1, 58.3 e 72.2% para três cenários diferentes de acordo com a distribuição das regiões na sala. Para a sala mobiliada, uma melhoria de precisão relativa de posicionamento de 214, 170 e 100% é observada para erro de posicionamento de 0.1, 0.2 e 0.3 m, respetivamente. Finalmente, foi proposto um sistema VLP indoor baseado em redes neurais convolucionais (CNN). O sistema foi demonstrado experimentalmente usando luminárias LED como transmissores e uma camara com obturador rotativo como recetor. O algoritmo de detecção usou um detector de disparo único (SSD) baseado numa CNN pré configurada (ou seja, MobileNet ou ResNet) para classificação. O sistema foi validado usando uma configuração de teste de tamanho real contendo oito luminárias LED. Os resultados obtidos mostraram que o erro de posicionamento quadrático médio alcançado é de 4.67 e 5.27 cm com os modelos SSD MobileNet e SSD ResNet, respetivamente. Os resultados da validação mostram que o sistema pode processar 67 imagens por segundo, permitindo o posicionamento em tempo real.Programa Doutoral em Engenharia Eletrotécnic

Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection