Design Of Computer Vision Systems For Optimizing The Threat Detection Accuracy

This dissertation considers computer vision (CV) systems in which a central monitoring station receives and analyzes the video streams captured and delivered wirelessly by multiple cameras. It addresses how the bandwidth can be allocated to various cameras by presenting a cross-layer solution that optimizes the overall detection or recognition accuracy. The dissertation presents and develops a real CV system and subsequently provides a detailed experimental analysis of cross-layer optimization. Other unique features of the developed solution include employing the popular HTTP streaming approach, utilizing homogeneous cameras as well as heterogeneous ones with varying capabilities and limitations, and including a new algorithm for estimating the effective medium airtime. The results show that the proposed solution significantly improves the CV accuracy. Additionally, the dissertation features an improved neural network system for object detection. The proposed system considers inherent video characteristics and employs different motion detection and clustering algorithms to focus on the areas of importance in consecutive frames, allowing the system to dynamically and efficiently distribute the detection task among multiple deployments of object detection neural networks. Our experimental results indicate that our proposed method can enhance the mAP (mean average precision), execution time, and required data transmissions to object detection networks. Finally, as recognizing an activity provides significant automation prospects in CV systems, the dissertation presents an efficient activity-detection recurrent neural network that utilizes fast pose/limbs estimation approaches. By combining object detection with pose estimation, the domain of activity detection is shifted from a volume of RGB (Red, Green, and Blue) pixel values to a time-series of relatively small one-dimensional arrays, thereby allowing the activity detection system to take advantage of highly capable neural networks that have been trained on large GPU clusters for thousands of hours. Consequently, capable activity detection systems with considerably fewer training sets and processing hours can be built

5G-Enabled Autonomous Platooning on Robotic Vehicle Testbed

Humanity is progressively moving towards a more intuitive and technological future. The area of Intelligent and Cooperative Transport Systems has revealed itself as one of the areas in great evolution, through technologies of autonomous driving and intravehicle communication. With the main goal of providing accident-free environments as well as optimizing the movement of vehicles on roads all over the world, Vehicle to Everything (V2X) communication is very important when it comes to all kinds of vehicular applications. The CMU|PT FLOYD project focuses on this area, with the aim of developing new systems for possible future implementation. In this report, a vehicular application using a 5G-capable module to perform Vehicle to Infrastructure (V2I) communications was evaluated. This vehicular application is based on an emergency braking scenario, whereby detecting an approaching vehicle in a place where an accident occurred, a message is sent over the network that is picked up by the main vehicle, triggering braking. It should be noted that this sending will be made through the module with 5G capacity, thus being an innovative application. Complementary to this scenario is the tracking of a vehicle by another vehicle, thus making a more complex emergency braking application with a cooperative platoon. This platoon will be maintained through sensors present in the following vehicle, such as LiDAR and ZED camera. With this, image processing and a sensor fusion was done in order to keep the follower at a safe distance but with the ability to follow the leader. In order to validate and test this entire solution, robotic testbeds were used as a low-cost solution, allowing a concrete evaluation, with enlightening physical results of the entire application performed.A humanidade, está a caminhar, progressivamente, para um futuro mais intuitivo e tecnológico. A área dos Sistemas Inteligentes e Cooperativos de Transporte tem-se revelado como uma das áreas em grande evolução, através de tecnologias de condução autónoma e comunicação intra-veicular. Com o objetivo principal de proporcionar ambientes sem acidentes, assim como otimizar o movimento de veículos nas estradas de todo o mundo, a comunicação V2X é muito importante no que toca a todo o tipo de aplicações veiculares. O projeto CMU|PT FLOYD centra-se nesta mesma área, com o intuito de desenvolver novos sistemas de possível implementação futura. Neste relatório, é avaliada assim uma aplicação veicular utilizando um módulo com capacidade 5G para realizar comunicações V2I. Essa aplicação veicular baseiase num cenário de travagem de emergência, em que ao detetar uma aproximação de um veículo num local onde ocorreu um acidente, é enviada uma mensagem pela rede que é captada pelo veículo principal, despoletando a travagem. De destacar que este envio será feito através do módulo com capacidade 5G, sendo desta forma uma aplicação inovadora. Complementado a este cenário está a realização do seguimento de um veículo por parte de um outro veículo, tornando assim uma aplicação mais complexa de travagem de emergência com um pelotão cooperativo. Este pelotão será mantido através de sensores presentes no veículo seguidor como o LiDAR e a ZED camera. Com isto, foi utilizado processamento de imagem e foi feita a fusão de sensores de forma a manter o seguidor a uma distância de segurança mas com capacidade de seguir o líder. Com o objetivo de validar e testar toda esta solução, foram utilizadas plataformas robóticas como solução de baixo custo, permitindo assim ter uma avaliação concreta, com resultados físicos esclarecedores de toda a aplicação realizada

Substracción de fondo y algoritmo yolo: Dos métodos para la detección de personas en entornos descontrolados

Introduction: This article is the result of research entitled “Signal processing system for the detection of people in agglomerations in areas of public space in the city of Cúcuta”, developed at the Universidad Francisco de Paula Santander in 2020.Problem: The high percentage of false positives and false negatives in people detection processes makes decision making in video surveillance, tracking and tracing applications complex. Objective: To determine which technique for the detection of people presents better results in terms of respon-se time and detection hits.Methodology: Two techniques for the detection of people in uncontrolled environments are validated in Python with videos taken inside the Universidad Francisco de Paula Santander: Background subtraction and the YOLO algorithm.Results: With the background subtraction technique, we obtained a hit rate of 84.07 % and an average response time of 0.815 seconds. Likewise, with the YOLO algorithm the hit rate and average response time are 90% and 4.59 seconds respectively.Conclusion: It is possible to infer the use of the background subtraction technique in hardware tools such as the Pi 3B+ Raspberry board for processes in which the analysis of information in real time is prioritized, while the YOLO algorithm presents the characteristics required in the processes in which the information is analyzed after the acquisition of the image.Originality: Through this research, aspects required for the real-time analysis of information obtained in pro-cesses of people detection in uncontrolled environments were analyzed. Limitations: The analyzed videos were taken only at the Universidad Francisco de Paula Santander. Also, the Raspberry Pi 3B+ board overheats when processing the video images, due to the full resource requirement of the device.Introducción: Este artículo es resultado de la investigación titulada “Sistema de procesamiento de señales para la detección de personas en aglomeraciones en zonas de espacio público de la ciudad de Cúcuta”, desarrollada en la Universidad Francisco de Paula Santander en el año 2020.Problema: El alto porcentaje de falsos positivos y falsos negativos en los procesos de detección de personas hace que la toma de decisiones en las aplicaciones de videovigilancia, seguimiento y localización sea compleja. Objetivo: Determinar qué técnica de detección de personas presenta mejores resultados en cuanto a tiempo de respuesta y aciertos en la detección.Metodología: Dos técnicas para la detección de personas en entornos no controlados son validadas en Python con videos tomados dentro de la Universidad Francisco de Paula Santander: la sustracción de fondo y el al-goritmo YOLO.Resultados: Con la técnica de sustracción de fondo se obtuvo una tasa de acierto del 84,07 % y un tiempo de respuesta medio de 0,815 segundos. Asimismo, con el algoritmo YOLO, la tasa de acierto y el tiempo de respuesta promedio son del 90% y 4,59 segundos respectivamente.Conclusión: Es posible inferir el uso de la técnica de sustracción de fondo en herramientas de hardware como la placa Raspberry Pi 3B+ para procesos en los que se prioriza el análisis de la información en tiempo real, mientras que el algoritmo YOLO presenta las características requeridas en los procesos en los que se analiza la información después de la adquisición de la imagen.Originalidad: A través de esta investigación se analizaron los aspectos necesarios para el análisis en tiempo real de la información obtenida en los procesos de detección de personas en ambientes no controlados

U-DiVE: Design and evaluation of a distributed photorealistic virtual reality environment

This dissertation presents a framework that allows low-cost devices to visualize and interact with photorealistic scenes. To accomplish this task, the framework makes use of Unity’s high-definition rendering pipeline, which has a proprietary Ray Tracing algorithm, and Unity’s streaming package, which allows an application to be streamed within its editor. The framework allows the composition of a realistic scene using a Ray Tracing algorithm, and a virtual reality camera with barrel shaders, to correct the lens distortion needed for the use on an inexpensive cardboard. It also includes a method to collect the mobile device’s spatial orientation through a web browser to control the user’s view, delivered via WebRTC. The proposed framework can produce low-latency, realistic and immersive environments to be accessed through low-cost HMDs and mobile devices. To evaluate the structure, this work includes the verification of the frame rate achieved by the server and mobile device, which should be higher than 30 FPS for a smooth experience. In addition, it discusses whether the overall quality of experience is acceptable by evaluating the delay of image delivery from the server up to the mobile device, in face of user’s movement. Our tests showed that the framework reaches a mean latency around 177 (ms) with household Wi-Fi equipment and a maximum latency variation of 77.9 (ms), among the 8 scenes tested.Esta dissertação apresenta um framework que permite que dispositivos de baixo custo visualizem e interajam com cenas fotorrealísticas. Para realizar essa tarefa, o framework faz uso do pipeline de renderização de alta definição do Unity, que tem um algoritmo de rastreamento de raio proprietário, e o pacote de streaming do Unity, que permite o streaming de um aplicativo em seu editor. O framework permite a composição de uma cena realista usando um algoritmo de Ray Tracing, e uma câmera de realidade virtual com shaders de barril, para corrigir a distorção da lente necessária para usar um cardboard de baixo custo. Inclui também um método para coletar a orientação espacial do dispositivo móvel por meio de um navegador Web para controlar a visão do usuário, entregue via WebRTC. O framework proposto pode produzir ambientes de baixa latência, realistas e imersivos para serem acessados por meio de HMDs e dispositivos móveis de baixo custo. Para avaliar a estrutura, este trabalho considera a verificação da taxa de quadros alcançada pelo servidor e pelo dispositivo móvel, que deve ser superior a 30 FPS para uma experiência fluida. Além disso, discute se a qualidade geral da experiência é aceitável, ao avaliar o atraso da entrega das imagens desde o servidor até o dispositivo móvel, em face da movimentação do usuário. Nossos testes mostraram que o framework atinge uma latência média em torno dos 177 (ms) com equipamentos wi-fi de uso doméstico e uma variação máxima das latências igual a 77.9 (ms), entre as 8 cenas testadas