Exploitation of time-of-flight (ToF) cameras

This technical report reviews the state-of-the art in the field of ToF cameras, their advantages, their limitations, and their present-day applications sometimes in combination with other sensors. Even though ToF cameras provide neither higher resolution nor larger ambiguity-free range compared to other range map estimation systems, advantages such as registered depth and intensity data at a high frame rate, compact design, low weight and reduced power consumption have motivated their use in numerous areas of research. In robotics, these areas range from mobile robot navigation and map building to vision-based human motion capture and gesture recognition, showing particularly a great potential in object modeling and recognition.Preprin

Cross-source Point Cloud Registration: Challenges, Progress and Prospects

The emerging topic of cross-source point cloud (CSPC) registration has attracted increasing attention with the fast development background of 3D sensor technologies. Different from the conventional same-source point clouds that focus on data from same kind of 3D sensor (e.g., Kinect), CSPCs come from different kinds of 3D sensors (e.g., Kinect and { LiDAR}). CSPC registration generalizes the requirement of data acquisition from same-source to different sources, which leads to generalized applications and combines the advantages of multiple sensors. In this paper, we provide a systematic review on CSPC registration. We first present the characteristics of CSPC, and then summarize the key challenges in this research area, followed by the corresponding research progress consisting of the most recent and representative developments on this topic. Finally, we discuss the important research directions in this vibrant area and explain the role in several application fields.Comment: Accepted by Neurocomputing 202

Ship Multimodel 3D Reconstruction and Corrosion Detection

3D reconstruction has been an area of increased interest due to the current higher demand in applications, such as virtual realities, 3D mapping, medical imaging, and many others. Although, there are still many problems associated with reconstructing a real-life object, such as capturing occluded zones, noise, and processing time. Furthermore, as deep learning technologies advance, there has been a growing interest in using such methods to replace human-driven tasks, namely corrosion inspection, as it decreases the risk of injury of the inspector, it is more efficient due to less time taken, and is cost-saving. This dissertation proposes a method for reconstructing a 3D model of ships using aerial RGB images and terrestrial RGB-D images, along with a system capable of detecting the corroded parts of the ship and highlighting them in the model. Using two different sensors in two different ground planes mitigates some of the occlusion problems and increases the final model’s accuracy. The current dissertation also aims to pick the methods that have the best trade-off between accuracy and computational speed. The final model can be advantageous for corrosion inspectors, as they will have the model of the ship, as well as the corroded zones which, with that information, can choose the steps to take next without the need to manually inspect the ship or even be in the same site as the ship. The final model is a fusion of three different 3D models. The model obtained from RGB images exploits Structure from Motion algorithm which recovers the 3D aspect of the ship from 2D images. As for the remaining models, RGB-D images were used in conjunction with the Open3D library to create 3D structures from both sides of the ship. The corrosion classifier model was trained in Google Colab and achieved an accuracy of 97.44 % on the test dataset. The images used to create the SfM 3D model were each divided into a total of 40 regions and fed into the classifier to simulate a less concise image detection algorithm instead of an image classification algorithm. The results were encoded into the 3D model, highlighting the corroded zones.A reconstrução 3D tem sido uma área com crescente interesse devido à maior demanda em aplicações como realidade virtual, mapeamento 3D, imagens médicas e muitos outros. Embora, existem ainda muitos problemas associados à reconstrução 3D de um objeto real. Exemplos desses são a captura de zonas oclusas, o ruído e o tempo de processamento necessário para efetuar a reconstrução. Adicionalmente, com o avanço das tecnologias de deep learning, tem havido um acrescido interesse em usar ditos métodos para substituir tarefas realizadas por humanos como, por exemplo, a inspeção de corrosão, pois diminui o risco de lesões ao inspetor, tem maior eficiência devido a um menor tempo gasto, e economiza os custos. Esta dissertação propõe um método de reconstrução de um modelo 3D de navios, utilizando imagens RGB aéreas e imagens RGB-D terrestres, juntamente com um sistema capaz de detetar as zonas com corrosão no navio e destacá-las no modelo. O uso de dois sensores diferentes em dois meios diferentes atenuará alguns dos problemas de oclusão e aumentará a precisão do modelo final. A presente dissertação também visa escolher os métodos que apresentam o melhor compromisso entre precisão e velocidade de processamento. O modelo final poderá ser vantajoso para os inspetores de corrosão, pois terão o modelo do navio, bem como as zonas com corrosão que, com essa informação, poderão escolher quais os passos a seguir, sem a necessidade de inspecionar manualmente o navio ou mesmo deslocar-se para o local do navio. O modelo final é uma fusão de três modelos 3D diferentes. O modelo obtido a partir de imagens RGB tirou partido do algoritmo Structure from Motion, que recupera o aspeto 3D do navio a partir de imagens 2D. Quanto aos modelos restantes, as imagens RGB-D foram utilizadas em conjunto com a biblioteca Open3D para criar estruturas 3D de ambos os lados do navio. O modelo de classificação de corrosão foi treinado em ambiente Google Colab e alcançou uma exatidão de 97.44% no dataset de teste. As imagens usadas para criar o modelo SfM 3D foram, cada uma, fracionadas num total de 40 regiões e dadas ao modelo de classificação com o intuito de simularum modelo de deteção de imagem menos conciso em vez de um modelo de classificação de imagem. Os resultados foram codificados no modelo 3D, destacando as zonas com corrosão

AFFECT-PRESERVING VISUAL PRIVACY PROTECTION

The prevalence of wireless networks and the convenience of mobile cameras enable many new video applications other than security and entertainment. From behavioral diagnosis to wellness monitoring, cameras are increasing used for observations in various educational and medical settings. Videos collected for such applications are considered protected health information under privacy laws in many countries. Visual privacy protection techniques, such as blurring or object removal, can be used to mitigate privacy concern, but they also obliterate important visual cues of affect and social behaviors that are crucial for the target applications. In this dissertation, we propose to balance the privacy protection and the utility of the data by preserving the privacy-insensitive information, such as pose and expression, which is useful in many applications involving visual understanding. The Intellectual Merits of the dissertation include a novel framework for visual privacy protection by manipulating facial image and body shape of individuals, which: (1) is able to conceal the identity of individuals; (2) provide a way to preserve the utility of the data, such as expression and pose information; (3) balance the utility of the data and capacity of the privacy protection. The Broader Impacts of the dissertation focus on the significance of privacy protection on visual data, and the inadequacy of current privacy enhancing technologies in preserving affect and behavioral attributes of the visual content, which are highly useful for behavior observation in educational and medical settings. This work in this dissertation represents one of the first attempts in achieving both goals simultaneously