Unifying terrain awareness for the visually impaired through real-time semantic segmentation.

Navigational assistance aims to help visually-impaired people to ambulate the environment safely and independently. This topic becomes challenging as it requires detecting a wide variety of scenes to provide higher level assistive awareness. Vision-based technologies with monocular detectors or depth sensors have sprung up within several years of research. These separate approaches have achieved remarkable results with relatively low processing time and have improved the mobility of impaired people to a large extent. However, running all detectors jointly increases the latency and burdens the computational resources. In this paper, we put forward seizing pixel-wise semantic segmentation to cover navigation-related perception needs in a unified way. This is critical not only for the terrain awareness regarding traversable areas, sidewalks, stairs and water hazards, but also for the avoidance of short-range obstacles, fast-approaching pedestrians and vehicles. The core of our unification proposal is a deep architecture, aimed at attaining efficient semantic understanding. We have integrated the approach in a wearable navigation system by incorporating robust depth segmentation. A comprehensive set of experiments prove the qualified accuracy over state-of-the-art methods while maintaining real-time speed. We also present a closed-loop field test involving real visually-impaired users, demonstrating the effectivity and versatility of the assistive framework

RADIATE: A Radar Dataset for Automotive Perception in Bad Weather

Datasets for autonomous cars are essential for the development and benchmarking of perception systems. However, most existing datasets are captured with camera and LiDAR sensors in good weather conditions. In this paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to facilitate research on object detection, tracking and scene understanding using radar sensing for safe autonomous driving. RADIATE includes 3 hours of annotated radar images with more than 200K labelled road actors in total, on average about 4.6 instances per radar image. It covers 8 different categories of actors in a variety of weather conditions (e.g., sun, night, rain, fog and snow) and driving scenarios (e.g., parked, urban, motorway and suburban), representing different levels of challenge. To the best of our knowledge, this is the first public radar dataset which provides high-resolution radar images on public roads with a large amount of road actors labelled. The data collected in adverse weather, e.g., fog and snowfall, is unique. Some baseline results of radar based object detection and recognition are given to show that the use of radar data is promising for automotive applications in bad weather, where vision and LiDAR can fail. RADIATE also has stereo images, 32-channel LiDAR and GPS data, directed at other applications such as sensor fusion, localisation and mapping. The public dataset can be accessed at http://pro.hw.ac.uk/radiate/.Comment: Accepted at IEEE International Conference on Robotics and Automation 2021 (ICRA 2021

Software architecture for self-driving navigation

Mención Internacional en el título de doctorThis dissertation is based on the development of the navigation software architecture for self-driving vehicles. The goal is very wide in terms of multidisciplinary fields over the different solutions provided, however, functional solutions for the implementation according to the software architecture has been proved and tested in the real research platform iCab (Intelligent Campus Automobile). The problems that the autonomous vehicles have to face are based accordingly as the three questions of navigation that each vehicle has to ask: Where am I, where should I go, and how can I go there. These questions are followed by the corresponding modules to solve that are divided into localization, planning, mapping, perception and control in addition to multitasking allocation, communication and Human-Machine Interaction. One more module is the self-awareness which is an optimal solution for detecting problems in the earliest stage. Throughout this document, the solution provided in form of a complete architecture for navigation describes the modules involved and the importance of software connections between them, generation of trajectories, mapping, localization and low level control. Finally, the results section describes scenarios and vehicle/software performance in terms of CPU for each module involved and the generation of trajectories, maps and control commands needed to move the vehicle from one point to another.Este documento es el resultado de cinco años de trabajo en el campo de los vehículos sin conductor donde, en el, se recoge el desarrollo de una arquitectura software de control para la navegación de este tipo de vehículos. El objetivo es muy ambicioso ya que para su desarrollo ha sido necesario el conocimiento de múltiples disciplinas como ingeniería electrónica, ingeniería informática, ingeniería de control, procesamiento de señales, mecánica y visión por computador. A pesar del vasto conocimiento necesario para lograr un vehículo funcional, se han alcanzado soluciones para cada uno de los problemas en que consiste la navegación autónoma, generando un vehículo autogobernado que toma decisiones por si mismo para evitar obstáculos y alcanzar los puntos de destino deseados. Los problemas principales que los vehículos autónomos tienen que hacer frente, están basados en tres preguntas principales: donde estoy, donde tengo que ir y como voy. Para responder a estas tres preguntas se ha dividido la arquitectura en los módulos siguientes: localización, planificación, mapeado del entorno y control junto con módulos extra para dotar al sistema de mas aptitudes y mejor funcionamiento como por ejemplo la comunicación entre vehículos, peatones e infraestructuras, la interacción humano máquina, la gestión de tareas con múltiples vehículos o la propia consciencia del vehículo en cuanto a su estado de baterías, mantenimiento, sensores conectados o desconectados, etc. A través de este documento, la solución proporcionada a cada uno de los módulos involucrados refleja la importancia de las conexiones de software y la comunicación entre procesos dentro de la arquitectura ya sea para la generación de trayectorias, la creación de los mapas a tiempo, la localización precisa en el entorno, o los comandos generados para gobernar el vehículo. Así mismo, en el apartado de resultados se pone de manifiesto la importancia de cumplir los plazos de compartición de mensajes y optimizar el sistema para no sobrecargar la CPU.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Felipe Jiménez Alonso.- Secretario: Agapito Ismael Ledezma Espino.- Vocal: Alessio Malizi

Epälambertilaiset pinnat ja niiden haasteet konenäössä

This thesis regards non-Lambertian surfaces and their challenges, solutions and study in computer vision. The physical theory for understanding the phenomenon is built first, using the Lambertian reflectance model, which defines Lambertian surfaces as ideally diffuse surfaces, whose luminance is isotropic and the luminous intensity obeys Lambert's cosine law. From these two assumptions, non-Lambertian surfaces violate at least the cosine law and are consequently specularly reflecting surfaces, whose perceived brightness is dependent from the viewpoint. Thus non-Lambertian surfaces violate also brightness and colour constancies, which assume that the brightness and colour of same real-world points stays constant across images. These assumptions are used, for example, in tracking and feature matching and thus non-Lambertian surfaces pose complications for object reconstruction and navigation among other tasks in the field of computer vision. After formulating the theoretical foundation of necessary physics and a more general reflectance model called the bi-directional reflectance distribution function, a comprehensive literature review into significant studies regarding non-Lambertian surfaces is conducted. The primary topics of the survey include photometric stereo and navigation systems, while considering other potential fields, such as fusion methods and illumination invariance. The goal of the survey is to formulate a detailed and in-depth answer to what methods can be used to solve the challenges posed by non-Lambertian surfaces, what are these methods' strengths and weaknesses, what are the used datasets and what remains to be answered by further research. After the survey, a dataset is collected and presented, and an outline of another dataset to be published in an upcoming paper is presented. Then a general discussion about the survey and the study is undertaken and conclusions along with proposed future steps are introduced

A review on deep learning techniques for 3D sensed data classification

Over the past decade deep learning has driven progress in 2D image understanding. Despite these advancements, techniques for automatic 3D sensed data understanding, such as point clouds, is comparatively immature. However, with a range of important applications from indoor robotics navigation to national scale remote sensing there is a high demand for algorithms that can learn to automatically understand and classify 3D sensed data. In this paper we review the current state-of-the-art deep learning architectures for processing unstructured Euclidean data. We begin by addressing the background concepts and traditional methodologies. We review the current main approaches including; RGB-D, multi-view, volumetric and fully end-to-end architecture designs. Datasets for each category are documented and explained. Finally, we give a detailed discussion about the future of deep learning for 3D sensed data, using literature to justify the areas where future research would be most valuable.Comment: 25 pages, 9 figures. Review pape

A survey on real-time 3D scene reconstruction with SLAM methods in embedded systems

The 3D reconstruction of simultaneous localization and mapping (SLAM) is an important topic in the field for transport systems such as drones, service robots and mobile AR/VR devices. Compared to a point cloud representation, the 3D reconstruction based on meshes and voxels is particularly useful for high-level functions, like obstacle avoidance or interaction with the physical environment. This article reviews the implementation of a visual-based 3D scene reconstruction pipeline on resource-constrained hardware platforms. Real-time performances, memory management and low power consumption are critical for embedded systems. A conventional SLAM pipeline from sensors to 3D reconstruction is described, including the potential use of deep learning. The implementation of advanced functions with limited resources is detailed. Recent systems propose the embedded implementation of 3D reconstruction methods with different granularities. The trade-off between required accuracy and resource consumption for real-time localization and reconstruction is one of the open research questions identified and discussed in this paper

Generación del dataset de imágenes etiquetadas SAUCE

En el mundo actual cada vez estamos más acostumbrados a la búsqueda de la autonomía, esa que debido a su alta complejidad resulta difícil de encontrar. En los medios urbanos de tráfico lograr esa autonomía resulta, por su gran variedad y extensión, un gran reto para los sistemas inteligentes de los vehículos. Respecto a esto, son muchas las soluciones y algoritmos presentados hasta el momento, con resultados muy dispares, costes o cargas computacionales que en muchas ocasiones se pueden evitar si se tuviera un sitio donde contrastar soluciones. Este trabajo se encuadra dentro del procesamiento digital y presenta un soporte ante esta gran variedad de sistemas inteligentes utilizados en la búsqueda de la autonomía de los vehículos, desarrolla un sitio de comparación con una base de datos de imágenes etiquetadas basadas en la arquitectura ROS de escenas semánticas en estéreovisión, y permite que usuarios externos puedan comparar el producto de sus algoritmos con dicha base de datos y contrasten sus resultados con los de otros usuarios en función de diversos parámetros (CPU, procesador, calidad del etiquetado resultante, etc.). El procedimiento que se ha utilizado para lograr este proyecto con eficacia consta de las siguientes fases: Creación de base de datos que consta de 100 etiquetas realizadas a mano una a una a partir de fotos originales realizadas por el vehículo autónomo iCab en el campus de la UC3M de Leganés, desarrollo del algoritmo de comparación cogiendo de base las OpenCV y lenguaje Python, realización de pruebas de comparación para evaluar la eficiencia del algoritmo y la creación de un sitio web (con base Wordpress) para poner a disposición de todos la posibilidad de comparar el resultado de sus algoritmos con los de nuestra base de datos.In the world today we are becoming more accustomed to the search for autonomy, which due to its high complexity is difficult to find. In the urban means of traffic achieving this autonomy is, by its great variety and extension, a great challenge for the intelligent systems of vehicles. Regarding this, there are many solutions and algorithms presented so far, with very different results, costs or computational loads that in many cases can be avoided if there is a place to test solutions. This work fits within the digital processing and presents a support to this great variety of intelligent systems used in the search of the autonomy of the vehicles, develops a comparison site with a labeled images based in ROS architecture for stereovision- based semantic scene, and allows External users can compare the product of their algorithms with that database and contrast their results with those of other users based on various parameters (CPU, processor, quality of the resulting labeling, etc.). The procedure that has been used to achieve this project effectively consists of the following phases: Creation of database consisting of 100 handmade tags one by one from original photos taken by the iCab standalone vehicle on the campus of the UC3M, development of the comparison algorithm based on the OpenCV and Python language, performing comparison tests to evaluate the efficiency of the algorithm and the creation of a website (based on Wordpress) to make available to all the possibility of Compare the results of your algorithms with those of our database.Ingeniería Electrónica Industrial y Automátic

Deep Learning of Semantic Image Labels on HDR Imagery in a Maritime Environment

Situational awareness in the maritime environment can be extremely challenging. The maritime environment is highly dynamic and largely undefined, requiring the perception of many potential hazards in the shared maritime environment. One particular challenge is the effect of direct-sunlight exposure and specular reflection causing degradation of camera reliability. It is for this reason then, in this work, the use of High-Dynamic Range imagery for deep learning of semantic image labels is studied in a littoral environment. This study theorizes that the use HDR imagery may be extremely beneficial for the purpose of situational awareness in maritime environments due to the inherent advantages of the technology. This study creates labels for a multi-class semantic segmentation process, and performs well on water and horizon identification in the littoral zone. Additionally, this work contributes proof that water can be reasonably identified using HDR imagery with semantic networks, which is useful for determining the navigable regions for a vessel. This result is a basis on which to build further semantic segmentation work upon in this environment, and could be further improved upon in future works with the introduction of additional data for multi-class segmentation problems