Toward 3D reconstruction of outdoor scenes using an MMW radar and a monocular vision sensor

International audienceIn this paper, we introduce a geometric method for 3D reconstruction of the exterior environment using a panoramic microwave radar and a camera. We rely on the complementarity of these two sensors considering the robustness to the environmental conditions and depth detection ability of the radar, on the one hand, and the high spatial resolution of a vision sensor, on the other. Firstly, geometric modeling of each sensor and of the entire system is presented. Secondly, we address the global calibration problem, which consists of finding the exact transformation between the sensors' coordinate systems. Two implementation methods are proposed and compared, based on the optimization of a non-linear criterion obtained from a set of radar-to-image target correspondences. Unlike existing methods, no special configuration of the 3D points is required for calibration. This makes the methods flexible and easy to use by a non-expert operator. Finally, we present a very simple, yet robust 3D reconstruction method based on the sensors' geometry. This method enables one to reconstruct observed features in 3D using one acquisition (static sensor), which is not always met in the state of the art for outdoor scene reconstruction.The proposed methods have been validated with synthetic and real data

Vision-Based Monocular SLAM in Micro Aerial Vehicle

Micro Aerial Vehicles (MAVs) are popular for their efficiency, agility, and lightweights. They can navigate in dynamic environments that cannot be accessed by humans or traditional aircraft. These MAVs rely on GPS and it will be difficult for GPS-denied areas where it is obstructed by buildings and other obstacles.  Simultaneous Localization and Mapping (SLAM) in an unknown environment can solve the aforementioned problems faced by flying robots.  A rotation and scale invariant visual-based solution, oriented fast and rotated brief (ORB-SLAM) is one of the best solutions for localization and mapping using monocular vision.  In this paper, an ORB-SLAM3 has been used to carry out the research on localizing micro-aerial vehicle Tello and mapping an unknown environment.  The effectiveness of ORB-SLAM3 was tested in a variety of indoor environments.   An integrated adaptive controller was used for an autonomous flight that used the 3D map, produced by ORB-SLAM3 and our proposed novel technique for robust initialization of the SLAM system during flight.  The results show that ORB-SLAM3 can provide accurate localization and mapping for flying robots, even in challenging scenarios with fast motion, large camera movements, and dynamic environments.  Furthermore, our results show that the proposed system is capable of navigating and mapping challenging indoor situations

PanoDepth - Panoramic Monocular Depth Perception Model and Framework

Depth perception has become a heavily researched area as companies and researchers are striving towards the development of self-driving cars. Self-driving cars rely on perceiving the surrounding area, which heavily depends on technology capable of providing the system with depth perception capabilities. In this paper, we explore developing a single camera (monocular) depth prediction model that is trained on panoramic depth images. Our model makes novel use of transfer learning efficient encoder models, pre-training on a larger dataset of flat depth images, and optimizing the model for use with a Jetson Nano. Additionally, we present a training and optimization framework to make developing and testing new monocular depth perception models easier and faster. While the model failed to achieve a high frame rate, the framework and models developed are a promising starting place for future work

Augmentation of Visual Odometry using Radar

As UAVs become viable for more applications, pose estimation continues to be critical. All UAVs need to know where they are at all times, in order to avoid disaster. However, in the event that UAVs are deployed in an area with poor visual conditions, such as in many disaster scenarios, many localization algorithms have difficulties working. This thesis presents VIL-DSO, a visual odometry method as a pose estimation solution, combining several different algorithms in order to improve pose estimation and provide metric scale. This thesis also presents a method for automatically determining an accurate physical transform between radar and camera data, and in doing so, allow for the projection of radar information into the image plane. Finally, this thesis presents EVIL-DSO, a method for localization that fuses visual-inertial odometry with radar information. The proposed EVIL-DSO algorithm uses radar information projected into the image plane in order to create a depth map for odometry to directly observe depth of features, which can then be used as part of the odometry algorithm to remove the need to perform costly depth estimations. Trajectory analysis of the proposed algorithm on outdoor data, compared to differential GPS data, shows that the proposed algorithm is more accurate in terms of root-mean-square error, as well as having a lower percentage of scale error. Runtime analysis shows that the proposed algorithm updates more frequently than other, similar, algorithms

Detecting road boundaries and drivable regions in challenging weather conditions

Road detection is a core component of self-driving vehicle perception, where it covers detecting road boundaries and drivable road regions. It can also help human drivers to drive safely in lower visibility. The majority of current road detection techniques use camera and lidar sensors. These sensors struggle in inclement weather conditions. MMwave radar works well in all weather conditions. However, due to the low resolution of the radar, it is currently limited to object detection for cruise control applications. This thesis investigates the impact of bad weather on vision-based systems and introduces a camera and radar-based method for efficient road detection. We propose a novel approach to overcome the sparse resolution of mmwave-radars and use it in the segmentation task. We augment the nuScenes dataset with fog and rain and use it for our validation. We achieve 20% and 18% better road boundary and drivable region detection in inclement weather

Interacción del diseño y la infraestructura de carreteras con los vehículos automatizados

[ES] El presente trabajo muestra la interacción del diseño y la infraestructura de carreteras con los vehículos automatizados. La aparición de los vehículos automatizados ha supuesto un importante avance que ha ido revolucionando no solo la tecnología, sino también el transporte y su infraestructura. La conducción semiautónoma se basa principalmente en el guiado de las marcas viales mediante un procesamiento digital de imágenes, lo que nos lleva a pensar si las infraestructuras actualmente están preparadas para estos vehículos. Debido a la entrada de los vehículos autonomos surge la necesidad de que las infraestructuras viales sean compatibles con estos nuevos sistemas. Para ello, se han realizado observaciones en más de 2,000 km de diversas autovías, carreteras multicarril y carreteras convencionales, observando 184 curvas, incluyendo ambos sentidos de circulación como curvas distintas, también 44 salidas y 41 entradas, entre ambos sentidos, y también 24 acuerdos convexos. Se realizaron observaciones de las localizaciones en donde el vehículo semiautónomo cedía el control al conductor y qué factores influían (perdida de guiado por marca vial de borde, geometría de la vía, etc.). Como resultado se ha encontrado una correlación entre la geometría de la via y la velocidad máxima a la que puede operar un vehículo semiautónomo sin ceder el control. De igual forma se concluye que la vigente norma de marcas viales presenta limitaciones para facilitar la continuidad de funcionamiento de la conducción semiautónoma, al disponer de huecos sin marca vial discontinua longitudinal en entradas y salidas. Así mismo, se concluye que existe una limitación entre la geometría y la distancia de visibilidad normativa de acuerdos convexos y el funcionamiento de los vehículos semiautónomos. A partir de esta investigación surge un nuevo concepto de velocidad: velocidad segura de conducción automatizada, el cual se encontró que es inferior a las velocidades de diseño, operación y reglamentarias. Así mismo, surge un nuevo concepto de consistencia de la conducción automatizada que considera las diferencias entre las expectativas de los conductores y la velocidad automatizada. Se define un nivel de servicio de la conducción automatizada para paliar las inconsistencias encontradas, el cual será previamente acreditado para cada tramo de carretera, y luego se informará a los conductores. También, se propone la eliminación de los huecos sin marcas viales, para favorecer la continuidad de guiado de la conducción automatizada. Por otra parte, se obtuvo que la distancia de visibilidad disponible del radar con tiempos de percepción y reacción de 0 a 0.7 segundos se equipara a la visibilidad de parada según la norma de diseño, para la misma velocidad.[CA] El present treball mostra la interacció del disseny i la infraestructura de carreteres amb els vehicles automatitzats. L'aparició dels vehicles automatitzats ha suposat un important avanç que ha anat revolucionant no sols la tecnologia, sinó també el transport i la seua infraestructura. La conducció semiautònoma es basa principalment en el guiat de les marques vials per mitjà d'un processament digital d'imatges, la qual cosa ens porta a pensar si les infraestructures actualment estan preparades per a estos vehicles. A causa de l'entrada dels vehicles autònoms sorgix la necessitat que les infraestructures vials siguen compatibles amb estos nous sistemes. Per a això, s'han realitzat observacions en més de 2,000 km de diverses autovies, carreteres multicarril i carreteres convencionals, observant 184 corbes, incloent ambdós sentits de circulació com a corbes distintes, també 44 eixides i 41 entrades, entre ambdós sentits, i també 24 acords convexos. Es van realitzar observacions de les localitzacions on el vehicle semiautònomo cedia el control al conductor i quins factors influïen (perduda de guiat per marca vial de bord, geometria de la via, etc.). Com resultat s'ha trobat una correlació entre la geometria de la via i la velocitat màxima a què pot operar un vehicle semiautónomo sense cedir el control. De la mateixa manera es conclou que la vigent norma de marques vials presenta limitacions per a facilitar la continuïtat de funcionament de la conducció semiautònoma, al disposar de buits sense marca vial discontínua longitudinal en entrades i eixides. Així mateix, es conclou que hi ha una limitació entre la geometria i la distància de visibilitat normativa d'acords convexos i el funcionament dels vehicles semiautònomos. A partir d'esta investigació sorgix un nou concepte de velocitat: velocitat segura de conducció automatitzada, el qual es va trobar que és inferior a les velocitats de disseny, operació i reglamentàries. Així mateix, sorgix un nou concepte de consistència de la conducció automatitzada que considera les diferències entre les expectatives dels conductors i la velocitat automatitzada. Es definix un nivell de servici de la conducció automatitzada per a pal·liar les inconsistències trobades, el qual serà prèviament acreditat per a cada tram de carretera, i després s'informarà els conductors. També, es proposa l'eliminació dels buits sense marques vials, per a afavorir la continuïtat de guiat de la conducció automatitzada. D'altra banda, es va obtindre que la distància de visibilitat disponible del radar amb temps de percepció i reacció de 0 a 0.7 segons s'equipara a la visibilitat de parada segons la norma de disseny, per a la mateixa velocitat.[EN] The present work shows the interaction of design and road infrastructure with automated vehicles. The appearance of automated vehicles has meant an important advance that has revolutionized not only technology, but also transport and its infrastructure. Semi-autonomous driving is based mainly on the guidance of road markings through digital image processing, which leads us to think whether the infrastructures are currently prepared for these vehicles. Due to the entry of autonomous vehicles, there is a need for road infrastructure to be compatible with these new systems. For this, observations have been made in more than 2,000 km of various highways, multi-lane roads and conventional roads, observing 184 curves, including both directions of circulation as different curves, also 44 exits and 41 entries, between both directions, and also 24 sag vertical curves. Observations were made of the locations where the semi-autonomous vehicle gave control to the driver and what factors influenced (loss of guidance by edge road marking, geometry of the road, etc.). As a result, a correlation has been found between the geometry of the road and the maximum speed at which a semi-autonomous vehicle can operate without yielding control. Likewise, it is concluded that the current norm of road markings has limitations to facilitate the continuity of operation of the semi-autonomous driving, by having gaps without discontinuous longitudinal road markings at entrances and exits. Likewise, it is concluded that there is a limitation between the geometry and the normative visibility distance of sag vertical curves and the operation of semi-autonomous vehicles. From this research a new concept of speed emerges: safe automated driving speed, which was found to be lower than the design, operation and regulatory speeds. Likewise, a new concept of automated driving consistency emerges that considers the differences between driver expectations and automated speed. A level of automated driving service is defined to alleviate the inconsistencies found, which will be previously accredited for each section of road, and then the drivers will be informed. Also, the elimination of the gaps without road markings is proposed, to favor the continuity of automated driving guidance. On the other hand, it was obtained that the available visibility distance of the radar with perception and reaction times from 0 to 0.7 seconds is equated to the stop visibility according to the design norm, for the same speed.Padovani Báez, PV. (2018). Interacción del diseño y la infraestructura de carreteras con los vehículos automatizados. http://hdl.handle.net/10251/117061Archivo delegad

Европейский и национальный контексты в научных исследованиях

В настоящем электронном сборнике «Европейский и национальный контексты в научных исследованиях. Технология» представлены работы молодых ученых по геодезии и картографии, химической технологии и машиностроению, информационным технологиям, строительству и радиотехнике. Предназначены для работников образования, науки и производства. Будут полезны студентам, магистрантам и аспирантам университетов.=In this Electronic collected materials “National and European dimension in research. Technology” works in the fields of geodesy, chemical technology, mechanical engineering, information technology, civil engineering, and radio-engineering are presented. It is intended for trainers, researchers and professionals. It can be useful for university graduate and post-graduate students