3D Visual Perception for Self-Driving Cars using a Multi-Camera System: Calibration, Mapping, Localization, and Obstacle Detection

Cameras are a crucial exteroceptive sensor for self-driving cars as they are low-cost and small, provide appearance information about the environment, and work in various weather conditions. They can be used for multiple purposes such as visual navigation and obstacle detection. We can use a surround multi-camera system to cover the full 360-degree field-of-view around the car. In this way, we avoid blind spots which can otherwise lead to accidents. To minimize the number of cameras needed for surround perception, we utilize fisheye cameras. Consequently, standard vision pipelines for 3D mapping, visual localization, obstacle detection, etc. need to be adapted to take full advantage of the availability of multiple cameras rather than treat each camera individually. In addition, processing of fisheye images has to be supported. In this paper, we describe the camera calibration and subsequent processing pipeline for multi-fisheye-camera systems developed as part of the V-Charge project. This project seeks to enable automated valet parking for self-driving cars. Our pipeline is able to precisely calibrate multi-camera systems, build sparse 3D maps for visual navigation, visually localize the car with respect to these maps, generate accurate dense maps, as well as detect obstacles based on real-time depth map extraction

Investigation of image enhancement techniques for the development of a self-contained airborne radar navigation system

This study was devoted to an investigation of the feasibility of applying advanced image processing techniques to enhance radar image characteristics that are pertinent to the pilot's navigation and guidance task. Millimeter (95 GHz) wave radar images for the overwater (i.e., offshore oil rigs) and overland (Heliport) scenario were used as a data base. The purpose of the study was to determine the applicability of image enhancement and scene analysis algorithms to detect and improve target characteristics (i.e., manmade objects such as buildings, parking lots, cars, roads, helicopters, towers, landing pads, etc.) that would be helpful to the pilot in determining his own position/orientation with respect to the outside world and assist him in the navigation task. Results of this study show that significant improvements in the raw radar image may be obtained using two dimensional image processing algorithms. In the overwater case, it is possible to remove the ocean clutter by thresholding the image data, and furthermore to extract the target boundary as well as the tower and catwalk locations using noise cleaning (e.g., median filter) and edge detection (e.g., Sobel operator) algorithms

Factors limiting sand dune restoration in Northwest Beach, Point Pelee National Park, Canada

Known as home to rare species of flora and fauna, and their critical habitats, Northwest beach of Point Pelee National Park has undergone significant ecological and infrastructural changes in the past decades. A number of important management challenges have emerged, including conservation of endangered Five-lined Skink (Plestiodon fasciatus) which inhabit the extensive dune system within the park. This research investigates key factors for sand dune ecosystem restoration in Northwest beach of Point Pelee with particular attention to the conservation of Skink habitat. Random stratified sampling method was used to collect sand and vegetation samples from the disturbed and natural areas. Sand samples were also collected from the sand piles, which is a part of dune restoration process initiated by the Parks Canada. Three aspects were considered: grain size distribution of dune sediments, vegetation assemblage and character of the dune associated species, land use and land cover change. Grain size distribution indicated that samples from most of the sand piles contained some amounts of clay/silt and pebble sized grains making it unfavourable for wind action, resulting in no significant contribution to dune formation. Most of the sand samples collected along the foredunes and water edge were appropriate for sediment transport. Shannon and Simpson’s Diversity Index was calculated as 1.48 and 0.67 for natural area as compared to 0.71 and 0.35 for the disturbed area, which indicate unfavourable species diversity for dune restoration in disturbed areas. The research also focused on the spatial and temporal changes in land use and land cover in NW beach area of Point Pelee using aerial photos for 1959, 1977, 2006 and 2015. Different time series of the aerial photos were chosen based on their availability. The Ecological land classification system for Southern Ontario were used to classify the aerial photos for land use and land cover (LULC). LULC classes included Shoreline vegetation, Deciduous thicket, Sand Barren and Dune Type, and Infrastructures (includes Transportation and services) for the entire Northwest Beach area. Segmentation and classification tools was used to classify four different time series of aerial photos. Grain size distribution and vegetation assemblage for dune associated species were calculated to determine the factors limiting habitat restoration process. Based on the results alternate management strategies for dune restoration in Point Pelee were recommended. The study offers key insights on the importance of timely detection, analysis and visualisation of dynamic changes for habitat restoration and maintaining ecological integrity of the Northwest beach area of Point Pelee

From a Competition for Self-Driving Miniature Cars to a Standardized Experimental Platform: Concept, Models, Architecture, and Evaluation

Context: Competitions for self-driving cars facilitated the development and research in the domain of autonomous vehicles towards potential solutions for the future mobility. Objective: Miniature vehicles can bridge the gap between simulation-based evaluations of algorithms relying on simplified models, and those time-consuming vehicle tests on real-scale proving grounds. Method: This article combines findings from a systematic literature review, an in-depth analysis of results and technical concepts from contestants in a competition for self-driving miniature cars, and experiences of participating in the 2013 competition for self-driving cars. Results: A simulation-based development platform for real-scale vehicles has been adapted to support the development of a self-driving miniature car. Furthermore, a standardized platform was designed and realized to enable research and experiments in the context of future mobility solutions. Conclusion: A clear separation between algorithm conceptualization and validation in a model-based simulation environment enabled efficient and riskless experiments and validation. The design of a reusable, low-cost, and energy-efficient hardware architecture utilizing a standardized software/hardware interface enables experiments, which would otherwise require resources like a large real-scale test track.Comment: 17 pages, 19 figues, 2 table

Full-automatic recognition of various parking slot markings using a hierarchical tree structure

A full-automatic method for recognizing parking slot markings is proposed. The proposed method recognizes various types of parking slot markings by modeling them as a hierarchical tree structure. This method mainly consists of two processes: bottom-up and top-down. First, the bottom-up process climbs up the hierarchical tree structure to excessively generate parking slot candidates so as not to lose the correct slots. This process includes corner detection, junction and slot generation, and type selection procedures. After that, the top-down process confirms the final parking slots by eliminating falsely generated slots, junctions, and corners based on the properties of the parking slot marking type by climbing down the hierarchical tree structure. The proposed method was evaluated in 608 real-world parking situations encompassing a variety of different parking slot markings. The experimental result reveals that the proposed method outperforms the previous semiautomatic method while requiring a small amount of computational costs even though it is fully automatic