Color transformation for improved traffic sign detection

This paper considers large scale traffic sign detection on a dataset consisting of high-resolution street-level panoramic photographs. Traffic signs are automatically detected and classified with a set of state-of-the-art algorithms. We introduce a color transformation to extend a Histogram of Oriented Gradients (HOG) based detection algorithm to further improve the performance. This transformation uses a specific set of reference colors that aligns with traffic sign characteristics, and measures the distance of each pixel to these reference colors. This results in an improved consistency on the gradients at the outer edge of the traffic sign. In an experiment with 33, 400 panoramic images, the number of misdetections decreased by 53.6% and 51.4% for red/blue circular signs, and by 19.6% and 28.4% for yellow speed bump signs, measured at a realistic detector operating point

Curve Sign Inventorying Method Using Smartphones and Deep Learning Technologies

The objective of the proposed research is to develop and assess a system using smartphones and deep learning technologies to automatically establish an intelligent and sustainable curve sign inventory from videos. The Manual on the Uniform Traffic Control Devices (MUTCD) is the nationwide regulator that defines the standards used for transportation asset installation and maintenance. The proposed system is one of the components of a larger methodology whose purpose is to accomplish a frequent and cost-effective MUTCD curve sign compliance checking and other curve safety checking in order to reduce the number of deadly crashes on curves. To automatically build an effective sign inventory from videos, four modules are needed: sign detection, classification, tracking and localization. For this purpose, a pipeline has been developed in the past by former students of the Transportation laboratory of Georgia Tech. However, this pipeline is not accurate enough and its different modules have never been critically tested and assessed. Therefore, the objective of this study is to improve the different modules and particularly the detection module, which is the most important module of the pipeline, and to critically assess these improved modules to determine the pipeline ability to build an effective sign inventory. The proposed system has been tested and assessed in real conditions on a mountain road with many curves and curve signs; it has shown that the detection module is able to detect every single curve sign with a very low number of detected non-curve signs (false positive), resulting in a precision of 0.97 and a recall of 1. The other modules also showed very promising results. Overall, this study demonstrates that the proposed system is suitable for building an accurate curve sign inventory that can be used by transportation agencies to get a precise idea of the condition of the curve sign networks on a particular road.M.S

Image-based recognition, 3D localization, and retro-reflectivity evaluation of high-quantity low-cost roadway assets for enhanced condition assessment

Systematic condition assessment of high-quantity low-cost roadway assets such as traffic signs, guardrails, and pavement markings requires frequent reporting on location and up-to-date status of these assets. Today, most Departments of Transportation (DOTs) in the US collect data using camera-mounted vehicles to filter, annotate, organize, and present the data necessary for these assessments. However, the cost and complexity of the collection, analysis, and reporting as-is conditions result in sparse and infrequent monitoring. Thus, some of the gains in efficiency are consumed by monitoring costs. This dissertation proposes to improve frequency, detail, and applicability of image-based condition assessment via automating detection, classification, and 3D localization of multiple types of high-quantity low-cost roadway assets using both images collected by the DOTs and online databases such Google Street View Images. To address the new requirements of US Federal Highway Administration (FHWA), a new method is also developed that simulates nighttime visibility of traffic signs from images taken during daytime and measures their retro-reflectivity condition. To initiate detection and classification of high-quantity low-cost roadway assets from street-level images, a number of algorithms are proposed that automatically segment and localize high-level asset categories in 3D. The first set of algorithms focus on the task of detecting and segmenting assets at high-level categories. More specifically, a method based on Semantic Texton Forest classifiers, segments each geo-registered 2D video frame at the pixel-level based on shape, texture, and color. A Structure from Motion (SfM) procedure reconstructs the road and its assets in 3D. Next, a voting scheme assigns the most observed asset category to each point in 3D. The experimental results from application of this method are promising, nevertheless because this method relies on using supervised ground-truth pixel labels for training purposes, scaling it to various types of assets is challenging. To address this issue, a non-parametric image parsing method is proposed that leverages lazy learning scheme for segmentation and recognition of roadway assets. The semi-supervised technique used in the proposed method does not need training and provides ground truth data in a more efficient manner. It is easily scalable to thousands of video frames captured during data collection. Once the high-level asset categories are detected, specific techniques needs to be exploited to detect and classify the assets at a higher level of granularity. To this end, performance of three computer vision algorithms are evaluated for classification of traffic signs in presence of cluttered backgrounds and static and dynamic occlusions. Without making any prior assumptions about the location of traffic signs in 2D, the best performing method uses histograms of oriented gradients and color together with multiple one-vs-all Support Vector Machines, and classifies these assets into warning, regulatory, stop, and yield sign categories. To minimize the reliance on visual data collected by the DOTs and improve frequency and applicability of condition assessment, a new end-to-end procedure is presented that applies the above algorithms and creates comprehensive inventory of traffic signs using Google Street View images. By processing images extracted using Google Street View API and discriminative classification scores from all images that see a sign, the most probable 3D location of each traffic sign is derived and is shown on the Google Earth using a dynamic heat map. A data card containing information about location, type, and condition of each detected traffic sign is also created. Finally, a computer vision-based algorithm is proposed that measures retro-reflectivity of traffic signs during daytime using a vehicle mounted device. The algorithm simulates nighttime visibility of traffic signs from images taken during daytime and measures their retro-reflectivity. The technique is faster, cheaper, and safer compared to the state-of-the-art as it neither requires nighttime operation nor requires manual sign inspection. It also satisfies measurement guidelines set forth by FHWA both in terms of granularity and accuracy. To validate the techniques, new detailed video datasets and their ground-truth were generated from 2.2-mile smart road research facility and two interstate highways in the US. The comprehensive dataset contains over 11,000 annotated U.S. traffic sign images and exhibits large variations in sign pose, scale, background, illumination, and occlusion conditions. The performance of all algorithms were examined using these datasets. For retro-reflectivity measurement of traffic signs, experiments were conducted at different times of day and for different distances. Results were compared with a method recommended by ASTM standards. The experimental results show promise in scalability of these methods to reduce the time and effort required for developing road inventories, especially for those assets such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods and also multiple categories of traffic signs. The applicability of Google Street View Images for inventory management purposes and also the technique for retro-reflectivity measurement during daytime demonstrate strong potential in lowering inspection costs and improving safety in practical applications

Road Surface Feature Extraction and Reconstruction of Laser Point Clouds for Urban Environment

Automakers are developing end-to-end three-dimensional (3D) mapping system for Advanced Driver Assistance Systems (ADAS) and autonomous vehicles (AVs). Using geomatics, artificial intelligence, and SLAM (Simultaneous Localization and Mapping) systems to handle all stages of map creation, sensor calibration and alignment. It is crucial to have a system highly accurate and efficient as it is an essential part of vehicle controls. Such mapping requires significant resources to acquire geographic information (GIS and GPS), optical laser and radar spectroscopy, Lidar, and 3D modeling applications in order to extract roadway features (e.g., lane markings, traffic signs, road-edges) detailed enough to construct a “base map”. To keep this map current, it is necessary to update changes due to occurring events such as construction changes, traffic patterns, or growth of vegetation. The information of the road play a very important factor in road traffic safety and it is essential for for guiding autonomous vehicles (AVs), and prediction of upcoming road situations within AVs. The data size of the map is extensive due to the level of information provided with different sensor modalities for that reason a data optimization and extraction from three-dimensional (3D) mobile laser scanning (MLS) point clouds is presented in this thesis. The research shows the proposed hybrid filter configuration together with the dynamic developed mechanism provides significant reduction of the point cloud data with reduced computational or size constraints. The results obtained in this work are proven by a real-world system

モービルマッピングシステムと航空測量を用いた都市空間高精度３次元モデリング

学位の種別: 課程博士審査委員会委員 : （主査）東京大学教授 瀬崎 薫, 東京大学教授 江崎 浩, 東京大学教授 苗村 健, 東京大学教授 柴崎 亮介, 東京大学准教授 上條 俊介, 国際電気通信基礎技術研究所 浅見 徹University of Tokyo(東京大学

Robust building identification from street views using deep convolutional neural networks

Street view imagery (SVI) is a rich source of information for architectural and urban analysis using computer vision techniques, but its integration with other building-level data sources requires an additional step of visual building identification. This step is particularly challenging in architecturally homogeneous, dense residential streets featuring narrow buildings, due to a combination of SVI geolocation errors and occlusions that significantly increase the risk of confusing a building with its neighboring buildings. This paper introduces a robust deep learning-based method to identify buildings across multiple street views taken at different angles and times, using global optimization to correct the position and orientation of street view panoramas relative to their surrounding building footprints. Evaluating the method on a dataset of 2000 street views shows that its identification accuracy (88%) outperforms previous deep learning-based methods (79%), while methods solely relying on geometric parameters correctly show the intended building less than 50% of the time. These results indicate that previous identification methods lack robustness to panorama pose errors when buildings are narrow, densely packed, and subject to occlusions, while collecting multiple views per building can be leveraged to increase the robustness of visual identification by ensuring that building views are consistent

Orientation and integration of images and image blocks with laser scanning data

Laser scanning and photogrammetry are methods for effective and accurate measurement and classification of urban and forest areas. Because these methods complement each other, then integration or integrated use brings additional benefits to real-life applications. However, finding tie features between data sets is a challenging task since laser scanning and imagery are far from each other in nature. The aim of this thesis was to create methods for solving relative orientations between laser scanning data and imagery that would assist in near-future applications integrating laser scanning and photogrammetry. Moreover, a further goal was to create methods enabling the use of data acquired from very different perspectives, such as terrestrial and airborne data. To meet these aims, an interactive orientation method enabling the use of single images, stereo images or larger image blocks was developed and tested. The multi-view approach usually has a significant advantage over the use of a single image. After accurate orientation of laser scanning data and imagery, versatile applications become available. Such applications include, e.g., automatic object recognition, accurate classification of individual trees, point cloud densification, automatic classification of land use, system calibration, and generation of photorealistic 3D models. Besides the orientation part, another aim of the research was to investigate how to fuse or use these two data types together in applications. As a result, examples that evaluated the behavior of laser point clouds in both urban and forestry areas, detection and visualization of temporal changes, enhanced data understanding, stereo visualization, multi-source and multi-angle data fusion, point cloud colorizing, and detailed examination of full waveform laser scanning data were given

3D City Models and urban information: Current issues and perspectives

Considering sustainable development of cities implies investigating cities in a holistic way taking into account many interrelations between various urban or environmental issues. 3D city models are increasingly used in different cities and countries for an intended wide range of applications beyond mere visualization. Could these 3D City models be used to integrate urban and environmental knowledge? How could they be improved to fulfill such role? We believe that enriching the semantics of current 3D city models, would extend their functionality and usability; therefore, they could serve as integration platforms of the knowledge related to urban and environmental issues allowing a huge and significant improvement of city sustainable management and development. But which elements need to be added to 3D city models? What are the most efficient ways to realize such improvement / enrichment? How to evaluate the usability of these improved 3D city models? These were the questions tackled by the COST Action TU0801 “Semantic enrichment of 3D city models for sustainable urban development”. This book gathers various materials developed all along the four year of the Action and the significant breakthroughs

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity