Uses and Challenges of Collecting LiDAR Data from a Growing Autonomous Vehicle Fleet: Implications for Infrastructure Planning and Inspection Practices

Autonomous vehicles (AVs) that utilize LiDAR (Light Detection and Ranging) and other sensing technologies are becoming an inevitable part of transportation industry. Concurrently, transportation agencies are increasingly challenged with the management and tracking of large-scale highway asset inventory. LiDAR has become popular among transportation agencies for highway asset management given its advantage over traditional surveying methods. The affordability of LiDAR technology is increasing day by day. Given this, there will be substantial challenges and opportunities for the utilization of big data resulting from the growth of AVs with LiDAR. A proper understanding of the data size generated from this technology will help agencies in making decisions regarding storage, management, and transmission of the data. The original raw data generated from the sensor shrinks a lot after filtering and processing following the Cache county Road Manual and storing into ASPRS recommended (.las) file format. In this pilot study, it is found that while considering the road centerline as the vehicle trajectory larger portion of the data fall into the right of way section compared to the actual vehicle trajectory in Cache County, UT. And there is a positive relation between the data size and vehicle speed in terms of the travel lanes section given the nature of the selected highway environment

Image-based recognition, 3D localization, and retro-reflectivity evaluation of high-quantity low-cost roadway assets for enhanced condition assessment

Systematic condition assessment of high-quantity low-cost roadway assets such as traffic signs, guardrails, and pavement markings requires frequent reporting on location and up-to-date status of these assets. Today, most Departments of Transportation (DOTs) in the US collect data using camera-mounted vehicles to filter, annotate, organize, and present the data necessary for these assessments. However, the cost and complexity of the collection, analysis, and reporting as-is conditions result in sparse and infrequent monitoring. Thus, some of the gains in efficiency are consumed by monitoring costs. This dissertation proposes to improve frequency, detail, and applicability of image-based condition assessment via automating detection, classification, and 3D localization of multiple types of high-quantity low-cost roadway assets using both images collected by the DOTs and online databases such Google Street View Images. To address the new requirements of US Federal Highway Administration (FHWA), a new method is also developed that simulates nighttime visibility of traffic signs from images taken during daytime and measures their retro-reflectivity condition. To initiate detection and classification of high-quantity low-cost roadway assets from street-level images, a number of algorithms are proposed that automatically segment and localize high-level asset categories in 3D. The first set of algorithms focus on the task of detecting and segmenting assets at high-level categories. More specifically, a method based on Semantic Texton Forest classifiers, segments each geo-registered 2D video frame at the pixel-level based on shape, texture, and color. A Structure from Motion (SfM) procedure reconstructs the road and its assets in 3D. Next, a voting scheme assigns the most observed asset category to each point in 3D. The experimental results from application of this method are promising, nevertheless because this method relies on using supervised ground-truth pixel labels for training purposes, scaling it to various types of assets is challenging. To address this issue, a non-parametric image parsing method is proposed that leverages lazy learning scheme for segmentation and recognition of roadway assets. The semi-supervised technique used in the proposed method does not need training and provides ground truth data in a more efficient manner. It is easily scalable to thousands of video frames captured during data collection. Once the high-level asset categories are detected, specific techniques needs to be exploited to detect and classify the assets at a higher level of granularity. To this end, performance of three computer vision algorithms are evaluated for classification of traffic signs in presence of cluttered backgrounds and static and dynamic occlusions. Without making any prior assumptions about the location of traffic signs in 2D, the best performing method uses histograms of oriented gradients and color together with multiple one-vs-all Support Vector Machines, and classifies these assets into warning, regulatory, stop, and yield sign categories. To minimize the reliance on visual data collected by the DOTs and improve frequency and applicability of condition assessment, a new end-to-end procedure is presented that applies the above algorithms and creates comprehensive inventory of traffic signs using Google Street View images. By processing images extracted using Google Street View API and discriminative classification scores from all images that see a sign, the most probable 3D location of each traffic sign is derived and is shown on the Google Earth using a dynamic heat map. A data card containing information about location, type, and condition of each detected traffic sign is also created. Finally, a computer vision-based algorithm is proposed that measures retro-reflectivity of traffic signs during daytime using a vehicle mounted device. The algorithm simulates nighttime visibility of traffic signs from images taken during daytime and measures their retro-reflectivity. The technique is faster, cheaper, and safer compared to the state-of-the-art as it neither requires nighttime operation nor requires manual sign inspection. It also satisfies measurement guidelines set forth by FHWA both in terms of granularity and accuracy. To validate the techniques, new detailed video datasets and their ground-truth were generated from 2.2-mile smart road research facility and two interstate highways in the US. The comprehensive dataset contains over 11,000 annotated U.S. traffic sign images and exhibits large variations in sign pose, scale, background, illumination, and occlusion conditions. The performance of all algorithms were examined using these datasets. For retro-reflectivity measurement of traffic signs, experiments were conducted at different times of day and for different distances. Results were compared with a method recommended by ASTM standards. The experimental results show promise in scalability of these methods to reduce the time and effort required for developing road inventories, especially for those assets such as guardrails and traffic lights that are not typically considered in 2D asset recognition methods and also multiple categories of traffic signs. The applicability of Google Street View Images for inventory management purposes and also the technique for retro-reflectivity measurement during daytime demonstrate strong potential in lowering inspection costs and improving safety in practical applications

Pavement Surface Evaluation Using Mobile Terrestrial LiDAR Scanning Systems

Periodic measurement of pavement surfaces for pavement management system (PMS) data collection is vital for state transportation agencies. Vehicle-based mobile light detection and ranging (LiDAR) systems can be used as a versatile tool to collect point data throughout a roadway corridor. The overall goal of this research is to investigate if mobile terrestrial LiDAR Scanning (MTLS) systems can be used as an efficient and effective method to create accurate digital pavement surfaces for. LiDAR data were collected by five MTLS vendors. In particular, the research is interested in three things: 1) how accurate MTLS is for collecting roadway cross slopes; 2) what is the potential for using MTLS digital pavement surfaces to do materials calculations for pavement rehabilitation projects; and 3) examine the benefit of using MTLS to identify pavement rutting locations. Cross slopes were measured at 23 test stations using traditional surveying methods (conventional leveling served as ground-truth) and compared with adjusted and unadjusted MTLS extracted cross slopes. The results indicate that both adjusted and unadjusted MTLS derived cross slopes meet suggested cross slope accuracies (±0.2%). Application of unadjusted MTLS instead of post-processed MTLS point clouds may decrease/eliminate the cost of a control surveys. The study also used a novel approach to process the MTLS data in a geographic information system (GIS) environment to create a 3-dimension raster representation of a roadway surface. MTLS data from each vendor was evaluated in terms of the accuracy and precision of their raster surface. The resultant surfaces were compared between vendors and with a raster surface created from a centerline profile and 100-ft. cross-section data obtained using traditional surveying methods. When comparing LiDAR data between compliant MTLS vendors, average raster cell height differences averaged 0.21 inches, indicating LiDAR data has considerable potential for creating accurate pavement material volume estimates. The application of MTLS data was also evaluated in terms of the accuracy of collected transverse profiles. Transverse profiles captured from MTLS systems have been compared to 2-inch interval field data collection using partial curve mapping (PCM), Frechet distance, area, curve length, and Dynamic Time Warping (DTW) techniques. The results indicated that there is potential for MTLS systems for use in creating an accurate transverse profile for potential identification of pavement rut areas. This research also identified a novel approach for determining pavement rut areas based on the shape of grid cells. This rather simplistic approach is easily implementable on a network wide basis depending on MTLS point cloud availability. The method does not require the calculation/estimation of an ideal surface to determine rut depths/locations

A Platform for Proactive, Risk-Based Slope Asset Management, Phase II

INE/AUTC 15.0

Evaluation of surface defect detection in reinforced concrete bridge decks using terrestrial LiDAR

Routine bridge inspections require labor intensive and highly subjective visual interpretation to determine bridge deck surface condition. Light Detection and Ranging (LiDAR) a relatively new class of survey instrument has become a popular and increasingly used technology for providing as-built and inventory data in civil applications. While an increasing number of private and governmental agencies possess terrestrial and mobile LiDAR systems, an understanding of the technology’s capabilities and potential applications continues to evolve. LiDAR is a line-of-sight instrument and as such, care must be taken when establishing scan locations and resolution to allow the capture of data at an adequate resolution for defining features that contribute to the analysis of bridge deck surface condition. Information such as the location, area, and volume of spalling on deck surfaces, undersides, and support columns can be derived from properly collected LiDAR point clouds. The LiDAR point clouds contain information that can provide quantitative surface condition information, resulting in more accurate structural health monitoring. LiDAR scans were collected at three study bridges, each of which displayed a varying degree of degradation. A variety of commercially available analysis tools and an independently developed algorithm written in ArcGIS Python (ArcPy) were used to locate and quantify surface defects such as location, volume, and area of spalls. The results were visual and numerically displayed in a user-friendly web-based decision support tool integrating prior bridge condition metrics for comparison. LiDAR data processing procedures along with strengths and limitations of point clouds for defining features useful for assessing bridge deck condition are discussed. Point cloud density and incidence angle are two attributes that must be managed carefully to ensure data collected are of high quality and useful for bridge condition evaluation. When collected properly to ensure effective evaluation of bridge surface condition, LiDAR data can be analyzed to provide a useful data set from which to derive bridge deck condition information

Road conditional mapping using terrestrial laser scanning method

Road transportation plays a vigorous part in the lives of people worldwide, because it bond people for commercial activities or pleasure by connecting small and large cities, urban and rural areas as well as connecting a country with its neighbour. To support the safe movement of people, goods and services, road and their features are carefully designed and constructed to increase road traffic safety, improve the efficient use of the overall network and reduce the harm such as death, injuries and property damage. Crack is the common surface distress of asphalt pavements it is necessary to detect the crack as early as possible to reduce maintenance cost. Terrestrial laser scanning is one of the most capable remote sensing techniques, which can be used to detect and analyse road distress at all levels The main objectives of this research were to acquire the road data using terrestrial laser scanning and close-range photogrammetry method, measure the width, length and area affected by the crack from point cloud data and also to verify the result using close-range photogrammetry and manual method. Ten lengths of the crack ware measured, ten width and area affected by the crack was also measured from point cloud data. The results obtained from point cloud data was verified using close-range photogrammetry and manual measurements. The results shows the potential of terrestrial laser scanning to detect, measure and analyse the road crack with root mean square error of the measured lengths between terrestrial laser scanning and close-range photogrammetry 0.015m and that of terrestrial laser scanning and manual method was 0.018m while the root mean square error of the measured widths between terrestrial laser scanning and close-range photogrammetry 0.001m and that of terrestrial laser scanning and manual method was 0.001m