Benchmarking Image Sensors Under Adverse Weather Conditions for Autonomous Driving

Adverse weather conditions are very challenging for autonomous driving because most of the state-of-the-art sensors stop working reliably under these conditions. In order to develop robust sensors and algorithms, tests with current sensors in defined weather conditions are crucial for determining the impact of bad weather for each sensor. This work describes a testing and evaluation methodology that helps to benchmark novel sensor technologies and compare them to state-of-the-art sensors. As an example, gated imaging is compared to standard imaging under foggy conditions. It is shown that gated imaging outperforms state-of-the-art standard passive imaging due to time-synchronized active illumination

Road Condition Mapping by Integration of Laser Scanning, RGB Imaging and Spectrometry

Roads are important infrastructure and are primary means of transportation. Control and maintenance of roads are substantial as the pavement surface deforms and deteriorates due to heavy load and influences of weather. Acquiring detailed information about the pavement condition is a prerequisite for proper planning of road pavement maintenance and rehabilitation. Many companies detect and localize the road pavement distresses manually, either by on-site inspection or by digitizing laser data and imagery captured by mobile mapping. The automation of road condition mapping using laser data and colour images is a challenge. Beyond that, the mapping of material properties of the road pavement surface with spectrometers has not yet been investigated. This study aims at automatic mapping of road surface condition including distress and material properties by integrating laser scanning, RGB imaging and spectrometry. All recorded data are geo-referenced by means of GNSS/ INS. Methods are developed for pavement distress detection that cope with a variety of different weather and asphalt conditions. Further objective is to analyse and map the material properties of the pavement surface using spectrometry data. No standard test data sets are available for benchmarking developments on road condition mapping. Therefore, all data have been recorded with a mobile mapping van which is set up for the purpose of this research. The concept for detecting and localizing the four main pavement distresses, i.e. ruts, potholes, cracks and patches is the following: ruts and potholes are detected using laser scanning data, cracks and patches using RGB images. For each of these pavement distresses, two or more methods are developed, implemented, compared to each other and evaluated to identify the most successful method. With respect to the material characteristics, spectrometer data of road sections are classified to indicate pavement quality. As a spectrometer registers almost a reflectivity curve in VIS, NIR and SWIR wavelength, indication of aging can be derived. After detection and localization of the pavement distresses and pavement quality classes, the road condition map is generated by overlaying all distresses and quality classes. As a preparatory step for rut and pothole detection, the road surface is extracted from mobile laser scanning data based on a height jump criterion. For the investigation on rut detection, all scanlines are processed. With an approach based on iterative 1D polynomial fitting, ruts are successfully detected. For streets with the width of 6 m to 10 m, a 6th order polynomial is found to be most suitable. By 1D cross-correlation, the centre of the rut is localized. An alternative method using local curvature shows a high sensitivity to the shape and width of a rut and is less successful. For pothole detection, the approach based on polynomial fitting generalized to two dimensions. As an alternative, a procedure using geodesic morphological reconstruction is investigated. Bivariate polynomial fitting encounters problems with overshoot at the boundary of the regions. The detection is very successful using geodesic morphology. For the detection of pavement cracks, three methods using rotation invariant kernels are investigated. Line Filter, High-pass Filter and Modified Local Binary Pattern kernels are implemented. A conceptual aspect of the procedure is to achieve a high degree of completeness. The most successful variant is the Line Filter for which the highest degree of completeness of 81.2 % is achieved. Two texture measures, the gradient magnitude and the local standard deviation are employed to detect pavement patches. As patches may differ with respect to homogeneity and may not always have a dark border with the intact pavement surface, the method using the local standard deviation is more suitable for detecting the patches. Linear discriminant analysis is utilized for asphalt pavement quality analysis and classification. Road pavement sections of ca. 4 m length are classified into two classes, namely: “Good” and “Bad” with the overall accuracy of 77.6 %. The experimental investigations show that the developed methods for automatic distress detection are very successful. By 1D polynomial fitting on laser scanlines, ruts are detected. In addition to ruts also pavement depressions like shoving can be revealed. The extraction of potholes is less demanding. As potholes appear relatively rare in the road networks of a city, the road segments which are affected by potholes are selected interactively. While crack detection by Line Filter works very well, the patch detection is more challenging as patches sometimes look very similar to the intact surface. The spectral classification of pavement sections contributes to road condition mapping as it gives hints on aging of the road pavement.Straßen bilden die primären Transportwege für Personen und Güter und sind damit ein wichtiger Bestandteil der Infrastruktur. Der Aufwand für Instandhaltung und Wartung der Straßen ist erheblich, da sich die Fahrbahnoberfläche verformt und durch starke Belastung und Wettereinflüsse verschlechtert. Die Erfassung detaillierter Informationen über den Fahrbahnzustand ist Voraussetzung für eine sachgemäße Planung der Fahrbahnsanierung und -rehabilitation. Viele Unternehmen detektieren und lokalisieren die Fahrbahnschäden manuell entweder durch Vor-Ort-Inspektion oder durch Digitalisierung von Laserdaten und Bildern aus mobiler Datenerfassung. Eine Automatisierung der Straßenkartierung mit Laserdaten und Farbbildern steht noch in den Anfängen. Zudem werden bisher noch nicht die Alterungszustände der Asphaltdecke mit Hilfe der Spektrometrie bewertet. Diese Studie zielt auf den automatischen Prozess der Straßenzustandskartierung einschließlich der Straßenschäden und der Materialeigenschaften durch Integration von Laserscanning, RGB-Bilderfassung und Spektrometrie ab. Alle aufgezeichneten Daten werden mit GNSS / INS georeferenziert. Es werden Methoden für die Erkennung von Straßenschäden entwickelt, die sich an unterschiedliche Datenquellen bei unterschiedlichem Wetter- und Asphaltzustand anpassen können. Ein weiteres Ziel ist es, die Materialeigenschaften der Fahrbahnoberfläche mittels Spektrometrie-Daten zu analysieren und abzubilden. Derzeit gibt es keine standardisierten Testdatensätze für die Evaluierung von Verfahren zur Straßenzustandsbeschreibung. Deswegen wurden alle Daten, die in dieser Studie Verwendung finden, mit einem eigens für diesen Forschungszweck konfigurierten Messfahrzeug aufgezeichnet. Das Konzept für die Detektion und Lokalisierung der wichtigsten vier Arten von Straßenschäden, nämlich Spurrillen, Schlaglöcher, Risse und Flickstellen ist das folgende: Spurrillen und Schlaglöcher werden aus Laserdaten extrahiert, Risse und Flickstellen aus RGB- Bildern. Für jede dieser Straßenschäden werden mindestens zwei Methoden entwickelt, implementiert, miteinander verglichen und evaluiert um festzustellen, welche Methode die erfolgreichste ist. Im Hinblick auf die Materialeigenschaften werden Spektrometriedaten der Straßenabschnitte klassifiziert, um die Qualität des Straßenbelages zu bewerten. Da ein Spektrometer nahezu eine kontinuierliche Reflektivitätskurve im VIS-, NIR- und SWIR-Wellenlängenbereich aufzeichnet, können Merkmale der Asphaltalterung abgeleitet werden. Nach der Detektion und Lokalisierung der Straßenschäden und der Qualitätsklasse des Straßenbelages wird der übergreifende Straßenzustand mit Hilfe von Durchschlagsregeln als Kombination aller Zustandswerte und Qualitätsklassen ermittelt. In einem vorbereitenden Schritt für die Spurrillen- und Schlaglocherkennung wird die Straßenoberfläche aus mobilen Laserscanning-Daten basierend auf einem Höhensprung-Kriterium extrahiert. Für die Untersuchung zur Spurrillen-Erkennung werden alle Scanlinien verarbeitet. Mit einem Ansatz, der auf iterativer 1D-Polynomanpassung basiert, werden Spurrillen erfolgreich erkannt. Für eine Straßenbreite von 8-10m erweist sich ein Polynom sechsten Grades als am besten geeignet. Durch 1D-Kreuzkorrelation wird die Mitte der Spurrille erkannt. Eine alternative Methode, die die lokale Krümmung des Querprofils benutzt, erweist sich als empfindlich gegenüber Form und Breite einer Spurrille und ist weniger erfolgreich. Zur Schlaglocherkennung wird der Ansatz, der auf Polynomanpassung basiert, auf zwei Dimensionen verallgemeinert. Als Alternative wird eine Methode untersucht, die auf der Geodätischen Morphologischen Rekonstruktion beruht. Bivariate Polynomanpassung führt zu Überschwingen an den Rändern der Regionen. Die Detektion mit Hilfe der Geodätischen Morphologischen Rekonstruktion ist dagegen sehr erfolgreich. Zur Risserkennung werden drei Methoden untersucht, die rotationsinvariante Kerne verwenden. Linienfilter, Hochpassfilter und Lokale Binäre Muster werden implementiert. Ein Ziel des Konzeptes zur Risserkennung ist es, eine hohe Vollständigkeit zu erreichen. Die erfolgreichste Variante ist das Linienfilter, für das mit 81,2 % der höchste Grad an Vollständigkeit erzielt werden konnte. Zwei Texturmaße, nämlich der Betrag des Grauwert-Gradienten und die lokale Standardabweichung werden verwendet, um Flickstellen zu entdecken. Da Flickstellen hinsichtlich der Homogenität variieren können und nicht immer eine dunkle Grenze mit dem intakten Straßenbelag aufweisen, ist diejenige Methode, welche die lokale Standardabweichung benutzt, besser zur Erkennung von Flickstellen geeignet. Lineare Diskriminanzanalyse wird zur Analyse der Asphaltqualität und zur Klassifikation benutzt. Straßenabschnitte von ca. 4m Länge werden zwei Klassen („Gut“ und „Schlecht“) mit einer gesamten Accuracy von 77,6 % zugeordnet. Die experimentellen Untersuchungen zeigen, dass die entwickelten Methoden für die automatische Entdeckung von Straßenschäden sehr erfolgreich sind. Durch 1D Polynomanpassung an Laser-Scanlinien werden Spurrillen entdeckt. Zusätzlich zu Spurrillen werden auch Unebenheiten des Straßenbelages wie Aufschiebungen detektiert. Die Extraktion von Schlaglöchern ist weniger anspruchsvoll. Da Schlaglöcher relativ selten in den Straßennetzen von Städten auftreten, werden die Straßenabschnitte mit Schlaglöchern interaktiv ausgewählt. Während die Rissdetektion mit Linienfiltern sehr gut funktioniert, ist die Erkennung von Flickstellen eine größere Herausforderung, da Flickstellen manchmal der intakten Straßenoberfläche sehr ähnlich sehen. Die spektrale Klassifizierung der Straßenabschnitte trägt zur Straßenzustandsbewertung bei, indem sie Hinweise auf den Alterungszustand des Straßenbelages liefert

Measuring urban growth, urban form and accessibility as indicators of urban sprawl in Hamilton, New Zealand

Hamilton City is currently the fourth most populous territorial authority in New Zealand. The city boundary was extended in 1989 in order to provide sufficient land for urban growth for at least 25 years. Despite being neither unplanned nor unchecked, urban growth within this boundary has been branded by the media as urban sprawl. Urban sprawl is a complex phenomenon with a wide range of definitions incorporating aesthetic judgements, unwanted externalities, policy consequences, land development patterns, and urban growth rates. The negative economic, environmental, social and public health effects of urban sprawl are widely considered to outweigh any positive effects, leading to the term having a negative connotation. Just as there are many ways to define urban sprawl, there are also many ways to measure the phenomenon including urban area growth rates, density measurements, and spatial geometry, as well as differences in access to public services, employment opportunities and commercial areas. The primary research question addressed in this study is whether the post-1989 urban growth in Hamilton should be categorised as urban sprawl? Remote sensing and GIS techniques have been used to measure urban growth, urban form and accessibility in order to address three sub-questions which reflect different ways of defining and measuring urban sprawl: • Has growth in the Hamilton urban area occurred at a greater rate than the city’s population growth? • Are new neighbourhoods more homogeneous, having a higher proportion of single family dwellings on larger land parcels, and do they lack street connectivity? • Do residents of old and new neighbourhoods have different access to essential services, commercial areas, employment areas, and public transport? Post-classification comparison of satellite imagery was used to measure urban growth. A supervised classification was performed on three Landsat images acquired in 1990, 2001 and 2014. Pixels in each image were classified into a common land cover schema comprising eight classes: urban-residential, urban-commercial/industrial, cleared land, grassland, natural vegetation, agriculture-crops, agriculture-fallow and water. Pixels were then reclassified to three classes (urban, non-urban and water) and the Hamilton urban area was quantified for each classification map by multiplying the number of urban pixels by the pixel size. An accuracy assessment showed the classification maps to have overall accuracy of 94-97% and Kappa estimation of 90-96%. Between 1990 and 2014 there was 45% growth in the Hamilton urban area, while census data showed 41% growth in the population residing within Hamilton City between 1991 and 2013. Hence growth in the Hamilton urban area has not occurred at a greater rate than the city’s population growth. Therefore, based on this method of measuring urban sprawl, the post-1989 urban growth in Hamilton should not be categorised as urban sprawl. Cadastral, land use and road data were used to calculate six metrics of urban form for each of the 37 residential neighbourhoods in Hamilton: land use mix, dwelling density, single-dwelling proportion, single-dwelling parcel size, internal street connectivity and external street connectivity. There are statistically significant differences in these metrics between old (developed prior to 1990) and new (developed from 1990 to 2014) neighbourhoods. New neighbourhoods are currently more homogeneous, they have a higher proportion of single family dwellings and lack street connectivity. However these single family dwellings do not occupy larger land parcels, which may be a function of several factors including subdivision policies, market demand and land prices. Therefore, based on five out of the six metrics of urban form, the post-1989 urban growth in Hamilton should be categorised as urban sprawl. Cadastral, facility and road data were used to calculate eleven metrics of accessibility for each residential neighbourhood. For residential land parcels the median distance to the nearest commercial land use, bus stop, primary school, employment area and medial clinic, and from the nearest police station, fire station and ambulance station was calculated for each neighbourhood. Pedestrian access (walkability) was calculated as the percentage of residential parcels in the neighbourhood within walking distance (800m) of a commercial land use, bus stop and primary school. There are statistically significant differences in these metrics between old and new neighbourhoods, and residents currently have different access to commercial areas, employment areas, public transport and some essential services. Residents of new neighbourhoods currently experience increased travelling distances and reduced accessibility and walkability. However there is currently no difference between old and new neighbourhoods in terms of access to police and ambulance services, suggesting that the locations of these facilities are well-balanced across the city. Therefore, based on nine out of the eleven metrics of accessibility, the post-1989 urban growth in Hamilton should be categorised as urban sprawl. The results of this study demonstrate the complexity of the urban sprawl phenomenon and that whether the post-1989 urban growth in Hamilton should be categorised as urban sprawl depends upon the particular definition of urban sprawl that is adopted and the measurement method used.According to the media urban sprawl is running rampant in Hamilton, New Zealand. There are claims this dangerous phenomenon is gobbling up some of the world’s best farmland, killing the heart of the city, and creating headaches for the future. But what exactly is urban sprawl? Is there any evidence for it in Hamilton or is the media just using the term as an attention-grabber? The United States Environmental Protection Agency defines urban sprawl as urban growth at a rate which exceeds population growth. Census data shows the population residing in Hamilton grew by 41% between 1991 and 2013. Landsat satellite images taken over a similar period show the Hamilton urban area grew by 45%. Most of this urban growth was in northeastern suburbs such as Rototuna, a previously rural area brought into the city in 1989 and specifically designated for urban development. Others define urban sprawl as a homogeneous pattern of land development, lacking a mix of land uses, dominated by single-family dwellings on large parcels of land, and containing too many winding streets and cul-de-sacs. This results in too much separation between land uses, leading to increased travelling distances for residents and a reduction in walkability. Hamilton’s northeastern suburbs certainly exhibit many of these characteristics. In comparison to the older central suburb of Hamilton East, Rototuna has much less commercial and community facility land use. Single-family dwellings occupy 65% of Rototuna’s land area compared to only 30% in Hamilton East, where multi-residential dwellings are far more common. The street network in Rototuna is dominated by cul-de-sacs. These differences in land use and street design have led to increased travelling distances for Rototuna residents. In fact, the median distance to the nearest of Hamilton’s main employment areas is almost three times that for residents of Hamilton East. And while all residents of Hamilton East are within walking distance of a commercial land use, only a third of Rototuna residents have this convenience. However, Hamilton’s northeastern suburbs differ from typical areas of urban sprawl when it comes to the size of land parcels. Median single-family dwelling parcel sizes are similar across the city, and are much smaller than the quarter acre once considered a requisite part of the New Zealand dream. So while Hamilton’s newer northeastern suburbs have some of the characteristics of urban sprawl, the city’s urban growth has not outstripped population growth. This highlights the complexity of the urban sprawl phenomenon and suggests the media’s claims of woe are part fact and part fiction

The perception system of intelligent ground vehicles in all weather conditions: A systematic literature review

Perception is a vital part of driving. Every year, the loss in visibility due to snow, fog, and rain causes serious accidents worldwide. Therefore, it is important to be aware of the impact of weather conditions on perception performance while driving on highways and urban traffic in all weather conditions. The goal of this paper is to provide a survey of sensing technologies used to detect the surrounding environment and obstacles during driving maneuvers in different weather conditions. Firstly, some important historical milestones are presented. Secondly, the state-of-the-art automated driving applications (adaptive cruise control, pedestrian collision avoidance, etc.) are introduced with a focus on all-weather activity. Thirdly, the most involved sensor technologies (radar, lidar, ultrasonic, camera, and far-infrared) employed by automated driving applications are studied. Furthermore, the difference between the current and expected states of performance is determined by the use of spider charts. As a result, a fusion perspective is proposed that can fill gaps and increase the robustness of the perception system

New Computational Methods for Automated Large-Scale Archaeological Site Detection

Aquesta tesi doctoral presenta una sèrie d'enfocaments, fluxos de treball i models innovadors en el camp de l'arqueologia computacional per a la detecció automatitzada a gran escala de jaciments arqueològics. S'introdueixen nous conceptes, enfocaments i estratègies, com ara lidar multitemporal, aprenentatge automàtic híbrid, refinament, curriculum learning i blob analysis; així com diferents mètodes d'augment de dades aplicats per primera vegada en el camp de l'arqueologia. S'utilitzen múltiples fonts, com ara imatges de satèl·lits multiespectrals, fotografies RGB de plataformes VANT, mapes històrics i diverses combinacions de sensors, dades i fonts. Els mètodes creats durant el desenvolupament d'aquest doctorat s'han avaluat en projectes en curs: Urbanització a Hispània i la Gàl·lia Mediterrània en el primer mil·lenni aC, detecció de monticles funeraris utilitzant algorismes d'aprenentatge automàtic al nord-oest de la Península Ibèrica, prospecció arqueològica intel·ligent basada en drons (DIASur), i cartografiat del patrimoni arqueològic al sud d'Àsia (MAHSA), per a la qual s'han dissenyat fluxos de treball adaptats als reptes específics del projecte. Aquests nous mètodes han aconseguit proporcionar solucions als problemes comuns d'estudis arqueològics presents en estudis similars, com la baixa precisió en detecció i les poques dades d'entrenament. Els mètodes validats i presentats com a part de la tesi doctoral s'han publicat en accés obert amb el codi disponible perquè puguin implementar-se en altres estudis arqueològics.Esta tesis doctoral presenta una serie de enfoques, flujos de trabajo y modelos innovadores en el campo de la arqueología computacional para la detección automatizada a gran escala de yacimientos arqueológicos. Se introducen nuevos conceptos, enfoques y estrategias, como lidar multitemporal, aprendizaje automático híbrido, refinamiento, curriculum learning y blob analysis; así como diferentes métodos de aumento de datos aplicados por primera vez en el campo de la arqueología. Se utilizan múltiples fuentes, como lidar, imágenes satelitales multiespectrales, fotografías RGB de plataformas VANT, mapas históricos y varias combinaciones de sensores, datos y fuentes. Los métodos creados durante el desarrollo de este doctorado han sido evaluados en proyectos en curso: Urbanización en Iberia y la Galia Mediterránea en el Primer Milenio a. C., Detección de túmulos mediante algoritmos de aprendizaje automático en el Noroeste de la Península Ibérica, Prospección Arqueológica Inteligente basada en Drones (DIASur), y cartografiado del Patrimonio del Sur de Asia (MAHSA), para los que se han diseñado flujos de trabajo adaptados a los retos específicos del proyecto. Estos nuevos métodos han logrado proporcionar soluciones a problemas comunes de la prospección arqueológica presentes en estudios similares, como la baja precisión en detección y los pocos datos de entrenamiento. Los métodos validados y presentados como parte de la tesis doctoral se han publicado en acceso abierto con su código disponible para que puedan implementarse en otros estudios arqueológicos.This doctoral thesis presents a series of innovative approaches, workflows and models in the field of computational archaeology for the automated large-scale detection of archaeological sites. New concepts, approaches and strategies are introduced such as multitemporal lidar, hybrid machine learning, refinement, curriculum learning and blob analysis; as well as different data augmentation methods applied for the first time in the field of archaeology. Multiple sources are used, such as lidar, multispectral satellite imagery, RGB photographs from UAV platform, historical maps, and several combinations of sensors, data, and sources. The methods created during the development of this PhD have been evaluated in ongoing projects: Urbanization in Iberia and Mediterranean Gaul in the First Millennium BC, Detection of burial mounds using machine learning algorithms in the Northwest of the Iberian Peninsula, Drone-based Intelligent Archaeological Survey (DIASur), and Mapping Archaeological Heritage in South Asia (MAHSA), for which workflows adapted to the project’ s specific challenges have been designed. These new methods have managed to provide solutions to common archaeological survey problems, presented in similar large-scale site detection studies, such as the low precision in previous detection studies and how to handle problems with few training data. The validated approaches for site detection presented as part of the PhD have been published as open access papers with freely available code so can be implemented in other archaeological studies

Remote Sensing in Applications of Geoinformation

Remote sensing, especially from satellites, is a source of invaluable data which can be used to generate synoptic information for virtually all parts of the Earth, including the atmosphere, land, and ocean. In the last few decades, such data have evolved as a basis for accurate information about the Earth, leading to a wealth of geoscientific analysis focusing on diverse applications. Geoinformation systems based on remote sensing are increasingly becoming an integral part of the current information and communication society. The integration of remote sensing and geoinformation essentially involves combining data provided from both, in a consistent and sensible manner. This process has been accelerated by technologically advanced tools and methods for remote sensing data access and integration, paving the way for scientific advances in a broadening range of remote sensing exploitations in applications of geoinformation. This volume hosts original research focusing on the exploitation of remote sensing in applications of geoinformation. The emphasis is on a wide range of applications, such as the mapping of soil nutrients, detection of plastic litter in oceans, urban microclimate, seafloor morphology, urban forest ecosystems, real estate appraisal, inundation mapping, and solar potential analysis