A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment

The Ultraviolet Imaging Telescope: Instrument and Data Characteristics

The Ultraviolet Imaging Telescope (UIT) was flown as part of the Astro observatory on the Space Shuttle Columbia in December 1990 and again on the Space Shuttle Endeavor in March 1995. Ultraviolet (1200-3300 Angstroms) images of a variety of astronomical objects, with a 40 arcmin field of view and a resolution of about 3 arcsec, were recorded on photographic film. The data recorded during the first flight are available to the astronomical community through the National Space Science Data Center (NSSDC); the data recorded during the second flight will soon be available as well. This paper discusses in detail the design, operation, data reduction, and calibration of UIT, providing the user of the data with information for understanding and using the data. It also provides guidelines for analyzing other astronomical imagery made with image intensifiers and photographic film.Comment: 44 pages, LaTeX, AAS preprint style and EPSF macros, accepted by PAS

An Overview about Emerging Technologies of Autonomous Driving

Since DARPA started Grand Challenges in 2004 and Urban Challenges in 2007, autonomous driving has been the most active field of AI applications. This paper gives an overview about technical aspects of autonomous driving technologies and open problems. We investigate the major fields of self-driving systems, such as perception, mapping and localization, prediction, planning and control, simulation, V2X and safety etc. Especially we elaborate on all these issues in a framework of data closed loop, a popular platform to solve the long tailed autonomous driving problems

Anomaly Detection in 3D Space for Autonomous Driving

Current state-of-the-art perception models do not always detect all objects in an image. Therefore, they cannot currently be relied upon in safety critical applications such as autonomous driving. Objects that cannot be detected are called anomalies. Current work on anomaly detection is primarily based on camera data. This work evaluates to what extent it is possible today to do anomaly detection in 3D on pseudo-lidar data. A pseudo-lidar is a model that estimates 3D depth for each pixel of an image. Currently, there is no approach available that performs anomaly detection on pseudo-lidar data. Research Question 1 (RQ1) considers whether dissimilarities between lidar and pseudo-lidar are an indicator of anomalies. For this purpose, it is evaluated whether there are larger deviations between pseudo-lidar and lidar point clouds for anomalies compared to non-anomalies. There is no multi-modal dataset for anomaly detection available which could be directly used. Therefore, in the multi-modal KITTI-360 dataset each instance that was not correctly segmented by a panoptic segmentation model, was labeled as anomaly. It is shown how the anomaly definition depends on the used segmentation criterion. The dataset contains 2D images with instance labels and corresponding 3D lidar point clouds with corresponding instance labels. With these labels, one can compare pseudo-lidar and lidar instance by instance. The 2D instance labels allow to define which instances are correctly segmented. In a next step the chamfer-distance between point clouds of each instance in lidar and pseudo-lidar are calculated. Lidar was used as physically captured ground truth. It has been found that the deviation between lidar and pseudo-lidar is similar for anomalies and non-anomalies. Thus, the dissimilarity between lidar and pseudo-lidar can not be used as an indicator for an anomaly. In Research Question 2 (RQ2) it was analyzed how good a pseudo-lidar can map anomalies of type novelties in 3D. Therefore, one augmented anomaly dataset, and two real world anomaly datasets were considered. All these datasets are image based. For answering the research questions, both a quantitative and qualitative analysis was carried out. As a quantitative analysis, Monte Carlo Dropout was applied onto these datasets to evaluate the uncertainty of the model. The 3D point clouds estimated with the pseudo-lidar were visualized for the qualitative analysis. The qualitative analysis shows that some anomalies can be mapped well in 3D and others are not mapped at all. Furthermore, it is shown that augmented anomalies can be sometimes mapped ambiguously in 3D. In the quantitative analysis, it is shown for all datasets considered that the pseudo-lidar is more certain for anomaly regions than for non-anomaly regions which is interpreted as a consequence of an overconfidence of the model. Furthermore, the anomaly concept is not always consistent across different modalities. The Research Question 3 (RQ3) analyzed whether anomalies can be found using flow estimation on pseudo-lidar predicted point clouds. An anomaly would be present in theory if the motion segmentation model contradicted with a panoptic segmentation model. For this purpose, it was investigated whether the pseudo-lidar estimated point clouds are consistent enough through time to do flow estimation on them. For this purpose, the multi-modal KITTI-360 dataset was used. For each instance in the pseudo-lidar it was determined how much a point cloud of an instance differs from the same instance in the next frame. For the consistency evaluation of static and dynamic instances, the ego motion has to be extracted. The pseudo-lidar prediction is consistent between frames if for static instances the distance is small and for dynamic instances the distance is equal to the motion of the instance. It has been shown that the pseudo-lidar makes inconsistent predictions over time, and therefore one cannot distinguish between static and dynamic instances based on pseudo-lidar point clouds. It follows that a flow-based approach to anomaly detection is not possible for point clouds predicted by current single image based pseudo-lidars

Identification and quantification of diffuse fresh submarine groundwater discharge via airborne thermal infrared remote sensing

Airborne thermal infrared (TIR) overflights were combined with shoreline radionuclide surveys to investigate submarine groundwater discharge (SGD) along the north shore of Long Island, NY between June 2013 and September 2014. Regression equations developed for three distinct geomorphological environments suggest a positive linear relationship between the rate of diffuse SGD and the spatial extent of the observed coastal TIR anomalies; such a relationship provides quantitative evidence of the ability to use TIR remote sensing as a tool to remotely identify and measure SGD. Landsat TIR scenes were unable to resolve any of the 18 TIR anomalies identified during the various airborne overflights. Two locations were studied in greater detail via 222Rn time series and manual seepage meters in order to understand why specific shoreline segments did not exhibit a TIR anomaly. SGD at the first site, located within a large, diffuse TIR anomaly, was composed of a mixture of fresh groundwater and circulated seawater with elevated levels of nitrate. In contrast, SGD at the second site, where no coastal TIR anomaly was observed, was composed of circulated seawater with negligible nitrate. Despite the compositional differences in seepage, both sites were similar in discharge magnitude, with average time series 222Rn derived SGD rates equal to 18 and 8 cm d−1 for the TIR site and non-TIR site, respectively. Results suggest that TIR remote sensing has the ability to identify locations of a mixture between diffuse fresh and circulated seawater SGD. If TIR anomalies can be demonstrated to represent a mixture between fresh and circulated seawater SGD, then the cumulative area of the TIR anomalies may be used to estimate the fresh fraction of SGD relative to the cumulative area of the seepage face, and thus allows for improved SGD derived nutrient flux calculations on a regional scale

Geospatial Analysis and Remote Sensing from Airplanes and Satellites for Cultural Resources Management

Cultural resource management consists of research to identify, evaluate, document and assess cultural resources, planning to assist in decision-making, and stewardship to implement the preservation, protection and interpretation of these decisions and plans. One technique that may be useful in cultural resource management archaeology is remote sensing. It is the acquisition of data and derivative information about objects or materials (targets) located on the Earth's surface or in its atmosphere by using sensor mounted on platforms located at a distance from the targets to make measurements on interactions between the targets and electromagnetic radiation. Included in this definition are systems that acquire imagery by photographic methods and digital multispectral sensors. Data collected by digital multispectral sensors on aircraft and satellite platforms play a prominent role in many earth science applications, including land cover mapping, geology, soil science, agriculture, forestry, water resource management, urban and regional planning, and environmental assessments. Inherent in the analysis of remotely sensed data is the use of computer-based image processing techniques. Geographical information systems (GIS), designed for collecting, managing, and analyzing spatial information, are also useful in the analysis of remotely sensed data. A GIS can be used to integrate diverse types of spatially referenced digital data, including remotely sensed and map data. In archaeology, these tools have been used in various ways to aid in cultural resource projects. For example, they have been used to predict the presence of archaeological resources using modern environmental indicators. Remote sensing techniques have also been used to directly detect the presence of unknown sites based on the impact of past occupation on the Earth's surface. Additionally, remote sensing has been used as a mapping tool aimed at delineating the boundaries of a site or mapping previously unknown features. All of these applications are pertinent to the goals of site discovery and assessment in cultural resource management

D5.1 SHM digital twin requirements for residential, industrial buildings and bridges

This deliverable presents a report of the needs for structural control on buildings (initial imperfections, deflections at service, stability, rheology) and on bridges (vibrations, modal shapes, deflections, stresses) based on state-of-the-art image-based and sensor-based techniques. To this end, the deliverable identifies and describes strategies that encompass state-of-the-art instrumentation and control for infrastructures (SHM technologies).Objectius de Desenvolupament Sostenible::8 - Treball Decent i Creixement EconòmicObjectius de Desenvolupament Sostenible::9 - Indústria, Innovació i InfraestructuraPreprin