A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure

To ensure the safety and the serviceability of civil infrastructure it is essential to visually inspect and assess its physical and functional condition. This review paper presents the current state of practice of assessing the visual condition of vertical and horizontal civil infrastructure; in particular of reinforced concrete bridges, precast concrete tunnels, underground concrete pipes, and asphalt pavements. Since the rate of creation and deployment of computer vision methods for civil engineering applications has been exponentially increasing, the main part of the paper presents a comprehensive synthesis of the state of the art in computer vision based defect detection and condition assessment related to concrete and asphalt civil infrastructure. Finally, the current achievements and limitations of existing methods as well as open research challenges are outlined to assist both the civil engineering and the computer science research community in setting an agenda for future research

Application of mathematical morphology to the analysis of X-ray NDE images

Ever since the beginning, man has been in the relentless pursuit of perfection. From stone age to space age, from caves to condominiums, from carts to planes, trains and automobiles, his drive for consummation has grown considerably. The high quality products that are available in the market at the turn of the twenty first century are living legacies of his unyielding endeavor for excellence. But one fact that most people do not realize is the amount of time and money devoted to quality control and non- destructive evaluation (NDE) that is responsible for the high quality of products. In the past, people used to tap earthenware and other materials as a means of non destructive testing for defects in the material. They could sense the defects by the nature of the sound propagated through the material. The ultrasonic method of NDE is an extension of this principle

Applications of pattern classification to time-domain signals

Many different kinds of physics are used in sensors that produce time-domain signals, such as ultrasonics, acoustics, seismology, and electromagnetics. The waveforms generated by these sensors are used to measure events or detect flaws in applications ranging from industrial to medical and defense-related domains. Interpreting the signals is challenging because of the complicated physics of the interaction of the fields with the materials and structures under study. often the method of interpreting the signal varies by the application, but automatic detection of events in signals is always useful in order to attain results quickly with less human error. One method of automatic interpretation of data is pattern classification, which is a statistical method that assigns predicted labels to raw data associated with known categories. In this work, we use pattern classification techniques to aid automatic detection of events in signals using features extracted by a particular application of the wavelet transform, the Dynamic Wavelet Fingerprint (DWFP), as well as features selected through physical interpretation of the individual applications. The wavelet feature extraction method is general for any time-domain signal, and the classification results can be improved by features drawn for the particular domain. The success of this technique is demonstrated through four applications: the development of an ultrasonographic periodontal probe, the identification of flaw type in Lamb wave tomographic scans of an aluminum pipe, prediction of roof falls in a limestone mine, and automatic identification of individual Radio Frequency Identification (RFID) tags regardless of its programmed code. The method has been shown to achieve high accuracy, sometimes as high as 98%

INTELLIGENT ROAD MAINTENANCE: A MACHINE LEARNING APPROACH FOR SURFACE DEFECT DETECTION

The emergence of increased sources for Big Data through consumer recording devices gives rise to a new basis for the management and governance of public infrastructures and policy de-sign. Road maintenance and detection of road surface defects, such as cracks, have traditionally been a time consuming and manual process. Lately, increased automation using easily acquirable front-view digital natural scene images is seen to be an alternative for taking timely maintenance decisions; reducing accidents and operating cost and increasing public safety. In this paper, we propose a machine learning based approach to handle the challenge of crack and related defect detection on road surfaces using front-view images captured from driver’s viewpoint under diverse conditions. We use a superpixel based method to first process the road images into smaller coherent image regions. These superpixels are then classified into crack and non-crack regions. Various texture-based features are combined for the classification mod-el. Classifiers such as Gradient Boosting, Artificial Neural Network, Random Forest and Linear Support Vector Machines are evaluated for the task. Evaluations on real datasets show that the approach successfully handles different road surface conditions and crack-types, while locating the defective regions in the scene images

A state-of-the-art of physics-informed neural networks in engineering

Técnicas de machine learning vêm ganhando cada vez mais espaço no cenário industrial no intuito de converter o crescente fluxo de informação (data) em melhorias de processos. Entre tais técnicas, as redes neuronais se destacam devido à sua capacidade de aproximador universal de funções, cuja performance pode ser enriquecida ao se fornecer conhecimentos físicos prévios: tem-se, então, o desenvolvimento das Physics-informed neural networks (PINN). Nesse contexto e observando-se um “gap” na produção de trabalhos relacionados ao tema e da difusão dessa temática na grade de formação dos cursos da Escola de Química, esse trabalho se propõe a realizar um estado da arte da técnica mencionada. Observou-se interesse particular das PINN para aplicações em mecânica dos fluidos e transferência de calor. Ademais, as PINN se mostram ferramentas importantes tanto para a resolução de problemas ditos “diretos” quanto “indiretos”. Por fim, através de exemplos práticos, constatou-se a capacidade de se aproximar funções de interesse particular na indústria química usando-se redes neurais sem nenhuma informação física do problema (obtenção do fator de atrito) e utilizando-se a equação diferencial que descreve o problema (resolução da equação de difusão em 1D)

Defect Detection and Classification in Sewer Pipeline Inspection Videos Using Deep Neural Networks

Sewer pipelines as a critical civil infrastructure become a concern for municipalities as they are getting near to the end of their service lives. Meanwhile, new environmental laws and regulations, city expansions, and budget constraints make it harder to maintain these networks. On the other hand, access and inspect sewer pipelines by human-entry based methods are problematic and risky. Current practice for sewer pipeline assessment uses various types of equipment to inspect the condition of pipelines. One of the most used technologies for sewer pipelines inspection is Closed Circuit Television (CCTV). However, application of CCTV method in extensive sewer networks involves certified operators to inspect hours of videos, which is time-consuming, labor-intensive, and error prone. The main objective of this research is to develop a framework for automated defect detection and classification in sewer CCTV inspection videos using computer vision techniques and deep neural networks. This study presents innovative algorithms to deal with the complexity of feature extraction and pattern recognition in sewer inspection videos due to lighting conditions, illumination variations, and unknown patterns of various sewer defects. Therefore, this research includes two main sub-models to first identify and localize anomalies in sewer inspection videos, and in the next phase, detect and classify the defects among the recognized anomalous frames. In the first phase, an innovative approach is proposed for identifying the frames with potential anomalies and localizing them in the pipe segment which is being inspected. The normal and anomalous frames are classified utilizing a one-class support vector machine (OC-SVM). The proposed approach employs 3D Scale Invariant Feature Transform (SIFT) to extract spatio-temporal features and capture scene dynamic statistics in sewer CCTV videos. The OC-SVM is trained by the frame-features which are considered normal, and the outliers to this model are considered abnormal frames. In the next step, the identified anomalous frames are located by recognizing the present text information in them using an end-to-end text recognition approach. The proposed localization approach is performed in two steps, first the text regions are detected using maximally stable extremal regions (MSER) algorithm, then the text characters are recognized using a convolutional neural network (CNN). The performance of the proposed model is tested using videos from real-world sewer inspection reports, where the accuracies of 95% and 86% were achieved for anomaly detection and frame localization, respectively. Identifying the anomalous frames and excluding the normal frames from further analysis could reduce the time and cost of detection. It also ensures the accuracy and quality of assessment by reducing the number of neglected anomalous frames caused by operator error. In the second phase, a defect detection framework is proposed to provide defect detection and classification among the identified anomalous frames. First, a deep Convolutional Neural Network (CNN) which is pre-trained using transfer learning, is used as a feature extractor. In the next step, the remaining convolutional layers of the constructed model are trained by the provided dataset from various types of sewer defects to detect and classify defects in the anomalous frames. The proposed methodology was validated by referencing the ground truth data of a dataset including four defects, and the mAP of 81.3% was achieved. It is expected that the developed model can help sewer inspectors in much faster and more accurate pipeline inspection. The whole framework would decrease the condition assessment time and increase the accuracy of sewer assessment reports

A recognition algorithm to detect pipe weld defects

Uzimajući kao objekt istraživanja propuštanje magnetskog toka (MFL) za snimanje grešaka u zavarima cijevi, predložen je algoritam raspoznavanja slika grešaka u zavarima utemeljen na matrici istovremene pojave sivih tonova (GGCM) te klasterskoj analizi i matematičkoj morfologiji. Postignuto je raspoznavanje različitih vrsta grešaka u zavarima. Prvo je korišten sustav kontinuiranog bezdodirnog skeniranja MFL za prikupljanje trodimenzijskog propuštanja magnetskog toka (MFL), a nakon toga je trodimenzijski MFL signal pretvoren u dvodimenzijsku sliku sivih tonova. Zatim su karakteristike MFL slike za dvodimenzijsku sliku u sivim tonovima izdvojene pomoću GGCM. Na temelju izdvojenih značajki slike, analizira se karakteristična količina pomoću particioniranja k-sredine, a zatim kroz kombinaciju izjednačenja histograma, Otsuovu metoda binarizacija, morfološkog uklanjanja malih objekata, otkrivanja rubova, a zatim strukturiranja morfološki optimiziranog algoritma ekstrakcije ruba za otkrivanje rubova na sivim tonovima. Kombiniranjem nekoliko metoda, strukturira se novi algoritam za poboljšanje učinka otkrivanja. Rezultati su pokazali da je ova metoda prilagodljiva i praktična. Ovaj je algoritam riješio poteškoće vezane uz MFL metodu koja se koristi u ispitivanju zavara radi otkrivanja grešaka u zavarima cijevi, a nadilazi granice primjene tradicionalne tehnologije obrade signala.Taking magnetic flux leakage (MFL) imaging of pipe weld defects as the research object, a weld defect image recognition algorithm based on grey-gradient co-occurrence matrix (GGCM) and cluster analysis and mathematical morphology is proposed. Recognition of different types of welding defects was achieved. Firstly, a continuous non-contact scanning MFL system for the pipe weld was used to collect the three-dimensional MFL. Secondly, the three-dimensional MFL signal was converted to a two-dimensional greyscale image. Then the MFL image characteristics of the two-dimensional grayscale image were extracted using GGCM. Based on extracted image features, the characteristic quantity was analysed by using k-means clustering and then through the combination of histogram equalization, Otsu’s method of binaryzation, morphologically removing small objects, edge detection, and then structuring a morphologically optimized edge extraction method for edge detection on the grayscale. Through combination of several methods, a new algorithm to improve the detection effect was structured. The results indicated that this algorithm is adaptable and practical. This algorithm solved difficulties associated with the MFL method being used in the weld testing to realize the recognition of pipe weld defects and break through the applicable limitations of traditional signal processing technology

Obstruction level detection of sewers videos using convolutional neural networks

Worldwide, sewer networks are designed to transport wastewater to a centralized treatment plant to be treated and returned to the environment. This is a critical process for preventing waterborne illnesses, providing safe drinking water and enhancing general sanitation in society. To keep a perfectly operational sewer network several inspections are manually performed by a Closed-Circuit Television system to report the obstruction level which may trigger a cleaning operative. In this work, we design a methodology to train a Convolutional Neural Network (CNN) for identifying the level of obstruction in pipes. We gathered a database of videos to generate useful frames to fed into the model. Our resulting classifier obtains deployment ready performances. To validate the consistency of the approach and its industrial applicability, we integrate the Layer-wise Relevance Propagation (LPR) algorithm, which endows a further understanding of the neural network behavior. The proposed system provides higher speed, accuracy, and consistency in the sewer process examination.This work is partially supported by the Consejo Nacional de Ciencia y Tecnologia (CONACYT), Estudiante No. CVU: 630716, by the RIS3CAT Utilities 4.0 SENIX project (COMRDI16-1-0055), cofounded by the European Regional Development Fund (FEDER) under the FEDER Catalonia Operative Programme 2014- 2020. It is also partially supported by the Spanish Government through Programa Severo Ochoa (SEV2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316-P project, and by the Generalitat de Catalunya (contracts 2017-SGR-1414).Peer ReviewedPostprint (published version

Sensing and Signal Processing in Smart Healthcare

In the last decade, we have witnessed the rapid development of electronic technologies that are transforming our daily lives. Such technologies are often integrated with various sensors that facilitate the collection of human motion and physiological data and are equipped with wireless communication modules such as Bluetooth, radio frequency identification, and near-field communication. In smart healthcare applications, designing ergonomic and intuitive human–computer interfaces is crucial because a system that is not easy to use will create a huge obstacle to adoption and may significantly reduce the efficacy of the solution. Signal and data processing is another important consideration in smart healthcare applications because it must ensure high accuracy with a high level of confidence in order for the applications to be useful for clinicians in making diagnosis and treatment decisions. This Special Issue is a collection of 10 articles selected from a total of 26 contributions. These contributions span the areas of signal processing and smart healthcare systems mostly contributed by authors from Europe, including Italy, Spain, France, Portugal, Romania, Sweden, and Netherlands. Authors from China, Korea, Taiwan, Indonesia, and Ecuador are also included