Second Place Award in recognition of outstanding achievement in the 2020 Annual Meeting Graduate Student Poster Competition sponsored by INSPIRE University Transportation Cente

Karim, Muhammad Monjurul

English

Missouri University of Science and Technology (Missouri S&T): Scholars' Mine

INTRODUCTIONProblem Statement:RESULTSThe test dataset has 212 images, with 1872 objects in 10 classes Objective:CONCLUSIONSDeveloped a semi-supervised deep learning NN for detecting and segmenting multiclass bridge elements from inspection video data• Saves time, reduces bias, allows to focus on knowledge-intensive tasks• Transfer learning and IS3T help adapt the model to new tasks• Keep human-in-the-loop to leverage human intelligence for AI model developmentACKNOWLEDGEMENTSINSPIRE UTC, EMSE department, and Intelligent Systems CenterBridge Inspection Video Data Analysis for Data-driven Asset ManagementMuhammad Monjurul Karim, Missouri University of Science and TechnologyTianyi Zhao, Ruwen Qin, Genda Chen, Suzanna Long, Zhaozheng Yin, Sushil LouisTemporal Coherence Analysis:• Select training samples from recovered hard data• Manual annotation + automatic annotation• Boost up the performance over iterationsTransfer learning IS3TIndex of iteration, 𝑙 0 1 2 3IoU = 0.5Precision (%) 80.3 81.7 90.7 91.8Recall (%) 74.4 90.3 90.1 93.6f1-Score (%) 77.2 85.8 90.4 92.7Method Training (min) precision (%) recall (%) f1 (%)Training from scratch 792+ 32.3 18.3 23.4Transfer learning 20 80.3 74.4 77.2Scale variationLack of illuminationSample Inspection datawith masks of detected ROIViewpoint changesWork time (min) Accuracy (%)w/o the AI tool 65 100w/ the AI  tool 0.27 93.7change ↓99.5% ↓6.3%APPROACHEngage inspectors in the development of the video data analysis AI tool.Transfer Learning: Efficiency of Transfer Learning:• Huge time saving and performance gainHelpfulness of the AI Tool for Inspection Image Data Analytics:Cost-Effectiveness Achieved by Engaging Inspectors:Automatically annotated by the trained NNInspector annotated high quality data• Mask R-CNN pre-trained on MS COCO dataset• Initial adaptation with 40 training images• Over 600k highway bridges in the National Bridge Inventory• Bridges are facing the prospect of rapid deterioration (39% exceed design life of 50 years, 9% require significant repair)• Bridges need to be inspected every two years• The current practice of bridge inspection has many issuesA cyber-physical system for analyze the data for an effective asset management• Digitally profile bridge conditions as a cyber system formanaging physical system (the bridge)• Provide the decision support for preservation• Compare detection results with neighboring frames• Recover false negative resultsIterative Semi-Supervised Self-Training (IS3T):Illustrative Examples:Blocking trafficTime consuming data collectionDifficultyof accessing elevated bridge structuresDangerous field activitiesBias of Human visual inspectionIterative Semi-supervised Self-trainingOne time manual annotationManual re-annotation of a subset in each iterationInitial (small) training datasetA pre-trained Mask R-CNNMask R-CNN for multiclass bridge element detection& segmentationApplied to Unlabeled testing datasetSegmentation resultsAdditional training dataSatisfied result?StopYNAssessment SamplingTransfer learningTemporal Coherence Analysis

Bridge Inspection Video Data Analysis for Data-Driven Asset Management

https://scholarsmine.mst.edu/cgi/viewcontent.cgi?article=1084&amp;context=inspire-meetings

Bridge Inspection Video Data Analysis for Data-Driven Asset Management

Abstract

Similar works

Full text

Available Versions

Missouri University of Science and Technology (Missouri S&T): Scholars' Mine