122 research outputs found
Training Framework of Robotic Operation and Image Analysis for Decision-Making in Bridge Inspection and Preservation
Inspection and preservation of existing transportation infrastructure to extend their service life is an effective way of mitigating the pressure of steadily growing transportation demands on the aging infrastructure. Their current practice, though, represents one of the most costly operations in state departments of transportation.
The INSPIRE University Transportation Center will develop a remotely-controlled robotic platform that helps with these labor-intensive tasks and allows engineers to focus on decision-making processes. An important mission of INSPIRE is to leverage users’ capability of implementing, and interacting with, the robotic platform. Therefore, a long-term plan has been made to create a framework of training engineers and policy makers as well as new workforce on robotic operation and image analysis for the inspection and maintenance of transportation infrastructure. The proposed project, as a component of the plan, involves the prototyping of such a framework based on camera-based bridge inspection and robot-based maintenance.
The overall goal of the project is to create a framework of training engineers and policy makers on robotic operation and image analysis for the inspection and preservation of transportation infrastructure. Specifically, the project aims to (1) provide the method for collecting camera-based bridge inspection data and the algorithms for data processing and pattern recognitions; and (2) create tools for assisting and training users on visually analyzing the processed image data and recognized patterns for inspection and preservation decision-making
A Training Framework of Robotic Operation and Image Analysis for Decision-Making in Bridge Inspection and Preservation
Inspection and preservation of existing transportation infrastructure to extend their service life is an effective way of mitigating the pressure of steadily growing transportation demands on the aging infrastructure. Their current practice, though, represents one of the most costly operations in state departments of transportation.
The INSPIRE University Transportation Center will develop a remotely-controlled robotic platform that helps with these labor-intensive tasks and allows engineers to focus on decision-making processes. An important mission of INSPIRE is to leverage users’ capability of implementing, and interacting with, the robotic platform. Therefore, a long-term plan has been made to create a framework of training engineers and policy makers as well as new workforce on robotic operation and image analysis for the inspection and maintenance of transportation infrastructure. The proposed project, as a component of the plan, involves the prototyping of such a framework based on camera-based bridge inspection and robot-based maintenance.
The overall goal of the project is to create a framework of training engineers and policy makers on robotic operation and image analysis for the inspection and preservation of transportation infrastructure. Specifically, the project aims to (1) provide the method for collecting camera-based bridge inspection data and the algorithms for data processing and pattern recognitions; and (2) create tools for assisting and training users on visually analyzing the processed image data and recognized patterns for inspection and preservation decision-making
Human-Robot Collaboration for Effective Bridge Inspection in the Artificial Intelligence Era
Advancements in sensor, Artificial Intelligence (AI), and robotic technologies have formed a foundation to enable a transformation from traditional engineering systems to complex adaptive systems. This paradigm shift will bring exciting changes to civil infrastructure systems and their builders, operators and managers. Funded by the INSPIRE University Transportation Center (UTC), Dr. Qin’s group investigated the holism of an AI-robot-inspector system for bridge inspection. Dr. Qin will discuss the need for close collaboration among the constituent components of the AI-robot-inspector system. In the workplace of bridge inspection using drones, the mobile robotic inspection platform rapidly collected big inspection video data that need to be processed prior to element-level inspections. She will illustrate how human intelligence and artificial intelligence can collaborate in creating an AI model both efficiently and effectively. Obtaining a large amount of expert-annotated data for model training is less desirable, if not unrealistic, in bridge inspection. This INSPIRE project addressed this annotation challenge by developing a semi-supervised self-learning (S3T) algorithm that utilizes a small amount of time and guidance from inspectors to help the model achieve an excellent performance. The project evaluated the improvement in job efficacy produced by the developed AI model. This presentation will conclude by introducing some of the on-going work to achieve the desired adaptability of AI models to new or revised tasks in bridge inspection as the National Bridge Inventory includes over 600,000 bridges of various types in material, shape, and age
Attaining Knowledge Workforce Agility in a Product Life Cycle Environment using Real Options
The product life cycle (PLC) phenomenon has placed significant pressures on high-tech industries which rely heavily on the knowledge workforce in transferring cutting-edge technologies into products. This thesis examines systems where market changes and production technology advances happen frequently and unpredictably during the PLC, causing difficulties in predicting an appropriate demand on the knowledge workforce and in maintaining reliable performance. Knowledge workforce agility (KWA) is identified as a desirable means for addressing the difficulties, and yet previous work on KWA is incomplete.
This thesis accomplishes several critical tasks for realizing the benefits of KWA in a representative PLC environment, semiconductor manufacturing. Real options (RO) is chosen as the approach towards exploiting KWA, since RO captures the essence of KWA-options in manipulating knowledge capacity, a human asset, or a self-cultivated organizational capability for pursuing interests associated with change. Accordingly, market demand change and workforce knowledge (WK) dynamics in adoption of technology advances are formulized as underlying stochastic processes during the PLC. This thesis models KWA as capacity options in a knowledge workforce and develops a RO approach of workforce training, either initial or continuous, for generating options. To quantify the elements of KWA that impact production, the role of the knowledge workforce in production and costs in obtaining KWA are characterized mathematically. It creates necessary RO valuation methods and techniques to optimize KWA.
An analytical examination of the PLC models identifies that KWA has potential to reduce negative impacts and generate opportunities in an environment of volatile demand, and to compensate unreliable performance of knowledge workforce in adoption of technology advances. The benefits of KWA are especially important when confronting highly volatile demand, a low initial adoption level, shrinking PLCs, a growing market size, intense and frequent WK dynamics, insufficient learning capability of employees, or diminishing returns from investments in learning. The thesis further assesses RO, as an agility-driven approach, by comparing it to a chase-demand heuristic and to the Bass forecasting model under demand uncertainty. The assessment demonstrates that the KWA attained from the RO approach, termed RO-based KWA, leads to a stably higher yield, to a persistently larger net present value (NPV), and to a NPV distribution that is more robust to highly volatile demand. Subsequently, a quantitative evaluation of KWA value shows that the RO-based KWA creates a considerable profit growth, either with uncertainty in demand or in the WK dynamics. In evaluation, RO modeling and the RO valuation are identified to be useful in creation of KWA value especially in highly uncertain PLC environments. This thesis illustrates the effectiveness of the numerical methods used for solving the dynamic system problem.
This research demonstrates an approach for optimizing KWA in PLC environments using RO. It provides an innovative solution for knowledge workforce planning in rapidly changing and highly unexpected environments. The work of this thesis is representative of studying KWA using quantitative techniques, where there is a dearth of quantitative studies in the literature
Development of a Causal Model for Improving Rural Seniors' Accessibility: Data Evidences
Seniors residing in rural areas often encounter limited accessibility to
opportunities, resources, and services. This paper introduces a model proposing
that both aging and rural residency are factors contributing to the restricted
accessibility faced by rural seniors. Leveraging data from the 2017 National
Household Travel Survey, the study examines three hypotheses pertaining to this
causal model. Multiple causal pathways emerge in the data analysis, with
mobility identified as a mediator in one of them. The study further identifies
specific challenges faced by rural seniors, such as the reduced accessibility
in reaching medical services and assisting others. These challenges stem
primarily from aging and geographic obstacles that not only diminish their
willingness to travel but also restrict more in the group from choosing
transportation modes with higher mobility. The insights gained from this study
serve as a foundation for devising effective methods to enhance transportation
accessibility for seniors in rural areas.Comment: 12 pages 5 table
Attention-Enhanced Co-Interactive Fusion Network (AECIF-Net) for Automated Structural Condition Assessment in Visual Inspection
Efficiently monitoring the condition of civil infrastructure requires
automating the structural condition assessment in visual inspection. This paper
proposes an Attention-Enhanced Co-Interactive Fusion Network (AECIF-Net) for
automatic structural condition assessment in visual bridge inspection.
AECIF-Net can simultaneously parse structural elements and segment surface
defects on the elements in inspection images. It integrates two task-specific
relearning subnets to extract task-specific features from an overall feature
embedding. A co-interactive feature fusion module further captures the spatial
correlation and facilitates information sharing between tasks. Experimental
results demonstrate that the proposed AECIF-Net outperforms the current
state-of-the-art approaches, achieving promising performance with 92.11% mIoU
for element segmentation and 87.16% mIoU for corrosion segmentation on the test
set of the new benchmark dataset Steel Bridge Condition Inspection Visual
(SBCIV). An ablation study verifies the merits of the designs for AECIF-Net,
and a case study demonstrates its capability to automate structural condition
assessment.Comment: Submitted to Automation in Constructio
Fusion-GRU: A Deep Learning Model for Future Bounding Box Prediction of Traffic Agents in Risky Driving Videos
To ensure the safe and efficient navigation of autonomous vehicles and
advanced driving assistance systems in complex traffic scenarios, predicting
the future bounding boxes of surrounding traffic agents is crucial. However,
simultaneously predicting the future location and scale of target traffic
agents from the egocentric view poses challenges due to the vehicle's egomotion
causing considerable field-of-view changes. Moreover, in anomalous or risky
situations, tracking loss or abrupt motion changes limit the available
observation time, requiring learning of cues within a short time window.
Existing methods typically use a simple concatenation operation to combine
different cues, overlooking their dynamics over time. To address this, this
paper introduces the Fusion-Gated Recurrent Unit (Fusion-GRU) network, a novel
encoder-decoder architecture for future bounding box localization. Unlike
traditional GRUs, Fusion-GRU accounts for mutual and complex interactions among
input features. Moreover, an intermediary estimator coupled with a
self-attention aggregation layer is also introduced to learn sequential
dependencies for long range prediction. Finally, a GRU decoder is employed to
predict the future bounding boxes. The proposed method is evaluated on two
publicly available datasets, ROL and HEV-I. The experimental results showcase
the promising performance of the Fusion-GRU, demonstrating its effectiveness in
predicting future bounding boxes of traffic agents
A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images
The vast network of bridges in the United States raises a high requirement
for maintenance and rehabilitation. The massive cost of manual visual
inspection to assess bridge conditions is a burden to some extent. Advanced
robots have been leveraged to automate inspection data collection. Automating
the segmentations of multiclass elements and surface defects on the elements in
the large volume of inspection image data would facilitate an efficient and
effective assessment of the bridge condition. Training separate single-task
networks for element parsing (i.e., semantic segmentation of multiclass
elements) and defect segmentation fails to incorporate the close connection
between these two tasks. Both recognizable structural elements and apparent
surface defects are present in the inspection images. This paper is motivated
to develop a multitask deep learning model that fully utilizes such
interdependence between bridge elements and defects to boost the model's task
performance and generalization. Furthermore, the study investigated the
effectiveness of the proposed model designs for improving task performance,
including feature decomposition, cross-talk sharing, and multi-objective loss
function. A dataset with pixel-level labels of bridge elements and corrosion
was developed for model training and testing. Quantitative and qualitative
results from evaluating the developed multitask deep model demonstrate its
advantages over the single-task-based model not only in performance (2.59%
higher mIoU on bridge parsing and 1.65% on corrosion segmentation) but also in
computational time and implementation capability.Comment: Accepted for presentation at the 2023 TRB Annual Meeting and
publication in the Transportation Research Record: Journal of the
Transportation Research Board (TRR
An Attention-guided Multistream Feature Fusion Network for Localization of Risky Objects in Driving Videos
Detecting dangerous traffic agents in videos captured by vehicle-mounted
dashboard cameras (dashcams) is essential to facilitate safe navigation in a
complex environment. Accident-related videos are just a minor portion of the
driving video big data, and the transient pre-accident processes are highly
dynamic and complex. Besides, risky and non-risky traffic agents can be similar
in their appearance. These make risky object localization in the driving video
particularly challenging. To this end, this paper proposes an attention-guided
multistream feature fusion network (AM-Net) to localize dangerous traffic
agents from dashcam videos. Two Gated Recurrent Unit (GRU) networks use object
bounding box and optical flow features extracted from consecutive video frames
to capture spatio-temporal cues for distinguishing dangerous traffic agents. An
attention module coupled with the GRUs learns to attend to the traffic agents
relevant to an accident. Fusing the two streams of features, AM-Net predicts
the riskiness scores of traffic agents in the video. In supporting this study,
the paper also introduces a benchmark dataset called Risky Object Localization
(ROL). The dataset contains spatial, temporal, and categorical annotations with
the accident, object, and scene-level attributes. The proposed AM-Net achieves
a promising performance of 85.73% AUC on the ROL dataset. Meanwhile, the AM-Net
outperforms current state-of-the-art for video anomaly detection by 6.3% AUC on
the DoTA dataset. A thorough ablation study further reveals AM-Net's merits by
evaluating the contributions of its different components.Comment: Submitted to IEEE-T-IT
Vision Sensor based Action Recognition for Improving Efficiency and Quality under the Environment of Industry 4.0
In the environment of industry 4.0, human beings are still an important influencing factor of efficiency and quality which are the core of product life cycle management. Hence, monitoring and analyzing humans\u27 actions are essential. This paper proposes a vision sensor based method to evaluate the accuracy of operators\u27 actions. Each action of operators is recognized in real time by a Convolutional Neural Network (CNN) based classification model in which hierarchical clustering is introduced to minimize the effects of action uncertainty. Warnings are triggered when incorrect actions occur in real time and applications of action analysis of workers on a reducer assembling line show the effectiveness of the proposed method. The research is expected to provide a guidance for operators to correct their actions to reduce the cost of quality defects and improve the efficiency of workforce
- …