A deep reinforcement learning model for predictive maintenance planning of road assets: Integrating LCA and LCCA

Road maintenance planning is an integral part of road asset management. One of the main challenges in Maintenance and Rehabilitation (M&R) practices is to determine maintenance type and timing. This research proposes a framework using Reinforcement Learning (RL) based on the Long Term Pavement Performance (LTPP) database to determine the type and timing of M&R practices. A predictive DNN model is first developed in the proposed algorithm, which serves as the Environment for the RL algorithm. For the Policy estimation of the RL model, both DQN and PPO models are developed. However, PPO has been selected in the end due to better convergence and higher sample efficiency. Indicators used in this study are International Roughness Index (IRI) and Rutting Depth (RD). Initially, we considered Cracking Metric (CM) as the third indicator, but it was then excluded due to the much fewer data compared to other indicators, which resulted in lower accuracy of the results. Furthermore, in cost-effectiveness calculation (reward), we considered both the economic and environmental impacts of M&R treatments. Costs and environmental impacts have been evaluated with paLATE 2.0 software. Our method is tested on a hypothetical case study of a six-lane highway with 23 kilometers length located in Texas, which has a warm and wet climate. The results propose a 20-year M&R plan in which road condition remains in an excellent condition range. Because the early state of the road is at a good level of service, there is no need for heavy maintenance practices in the first years. Later, after heavy M&R actions, there are several 1-2 years of no need for treatments. All of these show that the proposed plan has a logical result. Decision-makers and transportation agencies can use this scheme to conduct better maintenance practices that can prevent budget waste and, at the same time, minimize the environmental impacts

Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management

We present a multi-agent Deep Reinforcement Learning (DRL) framework for managing large transportation infrastructure systems over their life-cycle. Life-cycle management of such engineering systems is a computationally intensive task, requiring appropriate sequential inspection and maintenance decisions able to reduce long-term risks and costs, while dealing with different uncertainties and constraints that lie in high-dimensional spaces. To date, static age- or condition-based maintenance methods and risk-based or periodic inspection plans have mostly addressed this class of optimization problems. However, optimality, scalability, and uncertainty limitations are often manifested under such approaches. The optimization problem in this work is cast in the framework of constrained Partially Observable Markov Decision Processes (POMDPs), which provides a comprehensive mathematical basis for stochastic sequential decision settings with observation uncertainties, risk considerations, and limited resources. To address significantly large state and action spaces, a Deep Decentralized Multi-agent Actor-Critic (DDMAC) DRL method with Centralized Training and Decentralized Execution (CTDE), termed as DDMAC-CTDE is developed. The performance strengths of the DDMAC-CTDE method are demonstrated in a generally representative and realistic example application of an existing transportation network in Virginia, USA. The network includes several bridge and pavement components with nonstationary degradation, agency-imposed constraints, and traffic delay and risk considerations. Compared to traditional management policies for transportation networks, the proposed DDMAC-CTDE method vastly outperforms its counterparts. Overall, the proposed algorithmic framework provides near optimal solutions for transportation infrastructure management under real-world constraints and complexities

Deep Reinforcement Learning-based Project Prioritization for Rapid Post-Disaster Recovery of Transportation Infrastructure Systems

Among various natural hazards that threaten transportation infrastructure, flooding represents a major hazard in Region 6\u27s states to roadways as it challenges their design, operation, efficiency, and safety. The catastrophic flooding disaster event generally leads to massive obstruction of traffic, direct damage to highway/bridge structures/pavement, and indirect damages to economic activities and regional communities that may cause loss of many lives. After disasters strike, reconstruction and maintenance of an enormous number of damaged transportation infrastructure systems require each DOT to take extremely expensive and long-term processes. In addition, planning and organizing post-disaster reconstruction and maintenance projects of transportation infrastructures are extremely challenging for each DOT because they entail a massive number and the broad areas of the projects with various considerable factors and multi-objective issues including social, economic, political, and technical factors. Yet, amazingly, a comprehensive, integrated, data-driven approach for organizing and prioritizing post-disaster transportation reconstruction projects remains elusive. In addition, DOTs in Region 6 still need to improve the current practice and systems to robustly identify and accurately predict the detailed factors and their impacts affecting post-disaster transportation recovery. The main objective of this proposed research is to develop a deep reinforcement learning-based project prioritization system for rapid post-disaster reconstruction and recovery of damaged transportation infrastructure systems. This project also aims to provide a means to facilitate the systematic optimization and prioritization of the post-disaster reconstruction and maintenance plan of transportation infrastructure by focusing on social, economic, and technical aspects. The outcomes from this project would help engineers and decision-makers in Region 6\u27s State DOTs optimize and sequence transportation recovery processes at a regional network level with necessary recovery factors and evaluating its long-term impacts after disasters

A Decision Making Framework for Recommended Maintenance of Road Segments

With the rapid development of global road transportation, countries worldwide have completed the construction of road networks. However, the ensuing challenge lies in the maintenance of existing roads. It is well-known that countries allocate limited budgets to road maintenance projects, and road management departments face difficulties in making scientifically informed maintenance decisions. Therefore, integrating various artificial intelligence decision-making techniques to thoroughly explore historical maintenance data and adapt them to the context of road maintenance scientific decision-making has become an urgent issue. This integration aims to provide road management departments with more scientific tools and evidence for decision-making. The framework proposed in this paper primarily addresses the following four issues: 1) predicting the pavement performance of various routes, 2) determining the prioritization of maintenance routes, 3) making maintenance decisions based on the evaluation of the effects of past maintenance, and considering comprehensive technical and management indicators, and 4) determining the prioritization of maintenance sections based on the maintenance effectiveness and recommended maintenance effectiveness. By tackling these four problems, the framework enables intelligent decision-making for the optimal maintenance plan and maintenance sections, taking into account limited funding and historical maintenance management experience.Comment: 19 pages, 8 figures, 4 tables, and 2 algorithm

From Construction to Production: Enablers, Barriers and Opportunities for the Highways Supply Chain

The report presents the initial findings of a project part of the Lean Collaborative Research at Highways England with academia that aims at understanding enablers, barriers and opportunities to transform the current highways construction supply chain into a more manufacturing-like environment, where the benefits of production thinking can be achieved. The focus of the project is mostly on the adoption of off-site/modular (O/M) construction systems and advanced technologies, under a greater vision called “manufacturisation” of the highways supply chain

A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure

To ensure the safety and the serviceability of civil infrastructure it is essential to visually inspect and assess its physical and functional condition. This review paper presents the current state of practice of assessing the visual condition of vertical and horizontal civil infrastructure; in particular of reinforced concrete bridges, precast concrete tunnels, underground concrete pipes, and asphalt pavements. Since the rate of creation and deployment of computer vision methods for civil engineering applications has been exponentially increasing, the main part of the paper presents a comprehensive synthesis of the state of the art in computer vision based defect detection and condition assessment related to concrete and asphalt civil infrastructure. Finally, the current achievements and limitations of existing methods as well as open research challenges are outlined to assist both the civil engineering and the computer science research community in setting an agenda for future research