Assessing the Optimality of Decentralized Inspection and Maintenance Policies for Stochastically Degrading Engineering Systems

Long-term inspection and maintenance (I&M) planning, a multi-stage stochastic optimization problem, can be efficiently formulated as a partially observable Markov decision process (POMDP). How- ever, within this context, single-agent approaches do not scale well for large multi-component systems since the joint state, action and observation spaces grow exponentially with the number of components. To alleviate this curse of dimensionality, cooperative decentralized approaches, known as decentralized POMDPs, are often adopted and solved using multi-agent deep reinforcement learning (MADRL) algorithms. This paper examines the centralization vs. decentralization performance of MADRL formulations in I&M planning of multi-component systems. Towards this, we set up a comprehensive computational experimental program focused on k-out-of-n system configurations, a common and broadly applicable archetype of deteriorating engineering systems, to highlight the manifestations of MADRL strengths and pathologies when optimizing global returns under varying decentralization relaxations in such systems.Architectural Technolog

Optimizing deep reinforcement learning policies for deteriorating systems considering ordered action structuring and value of information

Inspection and maintenance (I&M) optimization entails many sources of computational complexity, among others, due to high-dimensional decision and state variables in multi-component systems, long planning horizons, stochasticity of objectives and constraints, and inherent uncertainties in measurements and models. This paper studies how the above can be addressed within the context of constrained Partially Observable Markov Decision Processes (POMDPs) and Deep Reinforcement Learning (DRL) in a unified fashion. Special emphasis is paid on how ordered action structuring of I&M actions can be exploited to decompose the respective policy parametrizations in actor-critic DRL schemes, resulting into fully decoupled maintenance and inspection actors. It is shown that the Value of Information (VoI) is naturally utilized in such POMDP control frameworks, as directly associated with the DRL advantage functions that emerge in the gradient computations of the inspection policy parameters. Overall, the presented approach, following the natural flow of engineering decisions, results in new architectural configurations for policy networks, facilitating more efficient training, while alleviating further the dimensionality burdens related to combinatorial definitions of I&M actions. The efficiency of the methodology is demonstrated in numerical experiments of a structural system subject to corrosion, where the optimization problem is formulated to concurrently account for state and model uncertainties as well as long-term probability of failure exceedance constraints. Results showcase that the obtained DRL policies considerably outperform standard decision rules.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Structural Design & Mechanic

Assessing life-cycle seismic fragility of corroding reinforced concrete bridges through dynamic Bayesian networks

Bridge structures are exposed to several chronic and abrupt stressors, among which the combined effects of corrosion and earthquakes pose a major threat to their long-term safety. Probabilistic risk assessment frameworks that quantify and propagate uncertainties inherent to these phenomena are necessary to mitigate this threat. This paper proposes a dynamic Bayesian network for state-dependent seismic fragility functions, capturing corrosion and seismic effects over time. Markovian transitions among deterioration states for different bridge components are developed, combining chloride diffusion and corrosion propagation models with non-stationary Gamma processes. State-dependent fragility curves are derived based on non-linear dynamic time-history analyses given possible degradation configurations of the structure, accounting for uncertainties in material, geometry, and deterioration parameters. Record-to-record variability is captured using synthetic ground motions. Results on a 4-span Gerber bridge showcase the suitability of the framework for describing life-cycle fragility, and its capacity for embedding in advanced algorithmic decision-making workflows is discussed.Architectural Technolog

Inference and maintenance planning of monitored structures through Markov chain Monte Carlo and deep reinforcement learning

A key computational challenge in maintenance planning for deteriorating structures is to concurrently secure (i) optimality of decisions over long planning horizons, and (ii) accuracy of realtime parameter updates in high-dimensional stochastic spaces. Both are often encumbered by the presence of discretized continuous-state models that describe the underlying deterioration processes, and the emergence of combinatorial decision spaces due to multi-component environments. Recent advances in Deep Reinforcement Learning (DRL) formulations for inspection and maintenance planning provide us with powerful frameworks to handle efficiently near-optimal decision-making in immense state and action spaces without the need for offline system knowledge. Moreover, Bayesian Model Updating (BMU), aided by advanced sampling methods, allows us to address dimensionality and accuracy issues related to discretized degradation processes. Building upon these concepts, we develop a joint framework in this work, coupling DRL, more specifically deep Q-learning and actor-critic algorithms, with BMU through Hamiltonian Monte Carlo. Single- and multi-component systems are examined, and it is shown that the proposed methodology yields reduced lifelong maintenance costs, and policies of high fidelity and sophistication compared to traditional optimized time- and condition-based maintenance strategies.Mechanics and Physics of StructuresArchitectural Technolog

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Determination of inspection and maintenance policies for minimizing long-term risks and costs in deteriorating engineering environments constitutes a complex optimization problem. Major computational challenges include the (i) curse of dimensionality, due to exponential scaling of state/action set cardinalities with the number of components; (ii) curse of history, related to exponentially growing decision-trees with the number of decision-steps; (iii) presence of state uncertainties, induced by inherent environment stochasticity and variability of inspection/monitoring measurements; (iv) presence of constraints, pertaining to stochastic long-term limitations, due to resource scarcity and other infeasible/undesirable system responses. In this work, these challenges are addressed within a joint framework of constrained Partially Observable Markov Decision Processes (POMDP) and multi-agent Deep Reinforcement Learning (DRL). POMDPs optimally tackle (ii)-(iii), combining stochastic dynamic programming with Bayesian inference principles. Multi-agent DRL addresses (i), through deep function parametrizations and decentralized control assumptions. Challenge (iv) is herein handled through proper state augmentation and Lagrangian relaxation, with emphasis on life-cycle risk-based constraints and budget limitations. The underlying algorithmic steps are provided, and the proposed framework is found to outperform well-established policy baselines and facilitate adept prescription of inspection and intervention actions, in cases where decisions must be made in the most resource- and risk-aware manner.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Structural Design & Mechanic

Preface

CAAD Futures is a biennial international conference on Computer-Aided Architectural Design under the umbrella of the CAAD Futures Foundation, and it is active world-wide in advancing and documenting related research. On 5–7 July 2023, the 20th CAAD Futures conference was hosted at Delft University of Technology. The CAAD Futures Foundation was established in 1985, holding the first conference on 18–19 September of that year at the very same University. The return of the conference to Delft for its 20thedition offered a chance to reflect on the past, present and future role of Computation in Architecture and the Built Environment. With reference to the theme of “INTERCONNECTIONS: Co-computing beyond boundaries”, CAAD Futures 2023 reflected on the role of computation to interconnect in and for Architectural Design.Digital TechnologiesArchitectural Technolog

Minimum Mass Cast Glass Structures Under Performance and Manufacturability Constraints

This work develops a computational method that produces algorithmically generated design forms, able to overcome inherent challenges related to the use of cast glass for the creation of monolithic structural components with light permeability. Structural Topology Optimization (TO) has a novel applicability potential, as decreased mass is associated with shorter annealing times and, thus, considerably improved manufacturability in terms of time, energy, and cost efficiency. However, realistic TO in such structures is currently hindered by existing mathematical formulations and commercial software capabilities. Incorporating annealing constraints into the optimization problem is an essential feature that needs to be accommodated, whereas the brittle nature of glass invokes asymmetric stress failure criteria that cannot be captured by conventional ductile plasticity surfaces or uniform stress constraints. This paper addresses the approximation problems in the evaluation of principal stresses while concurrently incorporating annealing-related manufacturing constraints into a unified TO formulation. A mass minimization objective is articulated, as this is the most critical factor for cast glass structures. To ensure the structural integrity and manufacturability of the component, the applied constraints refer both to the glass material/structural properties and to criteria that ensue from the annealing and fabrication processes. The developed code is based on the penalized artificial density interpolation scheme and the optimization problem is solved with the interior-point method. The proposed formulation is applied in a planar design domain to explore how different glass compositions and structural design strategies affect the final shape. Upon extraction of the optimized shape, the structural performance of the respective 3D structures is validated with respect to performance constraint violations using the Ansys software. Finally, brief guidelines on the practical aspects of the manufacturing process are provided.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care. Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Digital TechnologiesArchitectural Technolog

The role of value of information in multi-agent deep reinforcement learning for optimal decision-making under uncertainty

To preserve structural safety of deteriorating engineering systems through optimal maintenance, it is imperative to efficiently integrate structural health information with decision-making optimization frameworks. Although there may be abundance of available data, these are often uncertain and incomplete. In addition, joint inspection and maintenance (I&M) optimization is inherently complex due to high-dimensional state and action spaces, stochastic objectives, long planning horizons, and various constraints, among others. As shown recently, these computational challenges can be effectively addressed through optimization principles of Partially Observable Markov Decision Processes (POMDPs) and constrained Deep Reinforcement Learning (DRL). The POMDP framework provides a way of updating the decision-maker's perception about the system state by naturally incorporating the Value of Information (VoI) in the optimality equations. As such, optimal observation-gathering actions are those which guide maintenance decisions towards reduced life-cycle costs and risks. The role of VoI in DRL-driven I&M has also been shown to be central to the formation of policy gradients, which are necessary to obtain the optimal I&M plan with deep learning actor-critic architectures. Leveraging this property, a recently devised DRL architecture is further examined in this work, consisting of fully decoupled 'maintainer' and 'inspector' actors, which allow for greater efficacy and interpretability in multi-agent DRL settings. Several numerical analyses are carried out to assess the performance of the relevant architectures on stochastic systems with a varying number of components, multiple maintenance-inspection actions per component, and system-level failure risks.Architectural Technolog

Appraisal and mathematical properties of fragility analysis methods

Fragility analysis aims to compute the probabilities of a system exceeding certain damage conditions given different levels of hazard intensity. Fragility analysis is therefore a key process of performance-based earthquake engineering, with a number of approaches developed and widely recognized, including Incremental Dynamic Analysis (IDA), Multiple Stripe Analysis (MSA), and cloud analysis. Additionally, extended fragility analysis has recently been shown to possess important attributes of mathematical consistency and extensibility. This work provides a critical review of the different fragility methods by explaining the underlying probabilistic models and assumptions, as well as their connections to the extended fragility method. It is proven that IDA-based fragility curves provide an upper bound of the actual fragility, and cloud analysis manifests suboptimality issues arising from its underlying assumptions. MSA is identified to be a probit-linked Bernoulli regression model, similar to the one proposed by Shinozuka and coworkers. The latter, in turn, is shown to be a limiting subcase of the generalized linear model framework introduced within the extended fragility analysis. The paper first presents a simple case of one intensity measure and two damage condition states, and the discussion is subsequently extended to more general cases of multiple intensity measures and damage states. The discussed attributes are demonstrated in several numerical applications. Overall, this work aims to provide new insights on fragility methods, enabling efficient, accurate, and consistent estimations of structural performance, as well as promoting new research directions in earthquake engineering and other related fields.Structural Design & Mechanic

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I&M decision problem to the component level, often assuming statistical, structural, or cost independence among components, due to the computational complexity associated with global optimization methodologies under joint system-level state descriptions. In this paper, we propose an efficient algorithmic framework for inference and decision-making under uncertainty for engineering systems exposed to deteriorating environments, providing optimal management strategies directly at the system level. In our approach, the decision problem is formulated as a factored partially observable Markov decision process, whose dynamics are encoded in Bayesian network conditional structures. The methodology can handle environments under equal or general, unequal deterioration correlations among components, through Gaussian hierarchical structures and dynamic Bayesian networks, decoupling the originally joint system state space to component networks conditional on shared random variables. In terms of policy optimization, we adopt a deep decentralized multi-agent actor-critic (DDMAC) reinforcement learning approach, in which the policies are approximated by actor neural networks guided by a critic network. By including deterioration dependence in the simulated environment, and by formulating the cost model at the system level, DDMAC policies intrinsically consider the underlying system-effects. This is demonstrated through numerical experiments conducted for both a 9-out-of-10 system and a steel frame under fatigue deterioration. Results demonstrate that DDMAC policies offer substantial benefits when compared to state-of-the-art heuristic approaches. The inherent consideration of system-effects by DDMAC strategies is also interpreted based on the learned policies.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Architectural Technolog