Solving Markov decision processes for network-level post-hazard recovery via simulation optimization and rollout

Computation of optimal recovery decisions for community resilience assurance post-hazard is a combinatorial decision-making problem under uncertainty. It involves solving a large-scale optimization problem, which is significantly aggravated by the introduction of uncertainty. In this paper, we draw upon established tools from multiple research communities to provide an effective solution to this challenging problem. We provide a stochastic model of damage to the water network (WN) within a testbed community following a severe earthquake and compute near-optimal recovery actions for restoration of the water network. We formulate this stochastic decision-making problem as a Markov Decision Process (MDP), and solve it using a popular class of heuristic algorithms known as rollout. A simulation-based representation of MDPs is utilized in conjunction with rollout and the Optimal Computing Budget Allocation (OCBA) algorithm to address the resulting stochastic simulation optimization problem. Our method employs non-myopic planning with efficient use of simulation budget. We show, through simulation results, that rollout fused with OCBA performs competitively with respect to rollout with total equal allocation (TEA) at a meagre simulation budget of 5-10% of rollout with TEA, which is a crucial step towards addressing large-scale community recovery problems following natural disasters.Comment: Submitted to Simulation Optimization for Cyber Physical Energy Systems (Special Session) in 14th IEEE International Conference on Automation Science and Engineerin

Feedback Allocation For OFDMA Systems With Slow Frequency-domain Scheduling

We study the problem of allocating limited feedback resources across multiple users in an orthogonal-frequency-division-multiple-access downlink system with slow frequency-domain scheduling. Many flavors of slow frequency-domain scheduling (e.g., persistent scheduling, semi-persistent scheduling), that adapt user-sub-band assignments on a slower time-scale, are being considered in standards such as 3GPP Long-Term Evolution. In this paper, we develop a feedback allocation algorithm that operates in conjunction with any arbitrary slow frequency-domain scheduler with the goal of improving the throughput of the system. Given a user-sub-band assignment chosen by the scheduler, the feedback allocation algorithm involves solving a weighted sum-rate maximization at each (slow) scheduling instant. We first develop an optimal dynamic-programming-based algorithm to solve the feedback allocation problem with pseudo-polynomial complexity in the number of users and in the total feedback bit budget. We then propose two approximation algorithms with complexity further reduced, for scenarios where the problem exhibits additional structure.Comment: Accepted to IEEE Transactions on Signal Processin

Selecting the best stochastic systems for large scale engineering problems

Selecting a subset of the best solutions among large-scale problems is an important area of research. When the alternative solutions are stochastic in nature, then it puts more burden on the problem. The objective of this paper is to select a set that is likely to contain the actual best solutions with high probability. If the selected set contains all the best solutions, then the selection is denoted as correct selection. We are interested in maximizing the probability of this selection; P(CS). In many cases, the available computation budget for simulating the solution set in order to maximize P(CS) is limited. Therefore, instead of distributing these computational efforts equally likely among the alternatives, the optimal computing budget allocation (OCBA) procedure came to put more effort on the solutions that have more impact on the selected set. In this paper, we derive formulas of how to distribute the available budget asymptotically to find the approximation of P(CS). We then present a procedure that uses OCBA with the ordinal optimization (OO) in order to select the set of best solutions. The properties and performance of the proposed procedure are illustrated through a numerical example. Overall results indicate that the procedure is able to select a subset of the best systems with high probability of correct selection using small number of simulation samples under different parameter settings