Decentralized Stochastic Planning with Anonymity in Interactions

In this paper, we solve cooperative decentralized stochastic planning problems, where the interactions between agents (specified using transition and reward functions) are dependent on the number of agents (and not on the identity of the individual agents) involved in the interaction. A collision of robots in a narrow corridor, defender teams coordinating patrol activities to secure a target, etc. are examples of such anonymous interactions. Formally, we consider problems that are a subset of the well known Decentralized MDP (DEC-MDP) model, where the anonymity in interactions is specified within the joint reward and transition functions. In this paper, not only do we introduce a general model model called D-SPAIT to capture anonymity in interactions, but also provide optimization based optimal and local-optimal solutions for generalizable sub-categories of D-SPAIT.Singapore. National Research Foundation (Singapore-MIT Alliance for Research and Technology Center. Future Urban Mobility Program

Solving Uncertain MDPs with Objectives that are Separable over Instantiations of Model Uncertainty

National Research Foundation (NRF) Singapore under SMART and Future Mobilit

Dynamic repositioning to reduce lost demand in Bike Sharing Systems

National Research Foundation (NRF) Singapore under SMART Center for Future Mobilit

Dynamic Redeployment to Counter Congestion or Starvation in Vehicle Sharing Systems

Extensive usage of private vehicles has led to increased traffic congestion, carbon emissions, and usage of non-renewable resources. These concerns have led to the wide adoption of vehicle sharing (ex: bike sharing, car sharing) systems in many cities of the world. In vehicle-sharing systems, base stations (ex: docking stations for bikes) are strategically placed throughout a city and each of the base stations contain a pre-determined num-ber of vehicles at the beginning of each day. Due to the stochastic and individualistic movement of customers, there is typically either congestion (more than required) or starvation (fewer than required) of vehicles at cer-tain base stations. As demonstrated in our experimental results, this happens often and can cause a significant loss in demand. We propose to dynamically redeploy idle vehicles using carriers so as to minimize lost de-mand or alternatively maximize revenue for the vehicle sharing company. To that end, we contribute an opti-mization formulation to jointly address the redeploy-ment (of vehicles) and routing (of carriers) problems and provide two approaches that rely on decomposabil-ity and abstraction of problem domains to reduce the computation time significantly. Finally, we demonstrate the utility of our approaches on two real world data sets of bike-sharing companies.

Regret based Robust Solutions for Uncertain Markov Decision Processes

In this paper, we seek robust policies for uncertain Markov Decision Processes (MDPs). Most robust optimization approaches for these problems have focussed on the computation of {\em maximin} policies which maximize the value corresponding to the worst realization of the uncertainty. Recent work has proposed {\em minimax} regret as a suitable alternative to the {\em maximin} objective for robust optimization. However, existing algorithms for handling {\em minimax} regret are restricted to models with uncertainty over rewards only. We provide algorithms that employ sampling to improve across multiple dimensions: (a) Handle uncertainties over both transition and reward models; (b) Dependence of model uncertainties across state, action pairs and decision epochs; (c) Scalability and quality bounds. Finally, to demonstrate the empirical effectiveness of our sampling approaches, we provide comparisons against benchmark algorithms on two domains from literature. We also provide a Sample Average Approximation (SAA) analysis to compute a posteriori error bounds.Singapore. National Research Foundation (Singapore-MIT Alliance for Research and Technology Center. Future Urban Mobility Program)United States. Office of Naval Research (Grant N00014-12-1-0999

Sampling based approaches for minimizing regret in uncertain Markov Decision Problems (MDPs)

National Research Foundation (NRF) Singapore under Singapore-MIT Alliance for Research and Technology (SMART) Center for Future Mobilit

An adjustable sample average approximation algorithm for the stochastic production-inventory-routing problem

We consider a stochastic single item production-inventory-routing problem with a single producer, multiple clients, and multiple vehicles. At the clients, demand is allowed to be backlogged incurring a penalty cost. Demands are considered uncertain. A recourse model is presented, and valid inequalities are introduced to enhance the model. A new general approach that explores the sample average approximation (SAA) method is introduced. In the sample average approximation method, several sample sets are generated and solved independently in order to obtain a set of candidate solutions. Then, the candidate solutions are tested on a larger sample, and the best solution is selected among the candidates. In contrast to this approach, called static, we propose an adjustable approach that explores the candidate solutions in order to identify common structures. Using that information, part of the first-stage decision variables is fixed, and the resulting restricted problem is solved for a larger size sample. Several heuristic algorithms based on the mathematical model are considered within each approach. Computational tests based on randomly generated instances are conducted to test several variants of the two approaches. The results show that the new adjustable SAA heuristic performs better than the static one for most of the instances.publishe

Lagrangian relaxation bounds for a production-inventory-routing problem

We consider a single item Production-Inventory-Routing problem with a single producer/supplier and multiple retailers. Inventory management constraints are considered both at the producer and at the retailers, following a vendor managed inventory approach, where the supplier monitors the inventory at retailers and decides on the replenishment policy for each retailer. We assume a constant production capacity. Based on the mathematical formulation we discuss a classical Lagrangian relaxation which allows to decompose the problem into four subproblems, and a new Lagrangian decomposition which decomposes the problem into just a production-inventory subproblem and a routing subproblem. The new decomposition is enhanced with valid inequalities. A computational study is reported to compare the bounds from the two approaches

A branch-and-cut algorithm for a multi-item inventory distribution problem

This paper considers a multi-item inventory distribution problem motivated by a practical case occurring in the logistic operations of an hospital. There, a single warehouse supplies several nursing wards. The goal is to define a weekly distribution plan of medical products that minimizes the visits to wards, while respecting inventory capacities and safety stock levels. A mathematical formulation is introduced and several improvements such as tightening constraints, valid inequalities and an extended reformulation are discussed. In order to deal with real size instances, an hybrid heuristic based on mathematical models is introduced and the improvements are discussed. A branch-and-cut algorithm using all the discussed improvements is proposed. Finally, a computational experimentation is reported to show the relevance of the model improvements and the quality of the heuristic scheme