Dynamic Stochastic Orienteering Problems for Risk-Aware Applications

Singapore National Research Foundatio

Revisiting Risk-Sensitive MDPs: New Algorithms and Results

While Markov Decision Processes (MDPs) have been shown to be effective models for planning under uncertainty, the objective to minimize the expected cumulative cost is in-appropriate for high-stake planning problems. As such, Yu, Lin, and Yan (1998) introduced the Risk-Sensitive MDP (RS-MDP) model, where the objective is to find a policy that max-imizes the probability that the cumulative cost is within some user-defined cost threshold. In this paper, we revisit this prob-lem and introduce new algorithms that are based on classi-cal techniques, such as depth-first search and dynamic pro-gramming, and a recently introduced technique called Topo-logical Value Iteration (TVI). We demonstrate the applica-bility of our approach on randomly generated MDPs as wel

TRACCS: Trajectory-Aware Coordinated Urban Crowd-Sourcing

We investigate the problem of large-scale mobile crowd-tasking, where a large pool of citizen crowd-workers are used to perform a variety of location-specific urban logis-tics tasks. Current approaches to such mobile crowd-tasking are very decentralized: a crowd-tasking platform usually pro-vides each worker a set of available tasks close to the worker’s current location; each worker then independently chooses which tasks she wants to accept and perform. In contrast, we propose TRACCS, a more coordinated task assignment ap-proach, where the crowd-tasking platform assigns a sequence of tasks to each worker, taking into account their expected location trajectory over a wider time horizon, as opposed to just instantaneous location. We formulate such task assign-ment as an optimization problem, that seeks to maximize the total payoff from all assigned tasks, subject to a maximum bound on the detour (from the expected path) that a worker will experience to complete her assigned tasks. We develop credible computationally-efficient heuristics to address this optimization problem (whose exact solution requires solving a complex integer linear program), and show, via simulations with realistic topologies and commuting patterns, that a spe-cific heuristic (called Greedy-ILS) increases the fraction of assigned tasks by more than 20%, and reduces the average detour overhead by more than 60%, compared to the current decentralized approach

Intelligent Traffic Management: From Practical Stochastic Path Planning to Reinforcement Learning Based City-Wide Traffic Optimization

This research focuses on intelligent traffic management including stochastic path planning and city scale traffic optimization. Stochastic path planning focuses on finding paths when edge weights are not fixed and change depending on the time of day/week. Then we focus on minimizing the running time of the overall procedure at query time utilizing precomputation and approximation. The city graph is partitioned into smaller groups of nodes and represented by its exemplar. In query time, source and destination pairs are connected to their respective exemplars and the path between those exemplars is found. After this, we move toward minimizing the city wide traffic congestion by making structural changes include changing the number of lanes, using ramp metering, varying speed limit, and modifying signal timing is possible. We propose a multi agent reinforcement learning (RL) framework for improving traffic flow in city networks. Our framework utilizes two level learning: a) each single agent learns the initial policy and b) multiple agents (changing the environment at the same time) update their policy based on the interaction with the dynamic environment and in agreement with other agents. The goal of RL agents is to interact with the environment to learn the optimal modification for each road segment through maximizing the cumulative reward over the set of possible actions in state space

Orienteering Problem: A survey of recent variants, solution approaches and applications

National Research Foundation (NRF) Singapore under International Research Centres in Singapore Funding Initiativ