23 research outputs found
Metareasoning for Planning Under Uncertainty
The conventional model for online planning under uncertainty assumes that an
agent can stop and plan without incurring costs for the time spent planning.
However, planning time is not free in most real-world settings. For example, an
autonomous drone is subject to nature's forces, like gravity, even while it
thinks, and must either pay a price for counteracting these forces to stay in
place, or grapple with the state change caused by acquiescing to them. Policy
optimization in these settings requires metareasoning---a process that trades
off the cost of planning and the potential policy improvement that can be
achieved. We formalize and analyze the metareasoning problem for Markov
Decision Processes (MDPs). Our work subsumes previously studied special cases
of metareasoning and shows that in the general case, metareasoning is at most
polynomially harder than solving MDPs with any given algorithm that disregards
the cost of thinking. For reasons we discuss, optimal general metareasoning
turns out to be impractical, motivating approximations. We present approximate
metareasoning procedures which rely on special properties of the BRTDP planning
algorithm and explore the effectiveness of our methods on a variety of
problems.Comment: Extended version of IJCAI 2015 pape
Contract Scheduling With Predictions
Contract scheduling is a general technique that allows to design a system
with interruptible capabilities, given an algorithm that is not necessarily
interruptible. Previous work on this topic has largely assumed that the
interruption is a worst-case deadline that is unknown to the scheduler. In this
work, we study the setting in which there is a potentially erroneous prediction
concerning the interruption. Specifically, we consider the setting in which the
prediction describes the time that the interruption occurs, as well as the
setting in which the prediction is obtained as a response to a single or
multiple binary queries. For both settings, we investigate tradeoffs between
the robustness (i.e., the worst-case performance assuming adversarial
prediction) and the consistency (i.e, the performance assuming that the
prediction is error-free), both from the side of positive and negative results
Rectangle Search: An Anytime Beam Search (Extended Version)
Anytime heuristic search algorithms try to find a (potentially suboptimal)
solution as quickly as possible and then work to find better and better
solutions until an optimal solution is obtained or time is exhausted. The most
widely-known anytime search algorithms are based on best-first search. In this
paper, we propose a new algorithm, rectangle search, that is instead based on
beam search, a variant of breadth-first search. It repeatedly explores
alternatives at all depth levels and is thus best-suited to problems featuring
deep local minima. Experiments using a variety of popular search benchmarks
suggest that rectangle search is competitive with fixed-width beam search and
often performs better than the previous best anytime search algorithms.Comment: 30 pages, 200+ figure
Modeling Intelligent Control of Distributed Cooperative Inferencing
The ability to harness different problem-solving methods together into a cooperative system has the potential for significantly improving the performance of systems for solving NP-hard problems. The need exists for an intelligent controller that is able to effectively combine radically different problem-solving techniques with anytime and anywhere properties into a distributed cooperative environment. This controller requires models of the component algorithms in conjunction with feedback from those algorithms during run-time to manage a dynamic combination of tasks effectively. This research develops a domain-independent method for creating these models as well as a model for the controller itself. These models provide the means for the controller to select the most appropriate algorithms, both initially and during run-time. We utilize the algorithm performance knowledge contained in the algorithm models to aid in the selection process. This methodology is applicable to many NP-hard problems; applicability is only limited by the availability of anytime and anywhere algorithms for that domain. We demonstrate the capabilities of this methodology by applying it to a known NP-hard problem: uncertain inference over Bayesian Networks. Experiments using a collection of randomly generated networks and some common inference algorithms showed very promising results. Future directions for this research could involve the analysis of the impact of the accuracy of the algorithm models on the performance of the controller; the issue is whether the increased model accuracy would cause excessive system overhead, counteracting the potential increase in performance due to better algorithm selection
MANAGING QUERY AND UPDATE TRANSACTIONS UNDER QUALITY CONTRACTS IN WEB-DATABASES
In modern Web-database systems, users typically perform read-only queries, whereas all write-only data updates are performed in the background, concurrently with queries.For most of these services to be successful and their users to be kept satisfied, two criteria need to be met: user requests must be answered in a timely fashion and must return fresh data. This is relatively easy when the system is lightly loaded and, as such, both queries and updates can be executed quickly. However, this goal becomes practically hard to achieve in real systems due to the high volumes of queries and updates, especially in periods of flash crowds. In this work, we argue it is beneficial to allow users to specify their preferences and let the system optimize towards satisfying user preferences, instead of simply improving the average case. We believe that this user-centric approach will empower the system to gracefully deal with a broader spectrum of workloads.Towards user-centric web-databases, we propose a Quality Contracts framework to help users express their preferences over multiple quality specifications. Moreover, we propose a suite of algorithms to effectively perform load balancing and scheduling for both queries and updates according to user preferences. We evaluate the proposed framework and algorithms through a simulation with real traces from disk accesses and from a stock information website. Finally, to increase the applicability of Quality Contracts enhanced Web-database systems, we propose an algorithm to help users adapt to the Web-database system behavior and maximize their query success ratio