955 research outputs found
Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions
We study the tradeoff between consistency and robustness in the context of a
single-trajectory time-varying Markov Decision Process (MDP) with untrusted
machine-learned advice. Our work departs from the typical approach of treating
advice as coming from black-box sources by instead considering a setting where
additional information about how the advice is generated is available. We prove
a first-of-its-kind consistency and robustness tradeoff given Q-value advice
under a general MDP model that includes both continuous and discrete
state/action spaces. Our results highlight that utilizing Q-value advice
enables dynamic pursuit of the better of machine-learned advice and a robust
baseline, thus result in near-optimal performance guarantees, which provably
improves what can be obtained solely with black-box advice.Comment: 27 page
Double Coverage with Machine-Learned Advice
We study the fundamental online k-server problem in a learning-augmented
setting. While in the traditional online model, an algorithm has no information
about the request sequence, we assume that there is given some advice (e.g.
machine-learned predictions) on an algorithm's decision. There is, however, no
guarantee on the quality of the prediction and it might be far from being
correct.
Our main result is a learning-augmented variation of the well-known Double
Coverage algorithm for k-server on the line (Chrobak et al., SIDMA 1991) in
which we integrate predictions as well as our trust into their quality. We give
an error-dependent competitive ratio, which is a function of a user-defined
confidence parameter, and which interpolates smoothly between an optimal
consistency, the performance in case that all predictions are correct, and the
best-possible robustness regardless of the prediction quality. When given good
predictions, we improve upon known lower bounds for online algorithms without
advice. We further show that our algorithm achieves for any k an almost optimal
consistency-robustness tradeoff, within a class of deterministic algorithms
respecting local and memoryless properties.
Our algorithm outperforms a previously proposed (more general)
learning-augmented algorithm. It is remarkable that the previous algorithm
crucially exploits memory, whereas our algorithm is memoryless. Finally, we
demonstrate in experiments the practicability and the superior performance of
our algorithm on real-world data.Comment: Accepted at ITCS 202
Learning-Augmented Query Policies for Minimum Spanning Tree with Uncertainty
We study how to utilize (possibly erroneous) predictions in a model for computing under uncertainty in which an algorithm can query unknown data. Our aim is to minimize the number of queries needed to solve the minimum spanning tree problem, a fundamental combinatorial optimization problem that has been central also to the research area of explorable uncertainty. For all integral ? ? 2, we present algorithms that are ?-robust and (1+1/?)-consistent, meaning that they use at most ?OPT queries if the predictions are arbitrarily wrong and at most (1+1/?)OPT queries if the predictions are correct, where OPT is the optimal number of queries for the given instance. Moreover, we show that this trade-off is best possible. Furthermore, we argue that a suitably defined hop distance is a useful measure for the amount of prediction error and design algorithms with performance guarantees that degrade smoothly with the hop distance. We also show that the predictions are PAC-learnable in our model. Our results demonstrate that untrusted predictions can circumvent the known lower bound of 2, without any degradation of the worst-case ratio. To obtain our results, we provide new structural insights for the minimum spanning tree problem that might be useful in the context of query-based algorithms regardless of predictions. In particular, we generalize the concept of witness sets - the key to lower-bounding the optimum - by proposing novel global witness set structures and completely new ways of adaptively using those
Robust Learning for Smoothed Online Convex Optimization with Feedback Delay
We study a challenging form of Smoothed Online Convex Optimization, a.k.a.
SOCO, including multi-step nonlinear switching costs and feedback delay. We
propose a novel machine learning (ML) augmented online algorithm,
Robustness-Constrained Learning (RCL), which combines untrusted ML predictions
with a trusted expert online algorithm via constrained projection to robustify
the ML prediction. Specifically,we prove that RCL is able to
guarantee-competitiveness against any given expert for
any, while also explicitly training the ML model in a
robustification-aware manner to improve the average-case performance.
Importantly,RCL is the first ML-augmented algorithm with a provable robustness
guarantee in the case of multi-step switching cost and feedback delay.We
demonstrate the improvement of RCL in both robustness and average performance
using battery management for electrifying transportationas a case study.Comment: Accepted by NeurIPS 202
Sorting and Hypergraph Orientation under Uncertainty with Predictions
Learning-augmented algorithms have been attracting increasing interest, but have only recently been considered in the setting of explorable uncertainty where precise values of uncertain input elements can be obtained by a query and the goal is to minimize the number of queries needed to solve a problem. We study learning-augmented algorithms for sorting and hypergraph orientation under uncertainty, assuming access to untrusted predictions for the uncertain values. Our algorithms provide improved performance guarantees for accurate predictions while maintaining worst-case guarantees that are best possible without predictions. For sorting, our algorithm uses the optimal number of queries for accurate predictions and at most twice the optimal number for arbitrarily wrong predictions. For hypergraph orientation, for any γ ≥ 2, we give an algorithm that uses at most 1 + 1/γ times the optimal number of queries for accurate predictions and at most γ times the optimal number for arbitrarily wrong predictions. These tradeoffs are the best possible. We also consider different error metrics and show that the performance of our algorithms degrades smoothly with the prediction error in all the cases where this is possible
Online Algorithms with Uncertainty-Quantified Predictions
Online algorithms with predictions have become a trending topic in the field
of beyond worst-case analysis of algorithms. These algorithms incorporate
predictions about the future to obtain performance guarantees that are of high
quality when the predictions are good, while still maintaining bounded
worst-case guarantees when predictions are arbitrarily poor. In general, the
algorithm is assumed to be unaware of the prediction's quality. However, recent
developments in the machine learning literature have studied techniques for
providing uncertainty quantification on machine-learned predictions, which
describes how certain a model is about its quality. This paper examines the
question of how to optimally utilize uncertainty-quantified predictions in the
design of online algorithms. In particular, we consider predictions augmented
with uncertainty quantification describing the likelihood of the ground truth
falling in a certain range, designing online algorithms with these
probabilistic predictions for two classic online problems: ski rental and
online search. In each case, we demonstrate that non-trivial modifications to
algorithm design are needed to fully leverage the probabilistic predictions.
Moreover, we consider how to utilize more general forms of uncertainty
quantification, proposing a framework based on online learning that learns to
exploit uncertainty quantification to make optimal decisions in multi-instance
settings
Trusted Execution Environments in Protecting Machine Learning Models
The adaptation and application of machine learning (ML) has grown extensively in recent years, and has awakened concern about the safety of intellectual property (IP) related to the machine learning models. The training of machine learning models is a time-consuming and expensive task, that has increased the demand of better solutions to protect the intellectual property of the machine learning models. This thesis explores the promising potential of Trusted Execution Environments (TEE) like Intel's Software Guard Extensions (Intel SGX), in protecting intellectual property related to machine learning models. The concern of ML model safety arises especially when the software solution needs to be distributed to clients or machine learning operations needs to be done in an untrusted environment. The main focus of this thesis is on Intel's SGX, which is one of the most used TEE implementations. This thesis tries to answer to the questions on how TEEs can be used to protect IP of the ML models, what aspects need to be considered and what limitations may arise
Learning-Augmented Online TSP on Rings, Trees, Flowers and (Almost) Everywhere Else
We study the Online Traveling Salesperson Problem (OLTSP) with predictions. In OLTSP, a sequence of initially unknown requests arrive over time at points (locations) of a metric space. The goal is, starting from a particular point of the metric space (the origin), to serve all these requests while minimizing the total time spent. The server moves with unit speed or is "waiting" (zero speed) at some location. We consider two variants: in the open variant, the goal is achieved when the last request is served. In the closed one, the server additionally has to return to the origin. We adopt a prediction model, introduced for OLTSP on the line [Gouleakis et al., 2023], in which the predictions correspond to the locations of the requests and extend it to more general metric spaces.
We first propose an oracle-based algorithmic framework, inspired by previous work [Bampis et al., 2023]. This framework allows us to design online algorithms for general metric spaces that provide competitive ratio guarantees which, given perfect predictions, beat the best possible classical guarantee (consistency). Moreover, they degrade gracefully along with the increase in error (smoothness), but always within a constant factor of the best known competitive ratio in the classical case (robustness).
Having reduced the problem to designing suitable efficient oracles, we describe how to achieve this for general metric spaces as well as specific metric spaces (rings, trees and flowers), the resulting algorithms being tractable in the latter case. The consistency guarantees of our algorithms are tight in almost all cases, and their smoothness guarantees only suffer a linear dependency on the error, which we show is necessary. Finally, we provide robustness guarantees improving previous results
The Predicted-Deletion Dynamic Model: Taking Advantage of ML Predictions, for Free
The main bottleneck in designing efficient dynamic algorithms is the unknown
nature of the update sequence. In particular, there are some problems, like
3-vertex connectivity, planar digraph all pairs shortest paths, and others,
where the separation in runtime between the best partially dynamic solutions
and the best fully dynamic solutions is polynomial, sometimes even exponential.
In this paper, we formulate the predicted-deletion dynamic model, motivated
by a recent line of empirical work about predicting edge updates in dynamic
graphs. In this model, edges are inserted and deleted online, and when an edge
is inserted, it is accompanied by a "prediction" of its deletion time. This
models real world settings where services may have access to historical data or
other information about an input and can subsequently use such information make
predictions about user behavior. The model is also of theoretical interest, as
it interpolates between the partially dynamic and fully dynamic settings, and
provides a natural extension of the algorithms with predictions paradigm to the
dynamic setting.
We give a novel framework for this model that "lifts" partially dynamic
algorithms into the fully dynamic setting with little overhead. We use our
framework to obtain improved efficiency bounds over the state-of-the-art
dynamic algorithms for a variety of problems. In particular, we design
algorithms that have amortized update time that scales with a partially dynamic
algorithm, with high probability, when the predictions are of high quality. On
the flip side, our algorithms do no worse than existing fully-dynamic
algorithms when the predictions are of low quality. Furthermore, our algorithms
exhibit a graceful trade-off between the two cases. Thus, we are able to take
advantage of ML predictions asymptotically "for free.'
- …