6,409 research outputs found
Efficient Transductive Online Learning via Randomized Rounding
Most traditional online learning algorithms are based on variants of mirror
descent or follow-the-leader. In this paper, we present an online algorithm
based on a completely different approach, tailored for transductive settings,
which combines "random playout" and randomized rounding of loss subgradients.
As an application of our approach, we present the first computationally
efficient online algorithm for collaborative filtering with trace-norm
constrained matrices. As a second application, we solve an open question
linking batch learning and transductive online learningComment: To appear in a Festschrift in honor of V.N. Vapnik. Preliminary
version presented in NIPS 201
Hierarchies of Relaxations for Online Prediction Problems with Evolving Constraints
We study online prediction where regret of the algorithm is measured against
a benchmark defined via evolving constraints. This framework captures online
prediction on graphs, as well as other prediction problems with combinatorial
structure. A key aspect here is that finding the optimal benchmark predictor
(even in hindsight, given all the data) might be computationally hard due to
the combinatorial nature of the constraints. Despite this, we provide
polynomial-time \emph{prediction} algorithms that achieve low regret against
combinatorial benchmark sets. We do so by building improper learning algorithms
based on two ideas that work together. The first is to alleviate part of the
computational burden through random playout, and the second is to employ
Lasserre semidefinite hierarchies to approximate the resulting integer program.
Interestingly, for our prediction algorithms, we only need to compute the
values of the semidefinite programs and not the rounded solutions. However, the
integrality gap for Lasserre hierarchy \emph{does} enter the generic regret
bound in terms of Rademacher complexity of the benchmark set. This establishes
a trade-off between the computation time and the regret bound of the algorithm
Metric Learning for Individual Fairness
There has been much discussion concerning how "fairness" should be measured or enforced in classification. Individual Fairness [Dwork et al., 2012], which requires that similar individuals be treated similarly, is a highly appealing definition as it gives strong treatment guarantees for individuals. Unfortunately, the need for a task-specific similarity metric has prevented its use in practice. In this work, we propose a solution to the problem of approximating a metric for Individual Fairness based on human judgments. Our model assumes access to a human fairness arbiter who is free of explicit biases and possesses sufficient domain knowledge to evaluate similarity. Our contributions include definitions for metric approximation relevant for Individual Fairness, constructions for approximations from a limited number of realistic queries to the arbiter on a sample of individuals, and learning procedures to construct hypotheses for metric approximations which generalize to unseen samples under certain assumptions of learnability of distance threshold functions
Changing Bases: Multistage Optimization for Matroids and Matchings
This paper is motivated by the fact that many systems need to be maintained
continually while the underlying costs change over time. The challenge is to
continually maintain near-optimal solutions to the underlying optimization
problems, without creating too much churn in the solution itself. We model this
as a multistage combinatorial optimization problem where the input is a
sequence of cost functions (one for each time step); while we can change the
solution from step to step, we incur an additional cost for every such change.
We study the multistage matroid maintenance problem, where we need to maintain
a base of a matroid in each time step under the changing cost functions and
acquisition costs for adding new elements. The online version of this problem
generalizes online paging. E.g., given a graph, we need to maintain a spanning
tree at each step: we pay for the cost of the tree at time
, and also for the number of edges changed at
this step. Our main result is an -approximation, where is
the number of elements/edges and is the rank of the matroid. We also give
an approximation for the offline version of the problem. These
bounds hold when the acquisition costs are non-uniform, in which caseboth these
results are the best possible unless P=NP.
We also study the perfect matching version of the problem, where we must
maintain a perfect matching at each step under changing cost functions and
costs for adding new elements. Surprisingly, the hardness drastically
increases: for any constant , there is no
-approximation to the multistage matching maintenance
problem, even in the offline case
- …