14,870 research outputs found
Computing large market equilibria using abstractions
Computing market equilibria is an important practical problem for market
design (e.g. fair division, item allocation). However, computing equilibria
requires large amounts of information (e.g. all valuations for all buyers for
all items) and compute power. We consider ameliorating these issues by applying
a method used for solving complex games: constructing a coarsened abstraction
of a given market, solving for the equilibrium in the abstraction, and lifting
the prices and allocations back to the original market. We show how to bound
important quantities such as regret, envy, Nash social welfare, Pareto
optimality, and maximin share when the abstracted prices and allocations are
used in place of the real equilibrium. We then study two abstraction methods of
interest for practitioners: 1) filling in unknown valuations using techniques
from matrix completion, 2) reducing the problem size by aggregating groups of
buyers/items into smaller numbers of representative buyers/items and solving
for equilibrium in this coarsened market. We find that in real data
allocations/prices that are relatively close to equilibria can be computed from
even very coarse abstractions
Learning Scheduling Algorithms for Data Processing Clusters
Efficiently scheduling data processing jobs on distributed compute clusters
requires complex algorithms. Current systems, however, use simple generalized
heuristics and ignore workload characteristics, since developing and tuning a
scheduling policy for each workload is infeasible. In this paper, we show that
modern machine learning techniques can generate highly-efficient policies
automatically. Decima uses reinforcement learning (RL) and neural networks to
learn workload-specific scheduling algorithms without any human instruction
beyond a high-level objective such as minimizing average job completion time.
Off-the-shelf RL techniques, however, cannot handle the complexity and scale of
the scheduling problem. To build Decima, we had to develop new representations
for jobs' dependency graphs, design scalable RL models, and invent RL training
methods for dealing with continuous stochastic job arrivals. Our prototype
integration with Spark on a 25-node cluster shows that Decima improves the
average job completion time over hand-tuned scheduling heuristics by at least
21%, achieving up to 2x improvement during periods of high cluster load
Balancing the Tradeoff between Profit and Fairness in Rideshare Platforms During High-Demand Hours
Rideshare platforms, when assigning requests to drivers, tend to maximize profit for the system and/or minimize waiting time for riders. Such platforms can exacerbate biases that drivers may have over certain types of requests. We consider the case of peak hours when the demand for rides is more than the supply of drivers. Drivers are well aware of their advantage during the peak hours and can choose to be selective about which rides to accept. Moreover, if in such a scenario, the assignment of requests to drivers (by the platform) is made only to maximize profit and/or minimize wait time for riders, requests of a certain type (e.g. from a non-popular pickup location, or to a non-popular drop-off location) might never be assigned to a driver. Such a system can be highly unfair to riders. However, increasing fairness might come at a cost of the overall profit made by the rideshare platform. To balance these conflicting goals, we present a flexible, non-adaptive algorithm, \lpalg, that allows the platform designer to control the profit and fairness of the system via parameters and respectively. We model the matching problem as an online bipartite matching where the set of drivers is offline and requests arrive online. Upon the arrival of a request, we use \lpalg to assign it to a driver (the driver might then choose to accept or reject it) or reject the request. We formalize the measures of profit and fairness in our setting and show that by using \lpalg, the competitive ratios for profit and fairness measures would be no worse than and respectively. Extensive experimental results on both real-world and synthetic datasets confirm the validity of our theoretical lower bounds. Additionally, they show that \lpalg under some choice of can beat two natural heuristics, Greedy and Uniform, on \emph{both} fairness and profit
An adaptive multi-agent system for task reallocation in a MapReduce job
International audienceWe study the problem of task reallocation for load-balancing of MapReduce jobs in applications that process large datasets. In this context, we propose a novel strategy based on cooperative agents used to optimise the task scheduling in a single MapReduce job. The novelty of our strategy lies in the ability of agents to identify opportunities within a current unbalanced allocation, which in turn trigger concurrent and one-to-many negotiations amongst agents to locally reallocate some of the tasks within a job. Our contribution is that tasks are reallocated according to the proximity of the resources and they are performed in accordance to the capabilities of the nodes in which agents are situated. To evaluate the adaptivity and responsiveness of our approach, we implement a prototype test-bed and conduct a vast panel of experiments in a heterogeneous environment and by exploring varying hardware configurations. This extensive experimentation reveals that our strategy significantly improves the overall runtime over the classical Hadoop data processing
Multi-type Fair Resource Allocation for Distributed Multi-Robot Systems
Fair resource allocation is essential to ensure that all resource requesters acquire adequate resources and accomplish tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters. We apply the dominant resource fairness (DRF) principle in our solutions to two different systems: single-tasking robots with multi-robot tasks (STR-MRT) and multi-tasking robots with single-robot tasks (MTR-SRT). In STR-MRT, each robot can perform only one task at a time, tasks are divisible, and accomplishing each task requires one or more robots. In MTR-SRT, each robot can perform multiple tasks at a time, tasks are not divisible, and accomplishing each task requires only one robot.
We present centralized solutions to the fairness problem in STR-MRT. Meanwhile, we model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot subgroup is formed by robots that strategically select the same resource requester. For a requester associated with a specific subgroup, a consensus-based team formation algorithm further chooses the minimal set of robots to accomplish the task. We leverage the Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the commonly used Q-learning.
Finally, we propose two decentralized solutions to promote fair resource allocation in MTR-SRT, as a centralized solution already exists. We first propose a task-forwarding solution in which the robots need to negotiate the placement of each task. In our second solution, each robot first selects resource requesters and then independently allocates resources to tasks that arrive from the selected requesters. The resource-requester selection phase of the latter solution models a coordination game that is solved by reinforcement learning. The experimental results suggest that both approaches outperform their baselines
- …