Efficient Algorithms for the Data Exchange Problem

In this paper we study the data exchange problem where a set of users is interested in gaining access to a common file, but where each has only partial knowledge about it as side-information. Assuming that the file is broken into packets, the side-information considered is in the form of linear combinations of the file packets. Given that the collective information of all the users is sufficient to allow recovery of the entire file, the goal is for each user to gain access to the file while minimizing some communication cost. We assume that users can communicate over a noiseless broadcast channel, and that the communication cost is a sum of each user's cost function over the number of bits it transmits. For instance, the communication cost could simply be the total number of bits that needs to be transmitted. In the most general case studied in this paper, each user can have any arbitrary convex cost function. We provide deterministic, polynomial-time algorithms (in the number of users and packets) which find an optimal communication scheme that minimizes the communication cost. To further lower the complexity, we also propose a simple randomized algorithm inspired by our deterministic algorithm which is based on a random linear network coding scheme.Comment: submitted to Transactions on Information Theor

Minimum Cost Multicast with Decentralized Sources

In this paper we study the multisource multicast problem where every sink in a given directed acyclic graph is a client and is interested in a common file. We consider the case where each node can have partial knowledge about the file as a side information. Assuming that nodes can communicate over the capacity constrained links of the graph, the goal is for each client to gain access to the file, while minimizing some linear cost function of number of bits transmitted in the network. We consider three types of side-information settings:(ii) side information in the form of linearly correlated packets; and (iii) the general setting where the side information at the nodes have an arbitrary (i.i.d.) correlation structure. In this work we 1) provide a polynomial time feasibility test, i.e., whether or not all the clients can recover the file, and 2) we provide a polynomial-time algorithm that finds the optimal rate allocation among the links of the graph, and then determines an explicit transmission scheme for cases (i) and (ii)

The Lov\'asz Hinge: A Novel Convex Surrogate for Submodular Losses

Learning with non-modular losses is an important problem when sets of predictions are made simultaneously. The main tools for constructing convex surrogate loss functions for set prediction are margin rescaling and slack rescaling. In this work, we show that these strategies lead to tight convex surrogates iff the underlying loss function is increasing in the number of incorrect predictions. However, gradient or cutting-plane computation for these functions is NP-hard for non-supermodular loss functions. We propose instead a novel surrogate loss function for submodular losses, the Lov\'asz hinge, which leads to O(p log p) complexity with O(p) oracle accesses to the loss function to compute a gradient or cutting-plane. We prove that the Lov\'asz hinge is convex and yields an extension. As a result, we have developed the first tractable convex surrogates in the literature for submodular losses. We demonstrate the utility of this novel convex surrogate through several set prediction tasks, including on the PASCAL VOC and Microsoft COCO datasets