22,153 research outputs found
On Distributed Storage Allocations for Memory-Limited Systems
In this paper we consider distributed allocation problems with memory
constraint limits. Firstly, we propose a tractable relaxation to the problem of
optimal symmetric allocations from [1]. The approximated problem is based on
the Q-error function, and its solution approaches the solution of the initial
problem, as the number of storage nodes in the network grows. Secondly,
exploiting this relaxation, we are able to formulate and to solve the problem
for storage allocations for memory-limited DSS storing and arbitrary memory
profiles. Finally, we discuss the extension to the case of multiple data
objects, stored in the DSS.Comment: Submitted to IEEE GLOBECOM'1
Communication-efficient Distributed Multi-resource Allocation
In several smart city applications, multiple resources must be allocated
among competing agents that are coupled through such shared resources and are
constrained --- either through limitations of communication infrastructure or
privacy considerations. We propose a distributed algorithm to solve such
distributed multi-resource allocation problems with no direct inter-agent
communication. We do so by extending a recently introduced additive-increase
multiplicative-decrease (AIMD) algorithm, which only uses very little
communication between the system and agents. Namely, a control unit broadcasts
a one-bit signal to agents whenever one of the allocated resources exceeds
capacity. Agents then respond to this signal in a probabilistic manner. In the
proposed algorithm, each agent makes decision of its resource demand locally
and an agent is unaware of the resource allocation of other agents. In
empirical results, we observe that the average allocations converge over time
to optimal allocations.Comment: To appear in IEEE International Smart Cities Conference (ISC2 2018),
Kansas City, USA, September, 2018. arXiv admin note: substantial text overlap
with arXiv:1711.0197
Extending Message Passing Interface Windows to Storage
This work presents an extension to MPI supporting the one-sided communication
model and window allocations in storage. Our design transparently integrates
with the current MPI implementations, enabling applications to target MPI
windows in storage, memory or both simultaneously, without major modifications.
Initial performance results demonstrate that the presented MPI window extension
could potentially be helpful for a wide-range of use-cases and with
low-overhead
PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development
This paper describes PlinyCompute, a system for development of
high-performance, data-intensive, distributed computing tools and libraries. In
the large, PlinyCompute presents the programmer with a very high-level,
declarative interface, relying on automatic, relational-database style
optimization to figure out how to stage distributed computations. However, in
the small, PlinyCompute presents the capable systems programmer with a
persistent object data model and API (the "PC object model") and associated
memory management system that has been designed from the ground-up for high
performance, distributed, data-intensive computing. This contrasts with most
other Big Data systems, which are constructed on top of the Java Virtual
Machine (JVM), and hence must at least partially cede performance-critical
concerns such as memory management (including layout and de/allocation) and
virtual method/function dispatch to the JVM. This hybrid approach---declarative
in the large, trusting the programmer's ability to utilize PC object model
efficiently in the small---results in a system that is ideal for the
development of reusable, data-intensive tools and libraries. Through extensive
benchmarking, we show that implementing complex objects manipulation and
non-trivial, library-style computations on top of PlinyCompute can result in a
speedup of 2x to more than 50x or more compared to equivalent implementations
on Spark.Comment: 48 pages, including references and Appendi
- …