994 research outputs found
An optimally data efficient isomorphism inference algorithm
The time, space, and data complexity of an optimally data efficient isomorphism identification algorithm are presented. The data complexity, the amount of data required for an inference algorithm to terminate, is analyzed and shown to be the minimum possible for all possible isomorphism inference algorithms. The minimum data requirement is shown to be ⌈log2 (n)⌉, and a method for constructing this minimal sequence of data is presented. The average data requirement is shown to be approximately 2 log2(n). The time complexity is O(n2log2(n)) and the space requirement is O(n2
Streaming Graph Challenge: Stochastic Block Partition
An important objective for analyzing real-world graphs is to achieve scalable
performance on large, streaming graphs. A challenging and relevant example is
the graph partition problem. As a combinatorial problem, graph partition is
NP-hard, but existing relaxation methods provide reasonable approximate
solutions that can be scaled for large graphs. Competitive benchmarks and
challenges have proven to be an effective means to advance state-of-the-art
performance and foster community collaboration. This paper describes a graph
partition challenge with a baseline partition algorithm of sub-quadratic
complexity. The algorithm employs rigorous Bayesian inferential methods based
on a statistical model that captures characteristics of the real-world graphs.
This strong foundation enables the algorithm to address limitations of
well-known graph partition approaches such as modularity maximization. This
paper describes various aspects of the challenge including: (1) the data sets
and streaming graph generator, (2) the baseline partition algorithm with
pseudocode, (3) an argument for the correctness of parallelizing the Bayesian
inference, (4) different parallel computation strategies such as node-based
parallelism and matrix-based parallelism, (5) evaluation metrics for partition
correctness and computational requirements, (6) preliminary timing of a
Python-based demonstration code and the open source C++ code, and (7)
considerations for partitioning the graph in streaming fashion. Data sets and
source code for the algorithm as well as metrics, with detailed documentation
are available at GraphChallenge.org.Comment: To be published in 2017 IEEE High Performance Extreme Computing
Conference (HPEC
Multi-capacity bin packing with dependent items and its application to the packing of brokered workloads in virtualized environments
Providing resource allocation with performance
predictability guarantees is increasingly important in cloud
platforms, especially for data-intensive applications, in which
performance depends greatly on the available rates of data
transfer between the various computing/storage hosts underlying
the virtualized resources assigned to the application. Existing
resource allocation solutions either assume that applications
manage their data transfer between their virtualized resources, or
that cloud providers manage their internal networking resources.
With the increased prevalence of brokerage services in cloud
platforms, there is a need for resource allocation solutions that
provides predictability guarantees in settings, in which neither
application scheduling nor cloud provider resources can be
managed/controlled by the broker. This paper addresses this
problem, as we define the Network-Constrained Packing (NCP)
problem of finding the optimal mapping of brokered resources
to applications with guaranteed performance predictability. We
prove that NCP is NP-hard, and we define two special instances
of the problem, for which exact solutions can be found efficiently.
We develop a greedy heuristic to solve the general instance of the
NCP problem , and we evaluate its efficiency using simulations
on various application workloads, and network models.This work was done while author was at Boston University. It was partially supported by NSF CISE awards #1430145, #1414119, #1239021 and #1012798. (1430145 - NSF CISE; 1414119 - NSF CISE; 1239021 - NSF CISE; 1012798 - NSF CISE
Network-constrained packing of brokered workloads in virtualized environments
Providing resource allocation with performance predictability guarantees is increasingly important in cloud platforms, especially for data-intensive applications, in which performance depends greatly on the available rates of data transfer between the various computing/storage hosts underlying the virtualized resources assigned to the application. Existing resource allocation solutions either assume that applications manage their data transfer between their virtualized resources, or that cloud providers manage their internal networking resources.With the increased prevalence of brokerage services in cloud platforms, there is a need for resource allocation solutions that provides predictability guarantees in settings, in which neither application scheduling nor cloud provider resources can be managed/controlled by the broker. This paper addresses this problem, as we define the Network-Constrained Packing (NCP)problem of finding the optimal mapping of brokered resources to applications with guaranteed performance predictability. We prove that NCP is NP-hard, and we define two special instances of the problem, for which exact solutions can be found efficiently. We develop a greedy heuristic to solve the general instance of the NCP problem, and we evaluate its efficiency using simulations on various application workloads, and network models.This work is supported by NSF CISE CNS Award #1347522, # 1239021, # 1012798
- …