21,820 research outputs found
BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning
An ever increasing number of configuration parameters are provided to system
users. But many users have used one configuration setting across different
workloads, leaving untapped the performance potential of systems. A good
configuration setting can greatly improve the performance of a deployed system
under certain workloads. But with tens or hundreds of parameters, it becomes a
highly costly task to decide which configuration setting leads to the best
performance. While such task requires the strong expertise in both the system
and the application, users commonly lack such expertise.
To help users tap the performance potential of systems, we present
BestConfig, a system for automatically finding a best configuration setting
within a resource limit for a deployed system under a given application
workload. BestConfig is designed with an extensible architecture to automate
the configuration tuning for general systems. To tune system configurations
within a resource limit, we propose the divide-and-diverge sampling method and
the recursive bound-and-search algorithm. BestConfig can improve the throughput
of Tomcat by 75%, that of Cassandra by 63%, that of MySQL by 430%, and reduce
the running time of Hive join job by about 50% and that of Spark join job by
about 80%, solely by configuration adjustment
ACTS in Need: Automatic Configuration Tuning with Scalability Guarantees
To support the variety of Big Data use cases, many Big Data related systems
expose a large number of user-specifiable configuration parameters. Highlighted
in our experiments, a MySQL deployment with well-tuned configuration parameters
achieves a peak throughput as 12 times much as one with the default setting.
However, finding the best setting for the tens or hundreds of configuration
parameters is mission impossible for ordinary users. Worse still, many Big Data
applications require the support of multiple systems co-deployed in the same
cluster. As these co-deployed systems can interact to affect the overall
performance, they must be tuned together. Automatic configuration tuning with
scalability guarantees (ACTS) is in need to help system users. Solutions to
ACTS must scale to various systems, workloads, deployments, parameters and
resource limits. Proposing and implementing an ACTS solution, we demonstrate
that ACTS can benefit users not only in improving system performance and
resource utilization, but also in saving costs and enabling fairer
benchmarking
Partitioning Complex Networks via Size-constrained Clustering
The most commonly used method to tackle the graph partitioning problem in
practice is the multilevel approach. During a coarsening phase, a multilevel
graph partitioning algorithm reduces the graph size by iteratively contracting
nodes and edges until the graph is small enough to be partitioned by some other
algorithm. A partition of the input graph is then constructed by successively
transferring the solution to the next finer graph and applying a local search
algorithm to improve the current solution.
In this paper, we describe a novel approach to partition graphs effectively
especially if the networks have a highly irregular structure. More precisely,
our algorithm provides graph coarsening by iteratively contracting
size-constrained clusterings that are computed using a label propagation
algorithm. The same algorithm that provides the size-constrained clusterings
can also be used during uncoarsening as a fast and simple local search
algorithm.
Depending on the algorithm's configuration, we are able to compute partitions
of very high quality outperforming all competitors, or partitions that are
comparable to the best competitor in terms of quality, hMetis, while being
nearly an order of magnitude faster on average. The fastest configuration
partitions the largest graph available to us with 3.3 billion edges using a
single machine in about ten minutes while cutting less than half of the edges
than the fastest competitor, kMetis
Efficiently Generating Geometric Inhomogeneous and Hyperbolic Random Graphs
Hyperbolic random graphs (HRG) and geometric inhomogeneous random graphs (GIRG) are two similar generative network models that were designed to resemble complex real world networks. In particular, they have a power-law degree distribution with controllable exponent beta, and high clustering that can be controlled via the temperature T.
We present the first implementation of an efficient GIRG generator running in expected linear time. Besides varying temperatures, it also supports underlying geometries of higher dimensions. It is capable of generating graphs with ten million edges in under a second on commodity hardware. The algorithm can be adapted to HRGs. Our resulting implementation is the fastest sequential HRG generator, despite the fact that we support non-zero temperatures. Though non-zero temperatures are crucial for many applications, most existing generators are restricted to T = 0. We also support parallelization, although this is not the focus of this paper. Moreover, we note that our generators draw from the correct probability distribution, i.e., they involve no approximation.
Besides the generators themselves, we also provide an efficient algorithm to determine the non-trivial dependency between the average degree of the resulting graph and the input parameters of the GIRG model. This makes it possible to specify the desired expected average degree as input.
Moreover, we investigate the differences between HRGs and GIRGs, shedding new light on the nature of the relation between the two models. Although HRGs represent, in a certain sense, a special case of the GIRG model, we find that a straight-forward inclusion does not hold in practice. However, the difference is negligible for most use cases
Critical phenomena in complex networks
The combination of the compactness of networks, featuring small diameters,
and their complex architectures results in a variety of critical effects
dramatically different from those in cooperative systems on lattices. In the
last few years, researchers have made important steps toward understanding the
qualitatively new critical phenomena in complex networks. We review the
results, concepts, and methods of this rapidly developing field. Here we mostly
consider two closely related classes of these critical phenomena, namely
structural phase transitions in the network architectures and transitions in
cooperative models on networks as substrates. We also discuss systems where a
network and interacting agents on it influence each other. We overview a wide
range of critical phenomena in equilibrium and growing networks including the
birth of the giant connected component, percolation, k-core percolation,
phenomena near epidemic thresholds, condensation transitions, critical
phenomena in spin models placed on networks, synchronization, and
self-organized criticality effects in interacting systems on networks. We also
discuss strong finite size effects in these systems and highlight open problems
and perspectives.Comment: Review article, 79 pages, 43 figures, 1 table, 508 references,
extende
- …