7,836 research outputs found
Window-based Streaming Graph Partitioning Algorithm
In the recent years, the scale of graph datasets has increased to such a
degree that a single machine is not capable of efficiently processing large
graphs. Thereby, efficient graph partitioning is necessary for those large
graph applications. Traditional graph partitioning generally loads the whole
graph data into the memory before performing partitioning; this is not only a
time consuming task but it also creates memory bottlenecks. These issues of
memory limitation and enormous time complexity can be resolved using
stream-based graph partitioning. A streaming graph partitioning algorithm reads
vertices once and assigns that vertex to a partition accordingly. This is also
called an one-pass algorithm. This paper proposes an efficient window-based
streaming graph partitioning algorithm called WStream. The WStream algorithm is
an edge-cut partitioning algorithm, which distributes a vertex among the
partitions. Our results suggest that the WStream algorithm is able to partition
large graph data efficiently while keeping the load balanced across different
partitions, and communication to a minimum. Evaluation results with real
workloads also prove the effectiveness of our proposed algorithm, and it
achieves a significant reduction in load imbalance and edge-cut with different
ranges of dataset
Adaptive Partitioning for Large-Scale Dynamic Graphs
Abstract—In the last years, large-scale graph processing has gained increasing attention, with most recent systems placing particular emphasis on latency. One possible technique to improve runtime performance in a distributed graph processing system is to reduce network communication. The most notable way to achieve this goal is to partition the graph by minimizing the num-ber of edges that connect vertices assigned to different machines, while keeping the load balanced. However, real-world graphs are highly dynamic, with vertices and edges being constantly added and removed. Carefully updating the partitioning of the graph to reflect these changes is necessary to avoid the introduction of an extensive number of cut edges, which would gradually worsen computation performance. In this paper we show that performance degradation in dynamic graph processing systems can be avoided by adapting continuously the graph partitions as the graph changes. We present a novel highly scalable adaptive partitioning strategy, and show a number of refinements that make it work under the constraints of a large-scale distributed system. The partitioning strategy is based on iterative vertex migrations, relying only on local information. We have implemented the technique in a graph processing system, and we show through three real-world scenarios how adapting graph partitioning reduces execution time by over 50 % when compared to commonly used hash-partitioning. I
On combinatorial optimisation in analysis of protein-protein interaction and protein folding networks
Abstract: Protein-protein interaction networks and protein folding networks represent prominent research topics at the intersection of bioinformatics and network science. In this paper, we present a study of these networks from combinatorial optimisation point of view. Using a combination of classical heuristics and stochastic optimisation techniques, we were able to identify several interesting combinatorial properties of biological networks of the COSIN project. We obtained optimal or near-optimal solutions to maximum clique and chromatic number problems for these networks. We also explore patterns of both non-overlapping and overlapping cliques in these networks. Optimal or near-optimal solutions to partitioning of these networks into non-overlapping cliques and to maximum independent set problem were discovered. Maximal cliques are explored by enumerative techniques. Domination in these networks is briefly studied, too. Applications and extensions of our findings are discussed
Network Community Detection on Metric Space
Community detection in a complex network is an important problem of much
interest in recent years. In general, a community detection algorithm chooses
an objective function and captures the communities of the network by optimizing
the objective function, and then, one uses various heuristics to solve the
optimization problem to extract the interesting communities for the user. In
this article, we demonstrate the procedure to transform a graph into points of
a metric space and develop the methods of community detection with the help of
a metric defined for a pair of points. We have also studied and analyzed the
community structure of the network therein. The results obtained with our
approach are very competitive with most of the well-known algorithms in the
literature, and this is justified over the large collection of datasets. On the
other hand, it can be observed that time taken by our algorithm is quite less
compared to other methods and justifies the theoretical findings
Finding Non-overlapping Clusters for Generalized Inference Over Graphical Models
Graphical models use graphs to compactly capture stochastic dependencies
amongst a collection of random variables. Inference over graphical models
corresponds to finding marginal probability distributions given joint
probability distributions. In general, this is computationally intractable,
which has led to a quest for finding efficient approximate inference
algorithms. We propose a framework for generalized inference over graphical
models that can be used as a wrapper for improving the estimates of approximate
inference algorithms. Instead of applying an inference algorithm to the
original graph, we apply the inference algorithm to a block-graph, defined as a
graph in which the nodes are non-overlapping clusters of nodes from the
original graph. This results in marginal estimates of a cluster of nodes, which
we further marginalize to get the marginal estimates of each node. Our proposed
block-graph construction algorithm is simple, efficient, and motivated by the
observation that approximate inference is more accurate on graphs with longer
cycles. We present extensive numerical simulations that illustrate our
block-graph framework with a variety of inference algorithms (e.g., those in
the libDAI software package). These simulations show the improvements provided
by our framework.Comment: Extended the previous version to include extensive numerical
simulations. See http://www.ima.umn.edu/~dvats/GeneralizedInference.html for
code and dat
Recent Advances in Graph Partitioning
We survey recent trends in practical algorithms for balanced graph
partitioning together with applications and future research directions
- …