Efficient External-Memory Algorithms for Graph Mining

The explosion of big data in areas like the web and social networks has posed big challenges to research activities, including data mining, information retrieval, security etc. This dissertation focuses on a particular area, graph mining, and specifically proposes several novel algorithms to solve the problems of triangle listing and computation of neighborhood function in large-scale graphs. We first study the classic problem of triangle listing. We generalize the existing in-memory algorithms into a single framework of 18 triangle-search techniques. We then develop a novel external-memory approach, which we call Pruned Companion Files (PCF), that supports disk operation of all 18 algorithms. When compared to state-of-the-art available implementations MGT and PDTL, PCF runs 5-10 times faster and exhibits orders of magnitude less I/O. We next focus on I/O complexity of triangle listing. Recent work by Pagh etc. provides an appealing theoretical I/O complexity for triangle listing via graph partitioning by random coloring of nodes. Since no implementation of Pagh is available and little is known about the comparison between Pagh and PCF, we carefully implement Pagh, undertake an investigation into the properties of these algorithms, model their I/O cost, understand their shortcomings, and shed light on the conditions under which each method defeats the other. This insight leads us to develop a novel framework we call Trigon that surpasses the I/O performance of both techniques in all graphs and under all RAM conditions. We finally turn our attention to neighborhood function. Exact computation of neighborhood function is expensive in terms of CPU and I/O cost. Previous work mostly focuses on approximations. We show that our novel techniques developed for triangle listing can also be applied to this problem. We next study an application of neighborhood function to ranking of Internet hosts. Our method computes neighborhood functions for each host as an indication of its reputation. The evaluation shows that our method is robust to ranking manipulation and brings less spam to its top ranking list compared to PageRank and TrustRank

External Memory Algorithms for Factoring Sparse Matrices

We consider the factorization of sparse symmetric matrices in the context of a two-layer storage system: disk/core. When the core is sufficiently large the factorization can be performed in-core. In this case we must read the input, compute, and write the output, in this sequence. On the other hand, when the core is not large enough, the factorization becomes out-of-core, which means that data movement and computation must be interleaved. We identify two major out-of-core factorization scenarios: read-once/write-once (R1/W1) and read-many/write-many (RM/WM). The former requires minimum traffic, exactly as much as the in-core factorization: reading the input and writing the output. More traffic is required for the latter. We investigate three issues: the size of the core that determines the boundary between the two out-of-core scenarios, the in-core data structure reorganizations required by the R1/W1 factorization and the traffic required by the RM/WM factorization. We use three common factorization algorithms: left-looking, right-looking and multifrontal. In the R1/W1 scenario, our results indicate that for problems with good separators, such as those coming from the discretization of partial differential equations, ordered with nested dissection, right-looking and multifrontal factorization perform slightly better than left-looking factorization. There are, however, applications for which multifrontal is a bad choice, requiring too much temporary storage. On the other hand, right-looking factorization should be avoided in the RM/WM scenario. Left-looking is a good choice, but only if data is blocked along one dimension. Multifrontal performs well for both one and two dimensional blocks as long as not too much storage is required. We also explore a framework for a software implementation. We have implemented an in-core solver that relies on some object-oriented constructs. Most of the code is written in C++, except for some kernels written in Fortran 77. We intend to add out-of-core functionality to the code and data movement is a major concern. Implicit data movement represents the easy way, but, as some of our experiments show, good performance can be achieved only with explicit data movement. This complicates the code and we expect a substantial effort in order to implement an efficient out-of-core solver

Practical Parallel External Memory Algorithms via Simulation of Parallel Algorithms

This thesis introduces PEMS2, an improvement to PEMS (Parallel External Memory System). PEMS executes Bulk-Synchronous Parallel (BSP) algorithms in an External Memory (EM) context, enabling computation with very large data sets which exceed the size of main memory. Many parallel algorithms have been designed and implemented for Bulk-Synchronous Parallel models of computation. Such algorithms generally assume that the entire data set is stored in main memory at once. PEMS overcomes this limitation without requiring any modification to the algorithm by using disk space as memory for additional "virtual processors". Previous work has shown this to be a promising approach which scales well as computational resources (i.e. processors and disks) are added. However, the technique incurs significant overhead when compared with purpose-built EM algorithms. PEMS2 introduces refinements to the simulation process intended to reduce this overhead as well as the amount of disk space required to run the simulation. New functionality is also introduced, including asynchronous I/O and support for multi-core processors. Experimental results show that these changes significantly improve the runtime of the simulation. PEMS2 narrows the performance gap between simulated BSP algorithms and their hand-crafted EM counterparts, providing a practical system for using BSP algorithms with data sets which exceed the size of RAM

An Experimental Study of External Memory Algorithms for Connected Components

We empirically investigate algorithms for solving Connected Components in the external memory model. In particular, we study whether the randomized O(Sort(E)) algorithm by Karger, Klein, and Tarjan can be implemented to compete with practically promising and simpler algorithms having only slightly worse theoretical cost, namely Borůvka’s algorithm and the algorithm by Sibeyn and collaborators. For all algorithms, we develop and test a number of tuning options. Our experiments are executed on a large set of different graph classes including random graphs, grids, geometric graphs, and hyperbolic graphs. Among our findings are: The Sibeyn algorithm is a very strong contender due to its simplicity and due to an added degree of freedom in its internal workings when used in the Connected Components setting. With the right tunings, the Karger-Klein-Tarjan algorithm can be implemented to be competitive in many cases. Higher graph density seems to benefit Karger-Klein-Tarjan relative to Sibeyn. Borůvka’s algorithm is not competitive with the two others

External-Memory Algorithms for Processing Line Segments in Geographic Information Systems

The original publication is available at www.springerlink.comIn the design of algorithms for large-scale applications it is essential to consider the problem of minimizing I/O communication. Geographical information systems (GIS) are good examples of such large-scale applications as they frequently handle huge amounts of spatial data. In this paper we develop e cient new external-memory algorithms for a number of important problems involving line segments in the plane, including trapezoid decomposition, batched planar point location, triangulation, red-blue line segment intersection reporting, and general line segment intersection reporting. In GIS systems, the rst three problems are useful for rendering and modeling, and the latter two are frequently used for overlaying maps and extracting information from them

Efficient External-Memory Algorithms for Graph Mining

The explosion of big data in areas like the web and social networks has posed big challenges to research activities, including data mining, information retrieval, security etc. This dissertation focuses on a particular area, graph mining, and specifically proposes several novel algorithms to solve the problems of triangle listing and computation of neighborhood function in large-scale graphs. We first study the classic problem of triangle listing. We generalize the existing in-memory algorithms into a single framework of 18 triangle-search techniques. We then develop a novel external-memory approach, which we call Pruned Companion Files (PCF), that supports disk operation of all 18 algorithms. When compared to state-of-the-art available implementations MGT and PDTL, PCF runs 5-10 times faster and exhibits orders of magnitude less I/O. We next focus on I/O complexity of triangle listing. Recent work by Pagh etc. provides an appealing theoretical I/O complexity for triangle listing via graph partitioning by random coloring of nodes. Since no implementation of Pagh is available and little is known about the comparison between Pagh and PCF, we carefully implement Pagh, undertake an investigation into the properties of these algorithms, model their I/O cost, understand their shortcomings, and shed light on the conditions under which each method defeats the other. This insight leads us to develop a novel framework we call Trigon that surpasses the I/O performance of both techniques in all graphs and under all RAM conditions. We finally turn our attention to neighborhood function. Exact computation of neighborhood function is expensive in terms of CPU and I/O cost. Previous work mostly focuses on approximations. We show that our novel techniques developed for triangle listing can also be applied to this problem. We next study an application of neighborhood function to ranking of Internet hosts. Our method computes neighborhood functions for each host as an indication of its reputation. The evaluation shows that our method is robust to ranking manipulation and brings less spam to its top ranking list compared to PageRank and TrustRank

External memory algorithms for state space exploration in model checking and action planning

RAM is a scarce resource. Several real-world problems in model checking and action planning are beyond the reach of traditional RAM-based algorithms, due to the so-called state space explosion problem. This dissertation aims at designing a set of algorithms that mitigates the memory bottleneck problem in model checking and planning, through a controlled and efficient usage of secondary storage mediums, such as hard disks. We consider a broad variety of system models ranging from simple undirected and unweighted state spaces to highly complex Markov decision processes (MDP). Path/cycle search problem in the case of deterministic state spaces and policy search problem in the case of MDPs are the focal points of this thesis. The state spaces, or the implicit graphs, are not provided beforehand, but are generated on-the-fly through a set of initial states and a set of transformation rules. The proposed algorithms belong to the category of External Memory (EM) algorithms and are analyzed for their asymptotic I/O complexity. An EM guided search algorithm, called External A* (for being derived from the famous Best-First Search algorithm A*), is developed. External A* distinguishes itself from other external guided search approaches by being completely oblivious to the state space structure. Directed model checking has proved itself to be very effective in delivering shorter error trails and in memory savings. We incorporate external search into automata-based LTL model checking of concurrent systems through an extended variant of External A*. Accepting cycle detection lies at the heart of LTL model checking. Due to the inherent difficulty in cycle search in large graphs, earlier disk-based approaches distanced themselves from taking care of the full LTL model checking. In this dissertation, two algorithms for accepting cycle detection are put forward: a blind search algorithm based on Breadth-First traversal, and a guided algorithm evolved from External A*. To be able to utilize the full potential of modern multi-core architectures and easily accessible networks of workstations, External A* is further extended into a distributed algorithm. For model checking large real-time systems and optimal real-time scheduling, EM algorithms for exploration in timed automata and priced timed automata are presented. Graph-based action planning methods have achieved a significant level of maturity in the field of planning and scheduling. To integrate external heuristic search into planning, External Enforced Hill-Climbing is contributed. For optimal planning in PDDL3 domains involving preferences, a Cost-Optimal External Breadth-First Search is proposed. Nondeterministic and probabilistic state spaces are encountered both in model checking of stochastic systems and in planning under uncertainty. In such state spaces, one is interested not in a path but rather in a policy that maximizes the reward in reaching to a goal state. Due to the back-propagation of information in policy search, no efficient disk-based solution was ever contributed. We present an EM algorithm based on the standard Value Iteration procedure for policy search. The algorithm, External Value Iteration, is able to solve Bellman equations not only for large MDPs, but also for AND/OR graphs and Game trees. The algorithms developed in this dissertation have been successfully integrated in some state-of-the-art tools including the SPIN model checker, MIPS-XXL (based on FF) planning system and UPPAAL-CORA for real-time scheduling. The largest reported exploration consumed 3 Terabytes of hard disk, while using only 3 Gigabytes of RAM lasting for 479 hours -- time went down to 196 hours when 4 processors were engaged