Parallelisation for data-intensive applications over peer-to-peer networks

In Data Intensive Computing, properties of the data that are the input for an application decide running performance in most cases. Those properties include the size of the data, the relationships inside data, and so forth. There is a class of data intensive applications (BLAST, SETI@home, Folding@Home and so on so forth) whose performances solely depend on the amount of input data. Another important characteristic of those applications is that the input data can be split into units and these units are not related to each other during the runs of the applications. This characteristic helps this class of data intensive applications to be parallelised in the way where the input data is split into units and application runs on different computer nodes for certain portion of the units. SETI@home and Folding@Home have been successfully parallelised over peer-to-peer networks. However, they suffer from the problems of single point of failure and poor scalability. In order to solve these problems, we choose BLAST as our example data intensive applications and parallelise BLAST over a fully distributed peer-to-peer network. BLAST is a popular bioinformatics toolset which can be used to compare two DNA sequences. The major usage of BLAST is searching a query of sequences inside a database for their similarities so as to identify whether they are new. When comparing single pair of sequences, BLAST is efficient. However, due to growing size of the databases, executing BLAST jobs locally produces prohibitively poor performance. Thus, methods for parallelising BLAST are sought. Traditional BLAST parallelisation approaches are all based on clusters. Clusters employ a number of computing nodes and high bandwidth interlinks between nodes. Cluster-based BLAST exhibits higher performance; nevertheless, clusters suffer from limited resources and scalability problems. Clusters are expensive, prohibitively so when the growth of the sequence database are taken into account. It involves high cost and complication when increasing the number of nodes to adapt to the growth of BLAST databases. Hence a Peer-to-Peer-based BLAST service is required. This thesis demonstrates our parallelisation of BLAST over Peer-to-Peer networks (termed ppBLAST), which utilises the free storage and computing resources in the Peer-to-Peer networks to complete BLAST jobs in parallel. In order to achieve the goal, we build three layers in ppBLAST each of which is responsible for particular functions. The bottom layer is a DHT infrastructure with the support of range queries. It provides efficient range-based lookup service and storage for BLAST tasks. The middle layer is the BitTorrent-based database distribution. The upper layer is the core of ppBLAST which schedules and dispatches task to peers. For each layer, we conduct comprehensive research and the achievements are presented in this thesis. For the DHT layer, we design and implement our DAST-DHT. We analyse balancing, maximum number of children and the accuracy of the range query. We also compare the DAST with other range query methodology and state that if the number of children is adjusted to more two, the performance of DAST overcomes others. For the BitTorrent-like database distribution layer, we investigate the relationship between the seeding strategies and the selfish leechers (freeriders and exploiters). We conclude that OSS works better than TSS in a normal situation

Managing metadata in open learning repositories and P2P networks

[no abstract

Efficient, Locally-Enforceable Querier Privacy for Distributed Database Systems

Traditionally, the declarative nature of SQL is viewed as a major strength. It allows database users to simply describe what they want to retrieve without worrying about how the answer to their question is actually computed. However, in a decentralized setting, two different approaches to evaluating the same query may reveal vastly different information about the query being asked (and, hence, about the user) to participating servers. In the case that a user's query contains sensitive or private information, this is clearly problematic. In this dissertation, we address the problem of protecting query issuer privacy. We hypothesize that by extending SQL to allow for declarative specification of constraints on the attributes of query evaluation plans and accounting for such constraints during query optimization, users can produce efficient query evaluation plans that protect the private intensional regions of their queries without explicit server-side support. Towards supporting this hypothesis, we formalize a notion of intensional query privacy that we call (I, A)-privacy, and present PASQL, a set of extensions to SQL that allows users to specify (I, A)-privacy constraints to a query optimizer. We explore tradeoffs between the expressiveness of several PASQL variants, optimization time requirements, and the optimality of plans produced. We present two algorithms for optimizing queries with attached (I, A)-privacy constraints and formally establish their time and space complexities. We prove that one is capable of producing optimal results, though at the cost of greatly increased time and space requirements. We use the other as the basis of PAQO, our implementation of an (I, A)-privacy-aware query optimizer. We present an extensive experimental evaluation of PAQO to show that is is capable of efficiently generating plans to evaluate PASQL queries, and to confirm the results of our formal complexity analysis

Parallelisation for data-intensive applications over peer-to-peer networks

In Data Intensive Computing, properties of the data that are the input for an application decide running performance in most cases. Those properties include the size of the data, the relationships inside data, and so forth. There is a class of data intensive applications (BLAST, SETI@home, Folding@Home and so on so forth) whose performances solely depend on the amount of input data. Another important characteristic of those applications is that the input data can be split into units and these units are not related to each other during the runs of the applications. This characteristic helps this class of data intensive applications to be parallelised in the way where the input data is split into units and application runs on different computer nodes for certain portion of the units. SETI@home and Folding@Home have been successfully parallelised over peer-to-peer networks. However, they suffer from the problems of single point of failure and poor scalability. In order to solve these problems, we choose BLAST as our example data intensive applications and parallelise BLAST over a fully distributed peer-to-peer network. BLAST is a popular bioinformatics toolset which can be used to compare two DNA sequences. The major usage of BLAST is searching a query of sequences inside a database for their similarities so as to identify whether they are new. When comparing single pair of sequences, BLAST is efficient. However, due to growing size of the databases, executing BLAST jobs locally produces prohibitively poor performance. Thus, methods for parallelising BLAST are sought. Traditional BLAST parallelisation approaches are all based on clusters. Clusters employ a number of computing nodes and high bandwidth interlinks between nodes. Cluster-based BLAST exhibits higher performance; nevertheless, clusters suffer from limited resources and scalability problems. Clusters are expensive, prohibitively so when the growth of the sequence database are taken into account. It involves high cost and complication when increasing the number of nodes to adapt to the growth of BLAST databases. Hence a Peer-to-Peer-based BLAST service is required. This thesis demonstrates our parallelisation of BLAST over Peer-to-Peer networks (termed ppBLAST), which utilises the free storage and computing resources in the Peer-to-Peer networks to complete BLAST jobs in parallel. In order to achieve the goal, we build three layers in ppBLAST each of which is responsible for particular functions. The bottom layer is a DHT infrastructure with the support of range queries. It provides efficient range-based lookup service and storage for BLAST tasks. The middle layer is the BitTorrent-based database distribution. The upper layer is the core of ppBLAST which schedules and dispatches task to peers. For each layer, we conduct comprehensive research and the achievements are presented in this thesis. For the DHT layer, we design and implement our DAST-DHT. We analyse balancing, maximum number of children and the accuracy of the range query. We also compare the DAST with other range query methodology and state that if the number of children is adjusted to more two, the performance of DAST overcomes others. For the BitTorrent-like database distribution layer, we investigate the relationship between the seeding strategies and the selfish leechers (freeriders and exploiters). We conclude that OSS works better than TSS in a normal situation.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Efficient Range and Join Query Processing in Massively Distributed Peer-to-Peer Networks

Peer-to-peer (P2P) has become a modern distributed computing architecture that supports massively large-scale data management and query processing. Complex query operators such as range operator and join operator are needed by various distributed applications, including content distribution, locality-aware services, computing resource sharing, and many others. This dissertation tackles a number of problems related to range and join query processing in P2P systems: fault-tolerant range query processing under structured P2P architecture, distributed range caching under unstructured P2P architecture, and integration of heterogeneous data under unstructured P2P architecture. To support fault-tolerant range query processing so as to provide strong performance guarantees in the presence of network churn, effective replication schemes are developed at either the overlay network level or the query processing level. To facilitate range query processing, a prefetch-based caching approach is proposed to eliminate the performance bottlenecks incurred by those data items that are not well cached in the network. Finally, a purely decentralized partition-based join query operator is devised to realize bandwidth-efficient join query processing under unstructured P2P architecture. Theoretical analysis and experimental simulations demonstrate the effectiveness of the proposed approaches

Localisation de sources de données et optimisation de requêtes réparties en environnement pair-à-pair

Malgré leur succès dans le domaine du partage de fichiers, les systèmes P2P sont capables d'évaluer uniquement des requêtes simples basées sur la recherche d'un fichier en utilisant son nom. Récemment, plusieurs travaux de recherche sont effectués afin d'étendre ces systèmes pour qu'ils permettent le partage de données avec une granularité fine (i.e. un attribut atomique) et l'évaluation de requêtes complexes (i.e. requêtes SQL). A cause des caractéristiques des systèmes P2P (e.g. grande-échelle, instabilité et autonomie de nœuds), il n'est pas pratique d'avoir un catalogue global qui contient souvent des informations sur: les schémas, les données et les hôtes des sources de données. L'absence d'un catalogue global rend plus difficiles: (i) la localisation de sources de données en prenant en compte l'hétérogénéité de schémas et (ii) l'optimisation de requêtes. Dans notre thèse, nous proposons une approche pour l'évaluation des requêtes SQL en environnement P2P. Notre approche est fondée sur une ontologie de domaine et sur des formules de similarité pour résoudre l'hétérogénéité sémantique des schémas locaux. Quant à l'hétérogénéité structurelle de ces schémas, elle est résolue grâce à l'extension d'un algorithme de routage de requêtes (i.e. le protocole Chord) par des Indexes de structure. Concernant l'optimisation de requêtes, nous proposons de profiter de la phase de localisation de sources de données pour obtenir toutes les méta-données nécessaires pour générer un plan d'exécution proche de l'optimal. Afin de montrer la faisabilité et la validité de nos propositions, nous effectuons une évaluation des performances et nous discutons les résultats obtenus.Despite of their great success in the file sharing domain, P2P systems support only simple queries usually based on looking up a file by using its name. Recently, several research works have made to extend P2P systems to be able to share data having a fine granularity (i.e. atomic attribute) and to process queries written with a highly expressive language (i.e. SQL). The characteristics of P2P systems (e.g. large-scale, node autonomy and instability) make impractical to have a global catalog that stores often information about data, schemas and data source hosts. Because of the absence of a global catalog, two problems become more difficult: (i) locating data sources with taking into account the schema heterogeneity and (ii) query optimization. In our thesis, we propose an approach for processing SQL queries in a P2P environment. To solve the semantic heterogeneity between local schemas, our approach is based on domain ontology and on similarity formulas. As for the structural heterogeneity of local schemas, it is solved by the extension of a query routing method (i.e. Chord protocol) with Structure Indexes. Concerning the query optimization problem, we propose to take advantage of the data source localization phase to obtain all metadata required for generating a close to optimal execution plan. Finally, in order to show the feasibility and the validity of our propositions, we carry out performance evaluations and we discuss the obtained results