Otimização de consultas SPARQL em bases RDF distribuídas

Orientadora : Profa. Dra Carmem Satie HaraTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 07/04/2017Inclui referências : f. 83-85Resumo; O modelo de dados RDF vem sendo usado em diversas aplicações devido a sua simplicidade e exibilidade na modelagem de dados quando comparado aos modelos de dados tradicionais. Dado o grande volume de dados RDF existente atualmente, diversas abordagens de processamento de consultas têm sido propostas visando garantir a escalabilidade destas aplicações. De uma forma geral, estas abordagens propõem métodos de distribuição de dados a _m de promover o processamento distribuído e paralelo de consultas SPARQL em sistemas RDF. Embora a distribuição forneça escalabilidade de armazenamento, o custo de comunicação no processamento de consultas pode ser alto. Este trabalho propõe uma abordagem de processamento de consultas SPARQL que tem o objetivo de minimizar o custo de comunicação para o processamento de consultas em sistemas RDF distribuídos. A abordagem explora a existência de padrões de alocação (PAs) na distribuição de dados, fornecida por um método de distribuição controlada de dados, que determina como triplas RDF são agrupadas e armazenadas em um mesmo servidor. Sendo assim, durante a distribuição, fragmentos de bases RDF seguem a composição de um determinado PA. Logo, a abordagem de processamento proposta gera planos de execução de consultas baseando-se nestes padrões viabilizando a escolha de duas estratégias de comunicação durante o processamento de consultas: get-frag e send-result. Na primeira estratégia, dada uma consulta, um servidor requisita para servidores remotos fragmentos de dados para a resolução de consultas. Na segunda, o servidor envia resultados intermediários da consulta para outros servidores continuarem a sua execução. Essas estratégias são combinadas em um método, denominado de 2ways, que escolhe a estratégia de comunicação adequada sempre que a execução de consultas transitar entre fragmentos de dados. A escolha da estratégia depende do número de mensagens e do volume de dados a ser transmitido entre servidores. Resultados experimentais mostram que 2ways reduz o custo de comunicação de maneira efetiva e melhora o tempo de resposta do processamento de consultas SPARQL em sistemas RDF distribuídos. Por fim, considerando que bases RDF podem ser alteradas por meio de operações de exclusão/interseção de triplas, este trabalho estende a abordagem de processamento proposta considerando que nem sempre novos dados inseridos estarão de acordo com os PAs predefinidos. A abordagem de atualização define um tipo especial de PA, denominado de PaOverow, para o armazenamento de dados que não podem ser categorizados pelos PAs existentes. Logo, o PaOverow também deve ser considerado no planejamento e no processamento de consultas. Um estudo experimental inicial mostra que, como esperado, a adoção do PaOverow pode aumentar o tempo de resposta de consultas na abordagem de processamento proposta. Palavras-chave: RDF, SPARQL, Processamento Distribuído de Consultas, Otimização de Consultas.Abstract: RDF has been used by many applications due to its simplicity and exibility in data modeling. Due to the huge volume of RDF data that exists nowadays, many distributed query processing approaches have been proposed aiming to ensure scalability for these applications. In general, these approaches propose data distribution methods promoting distributed and parallel SPARQL query processing. However, while distribution may provide storage scalability, it may also incur high communication costs for processing queries. This work presents a parallel and distributed query processing approach that aims to minimize the communication cost. The approach explores the existence of data allocation patterns (PAs) for data distribution, provided by a controlled data distribution method, that determine how RDF triples should be grouped and stored on the same server. Fragments of the RDF datastore follow a given allocation pattern. The approach generates execution plans based on this distribution model making possible the choice of two communication strategies for query processing: get-frag and send-result. With the get-frag approach, a server requests remote servers to send fragments that contain data required by a query. The send-result approach, on the other hand, forwards intermediate results to other servers to continue the query processing. These strategies are combined on a method, called 2ways, that chooses the adequate communication strategy whenever queries traverse fragment boundaries. The choice of the communication strategy is based on the number of requisitions and the volume of the data to be transmitted. Experimental results show that our proposed technique e_ectively reduces the communication cost and improves the response time for processing SPARQL queries on a distributed RDF datastore. Finally, considering that RDF datasets are dynamic, and may be updated by delete/insert operations, this work extends the query processing approach considering that not all newly inserted data may conform to the prede_ned allocation patterns. We de_ne a special purpose type of PA, called PaOverow, for storing data that can not be categorized by existing PAs. Consequentelly, the PaOverow must be considered in query planning and processing. An initial experimental study shows that, as expected, the PaOverow adoption can increase the response time for processing queries on the proposed processing approach. Keywords: RDF, SPARQL, Distributed Query Processing, Query Optimization

Mining Heterogeneous Urban Data at Multiple Granularity Layers

The recent development of urban areas and of the new advanced services supported by digital technologies has generated big challenges for people and city administrators, like air pollution, high energy consumption, traffic congestion, management of public events. Moreover, understanding the perception of citizens about the provided services and other relevant topics can help devising targeted actions in the management. With the large diffusion of sensing technologies and user devices, the capability to generate data of public interest within the urban area has rapidly grown. For instance, different sensors networks deployed in the urban area allow collecting a variety of data useful to characterize several aspects of the urban environment. The huge amount of data produced by different types of devices and applications brings a rich knowledge about the urban context. Mining big urban data can provide decision makers with knowledge useful to tackle the aforementioned challenges for a smart and sustainable administration of urban spaces. However, the high volume and heterogeneity of data increase the complexity of the analysis. Moreover, different sources provide data with different spatial and temporal references. The extraction of significant information from such diverse kinds of data depends also on how they are integrated, hence alternative data representations and efficient processing technologies are required. The PhD research activity presented in this thesis was aimed at tackling these issues. Indeed, the thesis deals with the analysis of big heterogeneous data in smart city scenarios, by means of new data mining techniques and algorithms, to study the nature of urban related processes. The problem is addressed focusing on both infrastructural and algorithmic layers. In the first layer, the thesis proposes the enhancement of the current leading techniques for the storage and elaboration of Big Data. The integration with novel computing platforms is also considered to support parallelization of tasks, tackling the issue of automatic scaling of resources. At algorithmic layer, the research activity aimed at innovating current data mining algorithms, by adapting them to novel Big Data architectures and to Cloud computing environments. Such algorithms have been applied to various classes of urban data, in order to discover hidden but important information to support the optimization of the related processes. This research activity focused on the development of a distributed framework to automatically aggregate heterogeneous data at multiple temporal and spatial granularities and to apply different data mining techniques. Parallel computations are performed according to the MapReduce paradigm and exploiting in-memory computing to reach near-linear computational scalability. By exploring manifold data resolutions in a relatively short time, several additional patterns of data can be discovered, allowing to further enrich the description of urban processes. Such framework is suitably applied to different use cases, where many types of data are used to provide insightful descriptive and predictive analyses. In particular, the PhD activity addressed two main issues in the context of urban data mining: the evaluation of buildings energy efficiency from different energy-related data and the characterization of people's perception and interest about different topics from user-generated content on social networks. For each use case within the considered applications, a specific architectural solution was designed to obtain meaningful and actionable results and to optimize the computational performance and scalability of algorithms, which were extensively validated through experimental tests