Similarity search and mining in uncertain spatial and spatio-temporal databases

Both the current trends in technology such as smart phones, general mobile devices, stationary sensors and satellites as well as a new user mentality of utilizing this technology to voluntarily share information produce a huge flood of geo-spatial and geo-spatio-temporal data. This data flood provides a tremendous potential of discovering new and possibly useful knowledge. In addition to the fact that measurements are imprecise, due to the physical limitation of the devices, some form of interpolation is needed in-between discrete time instances. From a complementary perspective - to reduce the communication and bandwidth utilization, along with the storage requirements, often the data is subjected to a reduction, thereby eliminating some of the known/recorded values. These issues introduce the notion of uncertainty in the context of spatio-temporal data management - an aspect raising an imminent need for scalable and flexible data management. The main scope of this thesis is to develop effective and efficient techniques for similarity search and data mining in uncertain spatial and spatio-temporal data. In a plethora of research fields and industrial applications, these techniques can substantially improve decision making, minimize risk and unearth valuable insights that would otherwise remain hidden. The challenge of effectiveness in uncertain data is to correctly determine the set of possible results, each associated with the correct probability of being a result, in order to give a user a confidence about the returned results. The contrary challenge of efficiency, is to compute these result and corresponding probabilities in an efficient manner, allowing for reasonable querying and mining times, even for large uncertain databases. The paradigm used to master both challenges, is to identify a small set of equivalent classes of possible worlds, such that members of the same class can be treated as equivalent in the context of a given query predicate or data mining task. In the scope of this work, this paradigm will be formally defined, and applied to the most prominent classes of spatial queries on uncertain data, including range queries, k-nearest neighbor queries, ranking queries and reverse k-nearest neighbor queries. For this purpose, new spatial and probabilistic pruning approaches are developed to further speed up query processing. Furthermore, the proposed paradigm allows to develop the first efficient solution for the problem of frequent co-location mining on uncertain data. Special emphasis is taken on the temporal aspect of applications using modern data collection technologies. While the aforementioned techniques work well for single points of time, the prediction of query results over time remains a challenge. This thesis fills this gap by modeling an uncertain spatio-temporal object as a stochastic process, and by applying the above paradigm to efficiently query, index and mine historical spatio-temporal data.Moderne Technologien, z.B. Sattelitentechnologie und Technologie in Smart Phones, erzeugen eine Flut räumlicher Geo-Daten. Zudem ist in der Gesellschaft ein Trend zu beobachten diese erzeugten Daten freiwillig auf öffentlich zugänglichen Plattformen zur Verfügung zu stellen. Diese Datenflut hat immenses Potential, um neues und nützliches Wissen zu entdecken. Diese Daten sind jedoch grundsätzlich unsichere räumliche Daten. Die Unsicherheit ergibt sich aus mehreren Aspekten. Zum einen kommt es bei Messungen grundsätzlich zu Messungenauigkeiten, zum anderen ist zwischen diskreten Messzeitpunkten eine Interpolation nötig, die zusätzliche Unsicherheit erzeugt. Auerdem werden die Daten oft absichtlich reduziert, um Speicherplatz und Transfervolumen einzusparen, wodurch weitere Information verloren geht. Diese Unsicherheit schafft einen sofortigen Bedarf für skalierbare und flexible Methoden zur Verwaltung und Auswertung solcher Daten. Im Rahmen dieser Arbeit sollen effektive und effiziente Techniken zur Ähnlichkeitssuche und zum Data Mining bei unsicheren räumlichen und unsicheren räumlich-zeitlichen Daten erarbeitet werden. Diese Techniken liefern wertvolles Wissen, das auf verschiedenen Forschungsgebieten, als auch bei industriellen Anwendungen zur Entscheidungsfindung genutzt werden kann. Bei der Entwicklung dieser Techniken gibt es zwei Herausforderungen. Einerseits müssen die entwickelten Techniken effektiv sein, um korrekte Ergebnisse und Wahrscheinlichkeiten dieser Ergebnisse zurückzugeben. Andererseits müssen die entwickelten Techniken effizient sein, um auch in sehr großen Datenbanken Ergebnisse in annehmbarer Zeit zu liefern. Die Dissertation stellt ein neues Paradigma vor, das beide Herausforderungen meistert. Dieses Paradigma identifiziert mögliche Datenbankwelten, die bezüglich des gegebenen Anfrageprädikats äquivalent sind. Es wird formal definiert und auf die relevantesten räumlichen Anfragetypen angewendet, um effiziente Lösungen zu entwickeln. Dazu gehören Bereichanfragen, k-Nächste-Nachbarnanfragen, Rankinganfragen und Reverse k-Nächste-Nachbarnanfragen. Räumliche und probabilistische Pruningkriterien werden entwickelt, um insignifikante Ergebnisse früh auszuschlieen. Zudem wird die erste effiziente Lösung für das Problem des "Spatial Co-location Minings" auf unsicheren Daten präsentiert. Ein besonderer Schwerpunkt dieser Arbeit liegt auf dem temporalen Aspekt moderner Geo-Daten. Während obig genannte Techniken dieser Arbeit für einzelne Zeitpunkt sehr gut funktionieren, ist die effektive und effiziente Verwaltung von unsicheren räumlich zeitlichen Daten immer noch ein weitestgehend ungelöstes Problem. Diese Dissertation löst dieses Problem, indem unsichere räumlich-zeitliche Daten durch stochastische Prozesse modeliert werden. Auf diese stochastischen Prozesse lässt sich das oben genannte Paradigma anwenden, um unsichere räumlich-zeitliche Daten effizient anzufragen, zu indexieren, und zu minen

Modeling Algorithm Performance on Highly-threaded Many-core Architectures

The rapid growth of data processing required in various arenas of computation over the past decades necessitates extensive use of parallel computing engines. Among those, highly-threaded many-core machines, such as GPUs have become increasingly popular for accelerating a diverse range of data-intensive applications. They feature a large number of hardware threads with low-overhead context switches to hide the memory access latencies and therefore provide high computational throughput. However, understanding and harnessing such machines places great challenges on algorithm designers and performance tuners due to the complex interaction of threads and hierarchical memory subsystems of these machines. The achieved performance jointly depends on the parallelism exploited by the algorithm, the effectiveness of latency hiding, and the utilization of multiprocessors (occupancy). Contemporary work tries to model the performance of GPUs from various aspects with different emphasis and granularity. However, no model considers all of these factors together at the same time. This dissertation presents an analytical framework that jointly addresses parallelism, latency-hiding, and occupancy for both theoretical and empirical performance analysis of algorithms on highly-threaded many-core machines so that it can guide both algorithm design and performance tuning. In particular, this framework not only helps to explore and reduce the runtime configuration space for tuning kernel execution on GPUs, but also reflects performance bottlenecks and predicts how the runtime will trend as the problem and other parameters scale. The framework consists of a pair of analytical models with one focusing on higher-level asymptotic algorithm performance on GPUs and the other one emphasizing lower-level details about scheduling and runtime configuration. Based on the two models, we have conducted extensive analysis of a large set of algorithms. Two analysis provides interesting results and explains previously unexplained data. In addition, the two models are further bridged and combined as a consistent framework. The framework is able to provide an end-to-end methodology for algorithm design, evaluation, comparison, implementation, and prediction of real runtime on GPUs fairly accurately. To demonstrate the viability of our methods, the models are validated through data from implementations of a variety of classic algorithms, including hashing, Bloom filters, all-pairs shortest path, matrix multiplication, FFT, merge sort, list ranking, string matching via suffix tree/array, etc. We evaluate the models\u27 performance across a wide spectrum of parameters, data values, and machines. The results indicate that the models can be effectively used for algorithm performance analysis and runtime prediction on highly-threaded many-core machines

Anomaly detection in unknown environments using wireless sensor networks

This dissertation addresses the problem of distributed anomaly detection in Wireless Sensor Networks (WSN). A challenge of designing such systems is that the sensor nodes are battery powered, often have different capabilities and generally operate in dynamic environments. Programming such sensor nodes at a large scale can be a tedious job if the system is not carefully designed. Data modeling in distributed systems is important for determining the normal operation mode of the system. Being able to model the expected sensor signatures for typical operations greatly simplifies the human designer’s job by enabling the system to autonomously characterize the expected sensor data streams. This, in turn, allows the system to perform autonomous anomaly detection to recognize when unexpected sensor signals are detected. This type of distributed sensor modeling can be used in a wide variety of sensor networks, such as detecting the presence of intruders, detecting sensor failures, and so forth. The advantage of this approach is that the human designer does not have to characterize the anomalous signatures in advance. The contributions of this approach include: (1) providing a way for a WSN to autonomously model sensor data with no prior knowledge of the environment; (2) enabling a distributed system to detect anomalies in both sensor signals and temporal events online; (3) providing a way to automatically extract semantic labels from temporal sequences; (4) providing a way for WSNs to save communication power by transmitting compressed temporal sequences; (5) enabling the system to detect time-related anomalies without prior knowledge of abnormal events; and, (6) providing a novel missing data estimation method that utilizes temporal and spatial information to replace missing values. The algorithms have been designed, developed, evaluated, and validated experimentally in synthesized data, and in real-world sensor network applications

Predicting parking space availability based on heterogeneous data using Machine Learning techniques

Abstract. These days, smart cities are focused on improving their services and bringing quality to everyday life, leveraging modern ICT technologies. For this reason, the data from connected IoT devices, environmental sensors, economic platforms, social networking sites, governance systems, and others can be gathered for achieving such goals. The rapid increase in the number of vehicles in major cities of the world has made mobility in urban areas difficult, due to traffic congestion and parking availability issues. Finding a suitable parking space is often influenced by various factors such as weather conditions, traffic flows, and geographical information (markets, hospitals, parks, and others). In this study, a predictive analysis has been performed to estimate the availability of parking spaces using heterogeneous data from Cork County, Ireland. However, accumulating, processing, and analysing the produced data from heterogeneous sources is itself a challenge, due to their diverse nature and different acquisition frequencies. Therefore, a data lake has been proposed in this study to collect, process, analyse, and visualize data from disparate sources. In addition, the proposed platform is used for predicting the available parking spaces using the collected data from heterogeneous sources. The study includes proposed design and implementation details of data lake as well as the developed parking space availability prediction model using machine learning techniques

When Things Matter: A Data-Centric View of the Internet of Things

With the recent advances in radio-frequency identification (RFID), low-cost wireless sensor devices, and Web technologies, the Internet of Things (IoT) approach has gained momentum in connecting everyday objects to the Internet and facilitating machine-to-human and machine-to-machine communication with the physical world. While IoT offers the capability to connect and integrate both digital and physical entities, enabling a whole new class of applications and services, several significant challenges need to be addressed before these applications and services can be fully realized. A fundamental challenge centers around managing IoT data, typically produced in dynamic and volatile environments, which is not only extremely large in scale and volume, but also noisy, and continuous. This article surveys the main techniques and state-of-the-art research efforts in IoT from data-centric perspectives, including data stream processing, data storage models, complex event processing, and searching in IoT. Open research issues for IoT data management are also discussed

Natural Arabic language text understanding

The most challenging part of natural language understanding is the representation of meaning. The current representation techniques are not sufficient to resolve the ambiguities, especially when the meaning is to be used for interrogation at a later stage. Arabic language represents a challenging field for Natural Language Processing (NLP) because of its rich eloquence and free word order, but at the same time it is a good platform to capture understanding because of its rich computational, morphological and grammar rules. Among different representation techniques, Lexical Functional Grammar (LFG) theory is found to be best suited for this task because of its structural approach. LFG lays down a computational approach towards NLP, especially the constituent and the functional structures, and models the completeness of relationships among the contents of each structure internally, as well as among the structures externally. The introduction of Artificial Intelligence (AI) techniques, such as knowledge representation and inferencing, enhances the capture of meaning by utilising domain specific common sense knowledge embedded in the model of domain of discourse and the linguistic rules that have been captured from the Arabic language grammar. This work has achieved the following results: (i) It is the first attempt to apply the LFG formalism on a full Arabic declarative text that consists of more than one paragraph. (ii) It extends the semantic structure of the LFG theory by incorporating a representation based on the thematic-role frames theory. (iii) It extends to the LFG theory to represent domain specific common sense knowledge. (iv) It automates the production process of the functional and semantic structures. (v) It automates the production process of domain specific common sense knowledge structure, which enhances the understanding ability of the system and resolves most ambiguities in subsequent question-answer sessions

Landscape functional connectivity and animal movement: application of remote sensing for increasing efficiency of road mitigation measures

Roads are a major threat to wildlife due to induced mortality and restrictions to animal movement. A central issue in conservation biology is the accurate site identification for the implementation of multispecies mitigation measures, on roads. Those measures entail high costs and methodological challenges and their efficiency highly depend on the right location. The aim of this PhD is to inform, through remote sensing and connectivity modelling, how to increase the efficiency of planning mitigation measures to reduce roadkill and promote connectivity; and demonstrate the usefulness of remote sensing in defining suitable areas for the conservation of an endangered species that often occurs in the vicinity of roads. To do so, we first assessed whether occurrence-based strategies were able to infer functional connectivity, compared to those more complex and financially demanding based on telemetry, with respect to daily and dispersal movements. Secondly, we assessed whether remote sensing data were sufficiently informative to identify key habitats for a threatened species around road verges. Thirdly, we assessed the predictive and prioritisation ability of road mitigation units intercepting multispecies corridors to prevent vulnerability to roadkill. Findings revealed that simple models are suitable as complex ones for both daily and dispersal movements, allowing for costly-effective connectivity assessments. Results demonstrated the ability of free remote sensing data to identify microhabitat conditions in verges and surrounding landscape, for a threatened rodent, allowing for the delimitation of refugee areas and definition of monitoring strategies for the species. Undemanding data (occurrence and remote sensing) were able to describe species-specific ecological requirements for birds, bats and non-flying mammals as well as roadkill patterns, possibly due to similar overlapping corridors and habitats, despite some mismatches that occurred for highly mobile species. This framework ensured high efficiency in prioritisation of multispecies roadkill mitigation planning, resilient to long-term landscape dynamics; Conectividade funcional da paisagem e movimento animal: aplicação da detecção remota para aumentar a eficiência de medidas de mitigação em estradas. Resumo: As estradas constituem uma enorme ameaça para a vida selvagem devido à mortalidade. Uma questão central é a identificação dos locais para implementar medidas de mitigação multiespécies, em estradas. Essas medidas envolvem custos elevados e desafios metodológicos e sua eficiência depende muito da localização correcta. O objetivo deste doutoramento é informar, através de detecção remota e conectividade, como aumentar a eficiência do planeamento de medidas de mitigação para reduzir atropelamentos e promover a conectividade; e demonstrar a utilidade da detecção remota na definição de áreas adequadas para a conservação de espécies ameaçadas que podem ocorrer nas proximidades de estradas. Portanto, primeiro avaliamos se os dados resultantes de amostragens simples eram capazes de inferir conectividade funcional, em comparação com estratégias complexas, respeito aos movimentos diários e de dispersão. Segundo, avaliamos se os dados de detecção remota eram suficientemente informativos para identificar habitats-chave para uma espécie ameaçada em torno das margens das estradas. Terceiro, avaliamos a capacidade preditiva e de prioritização das unidades de mitigação de estradas que cruzam corredores multi-espécies para reduzir o risco de atropelamentos. Os resultados revelaram que os modelos simples são adequados quanto os complexos para os movimentos diários e de dispersão. Os resultados demonstraram a capacidade dos dados de detecção remota gratuitos em identificar condições de microhabitats nos habitats de berma e na paisagem circundante, para um roedor ameaçado, permitindo a delimitação de áreas de refúgio. Dados pouco exigentes (ocorrência e detecção remota) foram capazes de descrever os requisitos ecológicos específicos de aves, morcegos e mamíferos não voadores, bem como padrões de atropelamentos, possivelmente devido a corredores e habitats semelhantes, apesar de haver algumas incompatibilidades para espécies de maior mobilidade. Essa estrutura foi capaz de garantir uma elevada eficiência na prioritização de planeamento de mitigação de atropelamentos para multi-espécies, resiliente à dinâmica da paisagem de longo prazo