5 research outputs found
Adaptive Geospatial Joins for Modern Hardware
Geospatial joins are a core building block of connected mobility
applications. An especially challenging problem are joins between streaming
points and static polygons. Since points are not known beforehand, they cannot
be indexed. Nevertheless, points need to be mapped to polygons with low
latencies to enable real-time feedback.
We present an adaptive geospatial join that uses true hit filtering to avoid
expensive geometric computations in most cases. Our technique uses a
quadtree-based hierarchical grid to approximate polygons and stores these
approximations in a specialized radix tree. We emphasize on an approximate
version of our algorithm that guarantees a user-defined precision. The exact
version of our algorithm can adapt to the expected point distribution by
refining the index. We optimized our implementation for modern hardware
architectures with wide SIMD vector processing units, including Intel's brand
new Knights Landing. Overall, our approach can perform up to two orders of
magnitude faster than existing techniques
Adaptive geospatial joins for modern hardware
Geospatial joins are a core building block of connected
mobility applications. An especially challenging problem
are joins between streaming points and static polygons. Since
points are not known beforehand, they cannot be indexed.
Nevertheless, points need to be mapped to polygons with low
latencies to enable real-time feedback.
We present an adaptive geospatial join that uses true hit
filtering to avoid expensive geometric computations in most
cases. Our technique uses a quadtree-based hierarchical grid
to approximate polygons and stores these approximations in a
specialized radix tree. We emphasize on an approximate version
of our algorithm that guarantees a user-defined precision. The
exact version of our algorithm can adapt to the expected point
distribution by refining the index. We optimized our implementation
for modern hardware architectures with wide SIMD vector
processing units, including Intelâs brand new Knights Landing.
Overall, our approach can perform up to two orders of magnitude
faster than existing techniques
Adaptive main-memory indexing for high-performance point-polygon joins
Connected mobility applications rely heavily on geospatial joins that associate point data, such as locations of Uber cars, to static polygonal regions, such as city neighborhoods. These joins typically involve expensive geometric computations, which makes it hard to provide an interactive user experience. In this paper, we propose an adaptive polygon index that leverages true hit fltering to avoid expensive geometric computations in most cases. In particular, our approach closely approximates polygons by combining quadtrees with true hit filtering, and stores these approximations in a query-effcient radix tree. Based on this index, we introduce two geospatial join algorithms: an approximate one that guarantees a user-defined precision, and an exact one that adapts to the expected point distribution. In summary, our technique outperforms existing CPU-based joins by up to two orders of magnitude and is competitive with state-of-the-art GPU implementations
Weiterentwicklung analytischer Datenbanksysteme
This thesis contributes to the state of the art in analytical database systems. First, we identify and explore extensions to better support analytics on event streams. Second, we propose a novel polygon index to enable efficient geospatial data processing in main memory. Third, we contribute a new deep learning approach to cardinality estimation, which is the core problem in cost-based query optimization.Diese Arbeit trĂ€gt zum aktuellen Forschungsstand von analytischen Datenbanksystemen bei. Wir identifizieren und explorieren Erweiterungen um Analysen auf Eventströmen besser zu unterstĂŒtzen. Wir stellen eine neue Indexstruktur fĂŒr Polygone vor, die eine effiziente Verarbeitung von Geodaten im Hauptspeicher ermöglicht. Zudem prĂ€sentieren wir einen neuen Ansatz fĂŒr KardinalitĂ€tsschĂ€tzungen mittels maschinellen Lernens
Generalized database index structures on massively parallel processor architectures
Height-balanced search trees are ubiquitous in database management systems as well as in other applications that require efficient access methods in order to identify entries in large data volumes. They can be configured with various strategies for structuring the search space for a given data set and for pruning it when different kinds of search queries are answered. In order to facilitate the development of application-specific tree variants, index frameworks, such as GiST, exist that provide a reusable library of commonly shared tree management functionality. By specializing internal data organization strategies, the framework can be customized to create an index that is efficient for an application's data access characteristics. Because the majority of the framework's code can be reused development and testing efforts are significantly lower, compared to an implementation from scratch. However, none of the existing frameworks supports the execution of index operations on massively parallel processor architectures, such as GPUs. Enabling the use of such processors for generalized index frameworks is the goal of this thesis. By compiling state-of-the-art techniques from a wide range of CPU- and GPU-optimized indexes, a GiST extension is developed that abstracts the physical execution aspect of generic, tree-based search queries. Tree traversals are broken-down into vectorized processing primitives that can be scheduled to one of the available (co-)processors for execution. Further, a CPU-based implementation is provided as well as a new GPU-based algorithm that, unlike prior art in this area, does not require that the index is fully stored inside a GPU's main memory buffer. The applicability of the extended framework is assessed for image rendering engines and, based on microbenchmarks, the parallelized algorithm performance is compared for different CPU and GPU generations. It will be shown that cases exist, where the GPU clearly outperforms the CPU and vice versa. In order to leverage the strengths of each processor type, an adaptive scheduler is presented that can be calibrated to schedule index operations to the best-fitting device in a hybrid system. With the help of a tree traversal simulation different scheduling strategies are evaluated and it will be shown that the adaptive scheduler can be used to make near-optimal decisions.SuchbĂ€ume sind allgegenwĂ€rtig in Datenbanksystemen und anderen Anwendungen, die eine effiziente Möglichkeit benötigen um in groĂen DatensĂ€tzen nach EintrĂ€gen zu suchen, die bestimmte Suchkriterien erfĂŒllen. Sie können mit verschiedenen Strategien konfiguriert werden um den Suchraum zu strukturieren und die fĂŒr ein Suchergebnis irrelevante Bereiche von der Bearbeitung auszuschlieĂen. Die Entwicklung von anwendungsspezifischen Indexen wird durch Frameworks wie GiST unterstĂŒtzt. Jedoch unterstĂŒtzt keines der heute bereits existierenden Frameworks die Verwendung von hochgradig parallelen Prozessorarchitekturen wie GPUs. Solche Prozessoren fĂŒr generische Index Frameworks nutzbar zu machen, ist Ziel dieser Arbeit. Dazu werden Techniken aus verschiedensten CPU- und GPU-optimierten Indexen analysiert und fĂŒr die Entwicklung einer GiST-Erweiterung verwendet, welche die fĂŒr eine Suche in SuchbĂ€umen nötigen Berechnungen abstrahiert. Traversierungsoperationen werden dabei auf vektorisierte Primitive abgebildet, die auf parallelen Prozessoren implementiert werden können. Die Verwendung dieser Erweiterung wird beispielhaft an einem CPU Algorithmus demonstriert. Weiterhin wird ein neuer GPU-basierter Algorithmus vorgestellt, der im Vergleich zu bisherigen Verfahren, ein dynamisches Nachladen der Index Daten in den Hauptspeicher der GPU unterstĂŒtzt. Die PraktikabilitĂ€t des erweiterten Frameworks wird am Beispiel von Anwendungen aus der Computergrafik untersucht und die Performanz der verwendeten Algorithmen mit Hilfe eines Benchmarks auf verschiedenen CPU- und GPU-Modellen analysiert. Dabei wird gezeigt, unter welchen Bedingungen die parallele GPU-basierte AusfĂŒhrung schneller ist als die CPU-basierte Variante - und umgekehrt. Um die StĂ€rken beider Prozessortypen in einem hybriden System ausnutzen zu können, wird ein Scheduler entwickelt, der nach einer Kalibrierungsphase fĂŒr eine gegebene Operation den geeignetsten Prozessor wĂ€hlen kann. Mit Hilfe eines Simulators fĂŒr Baumtraversierungen werden verschiedenste Scheduling Strategien verglichen. Dabei wird gezeigt, dass die Entscheidungen des Schedulers kaum vom Optimum abweichen und, abhĂ€ngig von der simulierten Last, die erzielbaren DurchsĂ€tze fĂŒr die parallele AusfĂŒhrung mehrerer Suchoperationen durch hybrides Scheduling um eine GröĂenordnung und mehr erhöht werden können