A pattern language for parallelizing irregular algorithms

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia InformáticaIn irregular algorithms, data set’s dependences and distributions cannot be statically predicted. This class of algorithms tends to organize computations in terms of data locality instead of parallelizing control in multiple threads. Thus, opportunities for exploiting parallelism vary dynamically, according to how the algorithm changes data dependences. As such, effective parallelization of such algorithms requires new approaches that account for that dynamic nature. This dissertation addresses the problem of building efficient parallel implementations of irregular algorithms by proposing to extract, analyze and document patterns of concurrency and parallelism present in the Galois parallelization framework for irregular algorithms. Patterns capture formal representations of a tangible solution to a problem that arises in a well defined context within a specific domain. We document the said patterns in a pattern language, i.e., a set of inter-dependent patterns that compose well-documented template solutions that can be reused whenever a certain problem arises in a well-known context

COLTRANE: ConvolutiOnaL TRAjectory NEtwork for Deep Map Inference

The process of automatic generation of a road map from GPS trajectories, called map inference, remains a challenging task to perform on a geospatial data from a variety of domains as the majority of existing studies focus on road maps in cities. Inherently, existing algorithms are not guaranteed to work on unusual geospatial sites, such as an airport tarmac, pedestrianized paths and shortcuts, or animal migration routes, etc. Moreover, deep learning has not been explored well enough for such tasks. This paper introduces COLTRANE, ConvolutiOnaL TRAjectory NEtwork, a novel deep map inference framework which operates on GPS trajectories collected in various environments. This framework includes an Iterated Trajectory Mean Shift (ITMS) module to localize road centerlines, which copes with noisy GPS data points. Convolutional Neural Network trained on our novel trajectory descriptor is then introduced into our framework to detect and accurately classify junctions for refinement of the road maps. COLTRANE yields up to 37% improvement in F1 scores over existing methods on two distinct real-world datasets: city roads and airport tarmac.Comment: BuildSys 201

A GROWTH-BASED APPROACH TO THE AUTOMATIC GENERATION OF NAVIGATION MESHES

Providing an understanding of space in game and simulation environments is one of the major challenges associated with moving artificially intelligent characters through these environments. The usage of some form of navigation mesh has become the standard method to provide a representation of the walkable space in game environments to characters moving around in that environment. There is currently no standardized best method of producing a navigation mesh. In fact, producing an optimal navigation mesh has been shown to be an NP-Hard problem. Current approaches are a patchwork of divergent methods all of which have issues either in the time to create the navigation meshes (e.g., the best looking navigation meshes have traditionally been produced by hand which is time consuming), generate substandard quality navigation meshes (e.g., many of the automatic mesh production algorithms result in highly triangulated meshes that pose problems for character navigation), or yield meshes that contain gaps of areas that should be included in the mesh and are not (e.g., existing growth-based methods are unable to adapt to non-axis-aligned geometry and as such tend to provide a poor representation of the walkable space in complex environments). We introduce the Planar Adaptive Space Filling Volumes (PASFV) algorithm, Volumetric Adaptive Space Filling Volumes (VASFV) algorithm, and the Iterative Wavefront Edge Expansion Cell Decomposition (Wavefront) algorithm. These algorithms provide growth-based spatial decompositions for navigation mesh generation in either 2D (PASFV) or 3D (VASFV). These algorithms generate quick (on demand) decompositions (Wavefront), use quad/cube base spatial structures to provide more regular regions in the navigation mesh instead of triangles, and offer full coverage decompositions to avoid gaps in the navigation mesh by adapting to non-axis-aligned geometry. We have shown experimentally that the decompositions offered by PASFV and VASFV are superior both in character navigation ability, number of regions, and coverage in comparison to the existing and commonly used techniques of Space Filling Volumes, Hertel-Melhorn decomposition, Delaunay Triangulation, and Automatic Path Node Generation. Finally, we show that our Wavefront algorithm retains the superior performance of the PASFV and VASFV algorithms while providing faster decompositions that contain fewer degenerate and near degenerate regions. Unlike traditional navigation mesh generation techniques, the PASFV and VASFV algorithms have a real time extension (Dynamic Adaptive Space Filling Volumes, DASFV) which allows the navigation mesh to adapt to changes in the geometry of the environment at runtime. In addition, it is possible to use a navigation mesh for applications above and beyond character path planning and navigation. These multiple uses help to increase the return on the investment in creating a navigation mesh for a game or simulation environment. In particular, we will show how to use a navigation mesh for the acceleration of collision detection

ICASE/LaRC Workshop on Adaptive Grid Methods

Solution-adaptive grid techniques are essential to the attainment of practical, user friendly, computational fluid dynamics (CFD) applications. In this three-day workshop, experts gathered together to describe state-of-the-art methods in solution-adaptive grid refinement, analysis, and implementation; to assess the current practice; and to discuss future needs and directions for research. This was accomplished through a series of invited and contributed papers. The workshop focused on a set of two-dimensional test cases designed by the organizers to aid in assessing the current state of development of adaptive grid technology. In addition, a panel of experts from universities, industry, and government research laboratories discussed their views of needs and future directions in this field

Man-made Surface Structures from Triangulated Point Clouds

Photogrammetry aims at reconstructing shape and dimensions of objects captured with cameras, 3D laser scanners or other spatial acquisition systems. While many acquisition techniques deliver triangulated point clouds with millions of vertices within seconds, the interpretation is usually left to the user. Especially when reconstructing man-made objects, one is interested in the underlying surface structure, which is not inherently present in the data. This includes the geometric shape of the object, e.g. cubical or cylindrical, as well as corresponding surface parameters, e.g. width, height and radius. Applications are manifold and range from industrial production control to architectural on-site measurements to large-scale city models. The goal of this thesis is to automatically derive such surface structures from triangulated 3D point clouds of man-made objects. They are defined as a compound of planar or curved geometric primitives. Model knowledge about typical primitives and relations between adjacent pairs of them should affect the reconstruction positively. After formulating a parametrized model for man-made surface structures, we develop a reconstruction framework with three processing steps: During a fast pre-segmentation exploiting local surface properties we divide the given surface mesh into planar regions. Making use of a model selection scheme based on minimizing the description length, this surface segmentation is free of control parameters and automatically yields an optimal number of segments. A subsequent refinement introduces a set of planar or curved geometric primitives and hierarchically merges adjacent regions based on their joint description length. A global classification and constraint parameter estimation combines the data-driven segmentation with high-level model knowledge. Therefore, we represent the surface structure with a graphical model and formulate factors based on likelihood as well as prior knowledge about parameter distributions and class probabilities. We infer the most probable setting of surface and relation classes with belief propagation and estimate an optimal surface parametrization with constraints induced by inter-regional relations. The process is specifically designed to work on noisy data with outliers and a few exceptional freeform regions not describable with geometric primitives. It yields full 3D surface structures with watertightly connected surface primitives of different types. The performance of the proposed framework is experimentally evaluated on various data sets. On small synthetically generated meshes we analyze the accuracy of the estimated surface parameters, the sensitivity w.r.t. various properties of the input data and w.r.t. model assumptions as well as the computational complexity. Additionally we demonstrate the flexibility w.r.t. different acquisition techniques on real data sets. The proposed method turns out to be accurate, reasonably fast and little sensitive to defects in the data or imprecise model assumptions.Künstliche Oberflächenstrukturen aus triangulierten Punktwolken Ein Ziel der Photogrammetrie ist die Rekonstruktion der Form und Größe von Objekten, die mit Kameras, 3D-Laserscannern und anderern räumlichen Erfassungssystemen aufgenommen wurden. Während viele Aufnahmetechniken innerhalb von Sekunden triangulierte Punktwolken mit Millionen von Punkten liefern, ist deren Interpretation gewöhnlicherweise dem Nutzer überlassen. Besonders bei der Rekonstruktion künstlicher Objekte (i.S.v. engl. man-made = „von Menschenhand gemacht“ ist man an der zugrunde liegenden Oberflächenstruktur interessiert, welche nicht inhärent in den Daten enthalten ist. Diese umfasst die geometrische Form des Objekts, z.B. quaderförmig oder zylindrisch, als auch die zugehörigen Oberflächenparameter, z.B. Breite, Höhe oder Radius. Die Anwendungen sind vielfältig und reichen von industriellen Fertigungskontrollen über architektonische Raumaufmaße bis hin zu großmaßstäbigen Stadtmodellen. Das Ziel dieser Arbeit ist es, solche Oberflächenstrukturen automatisch aus triangulierten Punktwolken von künstlichen Objekten abzuleiten. Sie sind definiert als ein Verbund ebener und gekrümmter geometrischer Primitive. Modellwissen über typische Primitive und Relationen zwischen Paaren von ihnen soll die Rekonstruktion positiv beeinflussen. Nachdem wir ein parametrisiertes Modell für künstliche Oberflächenstrukturen formuliert haben, entwickeln wir ein Rekonstruktionsverfahren mit drei Verarbeitungsschritten: Im Rahmen einer schnellen Vorsegmentierung, die lokale Oberflächeneigenschaften berücksichtigt, teilen wir die gegebene vermaschte Oberfläche in ebene Regionen. Unter Verwendung eines Schemas zur Modellauswahl, das auf der Minimierung der Beschreibungslänge beruht, ist diese Oberflächensegmentierung unabhängig von Kontrollparametern und liefert automatisch eine optimale Anzahl an Regionen. Eine anschließende Verbesserung führt eine Menge von ebenen und gekrümmten geometrischen Primitiven ein und fusioniert benachbarte Regionen hierarchisch basierend auf ihrer gemeinsamen Beschreibungslänge. Eine globale Klassifikation und bedingte Parameterschätzung verbindet die datengetriebene Segmentierung mit hochrangigem Modellwissen. Dazu stellen wir die Oberflächenstruktur in Form eines graphischen Modells dar und formulieren Faktoren basierend auf der Likelihood sowie auf apriori Wissen über die Parameterverteilungen und Klassenwahrscheinlichkeiten. Wir leiten die wahrscheinlichste Konfiguration von Flächen- und Relationsklassen mit Hilfe von Belief-Propagation ab und schätzen eine optimale Oberflächenparametrisierung mit Bedingungen, die durch die Relationen zwischen benachbarten Primitiven induziert werden. Der Prozess ist eigens für verrauschte Daten mit Ausreißern und wenigen Ausnahmeregionen konzipiert, die nicht durch geometrische Primitive beschreibbar sind. Er liefert wasserdichte 3D-Oberflächenstrukturen mit Oberflächenprimitiven verschiedener Art. Die Leistungsfähigkeit des vorgestellten Verfahrens wird an verschiedenen Datensätzen experimentell evaluiert. Auf kleinen, synthetisch generierten Oberflächen untersuchen wir die Genauigkeit der geschätzten Oberflächenparameter, die Sensitivität bzgl. verschiedener Eigenschaften der Eingangsdaten und bzgl. Modellannahmen sowie die Rechenkomplexität. Außerdem demonstrieren wir die Flexibilität bzgl. verschiedener Aufnahmetechniken anhand realer Datensätze. Das vorgestellte Rekonstruktionsverfahren erweist sich als genau, hinreichend schnell und wenig anfällig für Defekte in den Daten oder falsche Modellannahmen

Filaments in observed and mock galaxy catalogues

Context. The main feature of the spatial large-scale galaxy distribution is an intricate network of galaxy filaments. Although many attempts have been made to quantify this network, there is no unique and satisfactory recipe for that yet. Aims. The present paper compares the filaments in the real data and in the numerical models, to see if our best models reproduce statistically the filamentary network of galaxies. Methods. We apply an object point process with interactions (the Bisous process) to trace and describe the filamentary network both in the observed samples (the 2dFGRS catalogue) and in the numerical models that have been prepared to mimic the data.We compare the networks. Results. We find that the properties of filaments in numerical models (mock samples) have a large variance. A few mock samples display filaments that resemble the observed filaments, but usually the model filaments are much shorter and do not form an extended network. Conclusions. We conclude that although we can build numerical models that are similar to observations in many respects, they may fail yet to explain the filamentary structure seen in the data. The Bisous-built filaments are a good test for such a structure.Comment: 13 pages, accepted for publication in Astronomy and Astrophysic

Simulations of cosmic reionization

In this thesis we investigate numerically how ionizing photons emitted by stars in galaxies cause the reionization of the Universe, the transition during which most of the gas in the Universe from a mostly neutral, to a highly ionised state it is in today. To this end, we discuss and improve two techniques for the transport of ionising radiation across cosmological volumes, analyse the sources of ionising photons at high redshifts predicted by a semi-analytical galaxy formation model (GALFORM), and combine these to make consistent model of how reionization proceeds. Our improvements to the hybrid characteristics (HC) radiative transport scheme are significant, making the code faster and more accurate, as demonstrated by our contribution to a code comparison paper (Iliev et al., 2009). Our improvements to the SimpleX radiative transport scheme allow for accurate and significantly better numerically converged calculations of the speeds of ionization fronts of cosmological HII regions. This is accomplished by a much more thorough analysis of how to properly model the density field on the unstructured density field in SimpleX. The dependence of the ionizing emissivity of GALFORM galaxies on various parameters of the model is examined. We show that massive stars formed in abundance because of the assumed top-heavy stellar initial mass function during starbursts in the Baugh et al. (2005) model, triggered by galaxy mergers, are the dominant source of ionizing photons. We show that the luminosity functions predicted by this model are in good agreement with the most recent Hubble Space Telescope results at z \gtrsim 8. The model also demonstrates that most photons are produced in faint galaxies which are not yet seen in the current data. We then combine the sources predicted by GALFORM with the SimpleX RT scheme to model inhomogeneous reionization including the effects of source suppression. We investigate how the morphology of reionization depends on the model for the sources, which may be crucial for future observations of this cosmic epoch

Extraction of routing relevant geodata using telemetry sensor data of agricultural vehicles

The mechanization of processes in agriculture is growing within the last hundred years. Within the last decades, the information technology in this sector constantly grew and data is one of the key factors for process optimization. For planning and execution of logistic processes and harvest campaigns, road maps and field geometries are essential to guide the large vehicles to their work places and to optimize harvest chains. Through nowadays available telemetry data of these vehicles, a new data source generates new opportunities. Mining geographic data from these movement data, that can improve agricultural work processes is one of the main objectives of this thesis. As a first step, data cleaning processes, and further preprocessing steps are shown. With classification algorithms, the continuous movement data will be separated into different work processes. Based on this data, algorithms to generate geographic geographic features, such as field boundaries have been analyzed and improved. As quality metric to compare the results, the Jaccard-Distance has been established. With the classified road representing measurements, the rural road networks were created and the results of different algorithmic approaches have been compared. The usability of volunteered geographic information to route the heterogeneous set of agricultural vehicles is shown in a third step. Due to the fact, that routes for e.g. harvesters are not ending at the field boundary, solutions for infield route graph generation have been given. The presented components provide the content and the services within a framework structure. The concluding prototype, a web based routing system demonstrates the interaction of all components and provides a consecutive routing from farm to field and within the field