Realistic reconstruction and rendering of detailed 3D scenarios from multiple data sources

During the last years, we have witnessed significant improvements in digital terrain modeling, mainly through photogrammetric techniques based on satellite and aerial photography, as well as laser scanning. These techniques allow the creation of Digital Elevation Models (DEM) and Digital Surface Models (DSM) that can be streamed over the network and explored through virtual globe applications like Google Earth or NASA WorldWind. The resolution of these 3D scenes has improved noticeably in the last years, reaching in some urban areas resolutions up to 1m or less for DEM and buildings, and less than 10 cm per pixel in the associated aerial imagery. However, in rural, forest or mountainous areas, the typical resolution for elevation datasets ranges between 5 and 30 meters, and typical resolution of corresponding aerial photographs ranges between 25 cm to 1 m. This current level of detail is only sufficient for aerial points of view, but as the viewpoint approaches the surface the terrain loses its realistic appearance. One approach to augment the detail on top of currently available datasets is adding synthetic details in a plausible manner, i.e. including elements that match the features perceived in the aerial view. By combining the real dataset with the instancing of models on the terrain and other procedural detail techniques, the effective resolution can potentially become arbitrary. There are several applications that do not need an exact reproduction of the real elements but would greatly benefit from plausibly enhanced terrain models: videogames and entertainment applications, visual impact assessment (e.g. how a new ski resort would look), virtual tourism, simulations, etc. In this thesis we propose new methods and tools to help the reconstruction and synthesis of high-resolution terrain scenes from currently available data sources, in order to achieve realistically looking ground-level views. In particular, we decided to focus on rural scenarios, mountains and forest areas. Our main goal is the combination of plausible synthetic elements and procedural detail with publicly available real data to create detailed 3D scenes from existing locations. Our research has focused on the following contributions: - An efficient pipeline for aerial imagery segmentation - Plausible terrain enhancement from high-resolution examples - Super-resolution of DEM by transferring details from the aerial photograph - Synthesis of arbitrary tree picture variations from a reduced set of photographs - Reconstruction of 3D tree models from a single image - A compact and efficient tree representation for real-time rendering of forest landscapesDurant els darrers anys, hem presenciat avenços significatius en el modelat digital de terrenys, principalment gràcies a tècniques fotogramètriques, basades en fotografia aèria o satèl·lit, i a escàners làser. Aquestes tècniques permeten crear Models Digitals d'Elevacions (DEM) i Models Digitals de Superfícies (DSM) que es poden retransmetre per la xarxa i ser explorats mitjançant aplicacions de globus virtuals com ara Google Earth o NASA WorldWind. La resolució d'aquestes escenes 3D ha millorat considerablement durant els darrers anys, arribant a algunes àrees urbanes a resolucions d'un metre o menys per al DEM i edificis, i fins a menys de 10 cm per píxel a les fotografies aèries associades. No obstant, en entorns rurals, boscos i zones muntanyoses, la resolució típica per a dades d'elevació es troba entre 5 i 30 metres, i per a les corresponents fotografies aèries varia entre 25 cm i 1m. Aquest nivell de detall només és suficient per a punts de vista aeris, però a mesura que ens apropem a la superfície el terreny perd tot el realisme. Una manera d'augmentar el detall dels conjunts de dades actuals és afegint a l'escena detalls sintètics de manera plausible, és a dir, incloure elements que encaixin amb les característiques que es perceben a la vista aèria. Així, combinant les dades reals amb instàncies de models sobre el terreny i altres tècniques de detall procedural, la resolució efectiva del model pot arribar a ser arbitrària. Hi ha diverses aplicacions per a les quals no cal una reproducció exacta dels elements reals, però que es beneficiarien de models de terreny augmentats de manera plausible: videojocs i aplicacions d'entreteniment, avaluació de l'impacte visual (per exemple, com es veuria una nova estació d'esquí), turisme virtual, simulacions, etc. En aquesta tesi, proposem nous mètodes i eines per ajudar a la reconstrucció i síntesi de terrenys en alta resolució partint de conjunts de dades disponibles públicament, per tal d'aconseguir vistes a nivell de terra realistes. En particular, hem decidit centrar-nos en escenes rurals, muntanyes i àrees boscoses. El nostre principal objectiu és la combinació d'elements sintètics plausibles i detall procedural amb dades reals disponibles públicament per tal de generar escenes 3D d'ubicacions existents. La nostra recerca s'ha centrat en les següents contribucions: - Un pipeline eficient per a segmentació d'imatges aèries - Millora plausible de models de terreny a partir d'exemples d’alta resolució - Super-resolució de models d'elevacions transferint-hi detalls de la fotografia aèria - Síntesis d'un nombre arbitrari de variacions d’imatges d’arbres a partir d'un conjunt reduït de fotografies - Reconstrucció de models 3D d'arbres a partir d'una única fotografia - Una representació compacta i eficient d'arbres per a navegació en temps real d'escenesPostprint (published version

Alike Scene Retrieval from Land-Cover Products Based on the Label Co-Occurrence Matrix (LCM) †

The management and application of remotely sensed data has become much more difficult due to the dramatically growing volume of remotely sensed imagery. To address this issue, content-based image retrieval (CBIR) has been applied to remote sensing image retrieval for information mining. As a consequence of the growing volume of remotely sensed imagery, the number of different types of image-derived products (such as land use/land cover (LULC) databases) is also increasing rapidly. Nevertheless, only a few studies have addressed the exploration and information mining of these products. In this letter, for the sake of making the most use of the LULC map, we propose an approach for the retrieval of alike scenes from it. Based on the proposed approach, we design a content-based map retrieval (CBMR) system for LULC. The main contributions of our work are listed below. Firstly, the proposed system can allow the user to select a region of interest as the reference scene with variable shape and size. In contrast, in the traditional CBIR/CBMR systems, the region of interest is usually of a fixed size, which is equal to the size of the analysis window for extracting features. In addition, the user can acquire various retrieval results by specifying the corresponding parameters. Finally, by combining the signatures in the base signature library, the user can acquire the retrieval result faster

Alike Scene Retrieval from Land-Cover Products Based on the Label Co-Occurrence Matrix (LCM) †

The management and application of remotely sensed data has become much more difficult due to the dramatically growing volume of remotely sensed imagery. To address this issue, content-based image retrieval (CBIR) has been applied to remote sensing image retrieval for information mining. As a consequence of the growing volume of remotely sensed imagery, the number of different types of image-derived products (such as land use/land cover (LULC) databases) is also increasing rapidly. Nevertheless, only a few studies have addressed the exploration and information mining of these products. In this letter, for the sake of making the most use of the LULC map, we propose an approach for the retrieval of alike scenes from it. Based on the proposed approach, we design a content-based map retrieval (CBMR) system for LULC. The main contributions of our work are listed below. Firstly, the proposed system can allow the user to select a region of interest as the reference scene with variable shape and size. In contrast, in the traditional CBIR/CBMR systems, the region of interest is usually of a fixed size, which is equal to the size of the analysis window for extracting features. In addition, the user can acquire various retrieval results by specifying the corresponding parameters. Finally, by combining the signatures in the base signature library, the user can acquire the retrieval result faster

Riverine sustainment 2012

Student Integrated ProjectIncludes supplementary materialThis technical report analyzed the Navy's proposed Riverine Force (RF) structure and capabilities for 2012. The Riverine Sustainment 2012 Team (RST) examined the cost and performance of systems of systems which increased RF sustainment in logistically barren environments. RF sustainment was decomposed into its functional areas of supply, repair, and force protection. The functional and physical architectures were developed in parallel and were used to construct an operational architecture for the RF. The RST used mathematical, agent-based and queuing models to analyze various supply, repair and force protection system alternatives. Extraction of modeling data revealed several key insights. Waterborne heavy lift connectors such as the LCU-2000 are vital in the re-supply of the RF when it is operating up river in a non-permissive environment. Airborne heavy lift connectors such as the MV-22 were ineffective and dominated by the waterborne variants in the same environment. Increase in manpower and facilities did appreciable add to the operational availability of the RF. Mean supply response time was the biggest factor effecting operational availability and should be kept below 24 hours to maintain operational availability rates above 80%. Current mortar defenses proposed by the RF are insufficient.N

Analysis of land change with parameterised multi-level class sets : exploring the semantic dimension

Een veelvoorkomend probleem in de landdynamiek is dat na verloop van tijd kennis vordert, technologie ontwikkelt en beleidsdoelstellingen veranderen. Dit betekent dat met elke kartering die wordt uitgevoerd, met een voor dat doel specifiek ontworpen classificatie, een nieuwe basis dataset wordt gemaakt in plaats van een continue gegevensreeks. Verschillen in de naamgeving van klassen, veranderingen in de definitie van de klasse, en de toevoeging of verwijdering van de klassen in de datasets over hetzelfde gebied in verschillende periodes leveren problemen op in de scheiding van de feitelijke veranderingen in de tijd van klaarblijkelijke veranderingen in de definities van categorieën. In de praktijk zullen de resultaten van verschillende onderzoeken echter moeten worden geharmoniseerd in tijd en ruimte

Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics

This paper proposes a method to enhance video object detection for indoor environments in robotics. Concretely, it exploits knowledge about the camera motion between frames to propagate previously detected objects to successive frames. The proposal is rooted in the concepts of planar homography to propose regions of interest where to find objects, and recursive Bayesian filtering to integrate observations over time. The proposal is evaluated on six virtual, indoor environments, accounting for the detection of nine object classes over a total of ∼ 7k frames. Results show that our proposal improves the recall and the F1-score by a factor of 1.41 and 1.27, respectively, as well as it achieves a significant reduction of the object categorization entropy (58.8%) when compared to a two-stage video object detection method used as baseline, at the cost of small time overheads (120 ms) and precision loss (0.92).</p

Deep Neural Networks and Data for Automated Driving

This open access book brings together the latest developments from industry and research on automated driving and artificial intelligence. Environment perception for highly automated driving heavily employs deep neural networks, facing many challenges. How much data do we need for training and testing? How to use synthetic data to save labeling costs for training? How do we increase robustness and decrease memory usage? For inevitably poor conditions: How do we know that the network is uncertain about its decisions? Can we understand a bit more about what actually happens inside neural networks? This leads to a very practical problem particularly for DNNs employed in automated driving: What are useful validation techniques and how about safety? This book unites the views from both academia and industry, where computer vision and machine learning meet environment perception for highly automated driving. Naturally, aspects of data, robustness, uncertainty quantification, and, last but not least, safety are at the core of it. This book is unique: In its first part, an extended survey of all the relevant aspects is provided. The second part contains the detailed technical elaboration of the various questions mentioned above