ARCHITECTURE ESTIMATION FROM SPARSE IMAGES USING GRAMMATICAL SHAPE PRIORS FOR CULTURAL HERITAGE

The estimation and reconstruction of 3D architectural structures is of great in- terest in computer vision, as well as cultural heritage. This dissertation proposes a novel approach to solve the di??cult problem of estimating architectural structures from sparse images and e??ciently generating 3D models from estimation results for cultural heritage. This approach takes as input one plan drawing image and a few fac¸ade images, and provides as output the volumetric 3D models which represent the structures in the sparse images. Support of this research goal has motivated new investigations in underlying structure estimation problems including detecting structural feature points in 2D images, decomposing plan drawings into semantically meaningful shapes for medieval castles, estimating rectangular and Gothic fac¸ades using shape priors, and estimating complete 3D models for architectural structures using a novel volumetric shape grammar. Major outstanding challenges in each of these topic areas are addressed resulting in contributions to current state-of-the-art as it applied to these di??cult problems

Spatially Aware Computing for Natural Interaction

Spatial information refers to the location of an object in a physical or digital world. Besides, it also includes the relative position of an object related to other objects around it. In this dissertation, three systems are designed and developed. All of them apply spatial information in different fields. The ultimate goal is to increase the user friendliness and efficiency in those applications by utilizing spatial information. The first system is a novel Web page data extraction application, which takes advantage of 2D spatial information to discover structured records from a Web page. The extracted information is useful to re-organize the layout of a Web page to fit mobile browsing. The second application utilizes the 3D spatial information of a mobile device within a large paper-based workspace to implement interactive paper that combines the merits of paper documents and mobile devices. This application can overlay digital information on top of a paper document based on the location of a mobile device within a workspace. The third application further integrates 3D space information with sound detection to realize an automatic camera management system. This application automatically controls multiple cameras in a conference room, and creates an engaging video by intelligently switching camera shots among meeting participants based on their activities. Evaluations have been made on all three applications, and the results are promising. In summary, this dissertation comprehensively explores the usage of spatial information in various applications to improve the usability

Computer-aided exploration of architectural design spaces: a digital sketchbook

Het ontwerpproces van architecten vormt vaak geen lineair pad van ontwerpopgave tot eindresultaat, maar wordt veeleer gekenmerkt door exploratie of het doorzoeken van meerdere alternatieven in een (conceptuele) ontwerpruimte. Dit proces wordt in de praktijk vaak ondersteund door manueel schetsen, waarbij de ontwerpers schetsboek kan gelezen worden als een reeks exploraties. Dit soort interactie met de ontwerpruimte wordt in veel mindere mate ondersteund door hedendaagse computerondersteunde ontwerpsystemen. De metafoor van een digitaal schetsboek, waarbij menselijke exploratie wordt versterkt door de (reken)kracht van een computer, is het centrale onderzoeksthema van dit proefschrift. Hoewel het opzet van een ontwerpruimte op het eerste gezicht schatplichtig lijkt aan het onderzoeksveld van de artificiële intelligentie (AI), wordt het ontwerpen hier ruimer geïnterpreteerd dan het oplossen van problemen. Als onderzoeksmethodologie worden vormengrammatica’s ingezet, die enerzijds nauw aanleunen bij de AI en een formeel raamwerk bieden voor de exploratie van ontwerpruimtes, maar tegelijkertijd ook weerstand bieden tegen de AI en een vorm van visueel denken en ambiguïteit toelaten. De twee bijhorende onderzoeksvragen zijn hoe deze vormengrammatica’s digitaal kunnen worden gerepresenteerd, en op welke manier de ontwerper-computer interactie kan gebeuren. De resultaten van deze twee onderzoeksvragen vormen de basis van een nieuw hulpmiddel voor architecten: het digitaal schetsboek

Automatic Reconstruction of Parametric, Volumetric Building Models from 3D Point Clouds

Planning, construction, modification, and analysis of buildings requires means of representing a building's physical structure and related semantics in a meaningful way. With the rise of novel technologies and increasing requirements in the architecture, engineering and construction (AEC) domain, two general concepts for representing buildings have gained particular attention in recent years. First, the concept of Building Information Modeling (BIM) is increasingly used as a modern means for representing and managing a building's as-planned state digitally, including not only a geometric model but also various additional semantic properties. Second, point cloud measurements are now widely used for capturing a building's as-built condition by means of laser scanning techniques. A particular challenge and topic of current research are methods for combining the strengths of both point cloud measurements and Building Information Modeling concepts to quickly obtain accurate building models from measured data. In this thesis, we present our recent approaches to tackle the intermeshed challenges of automated indoor point cloud interpretation using targeted segmentation methods, and the automatic reconstruction of high-level, parametric and volumetric building models as the basis for further usage in BIM scenarios. In contrast to most reconstruction methods available at the time, we fundamentally base our approaches on BIM principles and standards, and overcome critical limitations of previous approaches in order to reconstruct globally plausible, volumetric, and parametric models.Automatische Rekonstruktion von parametrischen, volumetrischen Gebäudemodellen aus 3D Punktwolken Für die Planung, Konstruktion, Modifikation und Analyse von Gebäuden werden Möglichkeiten zur sinnvollen Repräsentation der physischen Gebäudestruktur sowie dazugehöriger Semantik benötigt. Mit dem Aufkommen neuer Technologien und steigenden Anforderungen im Bereich von Architecture, Engineering and Construction (AEC) haben zwei Konzepte für die Repräsentation von Gebäuden in den letzten Jahren besondere Aufmerksamkeit erlangt. Erstens wird das Konzept des Building Information Modeling (BIM) zunehmend als ein modernes Mittel zur digitalen Abbildung und Verwaltung "As-Planned"-Zustands von Gebäuden verwendet, welches nicht nur ein geometrisches Modell sondern auch verschiedene zusätzliche semantische Eigenschaften beinhaltet. Zweitens werden Punktwolkenmessungen inzwischen häufig zur Aufnahme des "As-Built"-Zustands mittels Laser-Scan-Techniken eingesetzt. Eine besondere Herausforderung und Thema aktueller Forschung ist die Entwicklung von Methoden zur Vereinigung der Stärken von Punktwolken und Konzepten des Building Information Modeling um schnell akkurate Gebäudemodelle aus den gemessenen Daten zu erzeugen. In dieser Dissertation präsentieren wir unsere aktuellen Ansätze um die miteinander verwobenen Herausforderungen anzugehen, Punktwolken mithilfe geeigneter Segmentierungsmethoden automatisiert zu interpretieren, sowie hochwertige, parametrische und volumetrische Gebäudemodelle als Basis für die Verwendung im BIM-Umfeld zu rekonstruieren. Im Gegensatz zu den meisten derzeit verfügbaren Rekonstruktionsverfahren basieren unsere Ansätze grundlegend auf Prinzipien und Standards aus dem BIM-Umfeld und überwinden kritische Einschränkungen bisheriger Ansätze um vollständig plausible, volumetrische und parametrische Modelle zu erzeugen.</p

3D Reconstruction of Indoor Corridor Models Using Single Imagery and Video Sequences

In recent years, 3D indoor modeling has gained more attention due to its role in decision-making process of maintaining the status and managing the security of building indoor spaces. In this thesis, the problem of continuous indoor corridor space modeling has been tackled through two approaches. The first approach develops a modeling method based on middle-level perceptual organization. The second approach develops a visual Simultaneous Localisation and Mapping (SLAM) system with model-based loop closure. In the first approach, the image space was searched for a corridor layout that can be converted into a geometrically accurate 3D model. Manhattan rule assumption was adopted, and indoor corridor layout hypotheses were generated through a random rule-based intersection of image physical line segments and virtual rays of orthogonal vanishing points. Volumetric reasoning, correspondences to physical edges, orientation map and geometric context of an image are all considered for scoring layout hypotheses. This approach provides physically plausible solutions while facing objects or occlusions in a corridor scene. In the second approach, Layout SLAM is introduced. Layout SLAM performs camera localization while maps layout corners and normal point features in 3D space. Here, a new feature matching cost function was proposed considering both local and global context information. In addition, a rotation compensation variable makes Layout SLAM robust against cameras orientation errors accumulations. Moreover, layout model matching of keyframes insures accurate loop closures that prevent miss-association of newly visited landmarks to previously visited scene parts. The comparison of generated single image-based 3D models to ground truth models showed that average ratio differences in widths, heights and lengths were 1.8%, 3.7% and 19.2% respectively. Moreover, Layout SLAM performed with the maximum absolute trajectory error of 2.4m in position and 8.2 degree in orientation for approximately 318m path on RAWSEEDS data set. Loop closing was strongly performed for Layout SLAM and provided 3D indoor corridor layouts with less than 1.05m displacement errors in length and less than 20cm in width and height for approximately 315m path on York University data set. The proposed methods can successfully generate 3D indoor corridor models compared to their major counterpart

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

Building structural characterization using mobile terrestrial point cloud for flood risk anticipation

Compte tenu de la fréquence élevée et de l'impact majeur des inondations, les décideurs, les acteurs des municipalités et le ministère de la sécurité publique ont un besoin urgent de disposer d'outils permettant de prédire ou d'évaluer l'importance des inondations et leur impact sur la population. D'après les statistiques, le premier étage des bâtiments, ainsi que les ouvertures inférieures, sont plus susceptibles de subir des dommages lors d'une inondation. Ainsi, dans le cadre de l'évaluation de l'impact des inondations, il serait nécessaire d'identifier l'emplacement de l'ouverture la plus basse des bâtiments et surtout sa hauteur par rapport au sol. Le système de balayage laser mobile (MLS) monté sur un véhicule s'est avéré être l'une des sources les plus fiables pour caractériser les bâtiments. Il peut produire des millions de points géoréférencés en 3D avec un niveau de détail suffisant, grâce à son point de vue depuis la rue et sa proximité. De plus, l'augmentation du nombre de jeux de données, issues des MLS acquis dans les villes et les environnements ruraux, permet de développer des approches pour caractériser les maisons résidentielles à l'échelle provinciale. Plusieurs défis sont associés à l'extraction d'informations descriptives des façades de bâtiments à l'aide de données MLS. Ainsi, les occlusions devant une façade rendent impossible l'obtention de points 3D sur ces parties de la façade. Aussi, comme les fenêtres sont principalement constituées de verre, qui ne réfléchit pas les signaux laser, les points disponibles pour celles-ci sont généralement limités. De plus, les approches de détection exploitent la répétitivité et les positions symétriques des ouvertures sur la façade. Mais ces caractéristiques sont absentes pour des maisons rurales et résidentielles. Finalement, la variabilité de la densité de points dans les données MLS rend difficile le processus de détection lorsqu'on travaille à l'échelle d'une ville. Par conséquent, l'objectif principal de cette recherche est de concevoir et de développer une approche globale d'extraction efficace des ouvertures présentes sur une façade. La solution proposée se compose de trois phases: l'extraction des façades, la détection des ouvertures et l'identification des occlusions. La première phase utilise une approche de segmentation adaptative par croissance de régions pour extraire la boîte englobante 3D de la façade. La deuxième phase combine la détection de trous avec une technique de maillage pour extraire les boîtes englobantes 2D des ouvertures. La dernière phase, qui vise à discriminer les occlusions des ouvertures, est en cours d'achèvement. Des évaluations qualitatives et quantitatives ont été réalisées à l'aide d'un jeu de données réelles, fourni par Jakarto Cartographie 3D Inc., de la province de Québec, au Canada. Les statistiques ont révélé que l'approche proposée pouvait obtenir de bons taux de performance malgré la complexité du jeu de données, représentatif des données acquises en situation réelle. Les défis concernant l'auto-occlusion de certaines façades et la présence de grandes occlusions environnantes seront à étudier plus en profondeur afin d'obtenir des informations plus précises sur les ouvertures des façades.Given the high frequency and major impact of floods, decision-makers, stakeholders in municipalities and public security ministry are in the urgent need to have tools allowing to predict or assess the significance of flood events and their impact on the population. Based on statistics, the first floor of the buildings, as well as the lower openings, are more likely subject to potential damage during a flood event. Thus, in the context of flood impact assessment, it would be required identifying the location of the buildings' lowest opening and especially its height above the ground. The capacity to characterize building with a relevant level of detail depends on the data sources used for the modeling. Different sources of data have been employed to characterize buildings' façade and openings. Mobile Laser Scanning (MLS) system mounted on a vehicle has proved to be one of the most reliable sources in this domain. It can produce millions of 3D georeferenced points with sufficient level of detail of the building facades and its openings, due to its street-view and close-range distance. Moreover, the increase of MLS providers and acquisitions in towns and rural environments, makes it possible to develop approaches to characterize residential houses at a provincial scale. Although being effective, several challenges are associated with extracting descriptive information of building facades using MLS data. The presence of occlusion in front of a facade makes it impossible to obtain the 3D points of the covered parts of the facade. Given the fact that windows mostly consist of glass and laser signals could not be reflected from the glass, limited points are usually available for windows. While the repetitive pattern and symmetrical positions of the openings on the facade makes it easier for the detection system to extract them, this characteristic is missing on the facade on rural and residential houses. The inconsistency of the point density in MLS data make the detection process even harder when working at city scale. Accordingly, the main objective of this research is to design and develop a comprehensive approach that effectively extracts facade openings. In order to meet the research project objective, the proposed solution consists of three phases including facade extraction, opening detection, and occlusion recognition. The first phase employs an adaptive region growing segmentation approach to extract the 3D bounding box of the facade. The second phase combines a hole-based assumption with an XZ gridding technique to extract 2D bounding boxes of the openings. The last phase which recognizes holes related to the occlusion from the openings is currently being completed. Qualitative and quantitative evaluations were performed using a real-word dataset provided by Jakarto Cartographie 3D inc. of the Quebec Province, Canada. Statistics revealed that the proposed approach could obtain good performance rates despite the complexity of the dataset, representative of the data acquired in real situations. Challenges regarding facade's self-occlusion and the presence of large surrounding occlusions should be further investigated for obtaining more accurate opening information on the facade