Shape Retrieval Methods for Architectural 3D Models

This thesis introduces new methods for content-based retrieval of architecture-related 3D models. We thereby consider two different overall types of architectural 3D models. The first type consists of context objects that are used for detailed design and decoration of 3D building model drafts. This includes e.g. furnishing for interior design or barriers and fences for forming the exterior environment. The second type consists of actual building models. To enable efficient content-based retrieval for both model types that is tailored to the user requirements of the architectural domain, type-specific algorithms must be developed. On the one hand, context objects like furnishing that provide similar functions (e.g. seating furniture) often share a similar shape. Nevertheless they might be considered to belong to different object classes from an architectural point of view (e.g. armchair, elbow chair, swivel chair). The differentiation is due to small geometric details and is sometimes only obvious to an expert from the domain. Building models on the other hand are often distinguished according to the underlying floor- and room plans. Topological floor plan properties for example serve as a starting point for telling apart residential and commercial buildings. The first contribution of this thesis is a new meta descriptor for 3D retrieval that combines different types of local shape descriptors using a supervised learning approach. The approach enables the differentiation of object classes according to small geometric details and at the same time integrates expert knowledge from the field of architecture. We evaluate our approach using a database containing arbitrary 3D models as well as on one that only consists of models from the architectural domain. We then further extend our approach by adding a sophisticated shape descriptor localization strategy. Additionally, we exploit knowledge about the spatial relationship of object components to further enhance the retrieval performance. In the second part of the thesis we introduce attributed room connectivity graphs (RCGs) as a means to characterize a 3D building model according to the structure of its underlying floor plans. We first describe how RCGs are inferred from a given building model and discuss how substructures of this graph can be queried efficiently. We then introduce a new descriptor denoted as Bag-of-Attributed-Subgraphs that transforms attributed graphs into a vector-based representation using subgraph embeddings. We finally evaluate the retrieval performance of this new method on a database consisting of building models with different floor plan types. All methods presented in this thesis are aimed at an as automated as possible workflow for indexing and retrieval such that only minimum human interaction is required. Accordingly, only polygon soups are required as inputs which do not need to be manually repaired or structured. Human effort is only needed for offline groundtruth generation to enable supervised learning and for providing information about the orientation of building models and the unit of measurement used for modeling

The Software Architecture SCA3D: Adaptive 3D Visualization and Cooperation in Open Information Spaces

Die vorliegende Arbeit stellt mit der Software-Architektur SCA3D einen flexiblen Rahmen zur verteilten Visualisierung von komplexen und geschützten 3D Dokumenten vor. Digitale Daten bestimmen in immer stärkerem Mass moderne Arbeitsprozesse von der ersten Konzeptionsphase bis zur Fertigung eines Produktes. Hieraus ergeben sich Anforderungen für das verteilte Arbeiten mit digitalen Dokumenten: Erstens, die Notwendigkeit definierter Schnittstellen zwischen Verarbeitungssystemen; zweitens werden Methoden zur Langzeitspeicherung in Archiven, zum Retrieval und zur Verteilung von digitalen Dokumenten benötigt; drittens schliesslich sollte eine Adaption der multimedialen Dokumente bei der Verteilung in heterogenen Umgebungen unterstützt werden. Als Antwort auf diese Problemstellung stellt die Arbeit eine Software-Architektur zur verteilten 3D Visualisierung vor, die Dokumenteigenschaften in offenen, verteilten Umgebungen unterstützt und ein Komplexitäts- und Sicherheitsmanagement ermöglicht. Der Stand der Forschung auf dem Gebiet der verteilten Software-Architekturen zur 3D Visualisierung wird in zweifacher Weise erweitert. Zum einen wird ein neuer Ansatz zur Unterstützung von hybriden 2D/3D Client-Szenen in verteilten, kooperativen Arbeitsprozessen entwickelt. Hierzu zählt besonders die Realisierung eines Verfahrens zum anwendungsunabhängigen Zugriff auf verteilte 3D Dokumente über Proxy-Objekte (Document-Request-Broker). Zum anderen werden neue Methoden zur Verteilung der Verarbeitungslast (Rendering), zur Regulierung der Netzwerklast und zur Sicherung von 3D Dokumenten vorgestellt (Level-of-Information-Ansatz). Für die Praxis ergeben sich aus den Ergebnissen der Arbeit neue Möglichkeiten für Arbeitsprozesse in der digitalen Produktentwicklung, die eine breite Skala von Informationen über die verwendeten 3D Szenen nutzen können.This work presents the software architecture SCA3D as a flexible framework for distributed visualization of complex and protected 3D documents. Digital data have a determining influence on the modern work flow from the first conception phase to the final manufacturing of a product. Requirements for distributed work with digital data, which are regarded as digital documents, are the following: first, well-defined interfaces between processing systems, second, methods for long-term storage in archives, for retrieval and for distribution of digital documents and, third, adaptation of multimedia documents during distribution in heterogeneous environments. As a response to this challenge this dissertation presents a software architecture for distributed 3D visualization that supports document features in open distributed environments and provides functionality for complexity and security management of 3D documents. The status quo of the research on distributed software architectures for 3D visualization is extended by this work in two ways. First, it develops a new approach to supporting hybrid 2D/3D client scenes in distributed and cooperative work processes. Application-independent access to distributed documents is enabled in this context by a technique called document-request-broker. Second, it introduces new methods for complexity and security management that allow distribution of the processing load (rendering) and individual control of the network load (level-of-information approach). From a practical viewpoint the results of this work provide new possibilities for digital product development processes which can now use a wide scale of information about the 3D scenes being worked on

Sketch-based digital storyboards and floor plans for authoring computer-generated film pre-visuals

Pre-visualisation is an important tool for planning films during the pre-production phase of filmmaking. Existing pre-visualisation authoring tools do not effectively support the user in authoring pre-visualisations without impairing software usability. These tools require the user to either have programming skills, be experienced in modelling and animation, or use drag-and-drop style interfaces. These interaction methods do not intuitively fit with pre-production activities such as floor planning and storyboarding, and existing tools that apply a storyboarding metaphor do not automatically interpret user sketches. The goal of this research was to investigate how sketch-based user interfaces and methods from computer vision could be used for supporting pre-visualisation authoring using a storyboarding approach. The requirements for such a sketch-based storyboarding tool were determined from literature and an interview with Triggerfish Animation Studios. A framework was developed to support sketch-based pre-visualisation authoring using a storyboarding approach. Algorithms for describing user sketches, recognising objects and performing pose estimation were designed to automatically interpret user sketches. A proof of concept prototype implementation of this framework was evaluated in order to assess its usability benefit. It was found that the participants could author pre-visualisations effectively, efficiently and easily. The results of the usability evaluation also showed that the participants were satisfied with the overall design and usability of the prototype tool. The positive and negative findings of the evaluation were interpreted and combined with existing heuristics in order to create a set of guidelines for designing similar sketch-based pre-visualisation authoring tools that apply the storyboarding approach. The successful implementation of the proof of concept prototype tool provides practical evidence of the feasibility of sketch-based pre-visualisation authoring. The positive results from the usability evaluation established that sketch-based interfacing techniques can be used effectively with a storyboarding approach for authoring pre-visualisations without impairing software usability

Sketch-based digital storyboards and floor plans for authoring computer-generated film pre-visuals

Pre-visualisation is an important tool for planning films during the pre-production phase of filmmaking. Existing pre-visualisation authoring tools do not effectively support the user in authoring pre-visualisations without impairing software usability. These tools require the user to either have programming skills, be experienced in modelling and animation, or use drag-and-drop style interfaces. These interaction methods do not intuitively fit with pre-production activities such as floor planning and storyboarding, and existing tools that apply a storyboarding metaphor do not automatically interpret user sketches. The goal of this research was to investigate how sketch-based user interfaces and methods from computer vision could be used for supporting pre-visualisation authoring using a storyboarding approach. The requirements for such a sketch-based storyboarding tool were determined from literature and an interview with Triggerfish Animation Studios. A framework was developed to support sketch-based pre-visualisation authoring using a storyboarding approach. Algorithms for describing user sketches, recognising objects and performing pose estimation were designed to automatically interpret user sketches. A proof of concept prototype implementation of this framework was evaluated in order to assess its usability benefit. It was found that the participants could author pre-visualisations effectively, efficiently and easily. The results of the usability evaluation also showed that the participants were satisfied with the overall design and usability of the prototype tool. The positive and negative findings of the evaluation were interpreted and combined with existing heuristics in order to create a set of guidelines for designing similar sketch-based pre-visualisation authoring tools that apply the storyboarding approach. The successful implementation of the proof of concept prototype tool provides practical evidence of the feasibility of sketch-based pre-visualisation authoring. The positive results from the usability evaluation established that sketch-based interfacing techniques can be used effectively with a storyboarding approach for authoring pre-visualisations without impairing software usability