Sketch-based 3D Shape Retrieval using Convolutional Neural Networks

Retrieving 3D models from 2D human sketches has received considerable attention in the areas of graphics, image retrieval, and computer vision. Almost always in state of the art approaches a large amount of "best views" are computed for 3D models, with the hope that the query sketch matches one of these 2D projections of 3D models using predefined features. We argue that this two stage approach (view selection -- matching) is pragmatic but also problematic because the "best views" are subjective and ambiguous, which makes the matching inputs obscure. This imprecise nature of matching further makes it challenging to choose features manually. Instead of relying on the elusive concept of "best views" and the hand-crafted features, we propose to define our views using a minimalism approach and learn features for both sketches and views. Specifically, we drastically reduce the number of views to only two predefined directions for the whole dataset. Then, we learn two Siamese Convolutional Neural Networks (CNNs), one for the views and one for the sketches. The loss function is defined on the within-domain as well as the cross-domain similarities. Our experiments on three benchmark datasets demonstrate that our method is significantly better than state of the art approaches, and outperforms them in all conventional metrics.Comment: CVPR 201

Modelling and Simulation of Lily flowers using PDE Surfaces

This paper presents a partial differential equation (PDE)-based surface modelling and simulation framework for lily flowers. We use a PDE-based surface modelling technique to represent shape of a lily flower and PDE-based dynamic simulation to animate blossom and decay processes of lily flowers. To this aim, we first automatically construct the geometry of lily flowers from photos to obtain feature curves. Second, we apply a PDE-based surface modelling technique to generate sweeping surfaces to obtain geometric models of the flowers. Then, we use a physics-driven and data-based method and introduce the flower shapes at the initial and final positions into our proposed dynamic deformation model to generate a realistic deformation of flower blossom and decay. The results demonstrate that our proposed technique can create realistic flower models and their movements and shape changes against time efficiently with a small data size

User Interfaces and Difference Visualizations for Alternatives

Designers often create multiple iterations to evaluate alternatives. Todays computer-based tools do not support such easy exploration of a design space, despite the fact that such support has been advocated. This dissertation is centered on this. I begin by investigating the effectiveness of various forms of difference visualizations and support for merging changes within a system targeted at diagrams with node and edge attributes. I evaluated the benefits of the introduced difference visualization techniques in two user studies. I found that the basic side-by-side juxtaposition visualization was not effective and also not well received. For comparing diagrams with matching node positions, participants preferred the side-by-side option with a difference layer. For diagrams with non-matching positions animation was beneficial, but the combination with a difference layer was preferred. Thus, the difference layer technique was useful and a good complement to animation. I continue by investigating if explicit support for design alternatives better supports exploration and creativity in a generative design system. To investigate the new techniques to better support exploration, I built a new system that supports parallel exploration of alternative designs and generation of new structural combinations. I investigate the usefulness of my prototype in two user studies and interviews. The results and feedback suggest and confirm that supporting design alternatives explicitly enables designers to work more creatively. Generative models are often represented as DAGs (directed acyclic graphs) in a dataflow programming environment. Existing approaches to compare such DAGs do not generalize to multiple alternatives. Informed by and building on the first part of my dissertation, I introduce a novel user interface that enables visual differencing and editing alternative graphsspecifically more than two alternatives simultaneously, something that has not been presented before. I also explore multi-monitor support to demonstrate that the difference visualization technique scales well to up to 18 alternatives. The novel jamming space feature makes organizing alternatives on a 23 monitor system easier. To investigate the usability of the new difference visualization method I conducted an exploratory interview with three expert designers. The received comments confirmed that it meets their design goals

Improving Usability in Procedural Modeling

This work presents new approaches and algorithms for procedural modeling geared towards user convenience and improving usability, in order to increase artists’ productivity. Procedural models create geometry for 3D models from sets of rules. Existing approaches that allow to model trees, buildings, and terrain are reviewed and possible improvements are discussed. A new visual programming language for procedural modeling is discussed, where the user connects operators to visual programs called model graphs. These operators create geometry with textures, assign or evaluate variables or control the sequence of operations. When the user moves control points using the mouse in 3D space, the model graph is executed to change the geometry interactively. Thus, model graphs combine the creativity of freehand modeling with the power of programmed modeling while displaying the program structure more clearly than textbased approaches. Usability is increased as a result of these advantages. Also, an interactive editor for botanical trees is demonstrated. In contrast to previous tree modeling systems, we propose linking rules, parameters and geometry to semantic entities. This has the advantage that problems of associating parameters and instances are completely avoided. When an entity is clicked in the viewport, its parameters are displayed immediately, changes are applied to selected entities, and viewport editing operations are reflected in the parameter set. Furthermore, we store the entities in a hierarchical data structure and allow the user to activate recursive traversal via selection options for all editing operations. The user may choose to apply viewport or parameter changes to a single entity or many entities at once, and only the geometry for the affected entities needs to be updated. The proposed user interface simplifies the modeling process and increases productivity. Interactive editing approaches for 3D models often allow more precise control over a model than a global set of parameters that is used to generate a shape. However, usually scripted procedural modeling generates shapes directly from a fixed set of parameters, and interactive editing mostly uses a fixed set of tools. We propose to use scripts not only to generate models, but also for manipulating the models. A base script would set up the state of an object, and tool scripts would modify that state. The base script and the tool scripts generate geometry when necessary. Together, such a collection of scripts forms a template, and templates can be created for various types of objects. We examine how templates simplify the procedural modeling workflow by allowing for editing operations that are context-sensitive, flexible and powerful at the same time. Many algorithms have been published that produce geometry for fictional landscapes. There are algorithms which produce terrain with minimal setup time, allowing to adapt the level of detail as the user zooms into the landscape. However, these approaches lack plausible river networks, and algorithms that create eroded terrain with river networks require a user to supervise creation and minutes or hours of computation. In contrast to that, this work demonstrates an algorithm that creates terrain with plausible river networks and adaptive level of detail with no more than a few seconds of preprocessing. While the system can be configured using parameters, this text focuses on the algorithm that produces the rivers. However, integrating more tools for user-controlled editing of terrain would be possible.Verbesserung der Usability bei prozeduraler Modellierung Ziel der vorliegenden Arbeit ist es, prozedurale Modellierung durch neue neue Ansätze und Algorithmen einfacher, bequemer und anwendungsfreundlicher zu machen, und damit die Produktivität der Künstler zu erhöhen. Diese Anforderungen werden häufig unter dem Stichwort Usability zusammengefasst. Prozedurale Modelle spezifizieren 3D-Modelle über Regeln. Existierende Ansätze für Bäume, Gebäude und Terrain werden untersucht und es werden mögliche Verbesserungen diskutiert. Eine neue visuelle Programmiersprache für prozedurale Modelle wird vorgestellt, bei der Operatoren zu Modellgraphen verschaltet werden. Die Operatoren erzeugen texturierte Geometrie, weisen Variablen zu und werten sie aus, oder sie steuern den Ablauf der Operationen. Wenn der Benutzer Kontrollpunkte im Viewport mit der Maus verschiebt, wird der Modellgraph ausgeführt, um interaktiv neue Geometrie für das Modell zu erzeugen. Modellgraphen kombinieren die kreativen Möglichkeiten des freihändigen Editierens mit der Mächtigkeit der prozeduralen Modellierung. Darüber hinaus sind Modellgraphen eine visuelle Programmiersprache und stellen die Struktur der Algorithmen deutlicher dar als textbasierte Programmiersprachen. Als Resultat dieser Verbesserungen erhöht sich die Usability. Ein interaktiver Editor für botanische Bäume wird ebenfalls vorgestellt. Im Gegensatz zu früheren Ansätzen schlagen wir vor, Regeln, Parameter und Geometrie zu semantischen Entitäten zu verschmelzen. Auf diese Weise werden Zuordnungsprobleme zwischen Parametern und deren Instanzen komplett vermieden. Wenn im Viewport eine Instanz angeklickt wird, werden sofort ihre Parameter angezeigt, alle Änderungen wirken sich direkt auf die betroffenen Instanzen aus, und Änderungen im Viewport werden sofort in den Parametern reflektiert. Darüber hinaus werden die Entitäten in einer hierarchischen Datenstruktur gespeichert und alle Änderungen können rekursiv auf der Hierarchie ausgeführt werden. Dem Benutzer werden Selektionsoptionen zur Verfügung gestellt, über die er Änderungen an den Parametern oder Änderungen im Viewport an einzelnen oder vielen Instanzen gleichzeitig vornehmen kann. Anschließend muss das System nur die Geometrie der betroffenen Instanzen aktualisieren. Auch hier ist das Ziel, das User Interface möglichst an den Bedürfnissen des Benutzers auszurichten, um Vereinfachungen und eine Erhöhung der Produktivität zu erreichen. Interaktive Editieransätze für 3D-Modelle erlauben häufig eine präzisere Kontrolle über ein Modell als ein globaler Parametersatz, der für die Erzeugung des Modells genutzt wird. Trotzdem erzeugen prozedurale Modellierskripte ihre Modelle meist direkt aus einem festen Parametersatz, während interaktive Tools meist mit hartkodierten Operationen arbeiten. Wir schlagen vor, Skripte nicht nur zur Erzeugung der Modelle zu verwenden, sondern auch um die erzeugten Modelle zu editieren. Ein Basisskript soll die Statusinformationen eines Objekts anlegen, während weitere Skripte diesen Status verändern und passende Geometrie erzeugen. Diese Skripte bilden dann ein Template zum Erzeugen einer Klasse von Objekten. Verschiedene Objekttypen können jeweils ihr eigenes Template haben. Wir zeigen, wie Templates den Workflow mit prozeduralen Modellen vereinfachen können, indem Operationen geschaffen werden, die gleichzeitig kontext-sensitiv, mächtig und flexibel sind. Es existiert eine Reihe von Verfahren, um Geometrie für synthetische Landschaften zu erzeugen. Ein Teil der Algorithmen erzeugt Geometrie mit minimaler Vorberechnung und erlaubt es, den Detailgrad der Landschaft interaktiv an die Perspektive anzupassen. Leider fehlen den so erzeugten Landschaften plausible Flussnetze. Algorithmen, die erodiertes Terrain mit Flussnetzen erzeugen, müssen aufwendig vom Benutzer überwacht werden und brauchen Minuten oder Stunden Rechenzeit. Im Gegensatz dazu stellen wir einen Algorithmus vor, der plausible Flussnetze erzeugt, während sich der Betrachter interaktiv durch die Szene bewegt. Das System kann über Parameter gesteuert werden, aber der Fokus liegt auf dem Algorithmus zur Erzeugung der Flüsse. Dennoch wäre es möglich, Tools zum benutzergesteuerten Editieren von Terrain zu integrieren

Dynamic visualisation and modelling to support sustainable phosphate management

Under the WFD the UK water companies are responsible for providing effective treatment methods which will reduce the level of phosphate, a key nutrient in river eutrophication, in their respective waterways. The selection of effective treatment methods poses a sustainability problem as there are a number of economic and social aspects which must be balanced with the environmental standards set down by the WFD. This choice becomes even more complex as all of the stakeholders involved, the companies, the local government, local industry and the general public must be engaged in the decision-making process if they are to fully support whatever solution is chosen. This paper describes a sustainability assessment and visualisation toolkit, PhiZ, which has been designed to allow the assessment and communication of the sustainability of phosphate management scenarios. The PhiZ system takes the approach of combining a number of computational models covering the economic, social and environmental aspects of the scenarios with a unique visualisation based on 3D rendering techniques derived from the computer games industry, through which the results of the models can be communicated. This unique visualisation not only allows the ecological state of the river to be assessed over time but also allows a non-expert user to interactively investigate the impact of different management techniques on sustainability of the river catchment. PhiZ does not define sustainability but does provide a method for all the stakeholders involved to compare the relative sustainability of a number of management scenarios