Data-driven modelling of perceptual properties of 3D shapes

The recent surge in 3D content generation has led to the evolution of difficult to search, organise and re-use massive online 3D visual content libraries. We explore crowdsourcing and machine learning techniques to help alleviate these difficulties by focusing on the visual perceptual properties of 3D shapes. We study “style similarity” and “aesthetics” as two fundamental perceptual properties of 3D shapes and build data-driven models. We rely on crowdsourcing platforms to collect large number of human judgements on style matching and aesthetics of 3D shapes. The judgement data collected directly from humans is used to learn metrics of style matching and aesthetics. Our style similarity measure can be used to compute style distance between a pair of input 3D shapes. In contrast to previous work, we incorporate colour and texture in addition to geometric features to build a colour and texture aware style similarity metric. We also experiment with learning objective and personalised style metrics 3D shapes. The application prototypes we build demonstrate the use of style based search and scene composition. Further, our style distance metric is built iteratively to consume lesser amount of human style judgement data compared to previous methods. We study the problem of building a data-driven model of 3D shape aesthetics in two steps. We first focus on designing a study to crowdsource human aesthetics judgement data. We then formulate a deep learning based strategy to learn a measure of 3D shape aesthetics from collected data. The results of the study in first step helped us choose an appropriate shape representation i.e. voxels as an input to deep neural networks for learning a measure of visual aesthetics. In the same crowdsourcing study, we experiment with the use of polygonal, volumetric, and point based shape representations to create shape stimuli to collect and compare human shape aesthetics judgements. On analysis of the collected data we found that that humans can reliably distinguish more aesthetic shape in a pair even from coarser shape representations such as voxels. This observation implies that detailed shape representations are not needed to compare aesthetics in pairs. The aesthetic value of a 3D shape has traditionally been explored in terms of specific visual features (or handcrafted features) such as curvature and symmetry. For example, more symmetric and curved shapes are considered aesthetic compared to less curved and symmetric shapes. We call such properties as pre-existing notion (or rules) of aesthetics. In order to develop a measure of perceptual aesthetics of 3D shapes which is independent of any pre-existing notion or shape features, we train deep neural networks directly on human aesthetics judgement data. We demonstrate the usefulness of the learned measure by designing applications to rank a collection of shapes based on their aesthetics scores and interactively build scenes using shapes with high aesthetics scores. The overarching goal of this thesis is to demonstrate the use of machine learning and crowdsourcing approaches to build data-driven models of visual perceptual properties of 3D shapes for applications in search, organisation, scene composition, and visualisation of 3D shape data present in ever increasing online 3D shape content libraries. We believe that our exploration of perceptual properties of 3D shapes will motivate further research by looking into other important perceptual properties related to our vision system and will also fuel development of techniques to automatically enhance such properties of a given 3D shape

On the perceptual aesthetics of interactive objects

In this paper we measured the aesthetics of interactive objects (IOs), which are three-dimensional physical artefacts that exhibit autonomous behaviour when ‎handled. The aim of the research was threefold: firstly, to investigate whether aesthetic preference for distinctive objects' structures emerges in compound stimulation; secondly, to explore whether there exists aesthetic preference for distinctive objects’ behaviours; and lastly, to test whether there exists aesthetic preference for specific combinations of objects' structures and behaviours. The following variables were systematically manipulated: 1) IOs’ contour (rounded vs. angular); 2) IOs’ size (small vs. large); 3) IOs’ surface texture (rough vs. smooth); and 4) IOs’ behaviour (Lighting, Sounding, Vibrating, and Quiescent). Results show that behaviour was the dominant factor: it influenced aesthetics more than any other characteristic; Vibrating IOs were preferred over Lighting and Sounding IOs, supporting the importance of haptic processing in aesthetics. Results did not confirm the size and smoothness effects previously reported in vision and touch respectively, which suggests that for the aesthetics preference that emerges in isolated conditions may be different in compound stimulation. Finally, results corroborate the smooth curvature effect

Sketchy rendering for information visualization

We present and evaluate a framework for constructing sketchy style information visualizations that mimic data graphics drawn by hand. We provide an alternative renderer for the Processing graphics environment that redefines core drawing primitives including line, polygon and ellipse rendering. These primitives allow higher-level graphical features such as bar charts, line charts, treemaps and node-link diagrams to be drawn in a sketchy style with a specified degree of sketchiness. The framework is designed to be easily integrated into existing visualization implementations with minimal programming modification or design effort. We show examples of use for statistical graphics, conveying spatial imprecision and for enhancing aesthetic and narrative qualities of visual- ization. We evaluate user perception of sketchiness of areal features through a series of stimulus-response tests in order to assess users’ ability to place sketchiness on a ratio scale, and to estimate area. Results suggest relative area judgment is compromised by sketchy rendering and that its influence is dependent on the shape being rendered. They show that degree of sketchiness may be judged on an ordinal scale but that its judgement varies strongly between individuals. We evaluate higher-level impacts of sketchiness through user testing of scenarios that encourage user engagement with data visualization and willingness to critique visualization de- sign. Results suggest that where a visualization is clearly sketchy, engagement may be increased and that attitudes to participating in visualization annotation are more positive. The results of our work have implications for effective information visualization design that go beyond the traditional role of sketching as a tool for prototyping or its use for an indication of general uncertainty

Understanding and modeling of aesthetic response to shape and color in car body design

This study explored the phenomenon that a consumer's preference on color of car body may vary depending on shape of the car body. First, the study attempted to establish a theoretical framework that can account for this phenomenon. This framework is based on the (modern-) Darwinism approach to the so-called evolutionary psychology and aesthetics. It assumes that human's aesthetic sense works like an agent that seeks for environmental patterns that potentially afford to benefit the underlying needs of the agent, and this seeking process is evolutionary fitting. Second, by adopting the framework, a pattern called “fundamental aesthetic dimensions” was developed for identifying and modeling consumer’s aesthetic response to car body shape and color. Next, this study developed an effective tool that is capable in capturing and accommodating consumer’s color preference on a given car body shape. This tool was implemented by incorporating classic color theories and advanced digital technologies; it was named “Color-Shape Synthesizer”. Finally, an experiment was conducted to verify some of the theoretical developments. This study concluded (1) the fundamental aesthetics dimensions can be used for describing aesthetics in terms of shape and color; (2) the Color-Shape Synthesizer tool can be well applied in practicing car body designs; and (3) mapping between semantic representations of aesthetic response to the fundamental aesthetics dimensions can likely be a multiple-network structure