CSGNet: Neural Shape Parser for Constructive Solid Geometry

We present a neural architecture that takes as input a 2D or 3D shape and outputs a program that generates the shape. The instructions in our program are based on constructive solid geometry principles, i.e., a set of boolean operations on shape primitives defined recursively. Bottom-up techniques for this shape parsing task rely on primitive detection and are inherently slow since the search space over possible primitive combinations is large. In contrast, our model uses a recurrent neural network that parses the input shape in a top-down manner, which is significantly faster and yields a compact and easy-to-interpret sequence of modeling instructions. Our model is also more effective as a shape detector compared to existing state-of-the-art detection techniques. We finally demonstrate that our network can be trained on novel datasets without ground-truth program annotations through policy gradient techniques.Comment: Accepted at CVPR-201

Semi-Automated SVG Programming via Direct Manipulation

Direct manipulation interfaces provide intuitive and interactive features to a broad range of users, but they often exhibit two limitations: the built-in features cannot possibly cover all use cases, and the internal representation of the content is not readily exposed. We believe that if direct manipulation interfaces were to (a) use general-purpose programs as the representation format, and (b) expose those programs to the user, then experts could customize these systems in powerful new ways and non-experts could enjoy some of the benefits of programmable systems. In recent work, we presented a prototype SVG editor called Sketch-n-Sketch that offered a step towards this vision. In that system, the user wrote a program in a general-purpose lambda-calculus to generate a graphic design and could then directly manipulate the output to indirectly change design parameters (i.e. constant literals) in the program in real-time during the manipulation. Unfortunately, the burden of programming the desired relationships rested entirely on the user. In this paper, we design and implement new features for Sketch-n-Sketch that assist in the programming process itself. Like typical direct manipulation systems, our extended Sketch-n-Sketch now provides GUI-based tools for drawing shapes, relating shapes to each other, and grouping shapes together. Unlike typical systems, however, each tool carries out the user's intention by transforming their general-purpose program. This novel, semi-automated programming workflow allows the user to rapidly create high-level, reusable abstractions in the program while at the same time retaining direct manipulation capabilities. In future work, our approach may be extended with more graphic design features or realized for other application domains.Comment: In 29th ACM User Interface Software and Technology Symposium (UIST 2016

Upright posture and the meaning of meronymy: A synthesis of metaphoric and analytic accounts

Cross-linguistic strategies for mapping lexical and spatial relations from body partonym systems to external object meronymies (as in English ‘table leg’, ‘mountain face’) have attracted substantial research and debate over the past three decades. Due to the systematic mappings, lexical productivity and geometric complexities of body-based meronymies found in many Mesoamerican languages, the region has become focal for these discussions, prominently including contrastive accounts of the phenomenon in Zapotec and Tzeltal, leading researchers to question whether such systems should be explained as global metaphorical mappings from bodily source to target holonym or as vector mappings of shape and axis generated “algorithmically”. I propose a synthesis of these accounts in this paper by drawing on the species-specific cognitive affordances of human upright posture grounded in the reorganization of the anatomical planes, with a special emphasis on antisymmetrical relations that emerge between arm-leg and face-groin antinomies cross-culturally. Whereas Levinson argues that the internal geometry of objects “stripped of their bodily associations” (1994: 821) is sufficient to account for Tzeltal meronymy, making metaphorical explanations entirely unnecessary, I propose a more powerful, elegant explanation of Tzeltal meronymic mapping that affirms both the geometric-analytic and the global-metaphorical nature of Tzeltal meaning construal. I do this by demonstrating that the “algorithm” in question arises from the phenomenology of movement and correlative body memories—an experiential ground which generates a culturally selected pair of inverse contrastive paradigm sets with marked and unmarked membership emerging antithetically relative to the transverse anatomical plane. These relations are then selected diagrammatically for the classification of object orientations according to systematic geometric iconicities. Results not only serve to clarify the case in question but also point to the relatively untapped potential that upright posture holds for theorizing the emergence of human cognition, highlighting in the process the nature, origins and theoretical validity of markedness and double scope conceptual integration

Discovering Regularity in Point Clouds of Urban Scenes

Despite the apparent chaos of the urban environment, cities are actually replete with regularity. From the grid of streets laid out over the earth, to the lattice of windows thrown up into the sky, periodic regularity abounds in the urban scene. Just as salient, though less uniform, are the self-similar branching patterns of trees and vegetation that line streets and fill parks. We propose novel methods for discovering these regularities in 3D range scans acquired by a time-of-flight laser sensor. The applications of this regularity information are broad, and we present two original algorithms. The first exploits the efficiency of the Fourier transform for the real-time detection of periodicity in building facades. Periodic regularity is discovered online by doing a plane sweep across the scene and analyzing the frequency space of each column in the sweep. The simplicity and online nature of this algorithm allow it to be embedded in scanner hardware, making periodicity detection a built-in feature of future 3D cameras. We demonstrate the usefulness of periodicity in view registration, compression, segmentation, and facade reconstruction. The second algorithm leverages the hierarchical decomposition and locality in space of the wavelet transform to find stochastic parameters for procedural models that succinctly describe vegetation. These procedural models facilitate the generation of virtual worlds for architecture, gaming, and augmented reality. The self-similarity of vegetation can be inferred using multi-resolution analysis to discover the underlying branching patterns. We present a unified framework of these tools, enabling the modeling, transmission, and compression of high-resolution, accurate, and immersive 3D images