Interactive 3D Modeling with a Generative Adversarial Network

This paper proposes the idea of using a generative adversarial network (GAN) to assist a novice user in designing real-world shapes with a simple interface. The user edits a voxel grid with a painting interface (like Minecraft). Yet, at any time, he/she can execute a SNAP command, which projects the current voxel grid onto a latent shape manifold with a learned projection operator and then generates a similar, but more realistic, shape using a learned generator network. Then the user can edit the resulting shape and snap again until he/she is satisfied with the result. The main advantage of this approach is that the projection and generation operators assist novice users to create 3D models characteristic of a background distribution of object shapes, but without having to specify all the details. The core new research idea is to use a GAN to support this application. 3D GANs have previously been used for shape generation, interpolation, and completion, but never for interactive modeling. The new challenge for this application is to learn a projection operator that takes an arbitrary 3D voxel model and produces a latent vector on the shape manifold from which a similar and realistic shape can be generated. We develop algorithms for this and other steps of the SNAP processing pipeline and integrate them into a simple modeling tool. Experiments with these algorithms and tool suggest that GANs provide a promising approach to computer-assisted interactive modeling.Comment: Published at International Conference on 3D Vision 2017 (http://irc.cs.sdu.edu.cn/3dv/index.html

Two-person neuroscience and naturalistic social communication: The role of language and linguistic variables in brain-coupling research

Social cognitive neuroscience (SCN) seeks to understand the brain mechanisms through which we comprehend others? emotions and intentions in order to react accordingly. For decades, SCN has explored relevant domains by exposing individual participants to predesigned stimuli and asking them to judge their social (e.g., emotional) content. Subjects are thus reduced to detached observers of situations that they play no active role in. However, the core of our social experience is construed through real-time interactions requiring the active negotiation of information with other people. To gain more relevant insights into the workings of the social brain, the incipient field of two-person neuroscience (2PN) advocates the study of brain-to-brain coupling through multi-participant experiments. In this paper, we argue that the study of online language-based communication constitutes a cornerstone of 2PN. First, we review preliminary evidence illustrating how verbal interaction may shed light on the social brain. Second, we advance methodological recommendations to design experiments within language-based 2PN. Finally, we formulate outstanding questions for future research.Fil: García, Adolfo Martín. Universidad Nacional de Córdoba. Facultad de Lenguas; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva. Fundación Favaloro. Instituto de Neurociencia Cognitiva; Argentina. Universidad Diego Portales; ChileFil: Ibanez Barassi, Agustin Mariano. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Neurociencia Cognitiva. Fundación Favaloro. Instituto de Neurociencia Cognitiva; Argentina. Universidad Diego Portales; Chile. Universidad Autónoma del Caribe; Colombia. Australian Research Council Centre of Excellence in Cognition and its Disorders; Australi