Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.Comment: Accepted to ICLR 2019. Project Page: http://dpi.csail.mit.edu Video: https://www.youtube.com/watch?v=FrPpP7aW3L

Propagation Networks for Model-Based Control Under Partial Observation

There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt to unseen objects, scenes, and tasks. However, existing models like interaction networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNet), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. Experiments show that our propagation networks not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks. Compared with existing model-free deep reinforcement learning algorithms, model-based control with propagation networks is more accurate, efficient, and generalizable to new, partially observable scenes and tasks.Comment: Accepted to ICRA 2019. Project Page: http://propnet.csail.mit.edu Video: https://youtu.be/ZAxHXegkz4

Visual Object Networks: Image Generation with Disentangled 3D Representation

Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of objects with a disentangled 3D representation. Inspired by classic graphics rendering pipelines, we unravel our image formation process into three conditionally independent factors---shape, viewpoint, and texture---and present an end-to-end adversarial learning framework that jointly models 3D shapes and 2D images. Our model first learns to synthesize 3D shapes that are indistinguishable from real shapes. It then renders the object's 2.5D sketches (i.e., silhouette and depth map) from its shape under a sampled viewpoint. Finally, it learns to add realistic texture to these 2.5D sketches to generate natural images. The VON not only generates images that are more realistic than state-of-the-art 2D image synthesis methods, but also enables many 3D operations such as changing the viewpoint of a generated image, editing of shape and texture, linear interpolation in texture and shape space, and transferring appearance across different objects and viewpoints.Comment: NeurIPS 2018. Code: https://github.com/junyanz/VON Website: http://von.csail.mit.edu

A Compositional Object-Based Approach to Learning Physical Dynamics

We present the Neural Physics Engine (NPE), an object-based neural network architecture for learning predictive models of intuitive physics. We propose a factorization of a physical scene into composable object-based representations and also the NPE architecture whose compositional structure factorizes object dynamics into pairwise interactions. Our approach draws on the strengths of both symbolic and neural approaches: like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions, but as a neural network it can also be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that our model's compositional representation of the structure in physical interactions improves its ability to predict movement, generalize to different numbers of objects, and infer latent properties of objects such as mass.National Science Foundation (U.S.) (Award CCF-1231216)United States. Office of Naval Research (Grant N00014-16-1-2007

Assessment of motor coordination in primary education of Barcelona and province

El presente estudio tiene por objetivo valorar la coordinación motora de estudiantes de primaria de Barcelona y provincia. Para la evaluación se ha utilizado el test KTK en el que participaron 1254 personas, 670 niños y 584 niñas, de edades entre 7 y 10 años. Se realizó el análisis descriptivo y la comparación entre género y edad. Los resultados indican que más del 40% de la población estudiada presenta una coordinación por debajo de la normalidad, cerca de un 57% de la muestra fue clasificado con coordinación normal y solamente el 4,6% lo ha sido por encima de esta clasificación. Los chicos han presentado resultados significativamente mejores que las chicas. Los datos del estudio no permiten generalizar los resultados, sin embargo como es una muestra representativa, nos lleva a creer que la población de alumnos en educación primaria de Barcelona y su provincia tiene un nivel coordinativo por debajo del esperado para su edadThe present study aims to assess motor coordination of primary-school students of Barcelona and its province. For evaluation we used the KTK test. 1254 people, 670 boys and 584 girls with ages between 7 and 10 years participated in the study. Descriptive analysis and the comparison between gender and age were performed. The results indicate that about 40% of the population studied presents results below normal, around 57% were classified with normal coordination and only 4.6% has been above this rating. The boys have presented significantly better results than girls. The study data do not permit to generalize the results, however as a representative sample, it leads us to believe that the student population in elementary education from Barcelona and its province has a coordinative level below expected for their ageAgrupació de Recerca en Ciències de l’Educació. U

Microstructural development and mechanical properties of PM Ti-45Al-2Nb-2Mn-0.8vol.%TiB₂ processed by field assisted hot pressing

A gamma-TiAl intermetallic alloy, Ti-45Al-2Nb-2Mn(at.%)-0.8vol.%TiB₂, has been processed from gas atomized prealloyed powder by field assisted hot pressing (FAHP). An initial analysis of the prealloyed powder helped on the understanding of the intermetallic sintering behavior. Atomized powder consisted of alfa metastable phase that transformed into alfa2+gamma equilibrium phases by thermal treating. Different powder particle microstructures were found, which influence the microstructure development of the FAHP gamma-TiAl material depending on the sintering temperature. Duplex, nearly lamellar and fully lamellar microstructures were obtained at the sintering temperatures above 1000°C. Lower consolidation temperatures, below 1000°C, led to the formation of an Al rich phase at powder particle boundaries, which is deleterious to the mechanical properties. High compressive yield strength of 1050MPa was observed in samples with FAHP duplex microstructures at room temperature. Whereas nearly lamellar and fully lamellar microstructures showed yield strength values of 655 and 626MPa at room temperature and 440 and 425MPa at 750°C, respectively, which are superior in comparison to similar alloys processed by other techniques. These excellent properties can be explained due to the different volume fractions of the alfa2 and gamma phases and the refinement of the PM microstructures.Funding from the Spanish Ministry of Science and Innovation through projects MAT2009-14547-C02-01 and MAT2009-14547-C02-02 is acknowledged. The Madrid Regional Government partially supported this project through the ESTRUMAT grant (P2009/MAT-1585).Publicad

Learning Neural Acoustic Fields

Our environment is filled with rich and dynamic acoustic information. When we walk into a cathedral, the reverberations as much as appearance inform us of the sanctuary's wide open space. Similarly, as an object moves around us, we expect the sound emitted to also exhibit this movement. While recent advances in learned implicit functions have led to increasingly higher quality representations of the visual world, there have not been commensurate advances in learning spatial auditory representations. To address this gap, we introduce Neural Acoustic Fields (NAFs), an implicit representation that captures how sounds propagate in a physical scene. By modeling acoustic propagation in a scene as a linear time-invariant system, NAFs learn to continuously map all emitter and listener location pairs to a neural impulse response function that can then be applied to arbitrary sounds. We demonstrate that the continuous nature of NAFs enables us to render spatial acoustics for a listener at an arbitrary location, and can predict sound propagation at novel locations. We further show that the representation learned by NAFs can help improve visual learning with sparse views. Finally, we show that a representation informative of scene structure emerges during the learning of NAFs.Comment: Project page: https://www.andrew.cmu.edu/user/afluo/Neural_Acoustic_Fields