Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images

In this paper, we study the challenging problem of predicting the dynamics of objects in static images. Given a query object in an image, our goal is to provide a physical understanding of the object in terms of the forces acting upon it and its long term motion as response to those forces. Direct and explicit estimation of the forces and the motion of objects from a single image is extremely challenging. We define intermediate physical abstractions called Newtonian scenarios and introduce Newtonian Neural Network ($N^3$) that learns to map a single image to a state in a Newtonian scenario. Our experimental evaluations show that our method can reliably predict dynamics of a query object from a single image. In addition, our approach can provide physical reasoning that supports the predicted dynamics in terms of velocity and force vectors. To spur research in this direction we compiled Visual Newtonian Dynamics (VIND) dataset that includes 6806 videos aligned with Newtonian scenarios represented using game engines, and 4516 still images with their ground truth dynamics

More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch

For humans, the process of grasping an object relies heavily on rich tactile feedback. Most recent robotic grasping work, however, has been based only on visual input, and thus cannot easily benefit from feedback after initiating contact. In this paper, we investigate how a robot can learn to use tactile information to iteratively and efficiently adjust its grasp. To this end, we propose an end-to-end action-conditional model that learns regrasping policies from raw visuo-tactile data. This model -- a deep, multimodal convolutional network -- predicts the outcome of a candidate grasp adjustment, and then executes a grasp by iteratively selecting the most promising actions. Our approach requires neither calibration of the tactile sensors, nor any analytical modeling of contact forces, thus reducing the engineering effort required to obtain efficient grasping policies. We train our model with data from about 6,450 grasping trials on a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger. Across extensive experiments, our approach outperforms a variety of baselines at (i) estimating grasp adjustment outcomes, (ii) selecting efficient grasp adjustments for quick grasping, and (iii) reducing the amount of force applied at the fingers, while maintaining competitive performance. Finally, we study the choices made by our model and show that it has successfully acquired useful and interpretable grasping behaviors.Comment: 8 pages. Published on IEEE Robotics and Automation Letters (RAL). Website: https://sites.google.com/view/more-than-a-feelin

Transparency in Port-Hamiltonian-Based Telemanipulation

After stability, transparency is the major issue in the design of a telemanipulation system. In this paper, we exploit the behavioral approach in order to provide an index for the evaluation of transparency in port-Hamiltonian-based teleoperators. Furthermore, we provide a transparency analysis of packet switching scattering-based communication channels

A Hemispherical Contact Model for Simplifying 3D Occlusal Surfaces

Statement of problem Currently, dental articulators can recreate mandibular movements and occlusal contacts. However, whether virtual articulators can also provide information about occluding dental surfaces, functional movements, and the mandibular condyles is unclear. Purpose The purpose of this in vitro study was to evaluate the occluding surfaces on dental casts obtained from a patient and approximate them to a hemispherical contact model. Both models were tested by digitizing the Dentatus ARL dental articulator. Material and methods A combination of photogrammetry and structure from motion methods were used to scan a Dentatus ARL articulator and representative dental casts. Using computer-aided engineering and finite element analysis, contact points and action vectors to the forces on occluding surfaces and condyles were obtained for cast and hemispherical models. This experiment was performed using centric occlusion and 3 different condylar inclinations. The Kruskal-Wallis 1-way analysis of variance on ranks test was used to allow all pairwise comparisons between condylar inclination and mechanical action vector values in each location (α=.05). Results Action vectors from the cast model and each location of the hemispherical model were calculated to show the mechanical consequences and the similarity among models. Overall, no significant differences were observed for action vectors (A20 versus A40 versus A60) at each location (dental cast/hemisphere, right condylar, and left condylar) in the analysis of dental casts and the hemisphere model (.382≤P≤.999). Conclusions This study provided graphical information that may assist the dental professional in determining which occlusal contacts should be modified to attain condylar and balanced centric occlusion

Molecular modeling of an antigenic complex between a viral peptide and a class I major histocompatibility glycoprotein

Computer simulation of the conformations of short antigenic peptides (&lo residues) either free or bound to their receptor, the major histocompatibility complex (MHC)- encoded glycoprotein H-2 Ld, was employed to explain experimentally determined differences in the antigenic activities within a set of related peptides. Starting for each sequence from the most probable conformations disclosed by a pattern-recognition technique, several energyminimized structures were subjected to molecular dynamics simulations (MD) either in vacuo or solvated by water molecules. Notably, antigenic potencies were found to correlate to the peptides propensity to form and maintain an overall a-helical conformation through regular i,i + 4 hydrogen bonds. Accordingly, less active or inactive peptides showed a strong tendency to form i,i+3 hydrogen bonds at their Nterminal end. Experimental data documented that the C-terminal residue is critical for interaction of the peptide with H-2 Ld. This finding could be satisfactorily explained by a 3-D Q.S.A.R. analysis postulating interactions between ligand and receptor by hydrophobic forces. A 3-D model is proposed for the complex between a high-affinity nonapeptide and the H- 2 Ld receptor. First, the H-2 Ld molecule was built from X-ray coordinates of two homologous proteins: HLA-A2 and HLA-Aw68, energyminimized and studied by MD simulations. With HLA-A2 as template, the only realistic simulation was achieved for a solvated model with minor deviations of the MD mean structure from the X-ray conformation. Water simulation of the H-2 Ld protein in complex with the antigenic nonapeptide was then achieved with the template- derived optimal parameters. The bound peptide retains mainly its a-helical conformation and binds to hydrophobic residues of H-2 Ld that correspond to highly polymorphic positions of MHC proteins. The orientation of the nonapeptide in the binding cleft is in accordance with the experimentally determined distribution of its MHC receptor-binding residues (agretope residues). Thus, computer simulation was successfully employed to explain functional data and predicts a-helical conformation for the bound peptid