Tracking objects with point clouds from vision and touch

We present an object-tracking framework that fuses point cloud information from an RGB-D camera with tactile information from a GelSight contact sensor. GelSight can be treated as a source of dense local geometric information, which we incorporate directly into a conventional point-cloud-based articulated object tracker based on signed-distance functions. Our implementation runs at 12 Hz using an online depth reconstruction algorithm for GelSight and a modified second-order update for the tracking algorithm. We present data from hardware experiments demonstrating that the addition of contact-based geometric information significantly improves the pose accuracy during contact, and provides robustness to occlusions of small objects by the robot's end effector

Jacobian-based control of soft robots for manipulation using implicit surface models

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (page 47).Soft robot hands offer numerous advantages over rigid ones for manipulation, including robustness and safety. Yet, compared to rigid robots, soft robots are characterized by continuous mechanics, and finite-element approximations with many degrees of freedom present a significant obstacle for modern control approaches. The central question my thesis explores is whether we can capture the benefits of soft robot hands with relatively simple dynamical models. Specifically, we demonstrate a very simple model of a 2D soft manipulator that uses pulleys and cables to model deformable surfaces. This model captures much of the qualitative behavior of soft membranes, while also proving amenable to modern control techniques. We validate this model physically using a hardware set-up. We then demonstrate a simple quasi-static Jacobian controller which solves a second-order cone program to achieve the task of in-hand object repositioning.by Geronimo Mirano.M. Eng