Tactile control based on Gaussian images and its application in bi-manual manipulation of deformable objects

The field of in-hand robot manipulation of deformable objects is an open and key issue for the next-coming robots. Developing an adaptable and agile framework for the tasks where a robot grasps and manipulates different kinds of deformable objects, is a main goal in the literature. Many research works have been proposed to control the manipulation tasks using a model of the manipulated object. Despite these techniques are precise to model the deformations, they are time consuming and, using them in real environments is almost impossible because of the large amount of objects which the robot could find. In this paper, we propose a model-independent framework to control the movements of the fingers of the hands while the robot executes manipulation tasks with deformable objects. This technique is based on tactile images which are obtained as a common interface for different tactile sensors, and uses a servo-tactile control to stabilize the grasping points, avoid sliding and adapt the contacts’ configuration regarding to position and magnitude of the applied force. Tactile images are obtained using a combination of dynamic Gaussians, which allows the creation of a common representation for tactile data given by different sensors with different technologies and resolutions. The framework was tested on different manipulation tasks where the objects are deformed, and without using a model of them.Research supported by the Spanish Ministry of Economy, European FEDER funds, Valencia Regional Government and University of Alicante through the projects DPI2015-68087-R, PROMETEO/2013/085 and GRE 15-05. This work has been also supported by the French Government Research Program Investissements d’avenir, through the RobotEx Equipment of Excellence (ANR-10-EQPX-44) and the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01)

Model-free vision-based shaping of deformable plastic materials

We address the problem of shaping deformable plastic materials using non-prehensile actions. Shaping plastic objects is challenging, since they are difficult to model and to track visually. We study this problem, by using kinetic sand, a plastic toy material which mimics the physical properties of wet sand. Inspired by a pilot study where humans shape kinetic sand, we define two types of actions: \textit{pushing} the material from the sides and \textit{tapping} from above. The chosen actions are executed with a robotic arm using image-based visual servoing. From the current and desired view of the material, we define states based on visual features such as the outer contour shape and the pixel luminosity values. These are mapped to actions, which are repeated iteratively to reduce the image error until convergence is reached. For pushing, we propose three methods for mapping the visual state to an action. These include heuristic methods and a neural network, trained from human actions. We show that it is possible to obtain simple shapes with the kinetic sand, without explicitly modeling the material. Our approach is limited in the types of shapes it can achieve. A richer set of action types and multi-step reasoning is needed to achieve more sophisticated shapes.Comment: Accepted to The International Journal of Robotics Research (IJRR

Estimación de la forma de un objeto deformable mediante integración de visión y tacto

Desde hace varias décadas los robots industriales han demostrado su eficiencia para tareas desarrolladas en entornos perfectamente conocidos. No obstante, surgen importantes complicaciones operativas en entornos limitadamente conocidos, y esto dificulta de manera considerable su uso en muchas áreas de interés, limitando la expansión de la robótica. Esta tesis pretende aportar una pequeña contribución a este vasto problema y, para ello, focaliza su estudio sobre un problema concreto: la manipulación robotizada de objetos deformables. La operación sobre objetos rígidos no está exenta de problemas, muchos de ellos derivados de información imperfecta, pero al menos se sabe que la forma de estos objetos permanecerá inalterable cuando se los manipule. Es por ello que el trato con objetos deformables requiere abordar el problema desde una nueva perspectiva que pasa, necesariamente, por el uso de algún modelo del objeto que aporte información sobre su forma en función de la actuación del robot. Las dificultades surgen fundamentalmente por dos motivos: el modelo no es perfecto y la actuación del robot no es totalmente conocida. Esto obliga al uso de información externa con la que poder corregir ambas deficiencias, pero con ello tampoco queda resuelto el problema dado que esa nueva información tampoco es perfecta. Esta tesis aborda el problema de conseguir una buena información sobre la forma de una membrana de caucho natural fijada a una plataforma circular, cuando se la somete a una manipulación robotizada. Para ello se ha diseñado, implementado y validado el modelo exponencial de masas y muelles sobre el que se ha llevado a cabo un completo estudio del comportamiento de la membrana ante diferentes actuaciones del robot. A partir de ahí, y haciendo uso de información sensorial de fuerza, par y visión estereoscópica, se han diseñado, implementado y validado un conjunto de algoritmos de integración sensorial basados en el filtro de partículas y en la fusión probabilística, con los que se ha conseguido obtener una información de suficiente calidad como para operar debidamente.Industrial robots have proven their efficiency for several decades, when they operate in perfectly known environments. However, significant operational complications arise in limitedly known environments, and this difficult significantively its use in many areas of interest, hindering its expansion. This thesis aims to make a small contribution to this vast problem and, for this reason, the study focuses on a specific problem: the robotic manipulation of deformable objects. The operation on rigid objects is not exempt of problems. Many of them arise from an operation with non-perfect information, but at least we know that the shape of these objects remains unchanged when they are handled. But the handling of deformable objects requires the addressing of the problem from a new perspective, which necessarily needs the use of some object model that provides shape information based on the operation of the robot. The fundamental problem arises for two reasons: the model is not perfect and the operation of the robot is not fully known. This requires the use of independent information with which to correct both deficiencies, but even so, the problem is not totally solved because the new information is not perfect. This thesis addresses the problem of getting a proper information about the shape of a natural rubber membrane attached to a circular platform, when such membrane is subjected to a robotic manipulation. To address such problem, the exponential mass-spring model has been is designed, implemented and validated. A complete study has been made about the behavior of the membrane at different operations of the robot. Thereafter, making use of information obtained from force, torque and stereoscopic vision sensors, we have designed, implemented and validated a set of sensory integration algorithms. Such algorithms are based on particle filter and the probabilistic fusion, and it have allowed to obtain an information of sufficient quality to operate properly.Postprint (published version