Towards sensor-based manipulation of flexible objects

International audience— This paper presents the FLEXBOT project, a joint LIRMM-QUT effort to develop (in the near future) novel methodologies for robotic manipulation of flexible and deformable objects. To tackle this problem, and based on our past experiences, we propose to merge vision and force for manipulation control, and to rely on Model Predictive Control (MPC) and constrained optimization to program the object future shape. Index Terms— Control for object manipulation, learning from human demonstration, sensor fusion based on tactile, force and vision feedback. I. CONTEXT This abstract does not present experimental results, but aims at giving some preliminary hints on how flexible robot manipulation should be realized in the near future, particularly in the context of the FLEXBOT project, jointly submitted to the PHC FASIC Program 1 by LIRMM and QUT researchers. The objective of FLEXBOT is to solve one of the most challenging open problems in robotics. In fact, we aim at developing novel methodologies enabling robotic manipulation of flexible and deformable objects. The motivation comes from numerous applications, including the domestic, industrial, and medical examples 2 shown in Fig. 1. Many difficulties emerge when dealing with flexible manipulation. In the first place, the object deformation model (involving elasticity or plasticity) must be known, to derive the robot control inputs required for reconfiguring its shape. Ideally, this model should be derived online, while manipulating , with a simultaneous estimation and control approach, as commonly done in active perception and visual servoing. Hence perception, particularly from vision and force, will be indispensable. This leads to a second major difficulty: deformable object visual tracking. In fact, most current visual object tracking algorithms rely on rigidity, an assumption that is not valid here. A third challenge will consist in generating control inputs that comply with the shape the object is expected to have in the near future. In the next section, we provide a brief survey of the state of art on flexible object manipulation. We then conclude by proposing some novel methodologies for addressing the problem

In-hand recognition and manipulation of elastic objects using a servo-tactile control strategy

Grasping and manipulating objects with robotic hands depend largely on the features of the object to be used. Especially, features such as softness and deformability are crucial to take into account during the manipulation tasks. Indeed, positions of the fingers and forces to be applied by the robot hand when manipulating an object must be adapted to the caused deformation. For unknown objects, a previous recognition stage is usually needed to get the features of the object, and the manipulation strategies must be adapted depending on that recognition stage. To obtain a precise control in the manipulation task, a complex object model is usually needed and performed, for example using the Finite Element Method. However, these models require a complete discretization of the object and they are time-consuming for the performance of the manipulation tasks. For that reason, in this paper a new control strategy, based on a minimal spring model of the objects, is presented and used for the control of the robot hand. This paper also presents an adaptable tactile-servo control scheme that can be used in in-hand manipulation tasks of deformable objects. Tactile control is based on achieving and maintaining a force value at the contact points which changes according to the object softness, a feature estimated in an initial recognition stage.Research supported by Spanish Ministry of Economy, European FEDER funds, the Valencia Regional Government and University of Alicante, through projects DPI2012-32390, DPI2015-68087-R, PROMETEO/2013/085 and GRE 15-05

Simpler learning of robotic manipulation of clothing by utilizing DIY smart textile technology

Deformable objects such as ropes, wires, and clothing are omnipresent in society and industry but are little researched in robotics research. This is due to the infinite amount of possible state configurations caused by the deformations of the deformable object. Engineered approaches try to cope with this by implementing highly complex operations in order to estimate the state of the deformable object. This complexity can be circumvented by utilizing learning-based approaches, such as reinforcement learning, which can deal with the intrinsic high-dimensional state space of deformable objects. However, the reward function in reinforcement learning needs to measure the state configuration of the highly deformable object. Vision-based reward functions are difficult to implement, given the high dimensionality of the state and complex dynamic behavior. In this work, we propose the consideration of concepts beyond vision and incorporate other modalities which can be extracted from deformable objects. By integrating tactile sensor cells into a textile piece, proprioceptive capabilities are gained that are valuable as they provide a reward function to a reinforcement learning agent. We demonstrate on a low-cost dual robotic arm setup that a physical agent can learn on a single CPU core to fold a rectangular patch of textile in the real world based on a learned reward function from tactile information

A Mathematical Model of the Pneumatic Force Sensor for Robot-assisted Surgery

Restoring the sense of touch in robotic surgery is an emerging need several researchers tried to address. In this paper, we focused on the slave side proposing a pneumatic sensor to estimate contact forces occurring during the interaction between surgical instruments and anatomical areas. It consists of a tiny pneumatic balloon, which, after being inflated, appears near the tip of the instrument during the measurement phase only. This paper presents a mathematical method relating the intensity of the contact force to the variation of pressure inside the balloon. The latter was modeled as a spherical elastic membrane, whose behavior during contact was characterized taking into account both the deformation of the membrane and the compression of the contained gas. Geometrical considerations combined with an energetic approach allowed us to compute the force of interest. The effectiveness of our sensing device has been confirmed by experimental results, based on comparison with a high-performance commercial force sensor

Data-Driven Grasp Synthesis - A Survey

We review the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps. We divide the approaches into three groups based on whether they synthesize grasps for known, familiar or unknown objects. This structure allows us to identify common object representations and perceptual processes that facilitate the employed data-driven grasp synthesis technique. In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation. In the case of familiar objects, the techniques use some form of a similarity matching to a set of previously encountered objects. Finally for the approaches dealing with unknown objects, the core part is the extraction of specific features that are indicative of good grasps. Our survey provides an overview of the different methodologies and discusses open problems in the area of robot grasping. We also draw a parallel to the classical approaches that rely on analytic formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic

Activity profiling for minimally invasive surgery

Imperial Users onl