Calibration and External Force Sensing for Soft Robots using an RGB-D Camera

International audienceBenefiting from the deformability of soft robots, calibration and force sensing for soft robots are possible using an external vision-based system, instead of embedded mechatronic force sensors. In this paper, we first propose a calibration method to calibrate both the sensor-robot coordinate system and the actuator inputs. This task is addressed through a sequential optimization problem for both variables. We also introduce an external force sensing system based on a real-time Finite Element (FE) model with the assumption of static configurations, and which consists of two steps: force location detection and force intensity computation. The algorithm that estimates force location relies on the segmentation of the point cloud acquired by an RGB-D camera. Then, the force intensities can be computed by solving an inverse quasi-static problem based on matching the FE model with the point cloud of the soft robot. As for validation, the proposed strategies for calibration and force sensing have been tested using a parallel soft robot driven by four cables

A soft, sensorized gripper for delicate harvesting of small fruits

Harvesting fruits and vegetables is a complex task worth to be fully automated with robotic systems. It involves several precision tasks that have to be performed with accuracy and the appropriate amount of force. Classical mechanical grippers, due to the complex control and stiffness, cannot always be used to harvest fruits and vegetables. Instead, the use of soft materials could provide a visible advancement. In this work, we propose a soft, sensorized gripper for harvesting applications. The sensing is performed by tracking a set of markers integrated into the soft part of the gripper. Different machine learning-based approaches have been used to map the markers’ position and dimensions into forces in order to perform a close-loop control of the gripper. Results show that force can be measured with an error of 2.6% in a range from 0 to 4 N. The gripper was integrated into a robotic arm having an external vision system used to detect plants and fruits (strawberries in our case scenario). As a proof of concept, we evaluated the performance of the robotic system in a laboratory scenario. Plant and fruit identification reached a positive rate of 98.2% and 92.4%, respectively, while the correct picking of the fruits, by removing it from the stalk without a direct cut, achieved an 82% of successful rate

Image-based Optical Miniaturized Three-Axis Force Sensor for Cardiac Catheterization

In order to determine the cause of and to treat an abnormal heart rhythm, electrophysiological studies and ablation procedures of the heart, sensorized catheters are required. During catheterization, force sensors at the tip of the catheter are essential to provide quantitative information on the interacting force between the catheter tip and the heart tissue. In this study, we are proposing a small sized, robust, and low-cost three-axis force sensor for the catheter tip. The miniaturized force sensor uses fiber-optic technology (small sized multi-cores optical fiber and a CCD camera) based on image processing to read out the forces by measuring light intensity which are modulated as a function of the applied force. In addition, image processing techniques and a Kalman filter are used to reduce the noise of the light intensity signals. In this paper, we explain the design and fabrication of our three-axis force sensor and our approach for reducing noise levels by applying a Kalman filter model, and finally discuss the calibration procedure. Moreover, we provide an assessment of the performance of the proposed sensor

Rapid manufacturing of color-based hemispherical soft tactile fingertips

Tactile sensing can provide access to information about the contact (i.e. slippage, surface feature, friction), which is out of reach of vision but crucial for manipulation. To access this information, a dense measurement of the deformation of soft fingertips is necessary. Recently, tactile sensors that rely on a camera looking at a deformable membrane have demonstrated that a dense measurement of the contact is possible. However, their manufacturing can be time-consuming and labor-intensive. Here, we show a new design method that uses multi-color additive manufacturing and silicone casting to efficiently manufacture soft marker-based tactile sensors that are able to capture with high-resolution the three-dimensional deformation field at the interface. Each marker is composed of two superimposed color filters. The subtractive color mixing encodes the normal deformation of the membrane, and the lateral deformation is found by centroid detection. With this manufacturing method, we can reach a density of 400 markers on a 21 mm radius hemisphere, allowing for regular and dense measurement of the deformation. We calibrated and validated the approach by finding the curvature of objects with a threefold increase in accuracy as compared to previous implementations. The results demonstrate a simple yet effective approach to manufacturing artificial fingertips for capturing a rich image of the tactile interaction at the location of contact