Visual-Tactile Fusion for Robotic Stable Grasping

The stable grasp is the basis of robotic manipulation. It requires balance of the contact forces and the operated object. The status of the grasp determined by vision is direct according to the object’s shape or texture, but quite challenging. The tactile sensor can provide the effective way. In this work, we propose the visual-tactile fusion framework for predicting the grasp. Meanwhile, the object intrinsic property is also used. More than 2550 grasping trials using a novel robot hand with multiple tactile sensors are collected. And visual-tactile intrinsic deep neural network (DNN) is evaluated to prove the performance. The experimental results show the superiority of the proposed method

Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors

Simulation is widely applied in robotics research to save time and resources. There have been several works to simulate optical tactile sensors that leverage either a smoothing method or Finite Element Method (FEM). However, elastomer deformation physics is not considered in the former method, whereas the latter requires a massive amount of computational resources like a computer cluster. In this work, we propose a pluggable and low computational cost simulator using the Taichi programming language for simulating optical tactile sensors, named as Tacchi . It reconstructs elastomer deformation using particles, which allows deformed elastomer surfaces to be rendered into tactile images and reveals contact information without suffering from high computational costs. Tacchi facilitates creating realistic tactile images in simulation, e.g., ones that capture wear-and-tear defects on object surfaces. In addition, the proposed Tacchi can be integrated with robotics simulators for a robot system simulation. Experiment results showed that Tacchi can produce images with better similarity to real images and achieved higher Sim2Real accuracy compared to the existing methods. Moreover, it can be connected with MuJoCo and Gazebo with only the requirement of 1G memory space in GPU compared to a computer cluster applied for FEM. With Tacchi, physical robot simulation with optical tactile sensors becomes possible. All the materials in this paper are available at https://github.com/zixichen007115/Tacchi .Comment: 8 pages, 6 figures, accepted by IEEE Robotics and Automation Letter

Design of Zoom Optical System from Visible to NIR-II for Vivo Fluorescence Imaging Device

Macro vivo fluorescence imaging is becoming more and more important in the medical field. It is also necessary to design the optical design system for the visible light of the NIR-II detector. This paper discusses the design method of the wide spectrum achromatic optical system from visible light to NIR-II. Based on ZEMAX, a wide spectrum zoom system is designed to freely observe experimental targets in the fields of view of 3.1–18.6°. The four components layout was adopted by the wide spectrum zoom system, which is suitable for 8.2 mm (1/2 inch) CCD, with an f-number of 5.0~6.0, zoom range of 25 mm~150 mm, working spectral band of 400 nm~1700 nm, full field MTF ≥ 0.3 at the spatial frequency of 100 lp/mm, and the maximum distortion of ≤±3%. All optical elements adopt the standard spherical, which can correct all kinds of aberrations well and meet each part’s basic processing requirements

Development of a Wearable Device for Motion Capturing Based on Magnetic and Inertial Measurement Units

This paper presents a novel wearable device for gesture capturing based on inertial and magnetic measurement units that are made up of micromachined gyroscopes, accelerometers, and magnetometers. The low-cost inertial and magnetic measurement unit is compact and small enough to wear and there are altogether thirty-six units integrated in the device. The device is composed of two symmetric parts, and either the right part or the left one contains eighteen units covering all the segments of the arm, palm, and fingers. The offline calibration and online calibration are proposed to improve the accuracy of sensors. Multiple quaternion-based extended Kalman filters are designed to estimate the absolute orientations, and kinematic models of the arm-hand are considered to determine the relative orientations. Furthermore, position algorithm is deduced to compute the positions of corresponding joint. Finally, several experiments are implemented to verify the effectiveness of the proposed wearable device