Learning a Meta-Controller for Dynamic Grasping

Grasping moving objects is a challenging task that combines multiple submodules such as object pose predictor, arm motion planner, etc. Each submodule operates under its own set of meta-parameters. For example, how far the pose predictor should look into the future (i.e., look-ahead time) and the maximum amount of time the motion planner can spend planning a motion (i.e., time budget). Many previous works assign fixed values to these parameters either heuristically or through grid search; however, at different moments within a single episode of dynamic grasping, the optimal values should vary depending on the current scene. In this work, we learn a meta-controller through reinforcement learning to control the look-ahead time and time budget dynamically. Our extensive experiments show that the meta-controller improves the grasping success rate (up to 12% in the most cluttered environment) and reduces grasping time, compared to the strongest baseline. Our meta-controller learns to reason about the reachable workspace and maintain the predicted pose within the reachable region. In addition, it assigns a small but sufficient time budget for the motion planner. Our method can handle different target objects, trajectories, and obstacles. Despite being trained only with 3-6 randomly generated cuboidal obstacles, our meta-controller generalizes well to 7-9 obstacles and more realistic out-of-domain household setups with unseen obstacle shapes. Video is available at https://youtu.be/CwHq77wFQqI.Comment: 10 page

Tactile-based Object Retrieval From Granular Media

We introduce GEOTACT, a robotic manipulation method capable of retrieving objects buried in granular media. This is a challenging task due to the need to interact with granular media, and doing so based exclusively on tactile feedback, since a buried object can be completely hidden from vision. Tactile feedback is in itself challenging in this context, due to ubiquitous contact with the surrounding media, and the inherent noise level induced by the tactile readings. To address these challenges, we use a learning method trained end-to-end with simulated sensor noise. We show that our problem formulation leads to the natural emergence of learned pushing behaviors that the manipulator uses to reduce uncertainty and funnel the object to a stable grasp despite spurious and noisy tactile readings. We also introduce a training curriculum that enables learning these behaviors in simulation, followed by zero-shot transfer to real hardware. To the best of our knowledge, GEOTACT is the first method to reliably retrieve a number of different objects from a granular environment, doing so on real hardware and with integrated tactile sensing. Videos and additional information can be found at https://jxu.ai/geotact

A new phase unwrapping method for phase shifting profilometry with object in motion

Phase unwrapping is an important step for the phase shifting profilometry. The dual-frequency phase unwrapping method can unwrap the object with discontinues when the object is static by employing more fringe patterns. However, errors will occur when moving object is reconstructed. In this paper, a new phase unwrapping method with dual-frequency phase unwrapping method for the moving object measurement is proposed. The fringe pattern with low fringe pattern and high frequency are projected onto the moving object surface. Then, the phase values are retrieved for the two frequencies respectively. The relationship between the movement and phase value is analyzed and the phase variations caused by the movement is compensated. At last, the phase value is unwrapped by the traditional dual-frequency phase unwrapping method. The effectiveness of the proposed method is verified by simulations

Reconstruction of isolated moving objects with high 3D frame rate based on phase shifting profilometry

Recently, moving object reconstruction based on PSP has been attracted intensive research. The errors caused by the inner movement of PSP have been addressed successfully. However, when the object with discontinuities or isolated surface is measured and the temporal phase unwrapping method is applied, additional fringe patterns are required to unwrap the phase map. The object movement between the PSP fringe patterns and additional fringe patterns will cause unwrapping errors. This paper proposes a new method to reconstruct the moving object with discontinuous or isolated surface. The object movement is tracked and the influence on the phase map caused by the movement is analyzed. Then, the phase variation caused by the movement is obtained. The phase map of the object before movement is obtained by compensating the phase map of the object after movement based on the phase variations. Finally, the object is reconstructed by dual-frequency phase unwrapping method. A new projection strategy increasing the efficiency of the 3D frame rate is also presented in this paper. The 3D frame rate achieves half of the camera capture speed. The proposed method has high potential to be applied in industrial applications for real-time measurement of moving objects. Experiments are presented to verify the effectiveness