Deep Learning approaches for Robotic Grasp Detection and Image Super-Resolution

Department of Electrical EngineeringIn recent years, many papers mentioned that use Deep learning to objects detection and robot grasping detection have improved accuracy with higher image resolutions. We use the Deep learning to describe robot grasp detection and image supre-resolution related two papers. 0.0.1 Real-Time, Highly Accurate Robotic Grasp Detection using Fully Convolutional Neural Networks with High-Resolution Images Robotic grasp detection for novel objects is a challenging task, but for the last few years, deep learning based approaches have achieved remarkable performance improvements, up to 96.1% accuracy, with RGB-D data. In this paper, we propose fully convolutional neural network (FCNN) based methods for robotic grasp detection. Our methods also achieved state-of-the-art detection accuracy (up to 96.6%) with state-of-the-art real-time computation time for high-resolution images (6-20ms per 360 360 image) on Cornell dataset. Due to FCNN, our proposed method can be applied to images with any size for detecting multigrasps on multiobjects. Proposed methods were evaluated using 4-axis robot arm with small parallel gripper and RGB-D camera for grasping challenging small, novel objects. With accurate visionrobot coordinate calibration through our proposed learning-based, fully automatic approach, our proposed method yielded 90% success rate. 0.0.2 Efficient Module Based Single Image Super Resolution for Multiple Problems Example based single image super resolution (SR) is a fundamental task in computer vision. It is challenging, but recently, there have been significant performance improvements using deep learning approaches. In this article, we propose efficient module based single image SR networks (EMBSR) and tackle multiple SR problems in NTIRE 2018 challenge by recycling trained networks. Our proposed EMBSR allowed us to reduce training time with effectively deeper networks, to use modular ensemble for improved performance, and to separate subproblems for better erformance. We also proposed EDSR-PP, an improved version of previous ESDR by incorporating pyramid pooling so that global as well as local context information can be utilized. Lastly, we proposed a novel denoising / deblurring residual convolutional network (DnResNet) using residual block and batch normalization. Our proposed EMBSR with DnResNet demonstrated that multiple SR problems can be tackled efficiently and e ectively by winning the 2nd place for Track 2 and the 3rd place for Track 3. Our proposed method with EDSR-PP also achieved the ninth place for Track 1 with the fastest run time among top nine teams.clos

Efficient Fully Convolution Neural Network for Generating Pixel Wise Robotic Grasps With High Resolution Images

This paper presents an efficient neural network model to generate robotic grasps with high resolution images. The proposed model uses fully convolution neural network to generate robotic grasps for each pixel using 400 $\times$ 400 high resolution RGB-D images. It first down-sample the images to get features and then up-sample those features to the original size of the input as well as combines local and global features from different feature maps. Compared to other regression or classification methods for detecting robotic grasps, our method looks more like the segmentation methods which solves the problem through pixel-wise ways. We use Cornell Grasp Dataset to train and evaluate the model and get high accuracy about 94.42% for image-wise and 91.02% for object-wise and fast prediction time about 8ms. We also demonstrate that without training on the multiple objects dataset, our model can directly output robotic grasps candidates for different objects because of the pixel wise implementation.Comment: Submitted to ROBIO 201

Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenes

In this paper, we focus on the problem of feature learning in the presence of scale imbalance for 6-DoF grasp detection and propose a novel approach to especially address the difficulty in dealing with small-scale samples. A Multi-scale Cylinder Grouping (MsCG) module is presented to enhance local geometry representation by combining multi-scale cylinder features and global context. Moreover, a Scale Balanced Learning (SBL) loss and an Object Balanced Sampling (OBS) strategy are designed, where SBL enlarges the gradients of the samples whose scales are in low frequency by apriori weights while OBS captures more points on small-scale objects with the help of an auxiliary segmentation network. They alleviate the influence of the uneven distribution of grasp scales in training and inference respectively. In addition, Noisy-clean Mix (NcM) data augmentation is introduced to facilitate training, aiming to bridge the domain gap between synthetic and raw scenes in an efficient way by generating more data which mix them into single ones at instance-level. Extensive experiments are conducted on the GraspNet-1Billion benchmark and competitive results are reached with significant gains on small-scale cases. Besides, the performance of real-world grasping highlights its generalization ability. Our code is available at https://github.com/mahaoxiang822/Scale-Balanced-Grasp.Comment: Accepted at CoRL'202

Modular Anti-noise Deep Learning Network for Robotic Grasp Detection Based on RGB Images

While traditional methods relies on depth sensors, the current trend leans towards utilizing cost-effective RGB images, despite their absence of depth cues. This paper introduces an interesting approach to detect grasping pose from a single RGB image. To this end, we propose a modular learning network augmented with grasp detection and semantic segmentation, tailored for robots equipped with parallel-plate grippers. Our network not only identifies graspable objects but also fuses prior grasp analyses with semantic segmentation, thereby boosting grasp detection precision. Significantly, our design exhibits resilience, adeptly handling blurred and noisy visuals. Key contributions encompass a trainable network for grasp detection from RGB images, a modular design facilitating feasible grasp implementation, and an architecture robust against common image distortions. We demonstrate the feasibility and accuracy of our proposed approach through practical experiments and evaluations