PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds

Grasping for novel objects is important for robot manipulation in unstructured environments. Most of current works require a grasp sampling process to obtain grasp candidates, combined with local feature extractor using deep learning. This pipeline is time-costly, expecially when grasp points are sparse such as at the edge of a bowl. In this paper, we propose an end-to-end approach to directly predict the poses, categories and scores (qualities) of all the grasps. It takes the whole sparse point clouds as the input and requires no sampling or search process. Moreover, to generate training data of multi-object scene, we propose a fast multi-object grasp detection algorithm based on Ferrari Canny metrics. A single-object dataset (79 objects from YCB object set, 23.7k grasps) and a multi-object dataset (20k point clouds with annotations and masks) are generated. A PointNet++ based network combined with multi-mask loss is introduced to deal with different training points. The whole weight size of our network is only about 11.6M, which takes about 102ms for a whole prediction process using a GeForce 840M GPU. Our experiment shows our work get 71.43% success rate and 91.60% completion rate, which performs better than current state-of-art works.Comment: Accepted at the International Conference on Robotics and Automation (ICRA) 202

Grasp Stability Assessment Through Attention-Guided Cross-Modality Fusion and Transfer Learning

Extensive research has been conducted on assessing grasp stability, a crucial prerequisite for achieving optimal grasping strategies, including the minimum force grasping policy. However, existing works employ basic feature-level fusion techniques to combine visual and tactile modalities, resulting in the inadequate utilization of complementary information and the inability to model interactions between unimodal features. This work proposes an attention-guided cross-modality fusion architecture to comprehensively integrate visual and tactile features. This model mainly comprises convolutional neural networks (CNNs), self-attention, and cross-attention mechanisms. In addition, most existing methods collect datasets from real-world systems, which is time-consuming and high-cost, and the datasets collected are comparatively limited in size. This work establishes a robotic grasping system through physics simulation to collect a multimodal dataset. To address the sim-to-real transfer gap, we propose a migration strategy encompassing domain randomization and domain adaptation techniques. The experimental results demonstrate that the proposed fusion framework achieves markedly enhanced prediction performance (approximately 10%) compared to other baselines. Moreover, our findings suggest that the trained model can be reliably transferred to real robotic systems, indicating its potential to address real-world challenges.Comment: Accepted by IROS 202

Ubiquitous Robotic Technology for Smart Manufacturing System

As the manufacturing tasks become more individualized and more flexible, the machines in smart factory are required to do variable tasks collaboratively without reprogramming. This paper for the first time discusses the similarity between smart manufacturing systems and the ubiquitous robotic systems and makes an effort on deploying ubiquitous robotic technology to the smart factory. Specifically, a component based framework is proposed in order to enable the communication and cooperation of the heterogeneous robotic devices. Further, compared to the service robotic domain, the smart manufacturing systems are often in larger size. So a hierarchical planning method was implemented to improve the planning efficiency. A test bed of smart factory is developed. It demonstrates that the proposed framework is suitable for industrial domain, and the hierarchical planning method is able to solve large problems intractable with flat methods

Anthropomorphic Dual-Arm Coordinated Control for a Single-Port Surgical Robot Based on Dual-Step Optimization

Effective teleoperation of the small-scale and highly-integrated robots for single-port surgery (SPS) imposes unique control and human-robot interaction challenges. Traditional isometric teleoperation schemes mainly focus on end-to-end trajectory mapping, which is problematic when applied to SPS robotic control, especially for dual-arm coordinated operation. Inspired by the human arm configuration in boxing maneuvers, an optimized anthropomorphic coordinated control strategy based on a dual-step optimization approach is proposed. Theoretical derivation and solvability of the problem are addressed, and the effectiveness of the method is further demonstrated in detailed simulation and in-vitro experiments. The proposed control strategy has been shown to perform dexterous SPS bimanual manipulation more effectively, involving less instrument-interference and is free from singularities, thereby improving the safety and efficiency of SPS operations

Exploring the shared molecular mechanism of microvascular and macrovascular complications in diabetes: Seeking the hub of circulatory system injury

BackgroundMicrovascular complications, such as diabetic retinopathy (DR) and diabetic nephropathy (DN), and macrovascular complications, referring to atherosclerosis (AS), are the main complications of diabetes. Blindness or fatal microvascular diseases are considered to be identified earlier than fatal macrovascular complications. Exploring the intrinsic relationship between microvascular and macrovascular complications and the hub of pathogenesis is of vital importance for prolonging the life span of patients with diabetes and improving the quality of life.Materials and methodsThe expression profiles of GSE28829, GSE30529, GSE146615 and GSE134998 were downloaded from the Gene Expression Omnibus database, which contained 29 atherosclerotic plaque samples, including 16 AS samples and 13 normal controls; 22 renal glomeruli and tubules samples from diabetes nephropathy including 12 DN samples and 10 normal controls; 73 lymphoblastoid cell line samples, including 52 DR samples and 21 normal controls. The microarray datasets were consolidated and DEGs were acquired and further analyzed by bioinformatics techniques including GSEA analysis, GO-KEGG functional clustering by R (version 4.0.5), PPI analysis by Cytoscape (version 3.8.2) and String database, miRNA analysis by Diana database, and hub genes analysis by Metascape database. The drug sensitivity of characteristic DEGs was analyzed.ResultA total of 3709, 4185 and 8086 DEGs were recognized in AS, DN, DR, respectively, with 1820, 1666, 888 upregulated and 1889, 2519, 7198 downregulated. GO and KEGG pathway analyses of DEGs and GSEA analysis of common differential genes demonstrated that these significant sites focused primarily on inflammation-oxidative stress and immune regulation pathways. PPI networks show the connection and regulation on top-250 significant sites of AS, DN, DR. MiRNA analysis explored the non-coding RNA upstream regulation network and significant pathway in AS, DN, DR. The joint analysis of multiple diseases shows the common influenced pathways of AS, DN, DR and explored the interaction between top-1000 DEGs at the same time.ConclusionIn the microvascular and macrovascular complications of diabetes, immune-mediated inflammatory response, chronic inflammation caused by endothelial cell activation and oxidative stress are the three links linking atherosclerosis, diabetes retinopathy and diabetes nephropathy together. Our study has clarified the intrinsic relationship and common tissue damage mechanism of microcirculation and circulatory system complications in diabetes, and explored the mechanism center of these two vascular complications. It has far-reaching clinical and social value for reducing the incidence of fatal events and early controlling the progress of disabling and fatal circulatory complications in diabetes