GenPose: Generative Category-level Object Pose Estimation via Diffusion Models

Object pose estimation plays a vital role in embodied AI and computer vision, enabling intelligent agents to comprehend and interact with their surroundings. Despite the practicality of category-level pose estimation, current approaches encounter challenges with partially observed point clouds, known as the multihypothesis issue. In this study, we propose a novel solution by reframing categorylevel object pose estimation as conditional generative modeling, departing from traditional point-to-point regression. Leveraging score-based diffusion models, we estimate object poses by sampling candidates from the diffusion model and aggregating them through a two-step process: filtering out outliers via likelihood estimation and subsequently mean-pooling the remaining candidates. To avoid the costly integration process when estimating the likelihood, we introduce an alternative method that trains an energy-based model from the original score-based model, enabling end-to-end likelihood estimation. Our approach achieves state-of-the-art performance on the REAL275 dataset, surpassing 50% and 60% on strict 5d2cm and 5d5cm metrics, respectively. Furthermore, our method demonstrates strong generalizability to novel categories sharing similar symmetric properties without fine-tuning and can readily adapt to object pose tracking tasks, yielding comparable results to the current state-of-the-art baselines

GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping

The use of anthropomorphic robotic hands for assisting individuals in situations where human hands may be unavailable or unsuitable has gained significant importance. In this paper, we propose a novel task called human-assisting dexterous grasping that aims to train a policy for controlling a robotic hand's fingers to assist users in grasping objects. Unlike conventional dexterous grasping, this task presents a more complex challenge as the policy needs to adapt to diverse user intentions, in addition to the object's geometry. We address this challenge by proposing an approach consisting of two sub-modules: a hand-object-conditional grasping primitive called Grasping Gradient Field~(GraspGF), and a history-conditional residual policy. GraspGF learns `how' to grasp by estimating the gradient from a success grasping example set, while the residual policy determines `when' and at what speed the grasping action should be executed based on the trajectory history. Experimental results demonstrate the superiority of our proposed method compared to baselines, highlighting the user-awareness and practicality in real-world applications. The codes and demonstrations can be viewed at "https://sites.google.com/view/graspgf"

Score-PA: Score-based 3D Part Assembly

Autonomous 3D part assembly is a challenging task in the areas of robotics and 3D computer vision. This task aims to assemble individual components into a complete shape without relying on predefined instructions. In this paper, we formulate this task from a novel generative perspective, introducing the Score-based 3D Part Assembly framework (Score-PA) for 3D part assembly. Knowing that score-based methods are typically time-consuming during the inference stage. To address this issue, we introduce a novel algorithm called the Fast Predictor-Corrector Sampler (FPC) that accelerates the sampling process within the framework. We employ various metrics to assess assembly quality and diversity, and our evaluation results demonstrate that our algorithm outperforms existing state-of-the-art approaches. We release our code at https://github.com/J-F-Cheng/Score-PA_Score-based-3D-Part-Assembly.Comment: BMVC 202

Learning Gradient Fields for Scalable and Generalizable Irregular Packing

The packing problem, also known as cutting or nesting, has diverse applications in logistics, manufacturing, layout design, and atlas generation. It involves arranging irregularly shaped pieces to minimize waste while avoiding overlap. Recent advances in machine learning, particularly reinforcement learning, have shown promise in addressing the packing problem. In this work, we delve deeper into a novel machine learning-based approach that formulates the packing problem as conditional generative modeling. To tackle the challenges of irregular packing, including object validity constraints and collision avoidance, our method employs the score-based diffusion model to learn a series of gradient fields. These gradient fields encode the correlations between constraint satisfaction and the spatial relationships of polygons, learned from teacher examples. During the testing phase, packing solutions are generated using a coarse-to-fine refinement mechanism guided by the learned gradient fields. To enhance packing feasibility and optimality, we introduce two key architectural designs: multi-scale feature extraction and coarse-to-fine relation extraction. We conduct experiments on two typical industrial packing domains, considering translations only. Empirically, our approach demonstrates spatial utilization rates comparable to, or even surpassing, those achieved by the teacher algorithm responsible for training data generation. Additionally, it exhibits some level of generalization to shape variations. We are hopeful that this method could pave the way for new possibilities in solving the packing problem

Design, Synthesis, and In vitro Antitumor Activity Evaluation of Novel 4‐pyrrylamino Quinazoline Derivatives

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/88050/1/j.1747-0285.2011.01234.x.pd

Learning Semantic-Agnostic and Spatial-Aware Representation for Generalizable Visual-Audio Navigation

Visual-audio navigation (VAN) is attracting more and more attention from the robotic community due to its broad applications, \emph{e.g.}, household robots and rescue robots. In this task, an embodied agent must search for and navigate to the sound source with egocentric visual and audio observations. However, the existing methods are limited in two aspects: 1) poor generalization to unheard sound categories; 2) sample inefficient in training. Focusing on these two problems, we propose a brain-inspired plug-and-play method to learn a semantic-agnostic and spatial-aware representation for generalizable visual-audio navigation. We meticulously design two auxiliary tasks for respectively accelerating learning representations with the above-desired characteristics. With these two auxiliary tasks, the agent learns a spatially-correlated representation of visual and audio inputs that can be applied to work on environments with novel sounds and maps. Experiment results on realistic 3D scenes (Replica and Matterport3D) demonstrate that our method achieves better generalization performance when zero-shot transferred to scenes with unseen maps and unheard sound categories

Robotic Cane as a Soft SuperLimb for Elderly Sit-to-Stand Assistance

Many researchers have identified robotics as a potential solution to the aging population faced by many developed and developing countries. If so, how should we address the cognitive acceptance and ambient control of elderly assistive robots through design? In this paper, we proposed an explorative design of an ambient SuperLimb (Supernumerary Robotic Limb) system that involves a pneumatically-driven robotic cane for at-home motion assistance, an inflatable vest for compliant human-robot interaction, and a depth sensor for ambient intention detection. The proposed system aims at providing active assistance during the sit-to-stand transition for at-home usage by the elderly at the bedside, in the chair, and on the toilet. We proposed a modified biomechanical model with a linear cane robot for closed-loop control implementation. We validated the design feasibility of the proposed ambient SuperLimb system including the biomechanical model, our result showed the advantages in reducing lower limb efforts and elderly fall risks, yet the detection accuracy using depth sensing and adjustments on the model still require further research in the future. Nevertheless, we summarized empirical guidelines to support the ambient design of elderly-assistive SuperLimb systems for lower limb functional augmentation.Comment: 8 pages, 9 figures, accepted for IEEE RoboSoft 202

Porous polydimethylsiloxane films with specific surface wettability but distinct regular physical structures fabricated by 3D printing

Porous polydimethylsiloxane (PDMS) films with special surface wettability have potential applications in the biomedical, environmental, and structural mechanical fields. However, preparing porous PDMS films with a regular surface pattern using conventional methods, such as chemical foaming or physical pore formation, is challenging. In this study, porous PDMS films with a regular surface pattern are designed and prepared using 3D printing to ensure the formation of controllable and regular physical structures. First, the effect of the surface wettability of glass substrates with different surface energies (commercial hydrophilic glass and hydrophobic glass (F-glass) obtained by treating regular glass with 1H,1H,2H,2H-perfluorooctyl-trichlorosilane) on the structural characteristics of the 3D printed PDMS filaments is investigated systematically. Additionally, the effect of the printing speed and the surface wettability of the glass substrate on the PDMS filament morphology is investigated synchronously. Next, using the F-glass substrate and an optimized printing speed, the effects of the number of printed layers on both the morphologies of the individual PDMS filaments and porous PDMS films, and the surface wettability of the films are studied. This study reveals that regularly patterned porous PDMS films with distinct structural designs but the same controllable surface wettability, such as anisotropic surface wettability and superhydrophobicity, can be easily fabricated through 3D printing. This study provides a new method for fabricating porous PDMS films with a specific surface wettability, which can potentially expand the application of porous PDMS films

Physical and mental health impairments experienced by operating surgeons and camera-holder assistants during laparoscopic surgery: a cross-sectional survey

IntroductionSurgeons may experience physical and mental health problems because of their jobs, which may lead to chronic muscle damage, burnout, or even withdrawal. However, these are often ignored in camera-holder assistants during laparoscopic surgery. We aimed to analyze the differences between operating surgeons and camera-holder assistants.MethodsFrom January 1, 2022, to December 31, 2022, a cross-sectional survey was conducted to evaluate the muscle pain, fatigue, verbal scolding, and task load for operating surgeons and camera-holder assistants. The Nordic Musculoskeletal Questionnaire, the Space Administration Task Load Index, and the Surgical Task Load Index (SURG-TLX) were combined in the questionnaire.Results2,184 operations were performed by a total of 94 operating surgeons and 220 camera assistants. 81% of operating surgeons and 78% of camera-holder assistants reported muscle pain/discomfort during the procedure. The most affected anatomic region was the shoulders for operating surgeons, and the lower back for camera-holder assistants. Intraoperative fatigue was reported by 41.7% of operating surgeons and 51.7% of camera-holder assistants. 55.2% of camera-holder assistants reported verbal scolding from the operating surgeons, primarily attributed to lapses in laparoscope movement coordination. The SURG-TLX results showed that the distributions of mental, physical, and situational stress for operating surgeons and camera-holder assistants were comparable.ConclusionLike operating surgeons, camera-holder assistants also face similar physical and mental health impairments while performing laparoscopic surgery. Improvements to the working conditions of the camera-holder assistant should not be overlooked