Joint Rigid Registration of Multiple Generalized Point Sets With Anisotropic Positional Uncertainties in Image-Guided Surgery

In medical image analysis (MIA) and computer-assisted surgery (CAS), aligning two multiple point sets (PSs) together is an essential but also a challenging problem. For example, rigidly aligning multiple point sets into one common coordinate frame is a prerequisite for statistical shape modelling (SSM). Accurately aligning the pre-operative space with the intra-operative space in CAS is very crucial to successful interventions. In this article, we formally formulate the multiple generalized point set registration problem (MGPSR) in a probabilistic manner, where both the positional and the normal vectors are used. The six-dimensional vectors consisting of both positional and normal vectors are called as generalized points. In the formulated model, all the generalized PSs to be registered are considered to be the realizations of underlying unknown hybrid mixture models (HMMs). By assuming the independence of the positional and orientational vectors (i.e., the normal vectors), the probability density function (PDF) of an observed generalized point is computed as the product of Gaussian and Fisher distributions. Furthermore, to consider the anisotropic noise in surgical navigation, the positional error is assumed to obey a multi-variate Gaussian distribution. Finally, registering PSs is formulated as a maximum likelihood (ML) problem, and solved under the expectation maximization (EM) technique. By using more enriched information (i.e., the normal vectors), our algorithm is more robust to outliers. By treating all PSs equally, our algorithm does not bias towards any PS. To validate the proposed approach, extensive experiments have been conducted on surface points extracted from CT images of (i) a human femur bone model; (ii) a human pelvis bone model. Results demonstrate our algorithm's high accuracy, robustness to noise and outliers

Haptics-Enabled Forceps with Multi-Modal Force Sensing: Towards Task-Autonomous Surgery

Many robotic surgical systems have been developed with micro-sized biopsy forceps for tissue manipulation. However, these systems often lack force sensing at the tool side. This paper presents a vision-based force sensing method for micro-sized biopsy forceps. A miniature sensing module adaptive to common biopsy forceps is proposed, consisting of a flexure, a camera, and a customised target. The deformation of the flexure is obtained by the camera estimating the pose variation of the top-mounted target. Then, the external force applied to the sensing module is calculated using the flexure's displacement and stiffness matrix. Integrating the sensing module into the biopsy forceps, in conjunction with a single-axial force sensor at the proximal end, we equip the forceps with haptic sensing capabilities. Mathematical equations are derived to estimate the multi-modal force sensing of the haptics-enabled forceps, including pushing/pulling forces (Mode-I) and grasping forces (Mode-II). A series of experiments on phantoms and ex vivo tissues are conducted to verify the feasibility of the proposed design and method. Results indicate that the haptics-enabled forceps can achieve multi-modal force estimation effectively and potentially realize autonomous robotic tissue grasping procedures with controlled forces.Comment: 11 pages, 9 figures, submitted to T-MEC

Reliable Hybrid Mixture Model for Generalized Point Set Registration

Point set registration (PSR) is an essential problem in the field of surgical navigation and augmented reality (AR). In surgical navigation, the aim of registration is mapping the pre-operative space to the intra-operative space. This article introduces a reliable hybrid mixture model, in which the reliability of the normal vectors in the generalized point set (GPS) is examined and exploited. The motivation of considering the reliability of orientation information is that normal vectors cannot be estimated or measured accurately in the clinic. The point set (PS) is divided into two subsets according to the reliability of normal vectors. PSR is cast into the maximum likelihood estimation (MLE) problem. The expectation maximization (EM) framework is used to solve the MLE problem. In the E-step, the posterior probabilities between points in two PSs are computed. In the M-step, the transformation matrix and model components are updated by optimizing the objective function. We have demonstrated through extensive experiments on the human femur bone PS that the proposed algorithm outperforms the state-of-the-art ones in terms of accuracy, robustness, and convergence speed

Minimally-intrusive Navigation in Dense Crowds with Integrated Macro and Micro-level Dynamics

In mobile robot navigation, despite advancements, the generation of optimal paths often disrupts pedestrian areas. To tackle this, we propose three key contributions to improve human-robot coexistence in shared spaces. Firstly, we have established a comprehensive framework to understand disturbances at individual and flow levels. Our framework provides specialized computational strategies for in-depth studies of human-robot interactions from both micro and macro perspectives. By employing novel penalty terms, namely Flow Disturbance Penalty (FDP) and Individual Disturbance Penalty (IDP), our framework facilitates a more nuanced assessment and analysis of the robot navigation's impact on pedestrians. Secondly, we introduce an innovative sampling-based navigation system that adeptly integrates a suite of safety measures with the predictability of robotic movements. This system not only accounts for traditional factors such as trajectory length and travel time but also actively incorporates pedestrian awareness. Our navigation system aims to minimize disturbances and promote harmonious coexistence by considering safety protocols, trajectory clarity, and pedestrian engagement. Lastly, we validate our algorithm's effectiveness and real-time performance through simulations and real-world tests, demonstrating its ability to navigate with minimal pedestrian disturbance in various environments.Comment: 23 pages, 13 figure

Variable stiffness methods of flexible robots for minimally invasive surgery: A review

With high flexibility and slim body, flexible robots have been widely used in minimally invasive surgery because they can safely reach the lesion deep inside the human body through small incisions or natural orifices. However, high stiffness of robot body is also required for transferring force and maintaining the motion accuracy. To meet these two contradictory requirements, various methods have been implemented to enable adjustable stiffness for flexible surgical robots. In this review, we first summarize the anatomic constraints of common natural tracts of human body to provide a guidance for the design of variable stiffness flexible robots. And then, the variable stiffness methods have been categorized based on their basic principles of varying the stiffness. In the end, two variable stiffness methods with great potential and the moving strategy of variable stiffness flexible robots are discussed

Towards simultaneous coordinate calibrations for cooperative multiple robots

Tasks that are too hard for single robot can be easily carried out by multiple robots in a cooperative manner. If some/all robots have mobile bases, the cooperation is subjected to great uncertainties in both the robotic system and environment. Therefore, the relationships among all the base frames (robot-robot calibration) and the relationships between the end-effectors and the other devices such as cameras and tools (hand-eye and tool-flange calibrations) have to be calculated to enable the robots to cooperate. To address these challenges, in this paper, we propose a simultaneous hand-eye, tool-flange and robot-robot calibration method. Thorough simulations are conducted to show the superiority of the proposed simultaneous method under different noise levels and various numbers of robot movements. Furthermore, the comparison to two non-simultaneous calibration methods has also been carried out to show the efficiency and robustness of the proposed simultaneous method

Design and analysis of an intra-abdominal wall-lifting device for vNOTES

Natural orifice transluminal endoscopic surgery (NOTES) associated with less pain, shorter hospital stay, and outstanding cosmetic results has drawn considerable attention from the academic community. For NOTES, pneumoperitoneum is an essential technology to obtain intraperitoneal surgical space, which, however, can have some serious adverse hemodynamic effects. In this paper, we propose a novel intra-abdominal wall-lifting device that can replace the pneumoperitoneum functionally for transvaginal NOTES (vNOTES). The device consists mainly of two layers of elastic strips, namely, the outer strips that are used to lift the abdominal wall for surgical exposure and the inner strip with a camera mounted that can rotate to provide a wide surgical field of view. With the design, gasless vNOTES can be carried out with 82.74% of the operative field provided by pneumoperitoneum, eliminating all complications associated with gas insufflation. The design principle and kinematic analysis of the device are discussed in detail and validated by experiments. Finally, a prototype is fabricated to demonstrate the feasibility of the proposed concept

Simultaneous hand-eye, tool-flange, and robot-robot calibration for comanipulation by solving the AXB = YCZ problem

Multirobot comanipulation shows great potential in\ud surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand–eye calibration) and tools (tool–flange calibration), as well as the relationships among the base frames (robot–robot calibration) have to be determined timely to enable accurate robotic cooperation validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method