Semi-Automatic Infrared Calibration for Augmented Reality Systems in Surgery

Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Several studies attempt this calibration through manual alignment or with additional fiducial markers in the surgical scene. We propose a calibration system that offers a direct method for the calibration of AR head-mounted displays (HMDs) with CAOS systems, by using infrared-reflective marker-arrays widely used in CAOS. In our fast, user-agnostic setup, a HoloLens 2 detected the pose of marker arrays using infrared response and time-of-flight depth obtained through sensors onboard the HMD. Registration with a commercially available CAOS system was achieved when an IR marker-array was visible to both devices. Study tests found relative-tracking mean errors of 2.03 mm and 1.12{\deg} when calculating the relative pose between two static marker-arrays at short ranges. When using the calibration result to provide in-situ holographic guidance for a simulated wire-insertion task, a pre-clinical test reported mean errors of 2.07 mm and 1.54{\deg} when compared to a pre-planned trajectory.Comment: Published in conference proceedings for 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). For associated code visit: https://github.com/HL2-DIN

A long short-term memory network for vessel reconstruction based on laser doppler flowmetry via a steerable needle

Hemorrhage is one risk of percutaneous intervention in the brain that can be life-threatening. Steerable needles can avoid blood vessels thanks to their ability to follow curvilinear paths, although knowledge of vessel pose is required. To achieve this, we present the deployment of laser Doppler flowmetry (LDF) sensors as an in-situ vessel detection method for steerable needles. Since the perfusion value from an LDF system does not provide positional information directly, we propose the use of a machine learning technique based on a Long Short-term Memory (LSTM) network to perform vessel reconstruction online. Firstly, the LSTM is used to predict the diameter and position of an approaching vessel based on successive measurements of a single LDF probe. Secondly, a "no-go" area is predicted based on the measurement from four LDF probes embedded within a steerable needle, which accounts for the full vessel pose. The network was trained using simulation data and tested on experimental data, with 75 % diameter prediction accuracy and 0.27 mm positional Root Mean Square (RMS) Error for the single probe network, and 77 % vessel volume overlap for the 4-probe setup

Kinematics of continuum robots with constant curvature bending and extension capabilities

Continuum robots are becoming increasingly popular due to the capabilities they offer, especially when operating in cluttered environments, where their dexterity, maneuverability, and compliance represent a significant advantage. The subset of continuum robots that also belong to the soft robots category has seen rapid development in recent years, showing great promise. However, despite the significant attention received by these devices, various aspects of their kinematics remain unresolved, limiting their adoption and obscuring their potential. In this paper, the kinematics of continuum robots with the ability to bend and extend are studied, and analytical, closed-form solutions to both the direct and inverse kinematics are presented. The results obtained expose the redundancies of these devices, which are subsequently explored. The solution to the inverse kinematics derived here is shown to provide an analytical, closed-form expression describing the curve associated with these redundancies, which is also presented and analyzed. A condition on the reachable end-effector poses for robots with six actuation degrees-of-freedom (DOFs) is then distilled. The kinematics of robot layouts with over six actuation DOFs are subsequently considered. Finally, simulated results of the inverse kinematics are provided, verifying the study

A reduced actuation mecanum wheel platform for pipe inspection

This paper focuses on the design, development and assessment of a novel, 2 degrees-of-freedom magnetic pipe inspection robot. It consists of 4 mecanum wheels, with the diagonals functionally coupled and the system rotation constrained by the surface geometry, maintaining full translational mobility with reduced control and actuation requirements. The system uses positional encoding that is decoupled from the transmission system to overcome the main sources of positional/positioning errors when using mecanum wheels. The kinematic and dynamic models of the system are derived and integrated within the controller. The prototype robot is then tested and shown to follow a scan path at 20mm/s within ±1.5mm whilst correcting for gravitational drift and slip events

Markerless navigation system for orthopaedic knee surgery: a proof of concept study

Current computer-assisted surgical navigation systems mainly rely on optical markers screwed into the bone for anatomy tracking. The insertion of these percutaneous markers increases operating complexity and causes additional harm to the patient. A markerless tracking and registration algorithm has recently been proposed to avoid anatomical markers for knee surgery. The femur points were directly segmented from the recorded RGBD scene by a neural network and then registered to a pre-scanned femur model for the real-time pose. However, in a practical setup such a method can produce unreliable registration results, especially in rotation. Furthermore, its potential application in surgical navigation has not been demonstrated. In this paper, we first improved markerless registration accuracy by adopting a bounded-ICP (BICP) technique, where an estimate of the remote hip centre, acquired also in a markerless way, was employed to constrain distal femur alignment. Then, a proof-of-concept markerless navigation system was proposed to assist in typical knee drilling tasks. Two example setups for global anchoring were proposed and tested on a phantom leg. Our BICP-based markerless tracking and registration method has better angular accuracy and stability than the original method, bringing our straightforward, less invasive markerless navigation approach one step closer to clinical application. According to user tests, our proposed optically anchored navigation system achieves comparable accuracy with the state-of-the-art (3.64± 1.49 mm in position and 2.13±0.81° in orientation). Conversely, our visually anchored, optical tracker-free setup has a lower accuracy (5.86± 1.63 mm in position and 4.18±1.44° in orientation), but is more cost-effective and flexible in the operating room

Augmented-reality within computer assisted orthopaedic surgery workflows: a proof of concept study

The integration of augmented-reality (AR) in medical robotics has been shown to reduce cognitive burden and improve information management in the typically cluttered environment of computer-assisted surgery. A key benefit of such systems is the ability to generate a composite view of medical-informatics and the real environment, streamlining the pathway for delivering patient-specific data. Consequently, AR was integrated within an orthopaedic setting by designing a system that captured and replicated the user- interface of a commercially available surgical robot onto a commercial head mounted see through display. Thus, a clinician could simultaneously view the operating-site and real- time informatics when carrying out an assisted patellofemoral-arthroplasty (PFA). The system was tested with 10 surgeons to examine its usability and impact on procedure- completion times when conducting simulated PFA on sawbone models. A statistically insignificant mean increase in procedure completion-time (+23.7s, p=0.240) was found, and the results of a post-operative qualitative-evaluation indicated a strongly positive consensus on the system, with a large majority of subjects agreeing the system provided value to the procedure without incurring noticeable physical discomfort. Overall, this study provides an encouraging insight into the high levels of engagement AR has with a clinical audience as well as its ability to enhance future generations of medical robotics

Modelling the deformation of biologically inspired flexible structures for needle steering

Recent technical advances in minimally invasive surgery have been enabled by the development of new medical instruments and technologies. To date, the vast majority of mechanisms used within a clinical context are rigid, contrasting with the compliant nature of biological tissues. The field of robotics has seen an increased interest in flexible and compliant systems, and in this paper we investigate the behaviour of deformable multi-segment structures, which take their inspiration from the ovipositor design of parasitic wood wasps. These configurable structures have been shown to steer through highly compliant substrates, potentially enabling percutaneous access to the most delicate of tissues, such as the brain. The model presented here sheds light on how the deformation of the unique structure is related to its shape, and allows comparison between different potential designs. A finite element study is used to evaluate the proposed model, which is shown to provide a good fit (root-mean-square deviation 0.2636 mm for 4-segment case). The results show that both 3-segment and 4-segment designs are able to achieve deformation in all directions, however the magnitude of deformation is more consistent in the 4-segment case

A mechanics-based model for 3D steering of programmable bevel-tip needles

We present a model for the steering of programmable bevel-tip needles, along with a set of experiments demonstrating the 3D steering performance of a new, clinically viable, 4-segment, pre-production prototype. A multi-beam approach, based on Euler-Bernoulli beam theory, is used to model the novel multi-segment design of these needles. Finite element simulations for known loads are used to validate the multi-beam deflection model. A clinically sized (2.5 mm outer diameter), 4-segment programmable bevel-tip needle, manufactured by extrusion of a medical-grade polymer, is used to conduct an extensive set of experimental trials to evaluate the steering model. For the first time, we demonstrate the ability of the 4-segment needle design to steer in any direction with a maximum achievable curvature of 0.0192±0.0014 mm⁻¹. Finite element simulations confirm that the multi-beam approach produces a good model fit for tip deflections, with a root-mean-square deviation (RMSD) in modeled tip deflection of 0.2636 mm. We perform a parameter optimization to produce a best-fit steering model for the experimental trials, with a RMSD in curvature prediction of 1.12×10⁻³ mm⁻¹

Cyclic motion control for programmable bevel-tip needles 3D steering: a simulation study

Flexible, steerable, soft needles are desirable in Minimally Invasive Surgery to achieve complex trajectories while maintaining the benefits of percutaneous intervention compared to open surgery. One such needle is the multi-segment Programmable Bevel-tip Needle (PBN), which is inspired by the mechanical design of the ovipositor of certain wasps. PBNs can steer in 3D whilst minimizing the force applied to the surrounding substrate, due to the cyclic motion of the segments. Taking inspiration also from the control strategy of the wasp to perform insertions and lay their eggs, this paper presents the design of a cyclic controller that can steer a PBN to produce a desired trajectory in 3D. The performance of the controller is demonstrated in simulation in comparison to that of a direct controller without cyclic motion. It is shown that, while the same steering curvatures can be attained by both controllers, the time taken to achieve the configuration is longer for the cyclic controller, leading to issues of potential under-steering and longer insertion times