Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation dose to both patient and staff. This work proposes a marker-free "technician-in-the-loop" Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. We conduct experiments in a setting simulating orthopedic trauma surgery. Our proof-of-principle findings indicate that the proposed system can decrease the 2.76 X-ray images required per desired view down to zero, suggesting substantial reductions of radiation dose during C-arm repositioning. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future. The concept of technician-in-the-loop design will become relevant to various interventions considering the expected advancements of sensing and wearable computing in the near future

Augmented reality-based feedback for technician-in-the-loop C-arm repositioning

Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation. We propose a marker-free ‘technician-in-the-loop’ Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display system capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a target view, the recorded pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. Our proof-of-principle findings from a simulated trauma surgery indicate that the proposed system can decrease the 2.76 X-ray images required for re-aligning the scanner with an intra-operatively recorded C-arm view down to zero, suggesting substantial reductions of radiation dose. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future

Augmented reality for computer assisted orthopaedic surgery

In recent years, computer-assistance and robotics have established their presence in operating theatres and found success in orthopaedic procedures. Benefits of computer assisted orthopaedic surgery (CAOS) have been thoroughly explored in research, finding improvements in clinical outcomes, through increased control and precision over surgical actions. However, human-computer interaction in CAOS remains an evolving field, through emerging display technologies including augmented reality (AR) – a fused view of the real environment with virtual, computer-generated holograms. Interactions between clinicians and patient-specific data generated during CAOS are limited to basic 2D interactions on touchscreen monitors, potentially creating clutter and cognitive challenges in surgery. Work described in this thesis sought to explore the benefits of AR in CAOS through: an integration between commercially available AR and CAOS systems, creating a novel AR-centric surgical workflow to support various tasks of computer-assisted knee arthroplasty, and three pre–clinical studies exploring the impact of the new AR workflow on both existing and newly proposed quantitative and qualitative performance metrics. Early research focused on cloning the (2D) user-interface of an existing CAOS system onto a virtual AR screen and investigating any resulting impacts on usability and performance. An infrared-based registration system is also presented, describing a protocol for calibrating commercial AR headsets with optical trackers, calculating a spatial transformation between surgical and holographic coordinate frames. The main contribution of this thesis is a novel AR workflow designed to support computer-assisted patellofemoral arthroplasty. The reported workflow provided 3D in-situ holographic guidance for CAOS tasks including patient registration, pre-operative planning, and assisted-cutting. Pre-clinical experimental validation on a commercial system (NAVIO®, Smith & Nephew) for these contributions demonstrates encouraging early-stage results showing successful deployment of AR to CAOS systems, and promising indications that AR can enhance the clinician’s interactions in the future. The thesis concludes with a summary of achievements, corresponding limitations and future research opportunities.Open Acces

Mobile depth sensing technology and algorithms with application to occupational therapy healthcare

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe UK government is striving to shift its current healthcare delivery model from clini-cian–oriented services, to that of patient and self–care–oriented intervention strategies. It seeks to do so through Information Communication (ICT) and Computer Mediated Re-ality Technologies (CMRT) as a key strategy to overcome the ever–increasing scarcity of healthcare resources and costs. To this end, in the UK the use of paper–based information systems have exhibited their limitations in providing apposite care. At the national level, The Royal College of Occupational Therapists (RCOT) identify home visits and modifica-tions as key levers in a multifactorial health programme to evaluate interventions for older people with a history of falling or are identified as being prone to falling. Prescribing Assistive Equipment (AE) is one such mechanism that seeks to reduce the risk of falling whilst promoting the continued independence of physical dexterity and mobility in older adults at home. In the UK, the yearly cost of falls is estimated at £2.3 billion. Further evidence places a 30% to 60% abandonment rate on prescribed AE by and large due to a ‘poor fit’ and measurement inaccuracies. To remain aligned with the national strategy, and assist in the eradication of measurement inaccuracies, this thesis employs Mobile Depth Sensing and Motion Track-ing Devices (MDSMTDs) to assist OTs in in the process of digitally measuring the extrin-sic fall–risk factors for the provision of AE. The quintessential component in this assess-ment lies in the measurement of fittings and furniture items in the home. To digitise and aid in this process, the artefact presented in this thesis employs stereo computer–vision and camera calibration algorithms to extract edges in 3D space. It modifies the Sobel–Feldman convolution filter by reducing the magnitude response and employs the camera intrinsic parameters as a mechanism to calculate the distortion matrix for interpolation between the edges and the 3D point cloud. Further Augmented Reality User Experience (AR-UX) facets are provided to digitise current state of the art clinical guidance and over-lay its instructions onto the real world (i.e., 3D space). Empirical mixed methods assessment revealed that in terms of accuracy, the arte-fact exhibited enhanced performance gains over current paper–based guidance. In terms of accuracy consistency, the artefact can rectify measurement consistency inaccuracies, but there are still a wide range of factors that can influence the integrity of the point-cloud in respect of the device’s point-of-view, holding positions and measurement speed. To this end, OTs usability, and adoption preferences materialise in favour of the artefact. In conclusion, this thesis demonstrates that MDSMTDs are a promising alterna-tive to existing paper–based measurement practices as OTs appear to prefer the digital–based system and that they can take measurements more efficiently and accurately