Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human–robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

Artificial Intelligence for Emerging Technology in Surgery: Systematic Review and Validation

Surgery is a high-risk procedure of therapy and is associated to post trauma complications of longer hospital stay, estimated blood loss and long duration of surgeries. Reports have suggested that over 2.5% patients die during and post operation. This paper is aimed at systematic review of previous research on artificial intelligence (AI) in surgery, analyzing their results with suitable software to validate their research by obtaining same or contrary results. Six published research articles have been reviewed across three continents. These articles have been re-validated using software including SPSS and MedCalc to obtain the statistical features such as the mean, standard deviation, significant level, and standard error. From the significant values, the experiments are then classified according to the null (p0.05) hypotheses. The results obtained from the analysis have suggested significant difference in operating time, docking time, staging time, and estimated blood loss but show no significant difference in length of hospital stay, recovery time and lymph nodes harvested between robotic assisted surgery using AI and normal conventional surgery. From the evaluations, this research suggests that AI-assisted surgery improves over the conventional surgery as safer and more efficient system of surgery with minimal or no complications

Augmented Reality Assistance for Surgical Interventions using Optical See-Through Head-Mounted Displays

Augmented Reality (AR) offers an interactive user experience via enhancing the real world environment with computer-generated visual cues and other perceptual information. It has been applied to different applications, e.g. manufacturing, entertainment and healthcare, through different AR media. An Optical See-Through Head-Mounted Display (OST-HMD) is a specialized hardware for AR, where the computer-generated graphics can be overlaid directly onto the user's normal vision via optical combiners. Using OST-HMD for surgical intervention has many potential perceptual advantages. As a novel concept, many technical and clinical challenges exist for OST-HMD-based AR to be clinically useful, which motivates the work presented in this thesis. From the technical aspects, we first investigate the display calibration of OST-HMD, which is an indispensable procedure to create accurate AR overlay. We propose various methods to reduce the user-related error, improve robustness of the calibration, and remodel the calibration as a 3D-3D registration problem. Secondly, we devise methods and develop hardware prototype to increase the user's visual acuity of both real and virtual content through OST-HMD, to aid them in tasks that require high visual acuity, e.g. dental procedures. Thirdly, we investigate the occlusion caused by the OST-HMD hardware, which limits the user's peripheral vision. We propose to use alternative indicators to remind the user of unattended environment motion. From the clinical perspective, we identified many clinical use cases where OST-HMD-based AR is potentially helpful, developed applications integrated with current clinical systems, and conducted proof-of-concept evaluations. We first present a "virtual monitor'' for image-guided surgery. It can replace real radiology monitors in the operating room with easier user control and more flexibility in positioning. We evaluated the "virtual monitor'' for simulated percutaneous spine procedures. Secondly, we developed ARssist, an application for the bedside assistant in robotic surgery. The assistant can see the robotic instruments and endoscope within the patient body with ARssist. We evaluated the efficiency, safety and ergonomics of the assistant during two typical tasks: instrument insertion and manipulation. The performance for inexperienced users is significantly improved with ARssist, and for experienced users, the system significantly enhanced their confidence level. Lastly, we developed ARAMIS, which utilizes real-time 3D reconstruction and visualization to aid the laparoscopic surgeon. It demonstrates the concept of "X-ray see-through'' surgery. Our preliminary evaluation validated the application via a peg transfer task, and also showed significant improvement in hand-eye coordination. Overall, we have demonstrated that OST-HMD based AR application provides ergonomic improvements, e.g. hand-eye coordination. In challenging situations or for novice users, the improvements in ergonomic factors lead to improvement in task performance. With continuous effort as a community, optical see-through augmented reality technology will be a useful interventional aid in the near future

An evaluation of a checklist in Musculoskeletal (MSK) radiographic image interpretation when using Artificial Intelligence (AI)

Background: AI is being used increasingly in image interpretation tasks. There are challenges for its optimal use in reporting environments. Human reliance on technology and bias can cause decision errors. Trust issues exist amongst radiologists and radiographers in both over-reliance (automation bias) and reluctance in AI use for decision support. A checklist, used with the AI to mitigate against such biases, may optimise the use of AI technologies and promote good decision hygiene. Method: A checklist, to be used in image interpretation with AI assistance, was developed. Participants interpreted 20 examinations with AI assistance and then re- interpreted the 20 examinations with AI and a checklist. The MSK images were presented to radiographers as patient examinations to replicate the image interpretation task in clinical practice. Image diagnosis and confidence levels on the diagnosis provided were collected following each interpretation. The participant perception of the use of the checklist was investigated via a questionnaire.Results: Data collection and analysis are underway and will be completed at the European Congress of Radiology in Vienna, March 2023. The impact of the use of a checklist in image interpretation with AI will be evaluated. Changes in accuracy and confidence will be investigated and results will be presented. Participant feedback will be analysed to determine perceptions and impact of the checklist also. Conclusion: A novel checklist has been developed to aid the interpretation of images when using AI. The checklist has been tested for its use in assisting radiographers in MSK image interpretation when using AI.<br/

Low Back Pain (LBP)

Low back pain (LBP) is a major public health problem, being the most commonly reported musculoskeletal disorder (MSD) and the leading cause of compromised quality of life and work absenteeism. Indeed, LBP is the leading worldwide cause of years lost to disability, and its burden is growing alongside the increasing and aging population. The etiology, pathogenesis, and occupational risk factors of LBP are still not fully understood. It is crucial to give a stronger focus to reducing the consequences of LBP, as well as preventing its onset. Primary prevention at the occupational level remains important for highly exposed groups. Therefore, it is essential to identify which treatment options and workplace-based intervention strategies are effective in increasing participation at work and encouraging early return-to-work to reduce the consequences of LBP. The present Special Issue offers a unique opportunity to update many of the recent advances and perspectives of this health problem. A number of topics will be covered in order to attract high-quality research papers, including the following major areas: prevalence and epidemiological data, etiology, prevention, assessment and treatment approaches, and health promotion strategies for LBP. We have received a wide range of submissions, including research on the physical, psychosocial, environmental, and occupational perspectives, also focused on workplace interventions

Contemporary Topics in Patient Safety

As healthcare systems continue to evolve, it is clear that providing safe, high-quality care to patients is an extremely complex process. Ranging from multi-disciplinary teams to bedside care, virtually every aspect of the patient-care experience provides us with an opportunity for doing things better, from improving efficiency, safety, and overall outcomes to reducing costs and promoting team synergy. This book, the fifth in our patient safety series collection, consists of chapters that help explore key concepts related to both the safety and quality of care. In a departure from the vignette-driven format of our earlier books, this installment gravitates toward discussing frameworks, theoretical considerations, team-centric approaches, and a variety of other concepts that are critical to both our understanding and the implementation of safer and better-performing health systems. We also feel that the knowledge presented herein increasingly applies across the world, especially as global health systems evolve and mature over time. It is our goal to improve the recognition of potential opportunities that will highlight various aspects of the delivery of healthcare and thus contribute to better patient experiences, with safety at the forefront. Topics covered in this volume, as well as the previous volumes, highlight the critical importance of identifying and addressing opportunities for improvement, not as one-time events, but rather as continuous, hardwired institutional processes