Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.Comment: Accepted by Medical Image Analysi

Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound

We present the first system that provides real-time probe movement guidance for acquiring standard planes in routine freehand obstetric ultrasound scanning. Such a system can contribute to the worldwide deployment of obstetric ultrasound scanning by lowering the required level of operator expertise. The system employs an artificial neural network that receives the ultrasound video signal and the motion signal of an inertial measurement unit (IMU) that is attached to the probe, and predicts a guidance signal. The network termed US-GuideNet predicts either the movement towards the standard plane position (goal prediction), or the next movement that an expert sonographer would perform (action prediction). While existing models for other ultrasound applications are trained with simulations or phantoms, we train our model with real-world ultrasound video and probe motion data from 464 routine clinical scans by 17 accredited sonographers. Evaluations for 3 standard plane types show that the model provides a useful guidance signal with an accuracy of 88.8% for goal prediction and 90.9% for action prediction.Comment: Accepted at the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2020

Recent advances in robot-assisted echography: Combining perception, control and cognition

Echography imaging is an important technique frequently used in medical diagnostics due to low-cost, non-ionising characteristics, and pragmatic convenience. Due to the shortage of skilful technicians and injuries of physicians sustained from diagnosing several patients, robot-assisted echography (RAE) system is gaining great attention in recent decades. A thorough study of the recent research advances in the field of perception, control and cognition techniques used in RAE systems is presented in this study. This survey introduces the representative system structure, applications and projects, and products. Challenges and key technological issues faced by the traditional RAE system and how the current artificial intelligence and cobots attempt to overcome these issues are summarised. Furthermore, significant future research directions in this field have been identified by this study as cognitive computing, operational skills transfer, and commercially feasible system design

Implementation of safe human robot collaboration for ultrasound guided radiation therapy

This thesis shows that safe human-robot-interaction and Human Robot Collaboration is possible for Ultrasound (US) guided radiotherapy. Via the chosen methodology, all components (US, optical room monitoring and robot) could be linked and integrated and realized in a realistic clinical workflow. US guided radiotherapy offers a complement and alternative to existing image-guided therapy approaches. The real-time capability of US and high soft tissue contrast allow target structures to be tracked and radiation delivery to be modulated. However, Ultrasound guided radiation therapy (USgRT) is not yet clinically established but is still under development, as reliable and safe methods of image acquisition are not yet available. In particular, the loss of contact of the US probe to the patient surface poses a problem for patient movements such as breathing. For this purpose, a Breathing and motion compensation (BaMC) was developed in this work, which together with the safe control of a lightweight robot represents a new development for USgRT. The developed BaMC can be used to control the US probe with contact to the patient. The conducted experiments have confirmed that a steady contact with the patient surface and thus a continuous image acquisition can be ensured by the developed methodology. In addition, the image position in space can be accurately maintained in the submillimeter range. The BaMC seamlessly integrates into a developed clinical workflow. The graphical user interfaces developed for this purpose, as well as direct haptic control with the robot, provide an easy interaction option for the clinical user. The developed autonomous positioning of the transducer represents a good example of the feasibility of the approach. With the help of the user interface, an acoustic plane can be defined and autonomously approached via the robot in a time-efficient and precise manner. The tests carried out show that this methodology is suitable for a wide range of transducer positions. Safety in a human-robot interaction task is essential and requires individually customized concepts. In this work, adequate monitoring mechanisms could be found to ensure both patient and staff safety. In collision tests it could be shown that the implemented detection measures work and that the robot moves into a safe parking position. The forces acting on the patient could thus be pushed well below the limits required by the standard. This work has demonstrated the first important steps towards safe robot-assisted ultrasound imaging, which is not only applicable to USgRT. The developed interfaces provide the basis for further investigations in this field, especially in the area of image recognition, for example to determine the position of the target structure. With the proof of safety of the developed system, first study in human can now follow

Recent Advances in Artificial Intelligence-Assisted Ultrasound Scanning

Funded by the Spanish Ministry of Economic Affairs and Digital Transformation (Project MIA.2021.M02.0005 TARTAGLIA, from the Recovery, Resilience, and Transformation Plan financed by the European Union through Next Generation EU funds). TARTAGLIA takes place under the R&D Missions in Artificial Intelligence program, which is part of the Spain Digital 2025 Agenda and the Spanish National Artificial Intelligence Strategy.Ultrasound (US) is a flexible imaging modality used globally as a first-line medical exam procedure in many different clinical cases. It benefits from the continued evolution of ultrasonic technologies and a well-established US-based digital health system. Nevertheless, its diagnostic performance still presents challenges due to the inherent characteristics of US imaging, such as manual operation and significant operator dependence. Artificial intelligence (AI) has proven to recognize complicated scan patterns and provide quantitative assessments for imaging data. Therefore, AI technology has the potential to help physicians get more accurate and repeatable outcomes in the US. In this article, we review the recent advances in AI-assisted US scanning. We have identified the main areas where AI is being used to facilitate US scanning, such as standard plane recognition and organ identification, the extraction of standard clinical planes from 3D US volumes, and the scanning guidance of US acquisitions performed by humans or robots. In general, the lack of standardization and reference datasets in this field makes it difficult to perform comparative studies among the different proposed methods. More open-access repositories of large US datasets with detailed information about the acquisition are needed to facilitate the development of this very active research field, which is expected to have a very positive impact on US imaging.Depto. de Estructura de la Materia, Física Térmica y ElectrónicaFac. de Ciencias FísicasTRUEMinistry of Economic Affairs and Digital Transformation from the Recovery, Resilience, and Transformation PlanNext Generation EU fundspu

Improving access to ultrasound imaging in northern, remote communities

Access to healthcare services—including access to medical imaging—is an important determinant of health outcomes. This thesis aims to improve understanding of and address gaps in access to ultrasound imaging for patients in northern, remote communities, and advance a novel ultrasound technology with the ultimate goal of improving patient care and health outcomes. This thesis first brings greater understanding of patients’ perceptions of access and factors which shape access to ultrasound imaging in northern, remote communities in Saskatchewan, Canada. A qualitative study was performed using interpretive description as a methodological approach and a multi-dimensional conceptualization of access to care as a theoretical framework. The study identified barriers which patients in northern, remote communities face in accessing ultrasound imaging, and demonstrated that geographic remoteness from imaging facilities was a central barrier. To determine whether disparities in access to ultrasound imaging resulted in disparities in utilization of ultrasound services, two population-based studies assessed the association between sociodemographic and geographic factors and obstetrical and non-obstetrical ultrasound utilization in Saskatchewan. In the first study investigating obstetrical ultrasound utilization, multivariate logistic regression analysis demonstrated that women living in rural areas, remote areas, and low income neighbourhoods, as well as status First Nations women, were less likely to have a second trimester ultrasound, an important aspect of prenatal care. In a second study investigating non-obstetrical ultrasound utilization across the entire provincial population, multivariate Poisson regression analysis similarly demonstrated lower rates of non-obstetrical ultrasound utilization among individuals living in rural and remote areas, individuals residing in low income neighbourhoods, and status First Nations persons. To address the barriers which patients in northern, remote communities face in accessing ultrasound imaging and to minimize disparities in ultrasound imaging utilization as identified in previous studies in this thesis, telerobotic ultrasound technology was investigated as a solution to improve access to ultrasound imaging. Using this technology, radiologists and sonographers could remotely manipulate an ultrasound probe via a robotic arm, thereby remotely performing an ultrasound exam while patients remained in their home community. A clinical trial comparing conventional and telerobotic ultrasound approaches was undertaken, validating this technology for obstetrical ultrasound imaging. To determine the feasibility of using telerobotic technology to establish an ultrasound service delivery model to remotely provide diagnostic ultrasound exams in underserved communities, pilot telerobotic ultrasound clinics were developed in three northern, remote communities. Telerobotic ultrasound exams were sufficient for diagnosis in the majority of cases, minimizing travel or reducing wait times for these patients. This technology was subsequently evaluated during a COVID-19 outbreak in northern Saskatchewan, demonstrating the potential of this technology to provide critical ultrasound services to an underserved northern population and minimize health inequities during the COVID-19 pandemic. An economic evaluation was performed to compare a service delivery model using telerobotic ultrasound technology to alternative service delivery models. Telerobotic ultrasound combined with an itinerant sonographer service was found to be the lowest cost option from both a publicly funded healthcare payer perspective and a societal perspective for many northern, remote communities. This thesis provides key insights for health system leaders seeking improved understanding and novel solutions to improve access to ultrasound imaging in northern, remote communities. Findings suggest that telerobotic ultrasound is a viable solution to improve access to ultrasound imaging and reduce costs associated with ultrasound service delivery. Evidence in this thesis may be used to help improve ultrasound services and health equity for patients in underserved northern, remote communities. Continued respectful collaboration with northern, remote, Indigenous peoples and communities will be a critical aspect to ensure that ultrasound services meet community needs