Toward Fully Automated Robotic Platform for Remote Auscultation

Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an aging population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs. The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, light detection and ranging (LiDAR) camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camera-based multi-way registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive mechanism. Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm.Comment: 8 pages, 11 figure

Confidence-Driven Control of an Ultrasound Probe: Target-Specific Acoustic Window Optimization

International audienceWe propose a control framework to optimize the quality of robotic ultrasound imaging while tracking an anatomical target. We use a multitask approach to control the in-plane motion of a convex probe mounted on the end-effector of a robotic arm, based not only on the position of the target in the image, but also on features extracted from an ultrasound confidence map. The resulting control law therefore guarantees a good image quality, while keeping the target aligned with the central ultrasound scan-line. Potential applications of the proposed approach are, for example, teleoperated ultrasound examination, motion compensation for ultrasound-guided interventions , or automatic ultrasound acquisition. We demonstrate our approach with experiments on an ultrasound examination training phantom in motion

Técnicas imagiológicas para tele-ecografia assistida por robô

O objectivo deste trabalho consiste na reconstrução tridimensional (3D) de objectos a partir de um conjunto de imagens ecográficas obtidas a partir de movimentos de rotação de uma sonda ecográfica, acoplada na extremidade de um robô manipulador. A cada imagem adquirida foi extraída a sua matriz de transformação respectiva que dá a informação da sua posição e orientação relativamente à base do robô. O objecto de estudo foi um balão convencional em forma de coração, cheio de água que por sua vez foi mergulhado em água. Como estudo inicial, foi tida em conta informação acerca da manipulação robótica e controlo da sonda ecográfica no seu espaço de tarefa. Seguidamente, foram abordadas temáticas de processamento de imagem médica ecográfica, nomeadamente ecocardiográfica, e assim o levantamento das características e problemáticas que esse tipo de imagem implicava. Foram estudados vários métodos de segmentação de imagem bidimensional de ecocardiografia, entre eles, o método de level set de Chan e Vese e o método de Kass e Terzopoulos baseado em modelos de snakes. O modelo de Chan e Vese não foi completamente satisfatório para a segmentação das cavidades cardíacas, não fornecendo cavidades cardíacas fechadas. Por sua vez, o modelo snake funcionou muito bem na segmentação das imagens ecográficas do balão, induzindo uma reconstrução 3D offline de uma estrutura. Este trabalho passou também por um abrangente estudo científico que levou à implementação e simulação de um algoritmo de reconstrução 3D offline em linguagem Matlab2009b, fazendo uso das posições dos contornos obtidos da segmentação de Kass e Terzopoulos. Foram ainda realizados vários testes experimentais que validaram o método de reconstrução 3D proposto. Estes resultados abrem caminho para uma reconstrução 3D em tempo real de imagens ecográficas, utilizando um sistema robótic

Towards the development of safe, collaborative robotic freehand ultrasound

The use of robotics in medicine is of growing importance for modern health services, as robotic systems have the capacity to improve upon human tasks, thereby enhancing the treatment ability of a healthcare provider. In the medical sector, ultrasound imaging is an inexpensive approach without the high radiation emissions often associated with other modalities, especially when compared to MRI and CT imaging respectively. Over the past two decades, considerable effort has been invested into freehand ultrasound robotics research and development. However, this research has focused on the feasibility of the application, not the robotic fundamentals, such as motion control, calibration, and contextual awareness. Instead, much of the work is concentrated on custom designed robots, ultrasound image generation and visual servoing, or teleoperation. Research based on these topics often suffer from important limitations that impede their use in an adaptable, scalable, and real-world manner. Particularly, while custom robots may be designed for a specific application, commercial collaborative robots are a more robust and economical solution. Otherwise, various robotic ultrasound studies have shown the feasibility of using basic force control, but rarely explore controller tuning in the context of patient safety and deformable skin in an unstructured environment. Moreover, many studies evaluate novel visual servoing approaches, but do not consider the practicality of relying on external measurement devices for motion control. These studies neglect the importance of robot accuracy and calibration, which allow a system to safely navigate its environment while reducing the imaging errors associated with positioning. Hence, while the feasibility of robotic ultrasound has been the focal point in previous studies, there is a lack of attention to what occurs between system design and image output. This thesis addresses limitations of the current literature through three distinct contributions. Given the force-controlled nature of an ultrasound robot, the first contribution presents a closed-loop calibration approach using impedance control and low-cost equipment. Accuracy is a fundamental requirement for high-quality ultrasound image generation and targeting. This is especially true when following a specified path along a patient or synthesizing 2D slices into a 3D ultrasound image. However, even though most industrial robots are inherently precise, they are not necessarily accurate. While robot calibration itself has been extensively studied, many of the approaches rely on expensive and highly delicate equipment. Experimental testing showed that this method is comparable in quality to traditional calibration using a laser tracker. As demonstrated through an experimental study and validated with a laser tracker, the absolute accuracy of a collaborative robot was improved to a maximum error of 0.990mm, representing a 58.4% improvement when compared to the nominal model. The second contribution explores collisions and contact events, as they are a natural by-product of applications involving physical human-robot interaction (pHRI) in unstructured environments. Robot-assisted medical ultrasound is an example of a task where simply stopping the robot upon contact detection may not be an appropriate reaction strategy. Thus, the robot should have an awareness of body contact location to properly plan force-controlled trajectories along the human body using the imaging probe. This is especially true for remote ultrasound systems where safety and manipulability are important elements to consider when operating a remote medical system through a communication network. A framework is proposed for robot contact classification using the built-in sensor data of a collaborative robot. Unlike previous studies, this classification does not discern between intended vs. unintended contact scenarios, but rather classifies what was involved in the contact event. The classifier can discern different ISO/TS 15066:2016 specific body areas along a human-model leg with 89.37% accuracy. Altogether, this contact distinction framework allows for more complex reaction strategies and tailored robot behaviour during pHRI. Lastly, given that the success of an ultrasound task depends on the capability of the robot system to handle pHRI, pure motion control is insufficient. Force control techniques are necessary to achieve effective and adaptable behaviour of a robotic system in the unstructured ultrasound environment while also ensuring safe pHRI. While force control does not require explicit knowledge of the environment, to achieve an acceptable dynamic behaviour, the control parameters must be tuned. The third contribution proposes a simple and effective online tuning framework for force-based robotic freehand ultrasound motion control. Within the context of medical ultrasound, different human body locations have a different stiffness and will require unique tunings. Through real-world experiments with a collaborative robot, the framework tuned motion control for optimal and safe trajectories along a human leg phantom. The optimization process was able to successfully reduce the mean absolute error (MAE) of the motion contact force to 0.537N through the evolution of eight motion control parameters. Furthermore, contextual awareness through motion classification can offer a framework for pHRI optimization and safety through predictive motion behaviour with a future goal of autonomous pHRI. As such, a classification pipeline, trained using the tuning process motion data, was able to reliably classify the future force tracking quality of a motion session with an accuracy of 91.82 %

CO-ROBOTIC ULTRASOUND IMAGING: A COOPERATIVE FORCE CONTROL APPROACH

Ultrasound (US) imaging remains one of the most commonly used imaging modalities in medical practice due to its low cost and safety. However, 63-91% of ultrasonographers develop musculoskeletal disorders due to the effort required to perform imaging tasks. Robotic ultrasound (RUS), the application of robotic systems to assist ultrasonographers in ultrasound scanning procedures, has been proposed in literature and recently deployed in clinical settings using limited degree-of-freedom (DOF) systems. An example of this includes breast-scanning systems, which allow one-DOF translation of a large ultrasound array in order to capture patients’ breast scans and minimize sonographer effort while preserving a desired clinical outcome. Recently, the robotic industry has evolved to provide light-weight, compact, accurate, and cost-effective manipulators. We leverage this new reality in able to provide ultrasonographers with a full 6-DOF system that provides force assistance to facilitate US image acquisition. Admittance robot control allows for smooth human-machine interaction in a desired task. In the case of RUS, force control is capable of assisting sonographers in facilitating and even improving the imaging results of typical procedures. We propose a new system setup for collaborative force control in US applications. This setup consists of the 6-DOF UR5 industrial robot, and a 6-axes force sensor attached to the robot tooltip, which in turn has an US probe attached to it through a custom-designed probe attachment mechanism. Additionally, an independent one-axis load cell is placed inside this attachment device and used to measure the contact force between the probe and the patient’s anatomy in real time and independent of any other forces. As the sonographer guides the US probe, the robot collaborates with the hand motions, following the path of the user. When imaging, the robot can offer assistance to the sonographer by augmenting the forces applied by him or her, thereby lessening the physical effort required as well as the resulting strain. Additional benefits include force and velocity limiting for patient safety and robot motion constraints for particular imaging tasks. Initial results of a conducted user study show the feasibility of implementing the presented robot-assisted system in a clinical setting

Improving access to ultrasound imaging in northern, remote communities

Access to healthcare services—including access to medical imaging—is an important determinant of health outcomes. This thesis aims to improve understanding of and address gaps in access to ultrasound imaging for patients in northern, remote communities, and advance a novel ultrasound technology with the ultimate goal of improving patient care and health outcomes. This thesis first brings greater understanding of patients’ perceptions of access and factors which shape access to ultrasound imaging in northern, remote communities in Saskatchewan, Canada. A qualitative study was performed using interpretive description as a methodological approach and a multi-dimensional conceptualization of access to care as a theoretical framework. The study identified barriers which patients in northern, remote communities face in accessing ultrasound imaging, and demonstrated that geographic remoteness from imaging facilities was a central barrier. To determine whether disparities in access to ultrasound imaging resulted in disparities in utilization of ultrasound services, two population-based studies assessed the association between sociodemographic and geographic factors and obstetrical and non-obstetrical ultrasound utilization in Saskatchewan. In the first study investigating obstetrical ultrasound utilization, multivariate logistic regression analysis demonstrated that women living in rural areas, remote areas, and low income neighbourhoods, as well as status First Nations women, were less likely to have a second trimester ultrasound, an important aspect of prenatal care. In a second study investigating non-obstetrical ultrasound utilization across the entire provincial population, multivariate Poisson regression analysis similarly demonstrated lower rates of non-obstetrical ultrasound utilization among individuals living in rural and remote areas, individuals residing in low income neighbourhoods, and status First Nations persons. To address the barriers which patients in northern, remote communities face in accessing ultrasound imaging and to minimize disparities in ultrasound imaging utilization as identified in previous studies in this thesis, telerobotic ultrasound technology was investigated as a solution to improve access to ultrasound imaging. Using this technology, radiologists and sonographers could remotely manipulate an ultrasound probe via a robotic arm, thereby remotely performing an ultrasound exam while patients remained in their home community. A clinical trial comparing conventional and telerobotic ultrasound approaches was undertaken, validating this technology for obstetrical ultrasound imaging. To determine the feasibility of using telerobotic technology to establish an ultrasound service delivery model to remotely provide diagnostic ultrasound exams in underserved communities, pilot telerobotic ultrasound clinics were developed in three northern, remote communities. Telerobotic ultrasound exams were sufficient for diagnosis in the majority of cases, minimizing travel or reducing wait times for these patients. This technology was subsequently evaluated during a COVID-19 outbreak in northern Saskatchewan, demonstrating the potential of this technology to provide critical ultrasound services to an underserved northern population and minimize health inequities during the COVID-19 pandemic. An economic evaluation was performed to compare a service delivery model using telerobotic ultrasound technology to alternative service delivery models. Telerobotic ultrasound combined with an itinerant sonographer service was found to be the lowest cost option from both a publicly funded healthcare payer perspective and a societal perspective for many northern, remote communities. This thesis provides key insights for health system leaders seeking improved understanding and novel solutions to improve access to ultrasound imaging in northern, remote communities. Findings suggest that telerobotic ultrasound is a viable solution to improve access to ultrasound imaging and reduce costs associated with ultrasound service delivery. Evidence in this thesis may be used to help improve ultrasound services and health equity for patients in underserved northern, remote communities. Continued respectful collaboration with northern, remote, Indigenous peoples and communities will be a critical aspect to ensure that ultrasound services meet community needs