An Inertial-Optical Tracking System for Quantitative, Freehand, 3D Ultrasound

Three dimensional (3D) ultrasound has become an increasingly popular medical imaging tool over the last decade. It offers significant advantages over Two Dimensional (2D) ultrasound, such as improved accuracy, the ability to display image planes that are physically impossible with 2D ultrasound, and reduced dependence on the skill of the sonographer. Among 3D medical imaging techniques, ultrasound is the only one portable enough to be used by first responders, on the battlefield, and in rural areas. There are three basic methods of acquiring 3D ultrasound images. In the first method, a 2D array transducer is used to capture a 3D volume directly, using electronic beam steering. This method is mainly used for echocardiography. In the second method, a linear array transducer is mechanically actuated, giving a slower and less expensive alternative to the 2D array. The third method uses a linear array transducer that is moved by hand. This method is known as freehand 3D ultrasound. Whether using a 2D array or a mechanically actuated linear array transducer, the position and orientation of each image is known ahead of time. This is not the case for freehand scanning. To reconstruct a 3D volume from a series of 2D ultrasound images, assumptions must be made about the position and orientation of each image, or a mechanism for detecting the position and orientation of each image must be employed. The most widely used method for freehand 3D imaging relies on the assumption that the probe moves along a straight path with constant orientation and speed. This method requires considerable skill on the part of the sonographer. Another technique uses features within the images themselves to form an estimate of each image\u27s relative location. However, these techniques are not well accepted for diagnostic use because they are not always reliable. The final method for acquiring position and orientation information is to use a six Degree-of-Freedom (6 DoF) tracking system. Commercially available 6 DoF tracking systems use magnetic fields, ultrasonic ranging, or optical tracking to measure the position and orientation of a target. Although accurate, all of these systems have fundamental limitations in that they are relatively expensive and they all require sensors or transmitters to be placed in fixed locations to provide a fixed frame of reference. The goal of the work presented here is to create a probe tracking system for freehand 3D ultrasound that does not rely on any fixed frame of reference. This system tracks the ultrasound probe using only sensors integrated into the probe itself. The advantages of such a system are that it requires no setup before it can be used, it is more portable because no extra equipment is required, it is immune from environmental interference, and it is less expensive than external tracking systems. An ideal tracking system for freehand 3D ultrasound would track in all 6 DoF. However, current sensor technology limits this system to five. Linear transducer motion along the skin surface is tracked optically and transducer orientation is tracked using MEMS gyroscopes. An optical tracking system was developed around an optical mouse sensor to provide linear position information by tracking the skin surface. Two versions were evaluated. One included an optical fiber bundle and the other did not. The purpose of the optical fiber is to allow the system to integrate more easily into existing probes by allowing the sensor and electronics to be mounted away from the scanning end of the probe. Each version was optimized to track features on the skin surface while providing adequate Depth Of Field (DOF) to accept variation in the height of the skin surface. Orientation information is acquired using a 3 axis MEMS gyroscope. The sensor was thoroughly characterized to quantify performance in terms of accuracy and drift. This data provided a basis for estimating the achievable 3D reconstruction accuracy of the complete system. Electrical and mechanical components were designed to attach the sensor to the ultrasound probe in such a way as to simulate its being embedded in the probe itself. An embedded system was developed to perform the processing necessary to translate the sensor data into probe position and orientation estimates in real time. The system utilizes a Microblaze soft core microprocessor and a set of peripheral devices implemented in a Xilinx Spartan 3E field programmable gate array. The Xilinx Microkernel real time operating system performs essential system management tasks and provides a stable software platform for implementation of the inertial tracking algorithm. Stradwin 3D ultrasound software was used to provide a user interface and perform the actual 3D volume reconstruction. Stradwin retrieves 2D ultrasound images from the Terason t3000 portable ultrasound system and communicates with the tracking system to gather position and orientation data. The 3D reconstruction is generated and displayed on the screen of the PC in real time. Stradwin also provides essential system features such as storage and retrieval of data, 3D data interaction, reslicing, manual 3D segmentation, and volume calculation for segmented regions. The 3D reconstruction performance of the system was evaluated by freehand scanning a cylindrical inclusion in a CIRS model 044 ultrasound phantom. Five different motion profiles were used and each profile was repeated 10 times. This entire test regimen was performed twice, once with the optical tracking system using the optical fiber bundle, and once with the optical tracking system without the optical fiber bundle. 3D reconstructions were performed with and without the position and orientation data to provide a basis for comparison. Volume error and surface error were used as the performance metrics. Volume error ranged from 1.3% to 5.3% with tracking information versus 15.6% to 21.9% without for the version of the system without the optical fiber bundle. Volume error ranged from 3.7% to 7.6% with tracking information versus 8.7% to 13.7% without for the version of the system with the optical fiber bundle. Surface error ranged from 0.319 mm RMS to 0.462 mm RMS with tracking information versus 0.678 mm RMS to 1.261 mm RMS without for the version of the system without the optical fiber bundle. Surface error ranged from 0.326 mm RMS to 0.774 mm RMS with tracking information versus 0.538 mm RMS to 1.657 mm RMS without for the version of the system with the optical fiber bundle. The prototype tracking system successfully demonstrated that accurate 3D ultrasound volumes can be generated from 2D freehand data using only sensors integrated into the ultrasound probe. One serious shortcoming of this system is that it only tracks 5 of the 6 degrees of freedom required to perform complete 3D reconstructions. The optical system provides information about linear movement but because it tracks a surface, it cannot measure vertical displacement. Overcoming this limitation is the most obvious candidate for future research using this system. The overall tracking platform, meaning the embedded tracking computer and the PC software, developed and integrated in this work, is ready to take advantage of vertical displacement data, should a method be developed for sensing it

Inexpensive 1024-Channel 3D Telesonography System on FPGA

Volumetric ultrasound (US) is a very promising development of medical US imaging. An under-exploited advantage of volumetric US is the mitigation of the strict probe positioning constrains necessary to acquire 2D scans, potentially allowing the decoupling of US image acquisition and diagnosis. However, today’s 3D US systems are large and beset by high power and cost requirements, making them only available in well-equipped hospitals. In this study, we propose the first telesonography-capable medical imaging system that supports up to 1024 channels, on par with the state of the art. As a first embodiment, we have implemented our design in a single development FPGA board of 26.7cm×14cm×0.16cm, with an estimated power consumption of 6.1 W. Moreover, we have equipped our platform with an automatic positioning module to help any operator defining the scan location, hence allowing for better remote diagnosis. Our design supports two types of data inputs: real-time via an optical connection and offline over Ethernet. The reconstructed images can be visualized on an HDMI screen. The estimated cost of the proposed prototype materials is less than 4000e

Live Demonstration: Inexpensive 1024-Channel 3D Telesonography System on FPGA

Volumetric ultrasound (US) is a very promising development of medical US imaging. An under-exploited advantage of volumetric US is the mitigation of the strict probe positioning constrains necessary to acquire 2D scans, potentially allowing the decoupling of US image acquisition and diagnosis. However, today’s 3D US systems are large and beset by high power and cost requirements, making them only available in well-equipped hospitals. In this work [1], [2], we propose the first telesonography-capable medical imaging system that supports up to 1024 channels, on par with the state of the art. As a first embodiment, we have implemented our design in a single development FPGA board of 26.7cm×14cm×0.16cm, with an estimated power consumption of 6.1 W. The imager exploits a highly scalable architecture which can be either downscaled for 2D imaging, or further upscaled on a larger FPGA. Moreover, our design supports two types of data inputs: real-time via an optical connection and offline over Ethernet. The reconstructed images can be visualized on an HDMI screen. The estimated cost of the proposed prototype materials is less than 4000e. TABLE I shows the resources utilization of the current design and the extrapolated utilization in case of further upscaling on a larger FPGA

Apodization Scheme for Hardware-Efficient Beamformer

3D ultrasound is an emerging diagnostic technique that extends standard ultrasound imaging by capturing volumes, instead of planes. This brings completely new diagnostic opportunities, among which the possibility of disjoining image acquisition and analysis, thus enabling remote diagnosis, which would bring obvious medical and economic benefits. Unfortunately, 3D ultrasound is several orders of magnitude more computationally complex than 2D imaging. Therefore, algorithmic improvements to simplify the processing are mandatory in order to conceive cheap, portable, low-power imagers. The kernel of the 3D imaging process, called beamforming, consists essentially of computing delay and apodization profiles. We have previously devised an approximation of the delay calculation stage, which dramatically reduces hardware complexity. Unfortunately, this approximation introduces an intrinsic degree of inaccuracy that can be characterized as added image noise. In this paper, we identify an efficient approximated approach to the calculation of apodization profiles, that additionally minimizes (-76%) the error introduced during delay calculation. Together, these two techniques enable an efficient computation of 3D ultrasound images

Towards a Novel Ultrasound System Based on Low-Frequency Feature Extraction From a Fully-Printed Flexible Transducer

Ultrasound is a key technology in healthcare, and it is being explored for non-invasive, wearable, continuous monitoring of vital signs. However, its widespread adoption in this scenario is still hindered by the size, complexity, and power consumption of current devices. Moreover, such an application demands adaptability to human anatomy, which is hard to achieve with current transducer technology. This paper presents a novel ultrasound system prototype based on a fully printed, lead-free, and flexible polymer ultrasound transducer, whose bending radius promises good adaptability to the human anatomy. Our application scenario focuses on continuous blood flow monitoring. We implemented a hardware envelope filter to efficiently transpose high-frequency ultrasound signals to a lower-frequency spectrum. This reduces computational and power demands with little to no degradation in the task proposed for this work. We validated our method on a setup that mimics human blood flow by using a flow phantom and a peristaltic pump simulating 3 different heartbeat rhythms: 60, 90 and 120 beats per minute. Our ultrasound setup reconstructs peristaltic pump frequencies with errors of less than 0.05 Hz (3 bpm) from the set pump frequency, both for the raw echo and the enveloped echo. The analog pre-processing showed a promising reduction of signal bandwidth of more than 6x: pulse-echo signals of transducers excited at 12.5 MHz were reduced to about 2 MHz. Thus, allowing consumer MCUs to acquire and elaborate signals within mW-power range in an inexpensive fashion.Comment: 5 pages, 2 tables, 3 figures, Accepted at IEEE BioCAS 202

Vision Sensors and Edge Detection

Vision Sensors and Edge Detection book reflects a selection of recent developments within the area of vision sensors and edge detection. There are two sections in this book. The first section presents vision sensors with applications to panoramic vision sensors, wireless vision sensors, and automated vision sensor inspection, and the second one shows image processing techniques, such as, image measurements, image transformations, filtering, and parallel computing

Row-Column Capacitive Micromachined Ultrasonic Transducers for Medical Imaging

Ultrasound imaging plays an important role in modern medical diagnosis. Recent progress in real-time 3-D ultrasound imaging can offer critical information such as the accurate estimation of organ, cyst, or tumour volumes. However, compared to conventional 2-D ultrasound imaging, the large amount of data and circuit complexity found in 3-D ultrasound imaging results in very expensive systems. Therefore, a simplification scheme for 3-D ultrasound imaging technology is needed for a more wide-spread use and to advance clinical development of volumetric ultrasound. Row-column addressing 2-D array is one particular simplification scheme that requires only N + N addressing lines to activate each element in an N × N array. As a result, the fabrication, circuit, and processing complexity dramatically decrease. Capacitive micromachined ultrasonic transducer (CMUT) technology was chosen to fabricate the array as it offers micro-precision fabrication and a wide bandwidth, which make it an attractive transducer technology. The objective of this thesis is to investigate and demonstrate the imaging potential of row-column CMUT arrays for RT3D imaging. First, the motivation, physics, and modelling of both CMUTs and row-column arrays are described, followed by the demonstration of a customized row-column CMUT pseudo-real-time 3-D imaging system. One particular limitation about row-column arrays discovered as part of this dissertation work is the limited field-of-view of the row-column arrays’ imaging performance. A curved row-column CMUT array was proposed to improve the field-of-view, and the resulting modelling of the acoustic field and simulated reconstructed image are presented. Furthermore, a new fabrication process was proposed to construct a curved row-column CMUT array. The resulting device was tested to demonstrate its flexibility to achieve the necessary curvature. Finally, a new wafer bonding process is introduced to tackle the next generation of RC-CMUT fabrication. Many of the new fabrication techniques reported in this work are useful for CMUT fabrication engineers. The analysis on row-column array also provides additional insights for 2-D array simplification research

Fluorescence-guided surgical system using holographic display: From phantom studies to canine patients

SIGNIFICANCE: Holographic display technology is a promising area of research that can lead to significant advancements in cancer surgery. We present the benefits of combining bioinspired multispectral imaging technology with holographic goggles for fluorescence-guided cancer surgery. Through a series of experiments with 43D-printed phantoms, small animal models of cancer, and surgeries on canine patients with head and neck cancer, we showcase the advantages of this holistic approach. AIM: The aim of our study is to demonstrate the feasibility and potential benefits of utilizing holographic display for fluorescence-guided surgery through a series of experiments involving 3D-printed phantoms and canine patients with head and neck cancer. APPROACH: We explore the integration of a bioinspired camera with a mixed reality headset to project fluorescent images as holograms onto a see-through display, and we demonstrate the potential benefits of this technology through benchtop and RESULTS: Our complete imaging and holographic display system showcased improved delineation of fluorescent targets in phantoms compared with the 2D monitor display approach and easy integration into the veterinarian surgical workflow. CONCLUSIONS: Based on our findings, it is evident that our comprehensive approach, which combines a bioinspired multispectral imaging sensor with holographic goggles, holds promise in enhancing the presentation of fluorescent information to surgeons during intraoperative scenarios while minimizing disruptions

Développement d’un cathéter multimodal visant l’étude de la plaque d’athérosclérose

Résumé L’objectif de cette thèse est de concevoir et valider un système d’imagerie par cathéter visant l’étude de la plaque d’athérosclérose. L’innovation repose dans l’intégration de plusieurs modalités d’imagerie dans un seul appareil. En effet, le système combine des techniques d’imagerie anatomique et moléculaire. Ceci permet d’obtenir de l’information riche et diversifiée en temps réel à propos de la plaque d’athérosclérose. Le système conçu permet d’effectuer simultanément l’imagerie ultrasonore intravasculaire (IVUS), l’imagerie photoacoustique intravasculaire (IVPA), ainsi que l’imagerie de fluorescence intravasculaire (NIRF). L’élastographie intravasculaire (IVE) est également possible en post-traitement. L’hypothèse de travail est que la combinaison de l’ensemble de ces techniques d’imagerie permet une étude plus détaillée que l’utilisation d’une seule modalité. L’IVUS est une technique d’imagerie morphologique largement utilisée en recherche clinique qui permet d’obtenir en temps réel des séries de coupes transversales des artères à haute résolution. La paroi artérielle peut être étudiée afin d’identifier certains types de plaque. L’IVUS est la composante principale du système d’imagerie par cathéter conçu dans le cadre de ce projet. Bien que cette technique d’imagerie permette de visualiser l’anatomie générale de l’artère, elle permet difficilement de caractériser les composantes principales d’une plaque vulnérable. Afin de complémenter l’IVUS, des techniques d’imagerie moléculaire, l’IVPA et la NIRF, ont été incorporées au système. Le but est d’étudier le développement de la plaque à un stade d’évolution plus précoce, alors qu’il y a inflammation de la paroi artérielle, mais une accumulation insuffisante de dépôts lipidiques pour être visibles en IVUS. L’imagerie moléculaire a le potentiel de mieux caractériser les plaques vulnérables, qui sont plus susceptibles de subir des complications, telle une thrombose. Une des applications potentielles est l’étude de l’effet de nouveaux médicaments, qui ne se traduit pas nécessairement par un changement anatomique perceptible en IVUS, mais plutôt par un changement au niveau moléculaire. L’IVE permet d’obtenir de l’information quantitative à propos des propriétés élastiques de la paroi artérielle. Elle vise à complémenter l’IVUS en fournissant des propriétés mécaniques de l’artère et en évaluant la vulnérabilité de la plaque d’athérosclérose. La première contribution de ce travail contient une description détaillée du système d’imagerie par cathéter qui a été conçu. Une preuve de concept est ensuite présentée en exposant des résultats sur fantômes exprimant un contraste dans toutes les modalités d’imagerie intégrées au système : l’IVUS, l’IVPA, la NIRF et l’IVE.La deuxième contribution pousse la validation du système plus loin en évaluant le potentiel de détection de la plaque d’athérosclérose in vivo chez le lapin. La combinaison de l’IVUS et de la NIRF, avec injection d’un biomarqueur, soit le vert d’indocyanine (ICG), a permis la détection de certaines plaques et a été comparée avec des techniques d’imagerie ex vivo. La performance ainsi que la reproductibilité des mesures ont été évaluées. La troisième contribution est reliée à la colocalisation des images en IVUS et en NIRF obtenues chez le lapin avec des images ex vivo volumétriques à très haute résolution. Les images ex vivo sont comparées à celles obtenues avec le cathéter, afin de mieux apprécier les capacités et les limites du système d’imagerie in vivo conçu.----------Abstract The aim of this thesis is to design and validate a catheter imaging system for the study of the atherosclerotic plaque. The innovation resides in the integration of multiple imaging modalities in a single device. Indeed, the system combines anatomical and molecular imaging techniques. This allows obtaining rich and diversified information in real time about the atherosclerotic plaque.The designed system allows simultaneously performing intravascular ultrasound imaging (IVUS), intravascular photoacoustic imaging (IVPA) and intravascular fluorescence (NIRF). Intravascular elastography (IVE) is also possible in post-processing. The hypothesis of this work is that the combination of all these imaging techniques allows a more detailed study than the usage of a single modality.IVUS is a morphological imaging technique widely used in preclinical research that allows obtaining in real time series of cross sections of arteries at a high resolution. The artery wall can be studied to identify certain types of plaque. IVUS is the main component of the catheter imaging system designed in this project. While this imaging technique allows visualizing the general anatomy of the artery, it is not well suited for characterizing the main components of a vulnerable plaque. To complement IVUS, molecular imaging techniques, IVPA and NIRF, were incorporated to the system. The goal is to study the development of the plaque at an earlier evolution stage when there is inflammation in the artery wall, but an insufficient accumulation of lipids to be visible in IVUS. Molecular imaging has the potential to better characterize vulnerable plaques, which are more prone to complications, such as thrombosis. One of the potential applications is the study of the effect of new drugs, that doesn’t always translate by an anatomical change perceptible in IVUS, but rather a change at the molecular level. IVE allows obtaining quantitative information about elastic properties of the artery wall. It aims at complementing IVUS by providing mechanical properties of the artery and by evaluating the vulnerability of atherosclerotic plaque. The first contribution to this work contains a detailed description of the designed catheter imaging system. A proof of concept is then presented by exposing results on phantoms with contrasts in all the imaging modalities integrated to the system: IVUS, IVPA, NIRF and IVE. The second contribution further validates the system by evaluating the detection potential of atherosclerotic plaque in vivo on rabbits. The IVUS and NIRF combination, with the injection of a biomarker, indocyanine green (ICG), allowed the detection of certain plaques and was compared to ex vivo imaging techniques. The performance and the reproducibility of the measures were evaluated. The third contribution is related to the colocalisation of IVUS and NIRF images obtained in rabbits with volumetric ex vivo images at a very high resolution. Ex vivo images are compared to the ones obtained with the catheter, in order to better appreciate the capabilities and the limitations of the designed in vivo imaging system