A Review on Advances in Intra-operative Imaging for Surgery and Therapy: Imagining the Operating Room of the Future

none4openZaffino, Paolo; Moccia, Sara; De Momi, Elena; Spadea, Maria FrancescaZaffino, Paolo; Moccia, Sara; De Momi, Elena; Spadea, Maria Francesc

Fusion of interventional ultrasound & X-ray

In einer immer älter werdenden Bevölkerung wird die Behandlung von strukturellen Herzkrankheiten zunehmend wichtiger. Verbesserte medizinische Bildgebung und die Einführung neuer Kathetertechnologien führten dazu, dass immer mehr herkömmliche chirurgische Eingriffe am offenen Herzen durch minimal invasive Methoden abgelöst werden. Diese modernen Interventionen müssen durch verschiedenste Bildgebungsverfahren navigiert werden. Hierzu werden hauptsächlich Röntgenfluoroskopie und transösophageale Echokardiografie (TEE) eingesetzt. Röntgen bietet eine gute Visualisierung der eingeführten Katheter, was essentiell für eine gute Navigation ist. TEE hingegen bietet die Möglichkeit der Weichteilgewebedarstellung und kann damit vor allem zur Darstellung von anatomischen Strukturen, wie z.B. Herzklappen, genutzt werden. Beide Modalitäten erzeugen Bilder in Echtzeit und werden für die erfolgreiche Durchführung minimal invasiver Herzchirurgie zwingend benötigt. Üblicherweise sind beide Systeme eigenständig und nicht miteinander verbunden. Es ist anzunehmen, dass eine Bildfusion beider Welten einen großen Vorteil für die behandelnden Operateure erzeugen kann, vor allem eine verbesserte Kommunikation im Behandlungsteam. Ebenso können sich aus der Anwendung heraus neue chirurgische Worfklows ergeben. Eine direkte Fusion beider Systeme scheint nicht möglich, da die Bilddaten eine zu unterschiedliche Charakteristik aufweisen. Daher kommt in dieser Arbeit eine indirekte Registriermethode zum Einsatz. Die TEE-Sonde ist während der Intervention ständig im Fluoroskopiebild sichtbar. Dadurch wird es möglich, die Sonde im Röntgenbild zu registrieren und daraus die 3D Position abzuleiten. Der Zusammenhang zwischen Ultraschallbild und Ultraschallsonde wird durch eine Kalibrierung bestimmt. In dieser Arbeit wurde die Methode der 2D-3D Registrierung gewählt, um die TEE Sonde auf 2D Röntgenbildern zu erkennen. Es werden verschiedene Beiträge präsentiert, welche einen herkömmlichen 2D-3D Registrieralgorithmus verbessern. Nicht nur im Bereich der Ultraschall-Röntgen-Fusion, sondern auch im Hinblick auf allgemeine Registrierprobleme. Eine eingeführte Methode ist die der planaren Parameter. Diese verbessert die Robustheit und die Registriergeschwindigkeit, vor allem während der Registrierung eines Objekts aus zwei nicht-orthogonalen Richtungen. Ein weiterer Ansatz ist der Austausch der herkömmlichen Erzeugung von sogenannten digital reconstructed radiographs. Diese sind zwar ein integraler Bestandteil einer 2D-3D Registrierung aber gleichzeitig sehr zeitaufwendig zu berechnen. Es führt zu einem erheblichen Geschwindigkeitsgewinn die herkömmliche Methode durch schnelles Rendering von Dreiecksnetzen zu ersetzen. Ebenso wird gezeigt, dass eine Kombination von schnellen lernbasierten Detektionsalgorithmen und 2D-3D Registrierung die Genauigkeit und die Registrierreichweite verbessert. Zum Abschluss werden die ersten Ergebnisse eines klinischen Prototypen präsentiert, welcher die zuvor genannten Methoden verwendet.Today, in an elderly community, the treatment of structural heart disease will become more and more important. Constant improvements of medical imaging technologies and the introduction of new catheter devices caused the trend to replace conventional open heart surgery by minimal invasive interventions. These advanced interventions need to be guided by different medical imaging modalities. The two main imaging systems here are X-ray fluoroscopy and Transesophageal  Echocardiography (TEE). While X-ray provides a good visualization of inserted catheters, which is essential for catheter navigation, TEE can display soft tissues, especially anatomical structures like heart valves. Both modalities provide real-time imaging and are necessary to lead minimal invasive heart surgery to success. Usually, the two systems are detached and not connected. It is conceivable that a fusion of both worlds can create a strong benefit for the physicians. It can lead to a better communication within the clinical team and can probably enable new surgical workflows. Because of the completely different characteristics of the image data, a direct fusion seems to be impossible. Therefore, an indirect registration of Ultrasound and X-ray images is used. The TEE probe is usually visible in the X-ray image during the described minimal-invasive interventions. Thereby, it becomes possible to register the TEE probe in the fluoroscopic images and to establish its 3D position. The relationship of the Ultrasound image to the Ultrasound probe is known by calibration. To register the TEE probe on 2D X-ray images, a 2D-3D registration approach is chosen in this thesis. Several contributions are presented, which are improving the common 2D-3D registration algorithm for the task of Ultrasound and X-ray fusion, but also for general 2D-3D registration problems. One presented approach is the introduction of planar parameters that increase robustness and speed during the registration of an object on two non-orthogonal views. Another approach is to replace the conventional generation of digital reconstructedradiographs, which is an integral part of 2D-3D registration but also a performance bottleneck, with fast triangular mesh rendering. This will result in a significant performance speed-up. It is also shown that a combination of fast learning-based detection algorithms with 2D-3D registration will increase the accuracy and the capture range, instead of employing them solely to the  registration/detection of a TEE probe. Finally, a first clinical prototype is presented which employs the presented approaches and first clinical results are shown

REFLECTED IMAGE PROCESSING FOR SPECULAR WELD POOL SURFACE MEASUREMENT

The surface of the weld pool contains information that can be exploited to emulate a skilled human welder to better understand and control the welding process. Of the existing techniques, the method that uses the pool\u27s specular nature to an advantage and which is relatively more cost effective, and suitable for welding environment is the one that utilizes reflected images to reconstruct 3D weld pool surface by using structured light and image processing techniques. In this thesis, an improvement has been made to the existing method by changing welding direction to obtain a denser reflected dot-matrix pattern allowing more accurate surface measurement. Then, the reflected images, obtained by capturing the reflection of a structured laser dot-matrix pattern from the pool surface through a high-speed camera with a narrow band-pass filter, are processed by a newly proposed algorithm to find the position of each reflected dot relative to its actual projection dot. This is a complicated process owing to the increased density of dots and noise induced due to the harsh environment. The obtained correspondence map may later be used by a surface reconstruction algorithm to derive the three-dimensional pool surface based on the reflection law

Dependency of image quality on system configuration parameters in a stationary digital breast tomosynthesis system

Purpose: In principle, a stationary digital breast tomosynthesis (s-DBT) system has better image quality when compared to continuous motion DBT systems due to zero motion blur of the source. The authors have developed a s-DBT system by using a linear carbon nanotube x-ray source array. The purpose of the current study was to quantitatively evaluate the performance of the s-DBT system;and investigate the dependence of imaging quality on the system configuration parameters. Methods: Physical phantoms were used to assess the image quality of each configuration including inplane resolution as measured by the modulation transfer function (MTF), inplane contrast as measured by the signal difference to noise ratio (SdNR), and depth resolution as measured by the z-axis artifact spread function. Five parameters were varied to create five groups of configurations: (1) total angular span; (2) total number of projection images; (3) distribution of exposure (mAs) across the projection images; (4) entrance dose; (5) detector pixel size. Results: It was found that the z-axis depth resolution increased with the total angular span but was insensitive to the number of projection images, mAs distribution, entrance dose, and detector pixel size. The SdNR was not affected by the angular span or the number of projection images. A decrease in SdNR was observed when the mAs was not evenly distributed across the projection images. As expected, the SdNR increased with entrance dose and when larger pixel sizes were used. For a given detector pixel size, the inplane resolution was found to be insensitive to the total angular span, number of projection images, mAs distribution, and entrance dose. A 25% increase in the MTF was observed when the detector was operating in full resolution mode (70 μm pixel size) compared to 2 × 2 binned mode (140 μm pixel size). Conclusions: The results suggest that the optimal imaging configuration for a s-DBT system is a large angular span, an intermittent number of projection views, and a uniform mAs distribution over all views. With the detector operating at full resolution, a stationary DBT system can achieve an inplane resolution of 5.1 cycles per mm, which is significantly better than continuous motion DBT systems

Helmholtz Portfolio Theme Large-Scale Data Management and Analysis (LSDMA)

The Helmholtz Association funded the "Large-Scale Data Management and Analysis" portfolio theme from 2012-2016. Four Helmholtz centres, six universities and another research institution in Germany joined to enable data-intensive science by optimising data life cycles in selected scientific communities. In our Data Life cycle Labs, data experts performed joint R&D together with scientific communities. The Data Services Integration Team focused on generic solutions applied by several communities

Artificial Intelligence-based Motion Tracking in Cancer Radiotherapy: A Review

Radiotherapy aims to deliver a prescribed dose to the tumor while sparing neighboring organs at risk (OARs). Increasingly complex treatment techniques such as volumetric modulated arc therapy (VMAT), stereotactic radiosurgery (SRS), stereotactic body radiotherapy (SBRT), and proton therapy have been developed to deliver doses more precisely to the target. While such technologies have improved dose delivery, the implementation of intra-fraction motion management to verify tumor position at the time of treatment has become increasingly relevant. Recently, artificial intelligence (AI) has demonstrated great potential for real-time tracking of tumors during treatment. However, AI-based motion management faces several challenges including bias in training data, poor transparency, difficult data collection, complex workflows and quality assurance, and limited sample sizes. This review serves to present the AI algorithms used for chest, abdomen, and pelvic tumor motion management/tracking for radiotherapy and provide a literature summary on the topic. We will also discuss the limitations of these algorithms and propose potential improvements.Comment: 36 pages, 5 Figures, 4 Table

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

Post-stroke dysarthria (PSD) is a widespread outcome of a stroke. To help in the objective evaluation of dysarthria, the development of pathological voice recognition and technology has a lot of attention. Soft robotics therapy devices have been received as an alternative rehabilitation and hand grasp assistance for improving activity daily living (ADL). Despite the significant progress in this field, most soft robotic therapy devices use a complex, bulky, lack of pathological voice recognition model, large computational power, and stationary controller. This study aims to develop a portable wirelessly multi-controller with a simulated dysarthric vowel speech in Bahasa Indonesia and non-dysarthric micro speech recognition, using tiny machine learning (TinyMl) system for hardware efficiency. The speech interface using INMP441, compute with a lightweight Deep Convolutional Neural network (DCNN) design and embedded into ESP-32. Feature model using Short Time Fourier Transform (STFT) and fed into CNN. This method has proven useful in micro-speech recognition with low computational power in both speech scenarios with a level of accuracy above 90%. Realtime inference performance on ESP-32 using hand prosthetics, with 3-level household noise intensity respectively 24db,42db, and 62db, and has respectively resulted from 95%, 85%, and 50% Accuracy. Wireless connectivity success rate with both controllers is around 0.2 - 0.5 ms

Research and technology

As the NASA center responsible for assembly, checkout, servicing, launch, recovery, and operational support of Space Transportation System elements and payloads, Kennedy Space Center (KSC) is placing increasing emphasis on KSC's research and technology program. In addition to strengthening those areas of engineering and operations technology that contribute to safer, more efficient, and more economical execution of the current mission, the technological tools needed to execute KSC's mission relative to future programs are being developed. The Engineering Development Directorate encompasses most of the laboratories and other KSC resources that are key elements of research and technology program implementation and is responsible for implementation of the majority of the projects in this KSC 1990 annual report. Projects under the following topics are covered: (1) materials science; (2) hazardous emissions and contamination monitoring; (3) biosciences; (4) autonomous systems; (5) communications and control; (6) meteorology; (7) technology utilization; and (8) mechanics, structures, and cryogenics