FetusMapV2: Enhanced Fetal Pose Estimation in 3D Ultrasound

Fetal pose estimation in 3D ultrasound (US) involves identifying a set of associated fetal anatomical landmarks. Its primary objective is to provide comprehensive information about the fetus through landmark connections, thus benefiting various critical applications, such as biometric measurements, plane localization, and fetal movement monitoring. However, accurately estimating the 3D fetal pose in US volume has several challenges, including poor image quality, limited GPU memory for tackling high dimensional data, symmetrical or ambiguous anatomical structures, and considerable variations in fetal poses. In this study, we propose a novel 3D fetal pose estimation framework (called FetusMapV2) to overcome the above challenges. Our contribution is three-fold. First, we propose a heuristic scheme that explores the complementary network structure-unconstrained and activation-unreserved GPU memory management approaches, which can enlarge the input image resolution for better results under limited GPU memory. Second, we design a novel Pair Loss to mitigate confusion caused by symmetrical and similar anatomical structures. It separates the hidden classification task from the landmark localization task and thus progressively eases model learning. Last, we propose a shape priors-based self-supervised learning by selecting the relatively stable landmarks to refine the pose online. Extensive experiments and diverse applications on a large-scale fetal US dataset including 1000 volumes with 22 landmarks per volume demonstrate that our method outperforms other strong competitors.Comment: 16 pages, 11 figures, accepted by Medical Image Analysis(2023

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Detecting Red Blood Cells Morphological Abnormalities Using Genetic Algorithm and Kmeans

Vision is the most advanced of our senses, so it is not surprising that images play the single most important role in human perception. Computer-aided diagnosis is another important application of pattern recognition, aiming at assisting doctors in making diagnostic decisions. Many diseases which are not blood diseases in origin have hematological abnormalities and manifestation (have symptoms appeared on the blood). CBC (cell blood count test) for instance, is still the first test to be requested by the physicians or become in their mind. Blood abnormality can be in white blood cells, red blood cells and plasma. In this thesis, red blood cells are the suggested for detecting it is abnormality. The abnormality of blood cells shapes can't be detected easily, where the CBC (cell blood count) device give a count number and percentages not a description of the shapes of the blood cells, when the blood cells shapes wanted to be known, hematologist asked to view the blood films under the microscope which is time consuming task besides that the human error risk is high. Since the number of abnormal cells to normal cells in a given blood sample give a measure of the disease severity, detecting one cell with potential abnormality can give premature warning for future illness that can be avoided or treated earlier. This case can't be detected by hematologist. Computer involved in such task to save time and effort besides minimizing human error. This thesis name is "DETECTING RED BLOOD CELLS MORPHOLOGICAL ABNORMALITIES USING GENETIC ALGORITHM AND KMEANS". In this thesis, the thesis divided into four phases. First phase data collection where blood samples was drawn from healthy and sick people and then blood films made and viewed under microscope and an images captured for these blood films. Second phase preprocessing phase where the images prepared for the next phase. Third phase feature extraction was executed where these features are spatial domain and frequency domain features. Fourth phase is the classification phase where the features fed into the classifier to be classified. An acceptable detection rate is achieved by the proposed system. The genetic algorithm classifier success rate was 92.31% and the K-means classifier success rate was 94.00%

Optimization strategies for respiratory motion management in stereotactic body radiation therapy

Various challenges arise during the treatment of lung tumors with stereotactic body radiation therapy (SBRT), which is a form of hypofractionated high precision conformal radiation therapy delivered to small targets. The dose is applied in only a few fractions and respiratory organ and tumor motion is a source of uncertainty additional to interfractional set-up errors. Respiratory organ and tumor motion is highly patient-specific and it affects the whole radiotherapy treatment chain. In this thesis, motion management techniques for SBRT are evaluated and improved in a clinical setting. A clinical need for improvement has been present at the LMU university hospital for each issue addressed in this thesis: Initially, the usage of respiratory correlated computed tomography (4DCT), which is vital for SBRT treatment, was seen as impractical and prone to uncertainties in the data reconstruction in its current form. Therefore, the 4DCT reconstruction workflow has been improved to minimize these potential error sources. Secondly, treatment planning for tumors affected by respiratory motion was evaluated and subsequently improved. Finally, the treatment technique of respiratory gating was implemented at the clinic, which led to the need of evaluating the respiratory gating characteristics of the novel system configuration. At first, the 4DCT reconstruction workflow used in clinical practice was investigated, as in the presence of respiratory motion the knowledge of tumor position over time is essential in SBRT treatments. Using 4DCT, the full motion range of the individual tumor can be determined. However, certain 4DCT reconstruction methods can under- or overestimate tumor motion due to limitations in the data acquisition scheme and due to the incorrect sorting of certain X-ray computed tomography (CT) image slices into different respiratory phases. As the regular clinical workflow of cycle-based sorting (CBS) without maximum inspiration detection (and therefore no clear starting point for the individual breathing cycles) seemed to be affected by these potential errors, the usage of CBS with correct maximum detection and another sorting algorithm of the respiration states, so-called local amplitude-based sorting (LAS), both have been implemented for a reduction of image artifacts and improved 4DCT quality. The three phase binning algorithms have been investigated in a phantom study (using 10 different breathing waveforms) and in a patient study (with 10 different patients). The mis-representation of the tumor volume was reduced in both implemented sorting algorithms compared to the previously used CBS approach (without correct maximum detection) in the phantom and the patient study. The clinical recommendation was the use of CBS with improved maximum detection, as too many manual interventions would be needed for the LAS workflow. Secondly, a combination of the actual patient breathing trace during treatment, the log files generated by the linear accelerator (LINAC), and Monte Carlo (MC) four-dimensional (4D) dose calculations for each individual fraction was implemented as a 4D dose evaluation tool. This workflow was tested in a clinical environment for SBRT treatment planning on multiple CT datasets featuring: a native free-breathing 3DCT, an average intensity projection (AIP) as well as a maximum intensity projection (MIP), both obtained from the patient's 4DCT, and density overrides (DOs) in a 3DCT. This study has been carried out for 5 SBRT patients for three-dimensional conformal radiation therapy (3D-CRT) and volumetric modulated arc therapy (VMAT) treatment plans. The dose has been recalculated on each 4DCT breathing phase according the the patient's breathing waveform and accumulated to the gross tumor volume (GTV) at the end-of-exhale (EOE) breathing phase using deformable image registration. Even though the least differences in planned and recalculated dose were found for AIP and MIP treatment planning, the results indicate a strong dependency on individual tumor motion due to the variability of breathing motion in general, and on tumor size. The combination of the patient's individual breathing trace during each SBRT fraction with 4D MC dose calculation based on the LINAC log file information leads to a good approximation of actual dose delivery. Finally, in order to ensure precise and accurate treatment for respiratory gating techniques, the technical characteristics of the LINAC in combination with a breathing motion monitoring system as s surrogate for tumor motion have to be identified. The dose delivery accuracy and the latency of a surface imaging system in connection with a modern medical LINAC were investigated using a dynamic breathing motion phantom. The dosimetric evaluation has been carried out using a static 2D-diode array. The measurement of the dose difference between gated and ungated radiation delivery was found to be below 1% (for clinical relevant gating levels of about 30%). The beam-on latency, or time delay, determined using radiographic films was found to be up to 851 ms±100 ms. With these known parameters, an adjustment of the pre-selected gating level or the internal target volume (ITV) margins could be made. With the highly patient-specific character of respiratory motion, lung SBRT faces many additional challenges besides the specific issues addressed in this thesis. However, the findings of this thesis have improved clinical workflows at the Department of Radiation Oncology of the LMU University hospital. In a future perspective, a workflow using evaluation of the actual 4D dose in combination with accurate 4DCT image acquisition and specialized treatment delivery (such as respiratory gating) has the potential for a safe further reduction of treatment margins and increased sparing of organs-at-risk (OARs) in SBRT without compromising tumor dose targeting accuracy

The Probabilistic Active Shape Model: From Model Construction to Flexible Medical Image Segmentation

Automatic processing of three-dimensional image data acquired with computed tomography or magnetic resonance imaging plays an increasingly important role in medicine. For example, the automatic segmentation of anatomical structures in tomographic images allows to generate three-dimensional visualizations of a patient’s anatomy and thereby supports surgeons during planning of various kinds of surgeries. Because organs in medical images often exhibit a low contrast to adjacent structures, and because the image quality may be hampered by noise or other image acquisition artifacts, the development of segmentation algorithms that are both robust and accurate is very challenging. In order to increase the robustness, the use of model-based algorithms is mandatory, as for example algorithms that incorporate prior knowledge about an organ’s shape into the segmentation process. Recent research has proven that Statistical Shape Models are especially appropriate for robust medical image segmentation. In these models, the typical shape of an organ is learned from a set of training examples. However, Statistical Shape Models have two major disadvantages: The construction of the models is relatively difficult, and the models are often used too restrictively, such that the resulting segmentation does not delineate the organ exactly. This thesis addresses both problems: The first part of the thesis introduces new methods for establishing correspondence between training shapes, which is a necessary prerequisite for shape model learning. The developed methods include consistent parameterization algorithms for organs with spherical and genus 1 topology, as well as a nonrigid mesh registration algorithm for shapes with arbitrary topology. The second part of the thesis presents a new shape model-based segmentation algorithm that allows for an accurate delineation of organs. In contrast to existing approaches, it is possible to integrate not only linear shape models into the algorithm, but also nonlinear shape models, which allow for a more specific description of an organ’s shape variation. The proposed segmentation algorithm is evaluated in three applications to medical image data: Liver and vertebra segmentation in contrast-enhanced computed tomography scans, and prostate segmentation in magnetic resonance images

From nanometers to centimeters: Imaging across spatial scales with smart computer-aided microscopy

Microscopes have been an invaluable tool throughout the history of the life sciences, as they allow researchers to observe the miniscule details of living systems in space and time. However, modern biology studies complex and non-obvious phenotypes and their distributions in populations and thus requires that microscopes evolve from visual aids for anecdotal observation into instruments for objective and quantitative measurements. To this end, many cutting-edge developments in microscopy are fuelled by innovations in the computational processing of the generated images. Computational tools can be applied in the early stages of an experiment, where they allow for reconstruction of images with higher resolution and contrast or more colors compared to raw data. In the final analysis stage, state-of-the-art image analysis pipelines seek to extract interpretable and humanly tractable information from the high-dimensional space of images. In the work presented in this thesis, I performed super-resolution microscopy and wrote image analysis pipelines to derive quantitative information about multiple biological processes. I contributed to studies on the regulation of DNMT1 by implementing machine learning-based segmentation of replication sites in images and performed quantitative statistical analysis of the recruitment of multiple DNMT1 mutants. To study the spatiotemporal distribution of DNA damage response I performed STED microscopy and could provide a lower bound on the size of the elementary spatial units of DNA repair. In this project, I also wrote image analysis pipelines and performed statistical analysis to show a decoupling of DNA density and heterochromatin marks during repair. More on the experimental side, I helped in the establishment of a protocol for many-fold color multiplexing by iterative labelling of diverse structures via DNA hybridization. Turning from small scale details to the distribution of phenotypes in a population, I wrote a reusable pipeline for fitting models of cell cycle stage distribution and inhibition curves to high-throughput measurements to quickly quantify the effects of innovative antiproliferative antibody-drug-conjugates. The main focus of the thesis is BigStitcher, a tool for the management and alignment of terabyte-sized image datasets. Such enormous datasets are nowadays generated routinely with light-sheet microscopy and sample preparation techniques such as clearing or expansion. Their sheer size, high dimensionality and unique optical properties poses a serious bottleneck for researchers and requires specialized processing tools, as the images often do not fit into the main memory of most computers. BigStitcher primarily allows for fast registration of such many-dimensional datasets on conventional hardware using optimized multi-resolution alignment algorithms. The software can also correct a variety of aberrations such as fixed-pattern noise, chromatic shifts and even complex sample-induced distortions. A defining feature of BigStitcher, as well as the various image analysis scripts developed in this work is their interactivity. A central goal was to leverage the user's expertise at key moments and bring innovations from the big data world to the lab with its smaller and much more diverse datasets without replacing scientists with automated black-box pipelines. To this end, BigStitcher was implemented as a user-friendly plug-in for the open source image processing platform Fiji and provides the users with a nearly instantaneous preview of the aligned images and opportunities for manual control of all processing steps. With its powerful features and ease-of-use, BigStitcher paves the way to the routine application of light-sheet microscopy and other methods producing equally large datasets

From nanometers to centimeters: Imaging across spatial scales with smart computer-aided microscopy

Microscopes have been an invaluable tool throughout the history of the life sciences, as they allow researchers to observe the miniscule details of living systems in space and time. However, modern biology studies complex and non-obvious phenotypes and their distributions in populations and thus requires that microscopes evolve from visual aids for anecdotal observation into instruments for objective and quantitative measurements. To this end, many cutting-edge developments in microscopy are fuelled by innovations in the computational processing of the generated images. Computational tools can be applied in the early stages of an experiment, where they allow for reconstruction of images with higher resolution and contrast or more colors compared to raw data. In the final analysis stage, state-of-the-art image analysis pipelines seek to extract interpretable and humanly tractable information from the high-dimensional space of images. In the work presented in this thesis, I performed super-resolution microscopy and wrote image analysis pipelines to derive quantitative information about multiple biological processes. I contributed to studies on the regulation of DNMT1 by implementing machine learning-based segmentation of replication sites in images and performed quantitative statistical analysis of the recruitment of multiple DNMT1 mutants. To study the spatiotemporal distribution of DNA damage response I performed STED microscopy and could provide a lower bound on the size of the elementary spatial units of DNA repair. In this project, I also wrote image analysis pipelines and performed statistical analysis to show a decoupling of DNA density and heterochromatin marks during repair. More on the experimental side, I helped in the establishment of a protocol for many-fold color multiplexing by iterative labelling of diverse structures via DNA hybridization. Turning from small scale details to the distribution of phenotypes in a population, I wrote a reusable pipeline for fitting models of cell cycle stage distribution and inhibition curves to high-throughput measurements to quickly quantify the effects of innovative antiproliferative antibody-drug-conjugates. The main focus of the thesis is BigStitcher, a tool for the management and alignment of terabyte-sized image datasets. Such enormous datasets are nowadays generated routinely with light-sheet microscopy and sample preparation techniques such as clearing or expansion. Their sheer size, high dimensionality and unique optical properties poses a serious bottleneck for researchers and requires specialized processing tools, as the images often do not fit into the main memory of most computers. BigStitcher primarily allows for fast registration of such many-dimensional datasets on conventional hardware using optimized multi-resolution alignment algorithms. The software can also correct a variety of aberrations such as fixed-pattern noise, chromatic shifts and even complex sample-induced distortions. A defining feature of BigStitcher, as well as the various image analysis scripts developed in this work is their interactivity. A central goal was to leverage the user's expertise at key moments and bring innovations from the big data world to the lab with its smaller and much more diverse datasets without replacing scientists with automated black-box pipelines. To this end, BigStitcher was implemented as a user-friendly plug-in for the open source image processing platform Fiji and provides the users with a nearly instantaneous preview of the aligned images and opportunities for manual control of all processing steps. With its powerful features and ease-of-use, BigStitcher paves the way to the routine application of light-sheet microscopy and other methods producing equally large datasets

Fluoroscopic Navigation for Robot-Assisted Orthopedic Surgery

Robot-assisted orthopedic surgery has gained increasing attention due to its improved accuracy and stability in minimally-invasive interventions compared to a surgeon's manual operation. An effective navigation system is critical, which estimates the intra-operative tool-to-tissue pose relationship to guide the robotic surgical device. However, most existing navigation systems use fiducial markers, such as bone pin markers, to close the calibration loop, which requires a clear line of sight and is not ideal for patients. This dissertation presents fiducial-free, fluoroscopic image-based navigation pipelines for three robot-assisted orthopedic applications: femoroplasty, core decompression of the hip, and transforaminal lumbar epidural injections. We propose custom-designed image intensity-based 2D/3D registration algorithms for pose estimation of bone anatomies, including femur and spine, and pose estimation of a rigid surgical tool and a flexible continuum manipulator. We performed system calibration and integration into a surgical robotic platform. We validated the navigation system's performance in comprehensive simulation and ex vivo cadaveric experiments. Our results suggest the feasibility of applying our proposed navigation methods for robot-assisted orthopedic applications. We also investigated machine learning approaches that can benefit the medical imaging analysis, automate the navigation component or address the registration challenges. We present a synthetic X-ray data generation pipeline called SyntheX, which enables large-scale machine learning model training. SyntheX was used to train feature detection tasks of the pelvis anatomy and the continuum manipulator, which were used to initialize the registration pipelines. Last but not least, we propose a projective spatial transformer module that learns a convex shape similarity function and extends the registration capture range. We believe that our image-based navigation solutions can benefit and inspire related orthopedic robot-assisted system designs and eventually be used in the operating rooms to improve patient outcomes