Towards multiple 3D bone surface identification and reconstruction using few 2D X-ray images for intraoperative applications

This article discusses a possible method to use a small number, e.g. 5, of conventional 2D X-ray images to reconstruct multiple 3D bone surfaces intraoperatively. Each bone’s edge contours in X-ray images are automatically identified. Sparse 3D landmark points of each bone are automatically reconstructed by pairing the 2D X-ray images. The reconstructed landmark point distribution on a surface is approximately optimal covering main characteristics of the surface. A statistical shape model, dense point distribution model (DPDM), is then used to fit the reconstructed optimal landmarks vertices to reconstruct a full surface of each bone separately. The reconstructed surfaces can then be visualised and manipulated by surgeons or used by surgical robotic systems

Differently stained whole slide image registration technique with landmark validation

Abstract. One of the most significant features in digital pathology is to compare and fuse successive differently stained tissue sections, also called slides, visually. Doing so, aligning different images to a common frame, ground truth, is required. Current sample scanning tools enable to create images full of informative layers of digitalized tissues, stored with a high resolution into whole slide images. However, there are a limited amount of automatic alignment tools handling large images precisely in acceptable processing time. The idea of this study is to propose a deep learning solution for histopathology image registration. The main focus is on the understanding of landmark validation and the impact of stain augmentation on differently stained histopathology images. Also, the developed registration method is compared with the state-of-the-art algorithms which utilize whole slide images in the field of digital pathology. There are previous studies about histopathology, digital pathology, whole slide imaging and image registration, color staining, data augmentation, and deep learning that are referenced in this study. The goal is to develop a learning-based registration framework specifically for high-resolution histopathology image registration. Different whole slide tissue sample images are used with a resolution of up to 40x magnification. The images are organized into sets of consecutive, differently dyed sections, and the aim is to register the images based on only the visible tissue and ignore the background. Significant structures in the tissue are marked with landmarks. The quality measurements include, for example, the relative target registration error, structural similarity index metric, visual evaluation, landmark-based evaluation, matching points, and image details. These results are comparable and can be used also in the future research and in development of new tools. Moreover, the results are expected to show how the theory and practice are combined in whole slide image registration challenges. DeepHistReg algorithm will be studied to better understand the development of stain color feature augmentation-based image registration tool of this study. Matlab and Aperio ImageScope are the tools to annotate and validate the image, and Python is used to develop the algorithm of this new registration tool. As cancer is globally a serious disease regardless of age or lifestyle, it is important to find ways to develop the systems experts can use while working with patients’ data. There is still a lot to improve in the field of digital pathology and this study is one step toward it.Eri menetelmin värjättyjen virtuaalinäytelasien rekisteröintitekniikka kiintopisteiden validointia hyödyntäen. Tiivistelmä. Yksi tärkeimmistä digitaalipatologian ominaisuuksista on verrata ja fuusioida peräkkäisiä eri menetelmin värjättyjä kudosleikkeitä toisiinsa visuaalisesti. Tällöin keskenään lähes identtiset kuvat kohdistetaan samaan yhteiseen kehykseen, niin sanottuun pohjatotuuteen. Nykyiset näytteiden skannaustyökalut mahdollistavat sellaisten kuvien luonnin, jotka ovat täynnä kerroksittaista tietoa digitalisoiduista näytteistä, tallennettuna erittäin korkean resoluution virtuaalisiin näytelaseihin. Tällä hetkellä on olemassa kuitenkin vain kourallinen automaattisia työkaluja, jotka kykenevät käsittelemään näin valtavia kuvatiedostoja tarkasti hyväksytyin aikarajoin. Tämän työn tarkoituksena on syväoppimista hyväksikäyttäen löytää ratkaisu histopatologisten kuvien rekisteröintiin. Tärkeimpänä osa-alueena on ymmärtää kiintopisteiden validoinnin periaatteet sekä eri väriaineiden augmentoinnin vaikutus. Lisäksi tässä työssä kehitettyä rekisteröintialgoritmia tullaan vertailemaan muihin kirjallisuudessa esitettyihin algoritmeihin, jotka myös hyödyntävät virtuaalinäytelaseja digitaalipatologian saralla. Kirjallisessa osiossa tullaan siteeraamaan aiempia tutkimuksia muun muassa seuraavista aihealueista: histopatologia, digitaalipatologia, virtuaalinäytelasi, kuvantaminen ja rekisteröinti, näytteen värjäys, data-augmentointi sekä syväoppiminen. Tavoitteena on kehittää oppimispohjainen rekisteröintikehys erityisesti korkearesoluutioisille digitalisoiduille histopatologisille kuville. Erilaisissa näytekuvissa tullaan käyttämään jopa 40-kertaista suurennosta. Kuvat kudoksista on järjestetty eri menetelmin värjättyihin peräkkäisiin kuvasarjoihin ja tämän työn päämääränä on rekisteröidä kuvat pohjautuen ainoastaan kudosten näkyviin osuuksiin, jättäen kuvien tausta huomioimatta. Kudosten merkittävimmät rakenteet on merkattu niin sanotuin kiintopistein. Työn laatumittauksina käytetään arvoja, kuten kohteen suhteellinen rekisteröintivirhe (rTRE), rakenteellisen samankaltaisuuindeksin mittari (SSIM), sekä visuaalista arviointia, kiintopisteisiin pohjautuvaa arviointia, yhteensopivuuskohtia, ja kuvatiedoston yksityiskohtia. Nämä arvot ovat verrattavissa myös tulevissa tutkimuksissa ja samaisia arvoja voidaan käyttää uusia työkaluja kehiteltäessä. DeepHistReg metodi toimii pohjana tässä työssä kehitettävälle näytteen värjäyksen parantamiseen pohjautuvalle rekisteröintityökalulle. Matlab ja Aperio ImageScope ovat ohjelmistoja, joita tullaan hyödyntämään tässä työssä kuvien merkitsemiseen ja validointiin. Ohjelmointikielenä käytetään Pythonia. Syöpä on maailmanlaajuisesti vakava sairaus, joka ei katso ikää eikä elämäntyyliä. Siksi on tärkeää löytää uusia keinoja kehittää työkaluja, joita asiantuntijat voivat hyödyntää jokapäiväisessä työssään potilastietojen käsittelyssä. Digitaalipatologian osa-alueella on vielä paljon innovoitavaa ja tämä työ on yksi askel eteenpäin taistelussa syöpäsairauksia vastaan

Breast Cancer: Modelling and Detection

This paper reviews a number of the mathematical models used in cancer modelling and then chooses a specific cancer, breast carcinoma, to illustrate how the modelling can be used in aiding detection. We then discuss mathematical models that underpin mammographic image analysis, which complements models of tumour growth and facilitates diagnosis and treatment of cancer. Mammographic images are notoriously difficult to interpret, and we give an overview of the primary image enhancement technologies that have been introduced, before focusing on a more detailed description of some of our own recent work on the use of physics-based modelling in mammography. This theoretical approach to image analysis yields a wealth of information that could be incorporated into the mathematical models, and we conclude by describing how current mathematical models might be enhanced by use of this information, and how these models in turn will help to meet some of the major challenges in cancer detection

Multimodal image analysis of the human brain

Gedurende de laatste decennia heeft de snelle ontwikkeling van multi-modale en niet-invasieve hersenbeeldvorming technologieën een revolutie teweeg gebracht in de mogelijkheid om de structuur en functionaliteit van de hersens te bestuderen. Er is grote vooruitgang geboekt in het beoordelen van hersenschade door gebruik te maken van Magnetic Reconance Imaging (MRI), terwijl Elektroencefalografie (EEG) beschouwd wordt als de gouden standaard voor diagnose van neurologische afwijkingen. In deze thesis focussen we op de ontwikkeling van nieuwe technieken voor multi-modale beeldanalyse van het menselijke brein, waaronder MRI segmentatie en EEG bronlokalisatie. Hierdoor voegen we theorie en praktijk samen waarbij we focussen op twee medische applicaties: (1) automatische 3D MRI segmentatie van de volwassen hersens en (2) multi-modale EEG-MRI data analyse van de hersens van een pasgeborene met perinatale hersenschade. We besteden veel aandacht aan de verbetering en ontwikkeling van nieuwe methoden voor accurate en ruisrobuuste beeldsegmentatie, dewelke daarna succesvol gebruikt worden voor de segmentatie van hersens in MRI van zowel volwassen als pasgeborenen. Daarenboven ontwikkelden we een geïntegreerd multi-modaal methode voor de EEG bronlokalisatie in de hersenen van een pasgeborene. Deze lokalisatie wordt gebruikt voor de vergelijkende studie tussen een EEG aanval bij pasgeborenen en acute perinatale hersenletsels zichtbaar in MRI

Computer Aided Tools for the Design and Planning of Personalized Shoulder Arthroplasty

La artroplastia de hombro es el tercer procedimiento de reemplazo articular más común, después de la artroplastia de rodilla y cadera, y actualmentees el de más rápido crecimiento en el campo ortopédico. Las principales opciones quirúrgicas incluyen la artroplastia total de hombro (TSA), en la quese restaura la anatomía articular normal, y, para pacientes con un manguito rotador completamente desgarrado, la artroplastia inversa de hombro (RSA), en la que la bola y la cavidad de la articulación glenohumeral se cambian. A pesar del progreso reciente y los avances en el diseño, las tasas de complicaciones reportadas para RSA son más altas que las de la artroplastia de hombro convencional. Un enfoque específico para el paciente, en el que los médicos adaptan el tratamiento quirúrgico a las características del mismo y al estado preoperatorio, por ejemplo mediante implantes personalizados y planificación previa, puede ayudar a reducir los problemas postoperatorios y mejorar el resultado funcional. El objetivo principal de esta tesis es desarrollar y evaluar métodos novedosos para RSA personalizado, utilizando tecnologías asistidas por ordenador de última generación para estandarizar y automatizar las fases de diseño y planificación.Los implantes personalizados son una solución adecuada para el tratamiento de pacientes con pérdida extensa de hueso glenoideo. Sin embargo, los ingenieros clínicos se enfrentan a muchas variables en el diseño de implantes (número y tipo de tornillos, superficie de contacto, etc.) y una gran variabilidad anatómica y patológica. Actualmente, no existen herramientas objetivas para guiarlos a la hora de elegir el diseño óptimo, es decir, con suficiente estabilidad inicial del implante, lo que hace que el proceso de diseño sea tedioso, lento y dependiente del usuario. En esta tesis, se desarrolló una simulación de Virtual Bench Test (VBT) utilizando un modelo de elementos finitos para evaluar automáticamente la estabilidad inicial de los implantes de hombro personalizados. A través de un experimento de validación, se demostró que los ingenieros clínicos pueden utilizar el resultado de Virtual Bench Test como referencia para respaldar sus decisiones y adaptaciones durante el proceso de diseño del implante.Al diseñar implantes de hombro, el conocimiento de la morfología y la calidad ósea de la escápula en toda la población es fundamental. En particular, se tienen en cuenta las regiones con la mejor reserva ósea (hueso cortical) para definir la posición y orientación de los orificios de los tornillos, mientras se busca una fijación óptima. Como alternativa a las mediciones manuales, cuya generalización está limitada por el análisis de pequeños subconjuntos de pacientes potenciales, Statistical Shape Models (SSMs) se han utilizado comúnmente para describir la variabilidad de la forma dentro de una población. Sin embargo, estos SSMs normalmente no contienen información sobre el grosor cortical.Por lo tanto, se desarrolló una metodología para combinar la forma del hueso escapular y la morfología de la cortical en un SSM. Primero, se presentó y evaluó un método para estimar el espesor cortical, a partir de un análisis de perfil de Hounsfield Unit (HU). Luego, utilizando 32 escápulas sanas segmentadas manualmente, se creó y evaluó un modelo de forma estadística que incluía información de la cortical. La herramienta desarrollada se puede utilizar para implantar virtualmente un nuevo diseño y probar su congruencia dentro de una población virtual generada, reduciendo así el número de iteraciones de diseño y experimentos con cadáveres.Las mediciones del alargamiento de los músculos deltoides y del manguito rotador durante la planificación quirúrgica pueden ayudar a los médicos aseleccionar un diseño y una posición de implante adecuados. Sin embargo, tal evaluación requiere la indicación de puntos anatómicos como referencia para los puntos de unión de los músculos, un proceso que requiere mucho tiempo y depende del usuario, ya que a menudo se realiza manualmente. Además, las imágenes médicas, que se utilizan normalmente para la artroplastia de hombro,contienen en su mayoría solo el húmero proximal, lo que hace imposible indicarlos puntos de unión de los músculos que se encuentran fuera del campo de visión de la exploración. Por lo tanto, se desarrolló y evaluó un método totalmente automatizado, basado en SSM, para medir la elongación del deltoides y del manguito rotador. Su aplicabilidad clínica se demostró mediante la evaluación del rendimiento de la estimación automatizada de la elongación muscular para un conjunto de articulaciones artríticas del hombro utilizadas para la planificación preoperatoria de RSA, lo que confirma que es una herramienta adecuada para los cirujanos a la hora de evaluar y refinar las decisiones clínicas.En esta investigación, se dio un paso importante en la dirección de un enfoque más personalizado de la artroplastia inversa de hombro, en el que el manejo quirúrgico, es decir, el diseño y la posición del implante, se adapta a las características específicas del paciente y al estado preoperatorio. Al aplicar tecnologías asistidas por computadora en la práctica clínica, el proceso de diseño y planificación se puede automatizar y estandarizar, reduciendo así los costos y los plazos de entrega. Además, gracias a los métodos novedosos presentados en esta tesis, esperamos en el futuro una adopción más amplia del enfoque personalizado, con importantes beneficios tanto para los cirujanos como para los pacientes.Shoulder arthroplasty is the third most common joint replacement procedure, after knee and hip arthroplasty, and currently the most rapidly growing one in the orthopaedic field. The main surgical options include total shoulder arthroplasty (TSA), in which the normal joint anatomy is restored, and, for patients with a completely torn rotator cuff, reverse shoulder arthroplasty (RSA), in which the ball and the socket of the glenohumeral joint are switched. Despite the recent progress and advancement in design, the reported rates of complication for RSA are higher than those of conventional shoulder arthroplasty. A patient-specific approach, in which clinicians adapt the surgical management to patient characteristics and preoperative condition, e.g. through custom implants and pre-planning, can help to reduce postoperative problems and improve the functional outcome. The main goal of this thesis is to develop and evaluate novel methods for personalized RSA, using state-of-the-art computer aided technologies to standardize and automate the design and planning phases. Custom implants are a suitable solution when treating patients with extensive glenoid bone loss. However, clinical engineers are confronted with an enormous implant design space (number and type of screws, contact surface, etc.) and large anatomical and pathological variability. Currently, no objective tools exist to guide them when choosing the optimal design, i.e. with sufficient initial implant stability, thus making the design process tedious, time-consuming, and user-dependent. In this thesis, a Virtual Bench Test (VBT) simulation was developed using a finite element model to automatically evaluate the initial stability of custom shoulder implants. Through a validation experiment, it was shown that the virtual test bench output can be used by clinical engineers as a reference to support their decisions and adaptations during the implant design process. When designing shoulder implants, knowledge about bone morphology and bone quality of the scapula throughout a certain population is fundamental. In particular, regions with the best bone stock (cortical bone) are taken into account to define the position and orientation of the screw holes, while aiming for an optimal fixation. As an alternative to manual measurements, whose generalization is limited by the analysis of small sub-sets of the potential patients, Statistical Shape Models (SSMs) have been commonly used to describe shape variability within a population. However, these SSMs typically do not contain information about cortical thickness. Therefore, a methodology to combine scapular bone shape and cortex morphology in an SSM was developed. First, a method to estimate cortical thickness, starting from a profile analysis of Hounsfield Unit (HU), was presented and evaluated. Then, using 32 manually segmented healthy scapulae, a statistical shape model including cortical information was created and assessed. The developed tool can be used to virtually implant a new design and test its congruency inside a generated virtual population, thus reducing the number of design iterations and cadaver labs. Measurements of deltoid and rotator cuff muscle elongation during surgical planning can help clinicians to select a suitable implant design and position. However, such an assessment requires the indication of anatomical landmarks as a reference for the muscle attachment points, a process that is time-consuming and user-dependent, since often performed manually. Additionally, the medical images, which are normally used for shoulder arthroplasty, mostly contain only the proximal humerus, making it impossible to indicate those muscle attachment points which lie outside of the field of view of the scan. Therefore, a fully-automated method, based on SSM, for measuring deltoid and rotator cuff elongation was developed and evaluated. Its clinical applicability was demonstrated by assessing the performance of the automated muscle elongation estimation for a set of arthritic shoulder joints used for preoperative planning of RSA, thus confirming it a suitable tool for surgeons when evaluating and refining clinical decisions. In this research, a major step was taken into the direction of a more personalized approach to Reverse Shoulder Arthroplasty, in which the surgical management, i.e. implant design and position, is adapted to the patient-specific characteristics and preoperative condition. By applying computer aided technologies in the clinical practice, design and planning process can be automated and standardized, thus reducing costs and lead times. Additionally, thanks to the novel methods presented in this thesis, we expect in the future a wider adoption of the personalized approach, with important benefits both for surgeons and patients.<br /

Automatic 2-D/3-D Vessel Enhancement in Multiple Modality Images Using a Weighted Symmetry Filter

Automated detection of vascular structures is of great importance in understanding the mechanism, diagnosis and treatment of many vascular pathologies. However, automatic vascular detection continues to be an open issue because of difficulties posed by multiple factors such as poor contrast, inhomogeneous backgrounds, anatomical variations, and the presence of noise during image acquisition. In this paper, we propose a novel 2D/3D symmetry filter to tackle these challenging issues for enhancing vessels from different imaging modalities. The proposed filter not only considers local phase features by using a quadrature filter to distinguish between lines and edges, but also uses the weighted geometric mean of the blurred and shifted responses of the quadrature filter, which allows more tolerance of vessels with irregular appearance. As a result, this filter shows a strong response to the vascular features under typical imaging conditions. Results based on 8 publicly available datasets (six 2D datasets, one 3D dataset and one 3D synthetic dataset) demonstrate its superior performance to other state-ofthe- art methods

Curvilinear Structure Enhancement in Biomedical Images

Curvilinear structures can appear in many different areas and at a variety of scales. They can be axons and dendrites in the brain, blood vessels in the fundus, streets, rivers or fractures in buildings, and others. So, it is essential to study curvilinear structures in many fields such as neuroscience, biology, and cartography regarding image processing. Image processing is an important field for the help to aid in biomedical imaging especially the diagnosing the disease. Image enhancement is the early step of image analysis. In this thesis, I focus on the research, development, implementation, and validation of 2D and 3D curvilinear structure enhancement methods, recently established. The proposed methods are based on phase congruency, mathematical morphology, and tensor representation concepts. First, I have introduced a 3D contrast independent phase congruency-based enhancement approach. The obtained results demonstrate the proposed approach is robust against the contrast variations in 3D biomedical images. Second, I have proposed a new mathematical morphology-based approach called the bowler-hat transform. In this approach, I have combined the mathematical morphology with a local tensor representation of curvilinear structures in images. The bowler-hat transform is shown to give better results than comparison methods on challenging data such as retinal/fundus images. The bowler-hat transform is shown to give better results than comparison methods on challenging data such as retinal/fundus images. Especially the proposed method is quite successful while enhancing of curvilinear structures at junctions. Finally, I have extended the bowler-hat approach to the 3D version to prove the applicability, reliability, and ability of it in 3D

IE-Map: a novel in-vivo atlas and template of the human inner ear

Brain atlases and templates are core tools in scientific research with increasing importance also in clinical applications. Advances in neuroimaging now allowed us to expand the atlas domain to the vestibular and auditory organ, the inner ear. In this study, we present IE-Map, an in-vivo template and atlas of the human labyrinth derived from multi-modal high-resolution magnetic resonance imaging (MRI) data, in a fully non-invasive manner without any contrast agent or radiation. We reconstructed a common template from 126 inner ears (63 normal subjects) and annotated it with 94 established landmarks and semi-automatic segmentations of all relevant macroscopic vestibular and auditory substructures. We validated the atlas by comparing MRI templates to a novel CT/micro-CT atlas, which we reconstructed from 21 publicly available post-mortem images of the bony labyrinth. Templates in MRI and micro-CT have a high overlap, and several key anatomical measures of the bony labyrinth in IE-Map are in line with micro-CT literature of the inner ear. A quantitative substructural analysis based on the new template, revealed a correlation of labyrinth parameters with total intracranial volume. No effects of gender or laterality were found. We provide the validated templates, atlas segmentations, surface meshes and landmark annotations as open-access material, to provide neuroscience researchers and clinicians in neurology, neurosurgery, and otorhinolaryngology with a widely applicable tool for computational neuro-otology

A review of image processing methods for fetal head and brain analysis in ultrasound images

Background and objective: Examination of head shape and brain during the fetal period is paramount to evaluate head growth, predict neurodevelopment, and to diagnose fetal abnormalities. Prenatal ultrasound is the most used imaging modality to perform this evaluation. However, manual interpretation of these images is challenging and thus, image processing methods to aid this task have been proposed in the literature. This article aims to present a review of these state-of-the-art methods. Methods: In this work, it is intended to analyze and categorize the different image processing methods to evaluate fetal head and brain in ultrasound imaging. For that, a total of 109 articles published since 2010 were analyzed. Different applications are covered in this review, namely analysis of head shape and inner structures of the brain, standard clinical planes identification, fetal development analysis, and methods for image processing enhancement. Results: For each application, the reviewed techniques are categorized according to their theoretical approach, and the more suitable image processing methods to accurately analyze the head and brain are identified. Furthermore, future research needs are discussed. Finally, topics whose research is lacking in the literature are outlined, along with new fields of applications. Conclusions: A multitude of image processing methods has been proposed for fetal head and brain analysis. Summarily, techniques from different categories showed their potential to improve clinical practice. Nevertheless, further research must be conducted to potentiate the current methods, especially for 3D imaging analysis and acquisition and for abnormality detection. (c) 2022 Elsevier B.V. All rights reserved.FCT - Fundação para a Ciência e a Tecnologia(UIDB/00319/2020)This work was funded by projects “NORTE-01–0145-FEDER- 0 0 0 059 , NORTE-01-0145-FEDER-024300 and “NORTE-01–0145- FEDER-0 0 0 045 , supported by Northern Portugal Regional Opera- tional Programme (Norte2020), under the Portugal 2020 Partner- ship Agreement, through the European Regional Development Fund (FEDER). It was also funded by national funds, through the FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020 and by FCT and FCT/MCTES in the scope of the projects UIDB/05549/2020 and UIDP/05549/2020 . The authors also acknowledge support from FCT and the Euro- pean Social Found, through Programa Operacional Capital Humano (POCH), in the scope of the PhD grant SFRH/BD/136670/2018 and SFRH/BD/136721/2018

Multi-Atlas based Segmentation of Multi-Modal Brain Images

Brain image analysis is playing a fundamental role in clinical and population-based epidemiological studies. Several brain disorder studies involve quantitative interpretation of brain scans and particularly require accurate measurement and delineation of tissue volumes in the scans. Automatic segmentation methods have been proposed to provide reliability and accuracy of the labelling as well as performing an automated procedure. Taking advantage of prior information about the brain's anatomy provided by an atlas as a reference model can help simplify the labelling process. The segmentation in the atlas-based approach will be problematic if the atlas and the target image are not accurately aligned, or if the atlas does not appropriately represent the anatomical structure/region. The accuracy of the segmentation can be improved by utilising a group of atlases. Employing multiple atlases brings about considerable issues in segmenting a new subject's brain image. Registering multiple atlases to the target scan and fusing labels from registered atlases, for a population obtained from different modalities, are challenging tasks: image-intensity comparisons may no longer be valid, since image brightness can have highly diff ering meanings in dfferent modalities. The focus is on the problem of multi-modality and methods are designed and developed to deal with this issue specifically in image registration and label fusion. To deal with multi-modal image registration, two independent approaches are followed. First, a similarity measure is proposed based upon comparing the self-similarity of each of the images to be aligned. Second, two methods are proposed to reduce the multi-modal problem to a mono-modal one by constructing representations not relying on the image intensities. Structural representations work on the basis of using un-decimated complex wavelet representation in one method, and modified approach using entropy in the other one. To handle the cross-modality label fusion, a method is proposed to weight atlases based on atlas-target similarity. The atlas-target similarity is measured by scale-based comparison taking advantage of structural features captured from un-decimated complex wavelet coefficients. The proposed methods are assessed using the simulated and real brain data from computed tomography images and different modes of magnetic resonance images. Experimental results reflect the superiority of the proposed methods over the classical and state-of-the art methods