Sub-pixel Registration In Computational Imaging And Applications To Enhancement Of Maxillofacial Ct Data

In computational imaging, data acquired by sampling the same scene or object at different times or from different orientations result in images in different coordinate systems. Registration is a crucial step in order to be able to compare, integrate and fuse the data obtained from different measurements. Tomography is the method of imaging a single plane or slice of an object. A Computed Tomography (CT) scan, also known as a CAT scan (Computed Axial Tomography scan), is a Helical Tomography, which traditionally produces a 2D image of the structures in a thin section of the body. It uses X-ray, which is ionizing radiation. Although the actual dose is typically low, repeated scans should be limited. In dentistry, implant dentistry in specific, there is a need for 3D visualization of internal anatomy. The internal visualization is mainly based on CT scanning technologies. The most important technological advancement which dramatically enhanced the clinician\u27s ability to diagnose, treat, and plan dental implants has been the CT scan. Advanced 3D modeling and visualization techniques permit highly refined and accurate assessment of the CT scan data. However, in addition to imperfections of the instrument and the imaging process, it is not uncommon to encounter other unwanted artifacts in the form of bright regions, flares and erroneous pixels due to dental bridges, metal braces, etc. Currently, removing and cleaning up the data from acquisition backscattering imperfections and unwanted artifacts is performed manually, which is as good as the experience level of the technician. On the other hand the process is error prone, since the editing process needs to be performed image by image. We address some of these issues by proposing novel registration methods and using stonecast models of patient\u27s dental imprint as reference ground truth data. Stone-cast models were originally used by dentists to make complete or partial dentures. The CT scan of such stone-cast models can be used to automatically guide the cleaning of patients\u27 CT scans from defects or unwanted artifacts, and also as an automatic segmentation system for the outliers of the CT scan data without use of stone-cast models. Segmented data is subsequently used to clean the data from artifacts using a new proposed 3D inpainting approach

Lose The Views: Limited Angle CT Reconstruction via Implicit Sinogram Completion

Computed Tomography (CT) reconstruction is a fundamental component to a wide variety of applications ranging from security, to healthcare. The classical techniques require measuring projections, called sinograms, from a full 180$^\circ$ view of the object. This is impractical in a limited angle scenario, when the viewing angle is less than 180$^\circ$, which can occur due to different factors including restrictions on scanning time, limited flexibility of scanner rotation, etc. The sinograms obtained as a result, cause existing techniques to produce highly artifact-laden reconstructions. In this paper, we propose to address this problem through implicit sinogram completion, on a challenging real world dataset containing scans of common checked-in luggage. We propose a system, consisting of 1D and 2D convolutional neural networks, that operates on a limited angle sinogram to directly produce the best estimate of a reconstruction. Next, we use the x-ray transform on this reconstruction to obtain a "completed" sinogram, as if it came from a full 180$^\circ$ measurement. We feed this to standard analytical and iterative reconstruction techniques to obtain the final reconstruction. We show with extensive experimentation that this combined strategy outperforms many competitive baselines. We also propose a measure of confidence for the reconstruction that enables a practitioner to gauge the reliability of a prediction made by our network. We show that this measure is a strong indicator of quality as measured by the PSNR, while not requiring ground truth at test time. Finally, using a segmentation experiment, we show that our reconstruction preserves the 3D structure of objects effectively.Comment: Spotlight presentation at CVPR 201

Metal Artifact Reduction in Sinograms of Dental Computed Tomography

Use of metal objects such as dental implants, fillings, crowns, screws, nails, prosthesis and plates have increased in dentistry over the past 20 years, which raised a need for new methods for reducing the metal artifacts in medical images. Although there are several algorithms for metal artifact reduction, none of these algorithms are efficient enough to recover the original image free of all artifacts. This thesis presents two approaches for reducing metal artifacts through accurate segmentation of metal objects on dental computed tomography images. First approach was based on construction and tilting of a 3D jaw phantom, aiming to obtain fewer metals on each slice. 3D jaw phantom included the main anatomical structures of a jaw, and multiple metal fillings inserted on the teeth. Each jaw slice on the 3D phantom was tilted in order to mimic the (1) nodding movement, and (2) mouth opening/closing. Second approach was to segment the metals on an experimental dataset, consisting of a Cone-Beam Computed Tomography image, by using different segmentation algorithms. K-means clustering, Otsu’s thresholding method and logarithmic enhancement were used for extracting the metals from a real dental CT slice. Once the metal fillings on the jaw phantom were segmented out from the image, they were compensated by gap filling methods; Discrete Cosine Domain Gap Filling and inpainting. Qualitative and quantitative analyses were carried out for evaluating the performance of implemented segmentation methods. Efficiency of tilting alternatives on the segmentation of metal fillings was compared. In conclusion, jaw opening/closing movement between 24º-30º suggested a significant enhancement in segmentation, thus, metal artifact reduction on the jaw phantom. Inpainting method showed a better performance for both simulated and experimental dataset over the DCT domain gap filling method. Moreover, merging the logarithmic enhancement and inpainting showed superior results over other metal artifact reduction alternatives

Automatic alignment for three-dimensional tomographic reconstruction

In tomographic reconstruction, the goal is to reconstruct an unknown object from a collection of line integrals. Given a complete sampling of such line integrals for various angles and directions, explicit inverse formulas exist to reconstruct the object. Given noisy and incomplete measurements, the inverse problem is typically solved through a regularized least-squares approach. A challenge for both approaches is that in practice the exact directions and offsets of the x-rays are only known approximately due to, e.g. calibration errors. Such errors lead to artifacts in the reconstructed image. In the case of sufficient sampling and geometrically simple misalignment, the measurements can be corrected by exploiting so-called consistency conditions. In other cases, such conditions may not apply and we have to solve an additional inverse problem to retrieve the angles and shifts. In this paper we propose a general algorithmic framework for retrieving these parameters in conjunction with an algebraic reconstruction technique. The proposed approach is illustrated by numerical examples for both simulated data and an electron tomography dataset

Demosaicing multi-energy patterned composite pixels for spectral CT

Tese de mestrado integrado em Engenharia Biomédica e Biofísica, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2016O desenvolvimento da Tomografia Computadorizada foi realizada na combinação de duas áreas científicas, computação e imagiologia com base em raios-x. Em 1895, o cientista Wilhelm Roentgen descobriu os raios-X: fotões de altas energias provenientes de transições eletrónicas nos átomos. Estes são radiações eletromagnéticas que se propagam à velocidade da luz e são ionizantes. Devido às suas propriedades, os raios-x foram imediatamente rentabilizados como uma ferramenta para explorar a composição da matéria. Os fotões interagem com a matéria por dois mecanismos dominantes, dependendo da energia da radiação eletromagnética: efeito fotoelétrico e efeito de Compton. O efeito fotoelétrico corresponde à interação dos fotões com os eletrões que se encontram nas órbitas de maior energia do átomo. O fotão transfere toda a sua energia para o eletrão, sendo parte dessa usada para superar a energia de ligação do eletrão e a energia restante é transferida para o mesmo eletrão sob a forma de energia cinética. O efeito de Compton corresponde à interação do fotão com o eletrão que se encontra numa das órbitas de menor energia. Depois da interação, o fotão é desviado e o eletrão é ejetado do átomo. O fotão desviado pode voltar a interagir com a matéria sob o efeito de Compton ou o efeito fotoelétrico, ou simplesmente não a interagir com a matéria. Os raios-X têm a sua intensidade diminuída em função das interações que ocorrem com o material que as absorve. A atenuação da energia destes acontece de maneira exponencial em função da espessura do material absorvente. Devido às propriedades físicas provocadas pelos raios-X, esta radiação foi estabelecida como uma ferramenta médica. A tomografia convencional consistiu numa técnica de diagnóstico na qual a aquisição de imagem é realizada a partir de um filme radiográfico, que resulta da projeção das estruturas anatómicas tridimensionais em imagens bidimensionais, com sobreposições de informação anatómica. Em 1970, os cientistas Hounsfield e Cormack desenvolveram uma técnica, a Tomografia Computadorizada, que possuía logo de início a vantagem de corrigir o problema da sobreposição de informação. A Tomografia Computadorizada reconstrói as estruturas internas de um objeto a partir de múltiplas projeções utilizando algoritmos de reconstrução. A diferenciação e classificação de diferentes tipos de tecidos tornou-se extremamente desafiante nesta técnica, devido ao facto de que mesmo que dois materiais difiram em número atómico, dependendo da densidade de massa ou concentração, eles podem aparecer idênticos na imagem. Desta forma uma das soluções foi o estudo da Tomografia Computorizada Espectral, sendo esta uma técnica promissora no desenvolvimento da imagiologia pois potencia a deteção e caracterização dos tecidos anatómicos além dos níveis atualmente atingíveis com técnicas de TC convencionais. A TC espectral leva em consideração que a radiação transmitida transporta mais informações para além de mudanças de intensidade e que o coeficiente de atenuação depende não só do material, mas também da energia do fotão. A TC espectral difere das outras técnicas no sentido em que utiliza as características físicas dos materiais em estudo em mais de dois espectros de energia. Através da aquisição de imagens em diferentes níveis de energia, a técnica é capaz de diferenciar os vários elementos do corpo com base na densidade dos materiais ou nos números atómicos destes. As diferenças entre os vários tecidos são exibidas através de distintas cores na imagem final. Uma tecnologia importante utilizada na CT Espectral é a dos detetores de contagem de fotões, conhecidos por detetores híbridos. Estes detetores têm a particularidade de separar o espetro incidente em múltiplos espetros, cuja forma depende dos limiares de energia impostos. Estes detetores operam num modo de contagem, ou seja, em vez de operarem em modo de integração tal como os detetores convencionais, estes efetuam a contagem individual dos fotões da radiação incidente a partir de limiares de energia estipulados. A influência do ruído electrónico afeta a energia medida de cada fotão, contudo tendo em conta que estes detetores efetuam a contagem de fotões, o ruído eletrónico deixa de ter uma influência tão significativa na qualidade da imagem adquirida. “K-edge Imaging” é uma das abordagens utilizadas em sistemas de TC espectral; explora as propriedades físicas de agentes de contrastes utilizados em tomografia computorizada e as suas respetivas propriedades físicas. Os elementos utilizados para os agentes contrastes são elementos pesados e altamente atenuantes, e cujo efeito fotoelétrico ocorre ao mesmo alcance das energias utilizadas em TC. Deste modo, cada um desses elementos pesados tem um salto característico na sua atenuação de raios-X, o qual corresponde à energia que ocorre o efeito fotoelétrico. Como os eletrões envolvidos no efeito fotoelétrico pertencem à orbital K, o salto característico é designado por "K-edge". “K-edge Imaging” explora a escolha do espetro de energia aplicado de forma a abranger o salto característico destes elementos para identificar e localizar componentes específicos. No CPPM, o grupo imXgam desenvolveu uma micro-TC e uma PET / TC simultânea que incorpora a nova tecnologia de detetores híbridos desenvolvida pelo centro: o detetor XPAD3. Esta tecnologia não só permite trabalhar em modo de contagem de fotões, mas também é capaz de selecionar informação energética sobre os fotões detetados; consequentemente as capacidades do detector XPAD3 foram exploradas para desenvolver “K-edge Imaging”. Os artefactos que resultam de várias aquisições estão relacionados com o movimento. Para resolver esse problema, o CPPM desenvolveu um conceito de pixéis compostos, que consiste numa matriz de pixéis (3 × 3) com 3 diferentes limiares de energia. Embora, os pixéis compostos resolvam os artefactos de movimento, as imagens adquiridas perderam a resolução espacial. Assim, o projeto deste trabalho tem como objetivo a realização de "K-edge Imaging" em objectos em movimento em plena resolução espacial. Este projeto aborda o problema como um problema “Inpainting”, onde as medidas desconhecidas para cada limiar de energia serão estimadas a partir de medidas parciais. Há uma vasta literatura sobre o problema “Inpainting”, assim como noutra área de processamento de imagem, o “Demosaicing”. Estes são métodos de restauração que removem regiões danificadas ou reconstroem porções perdidas da imagem. O problema “Demosaicing” tem um interesse particular para este trabalho em virtude do método recuperar informação de imagens coloridas (imagens RGB). A utilização do método “Demosaicing” em imagens adquiridas por sistemas TC é praticamente inexistente, pelo que o objetivo deste projeto foi avaliar não só os métodos de restauração convencionais, mas também adaptar e avaliar o método “Demosaicing” às imagens adquiridas por sistemas TC. Desta forma, as imagens espectrais foram tratadas como imagens coloridas: cada imagem adquirida por um limiar de energia foi configurada como uma cor. A imagem resultante foi submetida ao processo de recuperação que consistiu em acoplar as três imagens obtidas por cada limiar de energia em uma imagem de cor( imagem RGB). Este trabalho exigiu, em primeiro lugar, o estudo do esquema de amostragem de imagens espectrais e a avaliação de desempenho dos métodos mais simples em relação ao ruído, ao fator de subamostragem e à resolução espacial. As técnicas mais sofisticadas como a “Inpainting” e ”Demosaicing” foram desenvolvidas e avaliadas especificamente para imagens espectrais tomográficas. Após a avaliação destas, foi realizado um “estado de arte” que comparou os métodos e, consequentemente, fez uma análise de qual o método mais adequado para imagens de TC espectral. A segunda parte deste projeto consistiu no estudo do padrão que os píxeis compostos devem seguir, de forma a definir um protocolo de aquisição. Para tal, foram testados dois tipos de padrões: regular e aleatório. A ideia de píxeis compostos foi obtida criando uma matriz com vários componentes que dependem do número de limiar de energias que se quer utilizar. Conforme mencionado, no CPPM é utilizado uma matriz de pixels com três limiares de energia, desta forma, neste projeto, a possibilidade de aumentar o número de limiares de energia foi também testado. Os objetivos do projeto foram alcançados uma vez que a avaliação dos métodos foi realizada e conclui-se que a nova abordagem apresentou melhores resultados que os métodos padrão. Conclui-se que as imagens adquiridas pelo método “Demosaicing” apresentam melhor resolução espacial. Relativamente ao padrão dos pixéis compostos verificou-se que em ambos a reconstrução apresentou bom desempenho. A análise do aumento de número de limiares de energia apontou para bons resultados, observados no uso de 4 níveis de energia, porém a nova abordagem “Demosaicing” teria de ser reformulada. De forma a alcançar os objetivos, este tema foi dividido em vários capítulos. No segundo capítulo foram introduzidos os conceitos físicos envolvidos na tomografia espectral, desde a produção dos raios-X até ao desenvolvimento da técnica propriamente dita. O terceiro capítulo abordou como o “estado de arte” foi efetuado, documentando o que foi realizado atualmente no campo em estudo. Nos capítulos 4 e 5 apresentou-se os materiais e métodos utilizados, assim como exposto as suas aplicações,e de forma mais particular a matemática e a programação envolvidas. No capítulo 6 apresentou-se os resultados alcançados e as respectivas observações. No último capítulo sumariou-se os resultados obtidos e as conclusões retiradas a partir destes.Computed Tomography is a diagnosis technique that uses X-ray radiation to create images of structures. This technique consists in reconstructing a quantitative map of the attenuation coefficients of the object sections from multiple projections using reconstruction algorithms. Since the attenuation coefficient is not unique for any material, the differentiation and classification of different tissue types by Computed Tomography has revealed to be extremely challenging. The solution has been provided through the development of an energy sensitive CT scanner, known as Spectral CT. This technique takes in consideration that the transmitted radiation carries more information than intensity changes, that the x-ray tube produces a wide range of energy spectrum and that the attenuation of radiation depends not only on the material but also on the photon energy. Spectral CT uses the attenuation characteristics at more than two energies which makes it possible to differentiate various elements in the body, based on their material density or atomic numbers. Therefore, this technique uses the new detector technology, the hybrid pixel detector. This detector allows the energy threshold setting. Combining the physical properties of different materials and the possibility of setting the energy threshold in the detectors, a new spectral imaging technique is used, K-edge imaging. This technique explores the discontinuity in the photoelectric effect, which is generated when photons interact with matter, and those interact with the shell electrons. Therefore, the Centre de Physique des Particules de Marseille developed a micro-CT and a simultaneous PET/CT scan based on hybrid pixel detector. The ability of tuning the energy threshold of each pixel independently was exploited to develop K-edge imaging and the proof of concept has been established on phantom and on living mice. In the context of pre-clinical imaging, objects are moving and the several acquisitions must be performed simultaneously to allow the registration set. For this purpose, CPPM had been working with composite pixels made of 9 (3× 3) pixels with 3 different thresholds. This solves the motion artefact problem at the price of loss in spatial resolution. Therefore, the research project of this work aims at performing K-edge imaging on moving object at full spatial resolution. The problem is seen as an Inpainting problem where unknown measure must be estimated from partial measurements. A huge literature exists in the Inpainting, and especially in the field of Demosaicing, which is particularity of interest in this research project. The project consists in a study of the sampling scheme of spectral CT images and to evaluate the performance of simplest methods with respect to noise and spatial resolution. More sophisticated techniques of Inpainting and Demosaicing were tested, which were developed specifically for spectral CT images by incorporating prior on image. Therefore, an evaluation performance of all the reconstruction methods was successfully made, and a state-of-art was established. In this research project, in order to create the composite pixels concept, a set of dynamic strategies of patterning composite pixels was achieved in order to define optimal protocols of acquisition

Implementation of a metal artifact reduction methods for small-animal CT

In the recent years, because of the constantly increasing knew discoveries in the fields of genomics and molecular biology and the development of new technologies, the use of animal models of human diseases has become more frequent. This combined with improvements in biomedical instrumentation and medical imaging has led to the development of micro CT systems enabling noninvasive investigations on animals. The work included in this thesis is framed on one of the lines of research carried out by the Biomedical Imaging and Instrumentation Group (BIIG) from the Bioengineering and Aerospace Department of Universidad Carlos III de Madrid working jointly with the Gregorio Marañón Hospital. This multidisciplinary group has developed a micro-CT system for small animals, which is used in different preclinical research lines within the group. One of these research lines focuses on the use of brain stimulation as Parkinson disease treatment. Rats have stainless electrodes implanted and fixed with screws in the lateral hypothalamus, through stereotaxic surgery. The CT subsystem of ARGUS is used then to corroborate the surgery was correct and the position of the electrodes is the right one. The presence of metallic objects creates severe streak artifacts in CT images affecting image quality and hindering the correct representation of anatomy. The beam hardening correction method, already integrated in the ARGUS system results insufficient for the correction of the artifacts derived from the presence of metals. Motivated by this context, the objective of this thesis is to implement an algorithm for metal artifact correction to be included in ARGUS. After reviewing the methods proposed in the literature the one proposed by Meyer et. al. in 2012 was implemented in MATLAB. The implemented MAR method was evaluated using simulations and real studies acquired with the ARGUS scanner, based on visual assessment, intensity profiles and mean squared error before and after the correction. The results of the evaluation showed an efficient elimination of streaks even for very strong artifact, as it is the case of gold implants. In all cases, bone edges were preserved when correcting with MAR and the metal structures are clearly delimited after correction.Ingeniería Biomédic

Advanced machine learning methods for oncological image analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally- invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head- neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra- dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis