Uncertainty quantification in non-rigid image registration via stochastic gradient Markov chain Monte Carlo

We develop a new Bayesian model for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backpropagation and the variational inference by backpropagation frameworks, in order to efficiently draw samples from the posterior distribution of transformation parameters. To address the modelling issues, we formulate a Bayesian model for image registration that overcomes the existing barriers when using a dense, high-dimensional, and diffeomorphic transformation parametrisation. This results in improved calibration of uncertainty estimates. We compare the model in terms of both image registration accuracy and uncertainty quantification to VoxelMorph, a state-of-the-art image registration model based on deep learning

Image registration via stochastic gradient markov chain monte carlo

We develop a fully Bayesian framework for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images along with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backprop and the variational inference by backprop frameworks in order to efficiently draw thousands of samples from the posterior distribution. Regarding the modelling issues, we carefully design a Bayesian model for registration to overcome the existing barriers when using a dense, high-dimensional, and diffeomorphic parameterisation of the transformation. This results in improved calibration of uncertainty estimates

Bayesian modelling of organ deformations in radiotherapy

Moderne strålebehandling mot kreft er skreddarsydd for å gje ein høg stråledose tilpassa svulsten (målvolumet), mens så lite dose som mogleg vert gitt til det friske vevet omkring. Den totale dosen vert levert over nokre veker i daglege "fraksjonar", noko som reduserer biverknader. Under og mellom desse fraksjonane rører dei indre organa på seg heile tida på grunn av pust, fylling av blæra, tarmar si rørsle og ekstern påverknad. Likevel vert posisjonen til målvolumet og relevante risikoorgan bestemt på grunnlag av eit statisk 3D-skann som er tatt før behandlinga startar. Den vanlege måten å sikre seg mot konsekvensar av denne rørsla er å legge til marginar rundt svulsten. Slik sikrar ein å treffe målvolumet, men til gjengjeld får det friske vevet meir dose. Marginane sin storleik er fastsett ved hjelp av statistikk over tidlegare behandla pasienter. Dei statistiske metodane som vert brukte er ofte enkle, og tek berre omsyn til rigid rørsle, altså at heile kroppen rører seg i eitt. Dessutan vert det ikkje teke omsyn til rørsla til risikoorgan. For å berekne dose til risikoorgana er det vanleg å anta at forma til organa i planleggingsskannet er representative for forma deira under behandling. Arbeidet i denne avhandlinga handlar om å bruka teknikkar frå Bayesiansk statistikk for å modellere korleis organ rører og deformerer seg mellom fraksjonane. Målet er å estimere nøyaktig den statistiske fordelinga av rørsle for eit eller fleire organ til ein pasient. Fordelinga gjev innsikt i korleis organa forandre seg medan behandlinga går for seg. Denne innsikta er nyttig for evaluering av stråleterapiplanar, statistisk prediksjon av biverknader, såkalla robust planlegging og å berekna størrelsen på marginar. Metodane som vert presentert er evaluerte for endetarmen (rektum) sine rørsler hjå prostatakreftpasientar. For desse pasientane er rektum eit viktig risikoorgan, som kan bli ramma både av akutte og seine biverknader, som lekkasje, bløding og smerter. Samanlikna med eksisterande metodar har den Bayesianske tilnærminga to fordelar: For det første gir kombinasjonen av populasjonsstatistikk og individuelle data meir nøyaktige anslag av den pasientspesifikke fordelinga. For det andre estimerer dei nye metodane den såkalla systematiske feilen i tillegg til variasjonar frå fraksjon til fraksjon. Den systematiske feilen er forskjellen mellom den estimerte forma på organet under planlegging, og gjennomsnittsforma til organet under bestråling. Denne typen feil var tema for artikkel I. Her fekk vi til å redusere den systematiske feilen til rektum hjå 33 av 37 prostatakreftpasientar ved å bruke ein metode som kombinerer forma på rektum under planlegginga og gjennomsnittsforma i populasjonen. Vi vurderte og om denne forbetringa hadde påverknad på estimering av summert dose til rektum. Metoden gav ikkje signifikant forbetring for to antatt relevante parametrar (ekvivalent uniform dose og D5%), men gav signifikant reduksjon av bias på det estimerte dose-volum-histogrammet i intervallet 52.5 Gy til 65 Gy. Hovudarbeidet i dette prosjektet er publisert i artikkel II. Der presenterer vi to modellar for organrørsle basert på Bayesianske metodar. Inndata til desse metodane er organformer som er henta frå 3D-skanningar. Metodane kan ta ulikt tal slike former, og produserer meir nøyaktige resultat jo fleire former dei får. Dei gjev anslag av gjennomsnittsforma og kor stor uvissa om denne forma er, i tillegg til anslag av fordelinga av variasjon av former frå fraksjon til fraksjon. Vi evaluerte metodane etter kor godt dei kunne berekne "dekningssannsyn", altså sannsynet for at organet skal dekke eit gitt punkt i pasientkoordinatsystemet til ei gitt tid. For denne berekninga måtte titusenvis av organformer gjerast om til såkalla binærmasker, som er 3D-matriser av punkter i pasient-koordinatsystemet der verdien til eit punkt er 1 dersom punktet er inne i organet, og 0 elles. Denne berekninga var mogleg på grunn av programvare som blei implementert for dette prosjektet, og som er presentert i artikkel III. Også her var det prostatakreftpasientar sitt rektum som vart brukt til evaluering. Berekningane til dei nye metodane var likare det sanne dekningssannsynet enn tilsvarande berekningar frå tidlegare metodar, i signifikant grad, i alle fall opp til tre input. Forskjellen mellom dei to nye algoritmane er i hovudsak kompleksiteten og nøyaktigheita, og valet mellom algoritmane i ein gitt bruk vil vere ei avveging mellom desse faktorane. Vi viste ein måte modellane kan verte brukte i artikkel IV, som handlar om pasientar som får re-bestråling for tilbakefall av prostatakreft. Her brukte vi modellane til å berekne forventa akkumulert dose til rektum frå dei to behandlingane, og også uvissa rundt den forventa dosen. Metoden er basert på representative former" av rektum, altså former som rektum kan ta som er sannsynlege, men lite fordelaktige. Desse formene kan brukast som visuell hjelp for onkologar og doseplanleggjarar, og metoden kan implementerast ved hjelp av eksisterande funksjonar i programvaren for behandlingsplanlegging. Overordna gir denne avhandlinga nye løysingar for den sentrale utfordringa med å redusere konsekvensar av organrørsle i stråleterapi. Dei presenterte modellane er dei første som utnyttar statistikk for populasjonen og data frå den enkelte pasienten samstundes, og som tar omsyn til både systematiske og tilfeldige feil.Modern radiotherapy tends to be highly conformal, meaning that a high and uniform dose is delivered to the target volume and as little dose as possible to the surrounding normal tissue. The total radiation dose is delivered across several smaller daily fractions, typically spanning several weeks. During and between these fractions, internal organs are constantly in motion due to factors such as breathing, changes to bladder filling state, intestinal movement and external influences. Nevertheless, the position of the target and relevant organs at risk (OARs) are determined based on a static 3D scan acquired before start of treatment. A common safeguard which is used to take such motion into account is the addition of margins around the target. These margins reduce the chance of missing parts of the target, yet increases dose to the healthy tissue surrounding the target. The margin size is based on statistics from previous patients. However, for the most part, the statistical methods used are very simple, and typically based on an assumption of rigid patient motion. Similarly, motion of the OARs is commonly neglected. For estimation of dose to the OARs, it is common to assume that the organ shape at the static scan is representative for its shape during treatment. The work in this thesis concerns the use of techniques from Bayesian statistics for modelling inter-fraction organ motion and deformation. The goal is to estimate accurately the statistical distribution of shapes for one or more organs for a given patient. The distribution provides knowledge of how the patient's organs might move and deform during the radiotherapy course. This information is useful for the evaluation of radiotherapy plans, prediction of adverse effects, so-called motion-robust radiotherapy planning, the generation of margins and more. The methods presented in this thesis have been evaluated for predicting deformations of the rectum of prostate cancer patients. For these patients, the rectum is a crucial OAR that is affected by both early and late side effects including leakage, bleeding and pain. Compared to existing methods, the Bayesian approach developed and implemented in this thesis offers two advantages: first, combining population statistics and individual data leads to more accurate estimates of the patient-specific distribution. Secondly, the new methods estimate the distribution of the so-called systematic error in addition to variations from fraction to fraction. The systematic error is the difference between the estimated shape/position of an organ at the planning stage and its average shape/position during therapy, and was the subject of paper I. Here, we were able to reduce the systematic error of the rectum in 33 out of 37 prostate cancer patients using a straightforward method to combine the shape of the rectum at the planning CT with the population mean shape. We also evaluated the impact of this improvement on the estimation of dose to the rectum. We found no significant improvement on the estimation of two presumably relevant dose parameters (equivalent uniform dose and D5%). However, we did find significant reduction in the bias of the estimated dose-volume histogram in the range from 52.5 Gy to 65 Gy. Paper II contains the central work of this project. It presents two organ deformation models based on Bayesian methods. The input data to these algorithms are organ shapes derived from 3D scans. The methods can take a varying number of such inputs from a given patient, and will produce more accurate results the more inputs they are given. They provide an estimate of the mean shape of the organ, as well as the uncertainty of this mean, in addition to the distribution of the variation of shapes from fraction to fraction. The methods were evaluated in the task of estimating coverage probabilities, i.e. the probability that the organ will cover a certain point in the patient coordinate system, for the rectum of prostate cancer patients. For this evaluation, tens of thousands of organ shapes needed to be converted to so-called binary masks, which are 3D arrays of points in the patient coordinate system where the value of each point is 1 if the point is inside the organ and 0 if it is outside. This was enabled by the highly efficient point-in-polyhedron software presented in paper III, which was developed for this project. The models were given varying number of scans, from 1 to 10, as input, and compared to two existing (non-Bayesian) models. The estimates of the coverage probability produced by the new models were significantly more similar to the ground truth than those produced by the existing models, at least up to three input scans. The main differences between the two new algorithms are their of conceptual complexity and accuracy, and the choice of method in a given application will therefore come down to a trade-off between these qualities. An application for the models derived in paper II, concerning patients receiving re-irradiation for recurrent prostate cancer, is presented in paper IV. We introduce a way of estimating the expectation and uncertainty of the accumulated dose to the rectum from the two treatment courses. The method is based on "representative shapes" of the rectum, that is, shapes that are probable and also particularly favourable or unfavourable in terms of dose. The advantage is that these shapes can be used as a visual aid for the oncologist or dose planner, and that the method can be implemented using existing features of treatment planning systems. Overall, this thesis provides novel solutions to the central challenge of organ motion mitigation in RT. The presented models are the first to simultaneously exploit population and patient specific organ motion and addressing both systematic and random errors.Doktorgradsavhandlin

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Advanced Algorithms for 3D Medical Image Data Fusion in Specific Medical Problems

Fúze obrazu je dnes jednou z nejběžnějších avšak stále velmi diskutovanou oblastí v lékařském zobrazování a hraje důležitou roli ve všech oblastech lékařské péče jako je diagnóza, léčba a chirurgie. V této dizertační práci jsou představeny tři projekty, které jsou velmi úzce spojeny s oblastí fúze medicínských dat. První projekt pojednává o 3D CT subtrakční angiografii dolních končetin. V práci je využito kombinace kontrastních a nekontrastních dat pro získání kompletního cévního stromu. Druhý projekt se zabývá fúzí DTI a T1 váhovaných MRI dat mozku. Cílem tohoto projektu je zkombinovat stukturální a funkční informace, které umožňují zlepšit znalosti konektivity v mozkové tkáni. Třetí projekt se zabývá metastázemi v CT časových datech páteře. Tento projekt je zaměřen na studium vývoje metastáz uvnitř obratlů ve fúzované časové řadě snímků. Tato dizertační práce představuje novou metodologii pro klasifikaci těchto metastáz. Všechny projekty zmíněné v této dizertační práci byly řešeny v rámci pracovní skupiny zabývající se analýzou lékařských dat, kterou vedl pan Prof. Jiří Jan. Tato dizertační práce obsahuje registrační část prvního a klasifikační část třetího projektu. Druhý projekt je představen kompletně. Další část prvního a třetího projektu, obsahující specifické předzpracování dat, jsou obsaženy v disertační práci mého kolegy Ing. Romana Petera.Image fusion is one of today´s most common and still challenging tasks in medical imaging and it plays crucial role in all areas of medical care such as diagnosis, treatment and surgery. Three projects crucially dependent on image fusion are introduced in this thesis. The first project deals with the 3D CT subtraction angiography of lower limbs. It combines pre-contrast and contrast enhanced data to extract the blood vessel tree. The second project fuses the DTI and T1-weighted MRI brain data. The aim of this project is to combine the brain structural and functional information that purvey improved knowledge about intrinsic brain connectivity. The third project deals with the time series of CT spine data where the metastases occur. In this project the progression of metastases within the vertebrae is studied based on fusion of the successive elements of the image series. This thesis introduces new methodology of classifying metastatic tissue. All the projects mentioned in this thesis have been solved by the medical image analysis group led by Prof. Jiří Jan. This dissertation concerns primarily the registration part of the first project and the classification part of the third project. The second project is described completely. The other parts of the first and third project, including the specific preprocessing of the data, are introduced in detail in the dissertation thesis of my colleague Roman Peter, M.Sc.

Constrained Stochastic State Estimation of Deformable 1D Objects: Application to Single-view 3D Reconstruction of Catheters with Radio-opaque Markers

International audienceMinimally invasive fluoroscopy-based procedures are the gold standard for diagnosis and treatment of various pathologies of the cardiovascular system. This kind of procedures imply for the clinicians to infer the 3D shape of the device from 2D images, which is known to be an ill-posed 10 problem. In this paper we present a method to reconstruct the 3D shape of the interventional device, with the aim of improving the navigation. The method combines a physics-based simulation with non-linear Bayesian filter. Whereas the physics-based model provides a prediction of the shape of the device navigating within the blood vessels (taking into account non-linear interactions be-15 tween the catheter and the surrounding anatomy), an Unscented Kalman Filter is used to correct the navigation model using 2D image features as external observations. The proposed framework has been evaluated on both synthetic and real data, under different model parameterizations, filter parameters tuning and external observations data-sets. Comparing the reconstructed 3D shape with a known ground truth, for the synthetic data-set, we obtained average values for 3D Hausdorff Distance of $0.81 ± 0.53 mm$, for the 3D mean distance at the segment of $0.37 ± 0.17$ mm and an average 3D tip error of $0.24 ± 0.13 mm$. For the real data-set,we obtained an average 3D Hausdorff distance of $1.74 ± 0.77 mm$, a average 3D mean distance at the distal segment of 0.91 ± 0.14 mm, an average 3D error on the tip of $0.53 ± 0.09 mm$. These results show the ability of our method to retrieve the 3D shape of the device, under a variety of filter parameterizations and challenging conditions: uncertainties on model parameterization, ambiguous views and non-linear complex phenomena such as stick and slip motions

Constrained Stochastic State Estimation for 3D Shape Reconstruction of Catheters and Guidewires in Fluoroscopic Images

Minimally invasive fluoroscopy-based procedures are the gold standard for diagnosis and treatment of various pathologies of the cardiovascular system. This kind of procedures imply for the clinicians to infer the 3D shape of the device from 2D images, which is known to be an ill-posed problem. In this paper we present a method to reconstruct the 3D shape of the interventional device, with the aim of improving the navigation. The method combines a physics-based simulation with non-linear Bayesian filter. Whereas the physics-based model provides a prediction of the shape of the device navigating within the blood vessels (taking into account non-linear interactions between the catheter and the surrounding anatomy), an Unscented Kalman Filter is used to correct the navigation model using 2D image features as external observations. The proposed framework has been evaluated on both synthetic and real data, under different model parameterization, filter parameters tuning and external observations data-sets. Comparing the reconstructed 3D shape with a known ground truth, for the synthetic data-set, we obtained an average 3D Hausdorff distance of 0.07 ± 0.37 mm; the 3D distance at the tip equal to 0.021 ± 0.009 mm and the 3D mean distance at the distal segment of the catheter equal to 0.02 ± 0.008 mm. For the real data-set, the obtained average 3D Hausdorff Distance was of 0.95 ± 0.35 mm, the average 3D distance at the tip is equal to 0.7 ± 0.45 mm with an average 3D mean distance at the distal segment of 0.7 ± 0.46 mm. These results show the ability of our method to retrieve the 3D shape of the device, under a variety of filter parameterizations and challenging conditions: errors on the friction coefficient, ambiguous views and non-linear complex phenomena such as stick and slip motions