Fiber Orientation Estimation Guided by a Deep Network

Diffusion magnetic resonance imaging (dMRI) is currently the only tool for noninvasively imaging the brain's white matter tracts. The fiber orientation (FO) is a key feature computed from dMRI for fiber tract reconstruction. Because the number of FOs in a voxel is usually small, dictionary-based sparse reconstruction has been used to estimate FOs with a relatively small number of diffusion gradients. However, accurate FO estimation in regions with complex FO configurations in the presence of noise can still be challenging. In this work we explore the use of a deep network for FO estimation in a dictionary-based framework and propose an algorithm named Fiber Orientation Reconstruction guided by a Deep Network (FORDN). FORDN consists of two steps. First, we use a smaller dictionary encoding coarse basis FOs to represent the diffusion signals. To estimate the mixture fractions of the dictionary atoms (and thus coarse FOs), a deep network is designed specifically for solving the sparse reconstruction problem. Here, the smaller dictionary is used to reduce the computational cost of training. Second, the coarse FOs inform the final FO estimation, where a larger dictionary encoding dense basis FOs is used and a weighted l1-norm regularized least squares problem is solved to encourage FOs that are consistent with the network output. FORDN was evaluated and compared with state-of-the-art algorithms that estimate FOs using sparse reconstruction on simulated and real dMRI data, and the results demonstrate the benefit of using a deep network for FO estimation.Comment: A shorter version is accepted by MICCAI 201

Trifocal Relative Pose from Lines at Points and its Efficient Solution

We present a new minimal problem for relative pose estimation mixing point features with lines incident at points observed in three views and its efficient homotopy continuation solver. We demonstrate the generality of the approach by analyzing and solving an additional problem with mixed point and line correspondences in three views. The minimal problems include correspondences of (i) three points and one line and (ii) three points and two lines through two of the points which is reported and analyzed here for the first time. These are difficult to solve, as they have 216 and - as shown here - 312 solutions, but cover important practical situations when line and point features appear together, e.g., in urban scenes or when observing curves. We demonstrate that even such difficult problems can be solved robustly using a suitable homotopy continuation technique and we provide an implementation optimized for minimal problems that can be integrated into engineering applications. Our simulated and real experiments demonstrate our solvers in the camera geometry computation task in structure from motion. We show that new solvers allow for reconstructing challenging scenes where the standard two-view initialization of structure from motion fails.Comment: This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while most authors were in residence at Brown University's Institute for Computational and Experimental Research in Mathematics -- ICERM, in Providence, R

From Diffusion to Tracts

Diffusion of water molecules within the brain tissue can be used to modulate the nuclear magnetic resonance signal that is used to form magnetic resonance images (MRI). As the signal itself can be noisy and its meaning challenging to interpret, mathematical models are generally fitted to these measurements to obtain the more accurate characterization of the brain microstructure. This, of course, requires that the mathematical model itself is sound in respect to the measurement setup. This dissertation focuses on the extensively used tensor models as they have been shown to unravel details of the physical diffusion phenomena along with various applications in the basic neuroscience, the clinical research, and even in the neurosurgery. One of the greatest challenges in the diffusion weighted MRI measurements is subject motion during the image acquisition as that can cause a complete loss of the measurement which is especially highlighted in ill or uncooperative patients studies. Due to the used acquisition technique, this loss extends to multiple measurements simultaneously resulting in an enormous gap in the sampling. Such gaps can be problematic for any model fitting, even for the currently available robust means developed to exclude outlier measurements from affecting the estimate. Hence in this dissertation, a tool coined as SOLID was developed to detect these outliers and to robustly process them during the tensor based model estimation. SOLID was implemented as a part of the widely used ExploreDTI toolbox to allow the rapid international distribution of the tool. Unfortunately, any reduction in the measurement sampling will lead to increasing error propagation during the model estimation. Mathematically this is detailed in terms of a condition number for the matrix inversion in the linear least squares fitting. Previously, the condition number has been used to optimize the diffusion weighted MRI acquisition gradient scheme but in this dissertation it was renovated into a novel quality control tool. The condition number of the matrix inversion that provides the model estimate can be calculated after the outliers are excluded to assess spatially and directionally varying error propagation to obviate any bias in subject or population studies. To motivate the importance of the robust methods and diffusion weighted MRI at large, neurocognitive studies with neonates’ visual abilities and bilinguals’ acquisition age of the second language were conducted as a part of this thesis. The findings in these studies indicated that premature birth affects the white matter structures across the brain whereas the age of acquisition of the second language affects only the speech related brain structures.Aivojen rakenteessa tapahtuvien muutosten mittaaminen on avainasemassa tutkittaessa esimerkiksi keskosena syntyneen lapsen kehitystä tai uusien taitojen, kuten kielten, oppimista. Ihmisaivojen tutkiminen on aiemmin rajoittunut aivojen toiminnan arviointiin aivosähkökäyrän ja neurokognitiivisten testien avulla. Viime vuosikymmenten kehitys magneettikuvaustekniikassa on tuonut mahdollisuuden tutkia kajoamattomasti myös aivojen rakennetta ja jopa seurata sen muutosta lapsen kasvaessa tai ihmisen oppiessa uusia taitoja. Yksi lupaavimmista aivojen tutkimusmenetelmistä on diffuusiopainotettu magneettikuvaus, jolle on löytynyt lukuisia käyttökohteita niin neurotieteessä, lääketieteellisissä tutkimuksissa kuin neurokirurgiassakin. Menetelmä perustuu vesimolekyylien lämpöliikkeen mittaamiseen aivoissa. Molekyylien liike on vapaata muun muassa valkean aineen rakenteiden myötäisesti, mutta lähes mahdotonta kohtisuoraan niiden lävitse. Jäljittämällä nämä reitit voidaan muodostaa tarkka malli aivojen rakenteesta. Mallin pohjalta on mahdollista laskea kuvaavia arvoja, jotka auttavat esimerkiksi määrittämään aivovaurion astetta. Diffuusiopainotetun magneettikuvauksen suurin haaste on menetelmän monimutkaisuus sekä mittauksen että analyysin osalta. Vain hyvin yksinkertaisissa tapauksissa asiantuntija voi arvioida suoraan diffuusiopainotetusta magneettikuvasta poikkeamia aivoissa. Yleensä käytetään matemaattisia menetelmiä kuvan tarkempaan analysointiin. Tällöin keskeistä on inversio-ongelman ratkaisu, missä potilaasta tehdyt mittaukset sovitetaan aivoja kuvaavaan matemaattiseen malliin. Sopivan mallin valinnalla on siis suuri vaikutus lopputuloksen hyödyllisyyteen. Diffuusiopainotettu magneettikuvaus on myös häiriöherkkä ja mittaukset sisältävät luonnostaan paljon kohinaa, jonka vaikutusta vähennetään tekemällä toistomittauksia. Toistomittaukset pidentävät kuvausaikaa, joka puolestaan voi olla haasteellinen potilaalle, koska potilaan pitää olla liikkumatta koko kuvauksen ajan. Potilaan pään pienikin liike voi johtaa huomattaviin mittavirheisiin, koska menetelmällä mitataan vesimolekyylien liikettä, jonka suuruus on vain kymmenien mikrometrien luokkaa. Tässä fysiikan väitöskirjassa keskityttiin diffuusiopainotetun magneettikuvauksen mallintamismenetelmien kehitystyöhön ja niiden käyttöönottoon Helsingin yliopistollisessa sairaalassa. Kehitimme kansainvälistä huomiota herättäneen SOLID-työkalun, jolla voidaan havaita sekä korjata potilaan liikkeestä aiheutuvia virheitä mittaustuloksissa. Tämän lisäksi esitimme laadunvalvonta menetelmän, jolla voidaan arvioida esimerkiksi potilaiden välisten mallinnustulosten vertailukelpoisuutta. Kehitettyjä menetelmiä testattiin ja sovellettiin kahdessa tutkimuksessa: Osoitimme, että vastasyntyneen lapsen kyky seurata katseellaan liikkuvaa kohdetta liittyy laaja-alaisiin muutoksiin aivojen valkean aineen rakenteessa. Lisäksi näytimme, että toisen kielen oppimisajankohta vaikuttaa aivojen puheentuottoon liittyvien aivorakenteiden muodostumiseen

Bayesian uncertainty quantification in linear models for diffusion MRI

Diffusion MRI (dMRI) is a valuable tool in the assessment of tissue microstructure. By fitting a model to the dMRI signal it is possible to derive various quantitative features. Several of the most popular dMRI signal models are expansions in an appropriately chosen basis, where the coefficients are determined using some variation of least-squares. However, such approaches lack any notion of uncertainty, which could be valuable in e.g. group analyses. In this work, we use a probabilistic interpretation of linear least-squares methods to recast popular dMRI models as Bayesian ones. This makes it possible to quantify the uncertainty of any derived quantity. In particular, for quantities that are affine functions of the coefficients, the posterior distribution can be expressed in closed-form. We simulated measurements from single- and double-tensor models where the correct values of several quantities are known, to validate that the theoretically derived quantiles agree with those observed empirically. We included results from residual bootstrap for comparison and found good agreement. The validation employed several different models: Diffusion Tensor Imaging (DTI), Mean Apparent Propagator MRI (MAP-MRI) and Constrained Spherical Deconvolution (CSD). We also used in vivo data to visualize maps of quantitative features and corresponding uncertainties, and to show how our approach can be used in a group analysis to downweight subjects with high uncertainty. In summary, we convert successful linear models for dMRI signal estimation to probabilistic models, capable of accurate uncertainty quantification.Comment: Added results from a group analysis and a comparison with residual bootstra

Optimization of the diffusion-weighted MRI processing pipeline for the longitudinal assessment of the brain microstructure in a rat model of Alzheimer’s disease

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Radiações em Diagnóstico e Terapia) Universidade de Lisboa, Faculdade de Ciências, 2019The mechanism that triggers Alzheimer’s disease (AD) is not well-established, with amyloid plaques, neurofibrillary tangles of tau protein, microgliosis and glucose hypometabolism all likely involved in the early cascade. One main advantage of animal models is the possibility to tease out the impact of each insult on the neurodegeneration. Following an intracerebroventricular (icv) injection of streptozotocin (STZ), rats and monkeys develop impaired brain glucose metabolism, i.e. “diabetes of the brain”. Nu-merous studies have reported AD-like features in icv-STZ animals, but this model has never been char-acterized in terms of Magnetic Resonance Imaging (MRI)-derived biomarkers beyond structural brain atrophy. White matter degeneration has been proposed as a promising biomarker for AD that well pre-cedes cortical atrophy and correlates strongly with disease severity. Therefore, this project proposes a longitudinal study of white matter degeneration in icv-STZ rats using diffusion MRI. An existing image processing pipeline was primarily used to obtain preliminary results and propose an optimization strat-egy to improve it in terms of data quality and reliability. These strategies were tested and implemented in the pipeline when confirmed to be valuable, in order to achieve results as reproducible as possible and find the spatio-temporal pattern of brain degeneration in this animal model. All experiments were approved by the local Service for Veterinary Affairs. Male Wistar rats (N=18) (236±11 g) underwent a bilateral icv-injection of either streptozotocin (3 mg/kg, STZ group, N=10) or buffer (control group, CTL, N=8). Rats were scanned at four timepoints following surgery on a 14 T Varian system. Diffusion data were acquired using a semi-adiabatic SE-EPI PGSE sequence as follows: 4 (b=0 ms/μm2), 12 (b=0.8 ms/μm2), 16 (b=1.3 ms/μm2) and 30 (b=2 ms/μm2) directions; TE/TR=48/2500 ms, 9 coronal 1 mm slices, δ/Δ=4/27 ms, FOV=23x17 mm2, matrix=128x64 and 4 shots. The existing image processing pipeline included image denoising and eddy-correction. Moreover, diffusion and kurtosis tensors were calculated for each voxel, producing parametric maps of fractional anisotropy (FA), mean, axial and radial diffusivity (MD, AxD and RD) and mean, axial and radial kur-tosis (MK, AK and RK). Additionally, the two-compartment WMTI-Watson model was further esti-mated to provide specificity to the microstructure assessment. The following metrics were derived from the model: volume water fraction , parallel intra-axonal diffusivity , parallel ,║ and perpendicular extra-axonal diffusivities ,ꓕ and dispersion of fiber orientations 2. Since the model allows for two mathematical solutions, the >,║ solution was retained based on recent evidence. Considering pre-vious findings, the corpus callosum, cingulum, fornix and fimbria were chosen as white matter regions of interest (ROIs) and automatically segmented using anatomical atlas-based registration. Mean diffu-sion metrics were calculated in each ROI for each dataset. CTL and STZ groups were compared using two-sided t-tests at each timepoint. Within-group longitudinal changes were assessed using one-way ANOVA. Because of the small cohort, statistical analysis excluded the last time point. In the course of this project, strategies to optimize the existing pipeline were developed and tested. The existing brain atlas template was supplemented with white matter labels, rat brain extraction was semi-automated, and bias field correction of anatomical data was added before registration. Ventricle enlargement is typically reported in icv-STZ animals and normally constitutes an issue of misalignment in registration. In order to better match the label ROIs with the respective underlying tissue, several registration procedures were tested with different FA and color-coded FA template images. Color-coded FA-based registration dramatically improved the segmentation of the corpus callosum and the fimbria and reliability of diffusion metrics extracted from these regions. Moreover, additional fiber metrics were extracted from a newly developed tractography pipeline to compare with tensors metrics and finally, tensors metrics were evaluated in the gray matter for a more comprehensive spatio-temporal character-ization of brain degeneration. Results from statistical analysis were obtained after implementing the successful optimization strat-egies into the pipeline. There were few significant differences within groups over time. However, be-tween-group differences at each time point were more pronounced. White matter microstructure altera-tions were consistent with previous studies of histology and cognitive performance of the icv-STZ model. Changes in tensors metrics indicate early axonal injury in the fimbria and fornix at 2 weeks after injection, a period of potential recovery at 6 weeks after injection and late axonal injury at 13 weeks in all ROIs. The WMTI-Watson biophysical model provided specificity to the underlying microstructure, by showing intra-axonal damage in the fimbria and corpus callosum as early as 2 weeks, followed by a recover period and definite axonal loss at 13 weeks after injection. Results from tensors metrics and the WMTI-Watson model are not only complementary, they are consistent with each other and with previously-established trends for structural thickness, memory per-formance, amyloid deposition and inflammation. The icv-STZ model displays white matter changes in tracts reportedly affected by AD, while the degeneration is induced primarily by impaired brain glucose metabolism. The icv-STZ constitutes an excellent model to reproduce sporadic AD and should allow to further explore the hypothesis of AD being “type III diabetes”. The combination of diffusion information extracted from tensor imaging and biophysical modelling is a promising set of tools to assess white matter in the AD brain and might be the upcoming strategy to assess the human brain. Regarding future work, it will focus on estimating the correlation between microstructural alterations and functional con-nectivity (from resting-state functional MRI), glucose hypometabolism (from FDG-PET), and patholog-ical features (from histological stainings) – all currently under processing at CIBM. Tractography is a cutting-edge methodology to assess brain connectivity and the pipeline created could be further devel-oped to improve understanding and support diffusion metrics. The relationship between white and gray matter will also improve the understanding of spatio-temporal degeneration and the progression nature of the disease.O mecanismo que desencadeia a doença de Alzheimer (DA) não é bem conhecido, contudo sabe-se que a presença de placas amilóides e de emaranhados neurofibrilares da proteína tau, microgliose e ainda hipometabolismo de glucose estão envolvidos na fase inicial da cascata de desenvolvimento da doença. A principal vantagem dos modelos animais é justamente a possibilidade de estudar individualmente o impacto de cada um destes mecanismos no processo de neurodegeneração. Após uma injeção intracere-broventricular (icv) de estreptozotocina (STZ), várias espécies de animais mostraram um metabolismo anormal de glucose no cérebro, processo que foi referido como “diabetes do cérebro”. Vários estudos demonstraram que animais icv-STZ são portadores de características típicas de DA, mas este modelo animal nunca foi estudado em termos de biomarcadores derivados de técnicas de imagem por ressonân-cia magnética (IRM), exceto atrofia estrutural do cérebro. Um biomarcador promissor de DA que se acredita preceder a atrofia do córtex cerebral é a degeneração da matéria branca do cérebro, uma vez que foi fortemente correlacionado com a progressão e gravidade da doença. Logo, este projeto propõe um estudo longitudinal da degeneração da matéria branca em ratazanas icv-STZ utilizando IRM de di-fusão. O plano de processamento de imagem existente foi utilizado primeiramente para obter resultados preliminares e viabilizar a proposta de estratégias de otimização da mesma, em termos de melhoramento da qualidade de imagem e credibilidade das variáveis extraídas das imagens resultantes. Estas estratégias foram testadas e implementadas no plano de processamento quando a sua performance confirmou ser de valor, para que os resultados fossem o mais reproduzíveis possível em caracterizar a distribuição espácio-temporal da degeneração do cérebro neste modelo animal. Todos os procedimentos aqui descritos foram aprovados pelo serviço local dos assuntos veterinários. Ratazanas macho Wistar (N=18, 236±11 g) foram submetidas a uma injeção icv de STZ (3 mg/kg) no caso do grupo infetado (N=10) ou de um buffer no caso do grupo de controlo (N=8). As ratazanas foram examinadas no scanner de IRM do tipo Varian de 14 T em quatro momentos no tempo: 2, 6, 13 e 21 semanas após a injeção. As imagens por difusão foram adquiridas com uma sequência semi-adiabática spin-echo EPI PGSE com os seguintes parâmetros: 4 (b=0), 12 (b=0.8 ms/μm2), 16 (b=1.3 ms/μm2) and 30 (b=2 ms/μm2) direções; TE/TR=48/2500 ms, 9 secções coronais de 1 mm, δ/Δ=4/27 ms, FOV=23x17 mm2, matriz=128x64 e 4 shots. O plano existente de processamento de imagem incluía a correção das imagens ao nível de ruído e correntes-eddy. Posteriormente, os tensores de difusão e curtose foram estimados para cada voxel e os mapas paramétricos de anisotropia fracional (FA), difusão média, axial e radial (MD, AD e RD) e cur-tose média, axial e radial (MK, AK e RK) foram calculados. Adicionalmente, um modelo de difusão de água nas fibras da matéria branca foi utilizado para providenciar maior especificidade ao estudo da microestrutura do cérebro. Como tal, o modelo de dois compartimentos denominado WMTI-Watson foi também estimado e as seguintes variáveis foram derivadas do mesmo: a fração do volume de água , a difusividade paralela intra-axonal , as difusividades paralela ,║ e perpendicular ,ꓕ extra-axonais e, finalmente, a orientação da dispersão axonal 2. Este modelo matemático tem duas soluções possíveis dada a sua natureza quadrática, pelo que a solução >,║ foi imposta com base em evidências re-centes. Considerando estudos anteriores, as regiões de interesse (RDIs) da matéria branca escolhidas para analisar a microestrutura cerebral foram o corpo caloso, o cíngulo, a fimbria e a fórnix. Estes foram automaticamente segmentados através de registo de imagem de um atlas das regiões do cérebro da rata-zana e as médias das medidas extraídas dos tensores de difusão e curtose e ainda do modelo biofísico neuronal foram calculadas em cada RDI para cada conjunto de imagens obtidas. Os dois grupos de teste e controlo foram comparados usando testes t de Student bilaterais em cada momento do tempo, e a comparação das alterações longitudinais em cada grupo foi feita usando uma ANOVA. Devido ao baixo número de amostras, o último momento no tempo às 21 semanas foi excluído da análise. No decorrer deste projeto, várias estratégias para otimizar o processamento de imagem ou comple-mentar a análise da informação disponível foram testadas. Nomeadamente, o atlas cerebral da ratazana foi aperfeiçoado relativamente às regiões de matéria branca, a segmentação do cérebro foi testada com algoritmos automáticos e a correção do bias field em imagens estruturais de IRM foi adicionada ao plano antes do registo de imagem. O aumento dos ventrículos cerebrais é uma característica frequente em animais icv-STZ, constituindo um problema de alinhamento nos métodos de registo de imagem. No sentido de otimizar a correspondência entre as regiões do atlas e as respetivas regiões na imagem estru-tural e por difusão, vários procedimentos de registo de imagem foram testados. O co-registo de imagem convencional utiliza imagens estruturais para normalizar o espaço das imagens por difusão, no entanto os mapas paramétricos de FA têm vindo a substituir este conceito dado o excelente contraste que provi-denciam entre a matéria branca e cinzenta do cérebro. Mapas de FA com diferentes direções predomi-nantes mostraram uma melhoria significante da segmentação do corpo caloso e da fimbria e também do poder estatístico das variáveis extraídas destas RDIs. Adicionalmente, um novo plano de processamento de tratografia foi construído de raiz no âmbito deste projeto para extrair variáveis adicionais das fibras de interesse e compará-las com as variáveis de difusão obtidas por análise voxel-a-voxel. Por último, as variáveis calculadas através dos tensores de difusão e curtose foram avaliadas na matéria cinzenta do cérebro para uma caracterização espácio-temporal da degeneração cerebral na DA. Os resultados da análise estatística foram obtidos após integrar no plano de processamento as estra-tégias que mostraram valorizar o projeto em termos de qualidade de imagem ou credibilidade das vari-áveis. Houve poucas diferenças significativas ao longo do tempo em cada grupo, no entanto as diferen-ças entre grupos foram bastante acentuadas. As alterações ao nível da microestrutura da matéria branca foram consistentes com estudos prévios em animais icv-STZ usando métodos histológicos e avaliações das suas capacidades cognitivas. Alterações nas variáveis extraídas dos tensores indicaram deficiência axonal inicial na fimbria e no fórnix 2 semanas após injeção no grupo de teste, um potencial período de recuperação às 6 semanas e novamente deficiência axonal às 13 semanas, sendo que neste período tardio todas as RDIs foram afetadas. O modelo biofísico WMTI-Watson confirmou aumentar especificidade ao estudo da microestrutura, visto que demostrou danos intra-axonais na fimbria e no corpo caloso 2 semanas após injeção, seguidos de um período de recuperação e de perda de estrutura axonal definitiva às 13 semanas em todas as RDIs. Não só estes dois métodos de análise de IRM de difusão se complementam, como são também con-sistentes entre eles e com as tendências de alterações ao longo do tempo descritas noutros estudos. Além disso, o animal icv-STZ mostrou alterações características da DA, mesmo tendo a degeneração cerebral sido induzida pela disrupção do metabolismo de glucose no cérebro. Como tal, este modelo animal é excelente para reproduzir a doença e deverá continuar a ser avaliado nas diferentes áreas multidiscipli-nares para explorar a hipótese de a DA ser desencadeada pela falha do sistema insulina/glucose. A com-binação da informação de difusão obtida dos tensores e da modelação da difusão neuronal provou ser uma ferramenta promissora no estudo das fibras da matéria branca do cérebro e poderá vir a ser o desafio futuro no que toca a investigação clínica da DA. Este estudo focar-se-á em correlacionar as alterações microestruturais aqui descritas com dados de conectividade funcional (obtida por IRM funcional em repouso), hipometabolismo de glucose (por FDG-PET) e outras características patológicas (por colora-ção histológica) – todos já em curso no CIBM. Tratografia é a metodologia topo de gama para aceder à conetividade cerebral e o plano de processamento gerado neste projeto poderá continuar a ser desenvol-vido no futuro para informação adicional, assim como a relação entre a matéria branca e cinzenta poderá suplementar a compreensão da progressão da doença no espaço e no tempo

Learning how to be robust: Deep polynomial regression

Polynomial regression is a recurrent problem with a large number of applications. In computer vision it often appears in motion analysis. Whatever the application, standard methods for regression of polynomial models tend to deliver biased results when the input data is heavily contaminated by outliers. Moreover, the problem is even harder when outliers have strong structure. Departing from problem-tailored heuristics for robust estimation of parametric models, we explore deep convolutional neural networks. Our work aims to find a generic approach for training deep regression models without the explicit need of supervised annotation. We bypass the need for a tailored loss function on the regression parameters by attaching to our model a differentiable hard-wired decoder corresponding to the polynomial operation at hand. We demonstrate the value of our findings by comparing with standard robust regression methods. Furthermore, we demonstrate how to use such models for a real computer vision problem, i.e., video stabilization. The qualitative and quantitative experiments show that neural networks are able to learn robustness for general polynomial regression, with results that well overpass scores of traditional robust estimation methods.Comment: 18 pages, conferenc

Multimodal Three Dimensional Scene Reconstruction, The Gaussian Fields Framework

The focus of this research is on building 3D representations of real world scenes and objects using different imaging sensors. Primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry, and multi-spectral image sequences including visual and thermal IR images that provide additional scene characteristics. The crucial technical challenge that we addressed is the automatic point-sets registration task. In this context our main contribution is the development of an optimization-based method at the core of which lies a unified criterion that solves simultaneously for the dense point correspondence and transformation recovery problems. The new criterion has a straightforward expression in terms of the datasets and the alignment parameters and was used primarily for 3D rigid registration of point-sets. However it proved also useful for feature-based multimodal image alignment. We derived our method from simple Boolean matching principles by approximation and relaxation. One of the main advantages of the proposed approach, as compared to the widely used class of Iterative Closest Point (ICP) algorithms, is convexity in the neighborhood of the registration parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. Physically the criterion is interpreted in terms of a Gaussian Force Field exerted by one point-set on the other. Such formulation proved useful for controlling and increasing the region of convergence, and hence allowing for more autonomy in correspondence tasks. Furthermore, the criterion can be computed with linear complexity using recently developed Fast Gauss Transform numerical techniques. In addition, we also introduced a new local feature descriptor that was derived from visual saliency principles and which enhanced significantly the performance of the registration algorithm. The resulting technique was subjected to a thorough experimental analysis that highlighted its strength and showed its limitations. Our current applications are in the field of 3D modeling for inspection, surveillance, and biometrics. However, since this matching framework can be applied to any type of data, that can be represented as N-dimensional point-sets, the scope of the method is shown to reach many more pattern analysis applications