Dynamic Denoising of Tracking Sequences

©2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or distribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.DOI: 10.1109/TIP.2008.920795In this paper, we describe an approach to the problem of simultaneously enhancing image sequences and tracking the objects of interest represented by the latter. The enhancement part of the algorithm is based on Bayesian wavelet denoising, which has been chosen due to its exceptional ability to incorporate diverse a priori information into the process of image recovery. In particular, we demonstrate that, in dynamic settings, useful statistical priors can come both from some reasonable assumptions on the properties of the image to be enhanced as well as from the images that have already been observed before the current scene. Using such priors forms the main contribution of the present paper which is the proposal of the dynamic denoising as a tool for simultaneously enhancing and tracking image sequences.Within the proposed framework, the previous observations of a dynamic scene are employed to enhance its present observation. The mechanism that allows the fusion of the information within successive image frames is Bayesian estimation, while transferring the useful information between the images is governed by a Kalman filter that is used for both prediction and estimation of the dynamics of tracked objects. Therefore, in this methodology, the processes of target tracking and image enhancement "collaborate" in an interlacing manner, rather than being applied separately. The dynamic denoising is demonstrated on several examples of SAR imagery. The results demonstrated in this paper indicate a number of advantages of the proposed dynamic denoising over "static" approaches, in which the tracking images are enhanced independently of each other

Design of a High-Speed Architecture for Stabilization of Video Captured Under Non-Uniform Lighting Conditions

Video captured in shaky conditions may lead to vibrations. A robust algorithm to immobilize the video by compensating for the vibrations from physical settings of the camera is presented in this dissertation. A very high performance hardware architecture on Field Programmable Gate Array (FPGA) technology is also developed for the implementation of the stabilization system. Stabilization of video sequences captured under non-uniform lighting conditions begins with a nonlinear enhancement process. This improves the visibility of the scene captured from physical sensing devices which have limited dynamic range. This physical limitation causes the saturated region of the image to shadow out the rest of the scene. It is therefore desirable to bring back a more uniform scene which eliminates the shadows to a certain extent. Stabilization of video requires the estimation of global motion parameters. By obtaining reliable background motion, the video can be spatially transformed to the reference sequence thereby eliminating the unintended motion of the camera. A reflectance-illuminance model for video enhancement is used in this research work to improve the visibility and quality of the scene. With fast color space conversion, the computational complexity is reduced to a minimum. The basic video stabilization model is formulated and configured for hardware implementation. Such a model involves evaluation of reliable features for tracking, motion estimation, and affine transformation to map the display coordinates of a stabilized sequence. The multiplications, divisions and exponentiations are replaced by simple arithmetic and logic operations using improved log-domain computations in the hardware modules. On Xilinx\u27s Virtex II 2V8000-5 FPGA platform, the prototype system consumes 59% logic slices, 30% flip-flops, 34% lookup tables, 35% embedded RAMs and two ZBT frame buffers. The system is capable of rendering 180.9 million pixels per second (mpps) and consumes approximately 30.6 watts of power at 1.5 volts. With a 1024×1024 frame, the throughput is equivalent to 172 frames per second (fps). Future work will optimize the performance-resource trade-off to meet the specific needs of the applications. It further extends the model for extraction and tracking of moving objects as our model inherently encapsulates the attributes of spatial distortion and motion prediction to reduce complexity. With these parameters to narrow down the processing range, it is possible to achieve a minimum of 20 fps on desktop computers with Intel Core 2 Duo or Quad Core CPUs and 2GB DDR2 memory without a dedicated hardware

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Enhanced Image-Aided Navigation Algorithm with Automatic Calibration and Affine Distortion Prediction

This research aims at improving two key steps within the image aided navigation process: camera calibration and landmark tracking. The camera calibration step is improved by automating the point correspondence calculation within the standard camera calibration algorithm, thereby reducing the required time for calibration while maintaining the output model accuracy. The feature landmark tracking step is improved by digitally simulating affine distortions on input images in order to calculate more accurate feature descriptors for improved feature matching in high relative viewpoint change. These techniques are experimentally demonstrated in an outdoor environment with a consumer-grade inertial sensor and three imaging sensors, one of which is orthogonal to the rest. Using a tactical-grade inertial sensor coupled with GPS position data for comparison, the improved image aided navigation algorithm is shown to reduce navigation errors by 24% in position, 16% in velocity and 35% in attitude when compared to the standard image-aided navigation algorithm

Characterization of carotid artery plaques using noninvasive vascular ultrasound elastography

L'athérosclérose est une maladie vasculaire complexe qui affecte la paroi des artères (par l'épaississement) et les lumières (par la formation de plaques). La rupture d'une plaque de l'artère carotide peut également provoquer un accident vasculaire cérébral ischémique et des complications. Bien que plusieurs modalités d'imagerie médicale soient actuellement utilisées pour évaluer la stabilité d'une plaque, elles présentent des limitations telles que l'irradiation, les propriétés invasives, une faible disponibilité clinique et un coût élevé. L'échographie est une méthode d'imagerie sûre qui permet une analyse en temps réel pour l'évaluation des tissus biologiques. Il est intéressant et prometteur d’appliquer une échographie vasculaire pour le dépistage et le diagnostic précoces des plaques d’artère carotide. Cependant, les ultrasons vasculaires actuels identifient uniquement la morphologie d'une plaque en termes de luminosité d'écho ou l’impact de cette plaque sur les caractéristiques de l’écoulement sanguin, ce qui peut ne pas être suffisant pour diagnostiquer l’importance de la plaque. La technique d’élastographie vasculaire non-intrusive (« noninvasive vascular elastography (NIVE) ») a montré le potentiel de détermination de la stabilité d'une plaque. NIVE peut déterminer le champ de déformation de la paroi vasculaire en mouvement d’une artère carotide provoqué par la pulsation cardiaque naturelle. En raison des différences de module de Young entre les différents tissus des vaisseaux, différents composants d’une plaque devraient présenter différentes déformations, caractérisant ainsi la stabilité de la plaque. Actuellement, les performances et l’efficacité numérique sous-optimales limitent l’acceptation clinique de NIVE en tant que méthode rapide et efficace pour le diagnostic précoce des plaques vulnérables. Par conséquent, il est nécessaire de développer NIVE en tant qu’outil d’imagerie non invasif, rapide et économique afin de mieux caractériser la vulnérabilité liée à la plaque. La procédure à suivre pour effectuer l’analyse NIVE consiste en des étapes de formation et de post-traitement d’images. Cette thèse vise à améliorer systématiquement la précision de ces deux aspects de NIVE afin de faciliter la prédiction de la vulnérabilité de la plaque carotidienne. Le premier effort de cette thèse a été dédié à la formation d'images (Chapitre 5). L'imagerie par oscillations transversales a été introduite dans NIVE. Les performances de l’imagerie par oscillations transversales couplées à deux estimateurs de contrainte fondés sur un modèle de déformation fine, soit l’ « affine phase-based estimator (APBE) » et le « Lagrangian speckle model estimator (LSME) », ont été évaluées. Pour toutes les études de simulation et in vitro de ce travail, le LSME sans imagerie par oscillation transversale a surperformé par rapport à l'APBE avec imagerie par oscillations transversales. Néanmoins, des estimations de contrainte principales comparables ou meilleures pourraient être obtenues avec le LSME en utilisant une imagerie par oscillations transversales dans le cas de structures tissulaires complexes et hétérogènes. Lors de l'acquisition de signaux ultrasonores pour la formation d'images, des mouvements hors du plan perpendiculaire au plan de balayage bidimensionnel (2-D) existent. Le deuxième objectif de cette thèse était d'évaluer l'influence des mouvements hors plan sur les performances du NIVE 2-D (Chapitre 6). À cette fin, nous avons conçu un dispositif expérimental in vitro permettant de simuler des mouvements hors plan de 1 mm, 2 mm et 3 mm. Les résultats in vitro ont montré plus d'artefacts d'estimation de contrainte pour le LSME avec des amplitudes croissantes de mouvements hors du plan principal de l’image. Malgré tout, nous avons néanmoins obtenu des estimations de déformations robustes avec un mouvement hors plan de 2.0 mm (coefficients de corrélation supérieurs à 0.85). Pour un jeu de données cliniques de 18 participants présentant une sténose de l'artère carotide, nous avons proposé d'utiliser deux jeux de données d'analyses sur la même plaque carotidienne, soit des images transversales et longitudinales, afin de déduire les mouvements hors plan (qui se sont avérés de 0.25 mm à 1.04 mm). Les résultats cliniques ont montré que les estimations de déformations restaient reproductibles pour toutes les amplitudes de mouvement, puisque les coefficients de corrélation inter-images étaient supérieurs à 0.70 et que les corrélations croisées normalisées entre les images radiofréquences étaient supérieures à 0.93, ce qui a permis de démontrer une plus grande confiance lors de l'analyse de jeu de données cliniques de plaques carotides à l'aide du LSME. Enfin, en ce qui concerne le post-traitement des images, les algorithmes NIVE doivent estimer les déformations des parois des vaisseaux à partir d’images reconstituées dans le but d’identifier les tissus mous et durs. Ainsi, le dernier objectif de cette thèse était de développer un algorithme d'estimation de contrainte avec une résolution de la taille d’un pixel ainsi qu'une efficacité de calcul élevée pour l'amélioration de la précision de NIVE (Chapitre 7). Nous avons proposé un estimateur de déformation de modèle fragmenté (SMSE) avec lequel le champ de déformation dense est paramétré avec des descriptions de transformées en cosinus discret, générant ainsi des composantes de déformations affines (déformations axiales et latérales et en cisaillement) sans opération mathématique de dérivées. En comparant avec le LSME, le SMSE a réduit les erreurs d'estimation lors des tests de simulations, ainsi que pour les mesures in vitro et in vivo. De plus, la faible mise en oeuvre de la méthode SMSE réduit de 4 à 25 fois le temps de traitement par rapport à la méthode LSME pour les simulations, les études in vitro et in vivo, ce qui pourrait permettre une implémentation possible de NIVE en temps réel.Atherosclerosis is a complex vascular disease that affects artery walls (by thickening) and lumens (by plaque formation). The rupture of a carotid artery plaque may also induce ischemic stroke and complications. Despite the use of several medical imaging modalities to evaluate the stability of a plaque, they present limitations such as irradiation, invasive property, low clinical availability and high cost. Ultrasound is a safe imaging method with a real time capability for assessment of biological tissues. It is clinically used for early screening and diagnosis of carotid artery plaques. However, current vascular ultrasound technologies only identify the morphology of a plaque in terms of echo brightness or the impact of the vessel narrowing on flow properties, which may not be sufficient for optimum diagnosis. Noninvasive vascular elastography (NIVE) has been shown of interest for determining the stability of a plaque. Specifically, NIVE can determine the strain field of the moving vessel wall of a carotid artery caused by the natural cardiac pulsation. Due to Young’s modulus differences among different vessel tissues, different components of a plaque can be detected as they present different strains thereby potentially helping in characterizing the plaque stability. Currently, sub-optimum performance and computational efficiency limit the clinical acceptance of NIVE as a fast and efficient method for the early diagnosis of vulnerable plaques. Therefore, there is a need to further develop NIVE as a non-invasive, fast and low computational cost imaging tool to better characterize the plaque vulnerability. The procedure to perform NIVE analysis consists in image formation and image post-processing steps. This thesis aimed to systematically improve the accuracy of these two aspects of NIVE to facilitate predicting carotid plaque vulnerability. The first effort of this thesis has been targeted on improving the image formation (Chapter 5). Transverse oscillation beamforming was introduced into NIVE. The performance of transverse oscillation imaging coupled with two model-based strain estimators, the affine phase-based estimator (APBE) and the Lagrangian speckle model estimator (LSME), were evaluated. For all simulations and in vitro studies, the LSME without transverse oscillation imaging outperformed the APBE with transverse oscillation imaging. Nonetheless, comparable or better principal strain estimates could be obtained with the LSME using transverse oscillation imaging in the case of complex and heterogeneous tissue structures. During the acquisition of ultrasound signals for image formation, out-of-plane motions which are perpendicular to the two-dimensional (2-D) scan plane are existing. The second objective of this thesis was to evaluate the influence of out-of-plane motions on the performance of 2-D NIVE (Chapter 6). For this purpose, we designed an in vitro experimental setup to simulate out-of-plane motions of 1 mm, 2 mm and 3 mm. The in vitro results showed more strain estimation artifacts for the LSME with increasing magnitudes of out-of-plane motions. Even so, robust strain estimations were nevertheless obtained with 2.0 mm out-of-plane motion (correlation coefficients higher than 0.85). For a clinical dataset of 18 participants with carotid artery stenosis, we proposed to use two datasets of scans on the same carotid plaque, one cross-sectional and the other in a longitudinal view, to deduce the out-of-plane motions (estimated to be ranging from 0.25 mm to 1.04 mm). Clinical results showed that strain estimations remained reproducible for all motion magnitudes since inter-frame correlation coefficients were higher than 0.70, and normalized cross-correlations between radiofrequency images were above 0.93, which indicated that confident motion estimations can be obtained when analyzing clinical dataset of carotid plaques using the LSME. Finally, regarding the image post-processing component of NIVE algorithms to estimate strains of vessel walls from reconstructed images with the objective of identifying soft and hard tissues, we developed a strain estimation method with a pixel-wise resolution as well as a high computation efficiency for improving NIVE (Chapter 7). We proposed a sparse model strain estimator (SMSE) for which the dense strain field is parameterized with Discrete Cosine Transform descriptions, thereby deriving affine strain components (axial and lateral strains and shears) without mathematical derivative operations. Compared with the LSME, the SMSE reduced estimation errors in simulations, in vitro and in vivo tests. Moreover, the sparse implementation of the SMSE reduced the processing time by a factor of 4 to 25 compared with the LSME based on simulations, in vitro and in vivo results, which is suggesting a possible implementation of NIVE in real time