Robust Modular Feature-Based Terrain-Aided Visual Navigation and Mapping

The visual feature-based Terrain-Aided Navigation (TAN) system presented in this thesis addresses the problem of constraining inertial drift introduced into the location estimate of Unmanned Aerial Vehicles (UAVs) in GPS-denied environment. The presented TAN system utilises salient visual features representing semantic or human-interpretable objects (roads, forest and water boundaries) from onboard aerial imagery and associates them to a database of reference features created a-priori, through application of the same feature detection algorithms to satellite imagery. Correlation of the detected features with the reference features via a series of the robust data association steps allows a localisation solution to be achieved with a finite absolute bound precision defined by the certainty of the reference dataset. The feature-based Visual Navigation System (VNS) presented in this thesis was originally developed for a navigation application using simulated multi-year satellite image datasets. The extension of the system application into the mapping domain, in turn, has been based on the real (not simulated) flight data and imagery. In the mapping study the full potential of the system, being a versatile tool for enhancing the accuracy of the information derived from the aerial imagery has been demonstrated. Not only have the visual features, such as road networks, shorelines and water bodies, been used to obtain a position ’fix’, they have also been used in reverse for accurate mapping of vehicles detected on the roads into an inertial space with improved precision. Combined correction of the geo-coding errors and improved aircraft localisation formed a robust solution to the defense mapping application. A system of the proposed design will provide a complete independent navigation solution to an autonomous UAV and additionally give it object tracking capability

Attitude-trajectory estimation for forward looking multi-beam sonar based on acoustic image registration

This work considers the processing of acoustic data from a multi-beam Forward Looking Sonar (FLS) on a moving underwater platform to estimate the platform’s attitude and trajectory. We propose an algorithm to produce an estimate of the attitude-trajectory for a FLS based on the optical flow between consecutive sonar frames. The attitude-trajectory can be used to locate an underwater platform, such as an Autonomous Underwater Vehicle (AUV), to a degree of accuracy suitable for navigation. It can also be used to build a mosaic of the underwater scene. The estimation is performed in three steps. Firstly, a selection of techniques based on the optical flow model are used to estimate a pixel displacement map (DM) between consecutive sonar frames represented in the native polar (range/bearing) format. The second step finds the best match between the estimated DM and DMs for a set of modeled sonar sensor motions. To reduce complexity, it is proposed to describe the DM with a small parameter vector derived from the displacement distribution. Thus, an estimate of the incremental sensor motion between frames is made. Finally, using a weighted regularized spline technique, the incremental inter-frame motions are integrated into an attitude-trajectory for the sonar sensor. To assess the accuracy of the attitude-trajectory estimate, it is used to register FLS frames from a field experiment dataset and build a high-quality mosaic of the underwater scene

Sparse variational regularization for visual motion estimation

The computation of visual motion is a key component in numerous computer vision tasks such as object detection, visual object tracking and activity recognition. Despite exten- sive research effort, efficient handling of motion discontinuities, occlusions and illumina- tion changes still remains elusive in visual motion estimation. The work presented in this thesis utilizes variational methods to handle the aforementioned problems because these methods allow the integration of various mathematical concepts into a single en- ergy minimization framework. This thesis applies the concepts from signal sparsity to the variational regularization for visual motion estimation. The regularization is designed in such a way that it handles motion discontinuities and can detect object occlusions

Markerless deformation capture of hoverfly wings using multiple calibrated cameras

This thesis introduces an algorithm for the automated deformation capture of hoverfly wings from multiple camera image sequences. The algorithm is capable of extracting dense surface measurements, without the aid of fiducial markers, over an arbitrary number of wingbeats of hovering flight and requires limited manual initialisation. A novel motion prediction method, called the ‘normalised stroke model’, makes use of the similarity of adjacent wing strokes to predict wing keypoint locations, which are then iteratively refined in a stereo image registration procedure. Outlier removal, wing fitting and further refinement using independently reconstructed boundary points complete the algorithm. It was tested on two hovering data sets, as well as a challenging flight manoeuvre. By comparing the 3-d positions of keypoints extracted from these surfaces with those resulting from manual identification, the accuracy of the algorithm is shown to approach that of a fully manual approach. In particular, half of the algorithm-extracted keypoints were within 0.17mm of manually identified keypoints, approximately equal to the error of the manual identification process. This algorithm is unique among purely image based flapping flight studies in the level of automation it achieves, and its generality would make it applicable to wing tracking of other insects

Motion tracking tMRI datasets to quantify abnormal left ventricle motion using finite element modelling

According to `The Atlas of Heart Disease and Stroke'[MMMG04] published by the World Health Organization, heart disease accounts for nearly half the deaths in both the developed and developing countries and is the world's single biggest killer. However, early detection of a diseased heart condition can prevent many of these fatalities. Regional wall motion abnormalities of the heart precede both ECG abnormalities and chest pain as an indicator of myocardial ischaemia and are an excellent indicator of coronary stenosis [GZM97]. These motion abnormalities of the heart muscle are difficult to observe and track, because the heart is a relatively smooth organ with few landmarks and non-rigid motion with a twisting motion or tangential component. The MRI tissue-tagging technique gives researchers the first glimpse into how the heart actually beats. This research uses the tagged MRI images of the heart to create a three dimensional model of a beating heart indicating the stress of a region. Tagged MRI techniques are still developing and vary vastly, meaning that there needs to be a methodology that can adapt to these changes rapidly and effectively, to meet the needs of the evolving technology. The focus of this research is to develop and test such a methodology by the means of a Strain Estimation Pipeline along with an effective way of validating any changes made to the individual processes that it comprises of

3-D lung deformation and function from respiratory-gated 4-D x-ray CT images : application to radiation treatment planning.

Many lung diseases or injuries can cause biomechanical or material property changes that can alter lung function. While the mechanical changes associated with the change of the material properties originate at a regional level, they remain largely asymptomatic and are invisible to global measures of lung function until they have advanced significantly and have aggregated. In the realm of external beam radiation therapy of patients suffering from lung cancer, determination of patterns of pre- and post-treatment motion, and measures of regional and global lung elasticity and function are clinically relevant. In this dissertation, we demonstrate that 4-D CT derived ventilation images, including mechanical strain, provide an accurate and physiologically relevant assessment of regional pulmonary function which may be incorporated into the treatment planning process. Our contributions are as follows: (i) A new volumetric deformable image registration technique based on 3-D optical flow (MOFID) has been designed and implemented which permits the possibility of enforcing physical constraints on the numerical solutions for computing motion field from respiratory-gated 4-D CT thoracic images. The proposed optical flow framework is an accurate motion model for the thoracic CT registration problem. (ii) A large displacement landmark-base elastic registration method has been devised for thoracic CT volumetric image sets containing large deformations or changes, as encountered for example in registration of pre-treatment and post-treatment images or multi-modality registration. (iii) Based on deformation maps from MOFIO, a novel framework for regional quantification of mechanical strain as an index of lung functionality has been formulated for measurement of regional pulmonary function. (iv) In a cohort consisting of seven patients with non-small cell lung cancer, validation of physiologic accuracy of the 4-0 CT derived quantitative images including Jacobian metric of ventilation, Vjac, and principal strains, (V?1, V?2, V?3, has been performed through correlation of the derived measures with SPECT ventilation and perfusion scans. The statistical correlations with SPECT have shown that the maximum principal strain pulmonary function map derived from MOFIO, outperforms all previously established ventilation metrics from 40-CT. It is hypothesized that use of CT -derived ventilation images in the treatment planning process will help predict and prevent pulmonary toxicity due to radiation treatment. It is also hypothesized that measures of regional and global lung elasticity and function obtained during the course of treatment may be used to adapt radiation treatment. Having objective methods with which to assess pre-treatment global and regional lung function and biomechanical properties, the radiation treatment dose can potentially be escalated to improve tumor response and local control

MR imaging of left-ventricular function : novel image acquisition and analysis techniques.

Many cardiac diseases, such as myocardial ischemia, secondary to coronary artery disease, may be identified and localized through the analysis of cardiac deformations. Early efforts for quantifying ventricular wall motion used surgical implantation and tracking of radiopaque markers with X-ray imaging in canine hearts [1]. Such techniques are invasive and affect the regional motion pattern of the ventricular wall during the marker tracking process and, clearly are not feasible clinically. Noninvasive imaging techniques are vital and have been widely applied to the clinic. MRI is a noninvasive imaging technique with the capability to monitor and assess the progression of cardiovascular diseases (CVD) so that effective procedures for the care and treatment of patients can be developed by physicians and researchers. It is capable of providing 3D analysis of global and regional cardiac function with great accuracy and reproducibility. In the past few years, numerous efforts have been devoted to cardiac motion recovery and deformation analysis from MR imaging sequences. In order to assess cardiac function, there are two categories of indices that are used: global and regional indices. Global indices include ejection fraction, cavity volume, and myocardial mass [2]. They are important indices for cardiac disease diagnosis. However, these global indices are not specific for regional analysis. A quantitative assessment of regional parameters may prove beneficial for the diagnosis of disease and evaluation of severity and the quantification of treatment [3]. Local measures, such as wall deformation and strain in all regions of the heart, can provide objective regional quantification of ventricular wall function and relate to the location and extent of ischemic injury. This dissertation is concerned with the development of novel MR imaging techniques and image postprocessing algorithms to analyze left ventricular deformations. A novel pulse sequence, termed Orthogonal CSPAMM (OCSPAMM), has been proposed which results in the same acquisition time as SPAMM for 2D deformation estimation while keeping the main advantages of CSPAMM [4,5]: i.e., maintaining tag contrast through-out the ECG cycle. Different from CSPAMM, in OCSPAMM the second tagging pulse orientation is rotated 90 degrees relative to the first one so that motion information can be obtained simultaneously in two directions. This reduces the acquisition time by a factor of two as compared to the traditional CSPAMM, in which two separate imaging sequences are applied per acquisition. With the application of OCSPAMM, the effect of tag fading encountered in SPAMM tagging due to Tl relaxation is mitigated and tag deformations can be visualized for the entire cardiac cycle, including diastolic phases. A multilevel B-spline fitting method (MBS) has been proposed which incorporates phase-based displacement information for accurate calculation of 2D motion and strain from tagged MRI [6, 7]. The proposed method combines the advantages of continuity and smoothness of MBS, and makes use of phase information derived from tagged MR images. Compared to previous 2D B-spline-based deformation analysis methods, MBS has the following advantages: 1) It can simultaneously achieve a smooth deformation while accurately approximating the given data set; 2) Computationally, it is very fast; and 3) It can produce more accurate deformation results. Since the tag intersections (intersections between two tag lines) can be extracted accurately and are more or less distributed evenly over the myocardium, MBS has proven effective for 2D cardiac motion tracking. To derive phase-based displacements, 2D HARP and SinMod analysis techniques [8,9] were employed. By producing virtual tags from HARP /SinMod and calculating intersections of virtual tag lines, more data points are obtained. In the reference frame, virtual tag lines are the isoparametric curves of an undeformed 2D B-spline model. In subsequent frames, the locations of intersections of virtual tag lines over the myocardium are updated with phase-based displacement. The advantage of the technique is that in acquiring denser myocardial displacements, it uses both real and virtual tag line intersections. It is fast and more accurate than 2D HARP and SinMod tracking. A novel 3D sine wave modeling (3D SinMod) approach for automatic analysis of 3D cardiac deformations has been proposed [10]. An accelerated 3D complementary spatial modulation of magnetization (CSPAMM) tagging technique [11] was used to acquire complete 3D+t tagged MR data sets of the whole heart (3 dynamic CSPAMM tagged MRI volume with tags in different orientations), in-vivo, in 54 heart beats and within 3 breath-holds. In 3D SinMod, the intensity distribution around each pixel is modeled as a cosine wave front. The principle behind 3D SinMod tracking is that both phase and frequency for each voxel are determined directly from the frequency analysis and the displacement is calculated from the quotient of phase difference and local frequency. The deformation fields clearly demonstrate longitudinal shortening during systole. The contraction of the LV base towards the apex as well as the torsional motion between basal and apical slices is clearly observable from the displacements. 3D SinMod can automatically process the image data to derive measures of motion, deformations, and strains between consecutive pair of tagged volumes in 17 seconds. Therefore, comprehensive 4D imaging and postprocessing for determination of ventricular function is now possible in under 10 minutes. For validation of 3D SinMod, 7 3D+t CSPAMM data sets of healthy subjects have been processed. Comparison of mid-wall contour deformations and circumferential shortening results by 3D SinMod showed good agreement with those by 3D HARP. Tag lines tracked by the proposed technique were also compared with manually delineated ones. The average errors calculated for the systolic phase of the cardiac cycles were in the sub-pixel range

Computer vision and optimization methods applied to the measurements of in-plane deformations

fi=vertaisarvioitu|en=peerReviewed

ToF cameras for active vision in robotics

ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited. After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/ foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages.This work was supported by the EU project GARNICS FP7-247947, by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510, and by the Catalan Research Commission through SGR-00155Peer Reviewe

Myocardial tagging by Cardiovascular Magnetic Resonance: evolution of techniques--pulse sequences, analysis algorithms, and applications

Cardiovascular magnetic resonance (CMR) tagging has been established as an essential technique for measuring regional myocardial function. It allows quantification of local intramyocardial motion measures, e.g. strain and strain rate. The invention of CMR tagging came in the late eighties, where the technique allowed for the first time for visualizing transmural myocardial movement without having to implant physical markers. This new idea opened the door for a series of developments and improvements that continue up to the present time. Different tagging techniques are currently available that are more extensive, improved, and sophisticated than they were twenty years ago. Each of these techniques has different versions for improved resolution, signal-to-noise ratio (SNR), scan time, anatomical coverage, three-dimensional capability, and image quality. The tagging techniques covered in this article can be broadly divided into two main categories: 1) Basic techniques, which include magnetization saturation, spatial modulation of magnetization (SPAMM), delay alternating with nutations for tailored excitation (DANTE), and complementary SPAMM (CSPAMM); and 2) Advanced techniques, which include harmonic phase (HARP), displacement encoding with stimulated echoes (DENSE), and strain encoding (SENC). Although most of these techniques were developed by separate groups and evolved from different backgrounds, they are in fact closely related to each other, and they can be interpreted from more than one perspective. Some of these techniques even followed parallel paths of developments, as illustrated in the article. As each technique has its own advantages, some efforts have been made to combine different techniques together for improved image quality or composite information acquisition. In this review, different developments in pulse sequences and related image processing techniques are described along with the necessities that led to their invention, which makes this article easy to read and the covered techniques easy to follow. Major studies that applied CMR tagging for studying myocardial mechanics are also summarized. Finally, the current article includes a plethora of ideas and techniques with over 300 references that motivate the reader to think about the future of CMR tagging