Distributed scene reconstruction from multiple mobile platforms

Recent research on mobile robotics has produced new designs that provide house-hold robots with omnidirectional motion. The image sensor embedded in these devices motivates the application of 3D vision techniques on them for navigation and mapping purposes. In addition to this, distributed cheapsensing systems acting as unitary entity have recently been discovered as an efficient alternative to expensive mobile equipment. In this work we present an implementation of a visual reconstruction method, structure from motion (SfM), on a low-budget, omnidirectional mobile platform, and extend this method to distributed 3D scene reconstruction with several instances of such a platform. Our approach overcomes the challenges yielded by the plaform. The unprecedented levels of noise produced by the image compression typical of the platform is processed by our feature filtering methods, which ensure suitable feature matching populations for epipolar geometry estimation by means of a strict quality-based feature selection. The robust pose estimation algorithms implemented, along with a novel feature tracking system, enable our incremental SfM approach to novelly deal with ill-conditioned inter-image configurations provoked by the omnidirectional motion. The feature tracking system developed efficiently manages the feature scarcity produced by noise and outputs quality feature tracks, which allow robust 3D mapping of a given scene even if - due to noise - their length is shorter than what it is usually assumed for performing stable 3D reconstructions. The distributed reconstruction from multiple instances of SfM is attained by applying loop-closing techniques. Our multiple reconstruction system merges individual 3D structures and resolves the global scale problem with minimal overlaps, whereas in the literature 3D mapping is obtained by overlapping stretches of sequences. The performance of this system is demonstrated in the 2-session case. The management of noise, the stability against ill-configurations and the robustness of our SfM system is validated on a number of experiments and compared with state-of-the-art approaches. Possible future research areas are also discussed

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'

Mitigating the limited view problem in photoacoustic tomography for a planar detection geometry by regularised iterative reconstruction

The use of a planar detection geometry in photoacoustic tomography results in the so-called limited-view problem due to the finite extent of the acoustic detection aperture. When images are reconstructed using one-step reconstruction algorithms, image quality is compromised by the presence of streaking artefacts, reduced contrast, image distortion and reduced signal-to-noise ratio. To mitigate this, model-based iterative reconstruction approaches based on least squares minimisation with and without total variation regularisation were evaluated using in-silico , experimental phantom, ex vivo and in vivo data. Compared to one-step reconstruction methods, it has been shown that iterative methods provide better image quality in terms of enhanced signal-to-artefact ratio, signal-to-noise ratio, amplitude accuracy and spatial fidelity. For the total variation approaches, the impact of the regularisation parameter on image feature scale and amplitude distribution was evaluated. In addition, the extent to which the use of Bregman iterations can compensate for the systematic amplitude bias introduced by total variation was studied. This investigation is expected to inform the practical application of model-based iterative image reconstruction approaches for improving photoacoustic image quality when using finite aperture planar detection geometries

Ultrasound tomography using pyroelectric and piezoelectric sensors

Acoustic absorption is one of several quantities which can differentiate healthy breast tissue from cancerous tissue. In order to accurately quantify the acoustic absorption, the sensor system must be able to accurately distinguish acoustic power loss due to absorption from other modes of attenuation. Traditional piezoelectric sensors are susceptible to phase-cancellation artifacts due to their directional signal response, and thus pyroelectric ultrasound sensors, which have a much flatter directional response, have been suggested as an alternate measurement device for improved absorption reconstructions in ultrasound tomography (UST). In this thesis we investigate the use of pyroelectric phase-insensitive sensors in UST — the thesis is divided into two parts. In the first part we present a model for a pyroelectric ultrasound sensor and investigate its directional response and sensitivity properties. The model’s time-series response and directional response are compared to real-world measurements to confirm accuracy. The second part focuses on the inverse problem aspect of ultrasound tomography, where we consider various reconstruction methods and sensor geometries to determine which situations can benefit from phase-insensitive data for acoustic absorption reconstruction. Reconstructions for both phase-insensitive as well as phase-sensitive sensors are analysed, with future work considerations for combined sensor systems

Local Features, Structure-from-motion and View Synthesis in Spherical Video

This thesis addresses the problem of synthesising new views from spherical video or image sequences. We propose an interest point detector and feature descriptor that allows us to robustly match local features between pairs of spherical images and use this as part of a structure-from-motion pipeline that allows us to estimate camera pose from a spherical video sequence. With pose estimates to hand, we propose methods for view stabilisation and novel viewpoint synthesis. In Chapter 3 we describe our contribution in the area of feature detection and description in spherical images. First, we present a novel representation for spherical images which uses a discrete geodesic grid composed of hexagonal pixels. Second, we extend the BRISK binary descriptor to the sphere, proposing methods for multiscale corner detection, sub-pixel position and sub-octave scale refinement and descriptor construction in the tangent space to the sphere. In Chapter 4 we describe our contributions in the area of spherical structure-from-motion. We revisit problems from multiview geometry in the context of spherical images. We propose methods suited to spherical camera geometry for the spherical-n-point problem and calibrated spherical reconstruction. We introduce a new probabilistic interpretation of spherical structure-from-motion which uses the von Mises-Fisher distribution in spherical feature point positions. This model provides an alternate objective function that we use in bundle adjustment. In Chapter 5 we describe our contributions in the area of view synthesis from spherical images. We exploit the camera pose estimates made by our pipeline and use these in two view synthesis applications. The first is view stabilisation where we remove the effect of viewing direction changes, often present in first person video. Second, we propose a method for synthesising novel viewpoints

Systèmes de localisation en temps réel basés sur les réseaux de communication sans fil

Des techniques fiables de radiolocalisation s’avèrent indispensables au développement d’un grand nombre de nouveaux systèmes pertinents. Les techniques de localisation basées sur les réseaux de communication sans-fil (WNs) sont particulièrement adéquates aux espaces confinés et fortement urbanisés. Le présent projet de recherche s’intéresse aux systèmes de localisation en temps réel (RTLS) basés sur les technologies de communication sans-fil existantes. Deux nouvelles techniques de radiolocalisation alternatives sont proposées pour améliorer la précision de positionnement des nœuds sans-fil mobiles par rapport aux méthodes conventionnelles basées sur la puissance des signaux reçus (RSS). La première méthode de type géométrique propose une nouvelle métrique de compensation entre les puissances de signaux reçus par rapport à des paires de stations réceptrices fixes. L’avantage de cette technique est de réduire l’effet des variations du milieu de propagation et des puissances d’émission des signaux sur la précision de localisation. La même métrique est sélectionnée pour former les signatures utilisées pour créer la carte radio de l’environnement de localisation durant la phase hors-ligne dans la deuxième méthode de type analyse de situation. Durant la phase de localisation en temps réel, la technique d’acquisition comprimée (CS) est appliquée pour retrouver les positions des nœuds mobiles à partir d’un nombre réduit d’échantillons de signaux reçus en les comparant à la carte radio préétablie. Le calcul d’algèbre multilinéaire proposé dans ce travail permet l’utilisation de ce type de métrique ternaire, équivalemment la différence des temps d’arrivée (TDOA), pour calculer les positions des cibles selon la technique de CS. Les deux méthodes sont ensuite validées par des simulations et des expérimentations effectuées dans des environnements à deux et à trois dimensions. Les expériences ont été menées dans un bâtiment multi-étages (MFB) en utilisant l’infrastructure sans-fil existante pour retrouver conjointement la position et l’étage des cibles en utilisant les techniques proposées. Un exemple emblématique de l’application des RTLS dans les zones urbaines est celui des systèmes de transport intelligents (ITS) pour améliorer la sécurité routière. Ce projet s’intéresse également à la performance d’une application de sécurité des piétons au niveau des intersections routières. L’accomplissement d’un tel système repose sur l’échange fiable, sous des contraintes temporelles sévères, des données de positionnement géographique entre nœuds mobiles pour se tenir mutuellement informés de leurs présences et positions afin de prévenir les risques de collision. Ce projet mène une étude comparative entre deux architectures d’un système ITS permettant la communication entre piétons et véhicules, directement et via une unité de l’infrastructure, conformément aux standards de communication dans les réseaux ad hoc véhiculaires (VANETs)

Speech Enhancement using Multiple Transducers

In this thesis, three methods of speech enhancement techniques are investigated with applications in extreme noise environments. Various beamforming techniques are evaluated for their performance characteristics in terms of signal to (distant) noise ratio and tolerance to design imperfections. Two suitable designs are identified with contrasting performance characteristics — the second order differential array, with excellent noise rejection but poor robustness; and a least squares design, with adequate noise rejection and good robustness. Adaptive filters are introduced in the context of a simple noise canceller and later a post-processor for a dual beamformer system. Modifications to the least mean squares (LMS) filter are introduced to tolerate cross-talk between microphones or beamformer outputs. An adaptive filter based post-processor beamforming system is designed and evaluated using a simulation involving speech in noisy environments. The beamforming methods developed are combined with the modified LMS adaptive filter to further reduce noise (if possible) based on correlations between noise signals in a beamformer directed to the talker and a complementary beamformer (nullformer) directed away from the talker. This system shows small, but not insignificant, improvements in noise reduction over purely beamforming based methods. Blind source separation is introduced briefly as a potential future method for enhancing speech in noisy environments. The FastICA algorithm is evaluated on existing data sets and found to perform similarly to the post-processing system developed in this thesis. Future avenues of research in this field are highlighted

Sparse separation of sources in 3D soundscapes

A novel blind source separation algorithm applicable to extracting sources from within 3D soundscapes is presented. The algorithm is based on constructing a binary mask based on directional information. The validity of filtering using binary masked based on the ω-disjoint assumption is examined for several typical scenarios. Results for these test environments show an improvement by an order of magnitude when compared to similar work using speech mixtures. Also presented is the novel application of a dual-tree complex wavelet transform to sparse source separation, providing an alternative transformation to the short-time Fourier transform often used in this area. Results are presented showing compara- ble signal-to-interference performance, and significantly improved signal-to-distortion performance when compared against the short time Fourier transform. Results presented for the separation algorithm include quantitative measures of the separation performance for robust comparison against other separation algorithms. Consideration is given to the related problem of localising sources within 3D sound- scapes. Two novel methods are presented, the first using a peak estimation on a spherical histogram constructed using a geodesic grid, the second by adapting a self learning plastic self-organising map to operate on the surface of a unit sphere. It is concluded that the separation algorithm presented is effective for soundscapes comprising ecological or zoological sources. Specific areas for further work are recog- nised, both in terms of isolated technologies and towards the integration of this work into an instrument for soundscape recognition, evaluation and identification.EThOS - Electronic Theses Online ServiceGBUnited Kingdo