Visual / acoustic detection and localisation in embedded systems

©Cranfield UniversityThe continuous miniaturisation of sensing and processing technologies is increasingly offering a variety of embedded platforms, enabling the accomplishment of a broad range of tasks using such systems. Motivated by these advances, this thesis investigates embedded detection and localisation solutions using vision and acoustic sensors. Focus is particularly placed on surveillance applications using sensor networks. Existing vision-based detection solutions for embedded systems suffer from the sensitivity to environmental conditions. In the literature, there seems to be no algorithm able to simultaneously tackle all the challenges inherent to real-world videos. Regarding the acoustic modality, many research works have investigated acoustic source localisation solutions in distributed sensor networks. Nevertheless, it is still a challenging task to develop an ecient algorithm that deals with the experimental issues, to approach the performance required by these systems and to perform the data processing in a distributed and robust manner. The movement of scene objects is generally accompanied with sound emissions with features that vary from an environment to another. Therefore, considering the combination of the visual and acoustic modalities would offer a significant opportunity for improving the detection and/or localisation using the described platforms. In the light of the described framework, we investigate in the first part of the thesis the use of a cost-effective visual based method that can deal robustly with the issue of motion detection in static, dynamic and moving background conditions. For motion detection in static and dynamic backgrounds, we present the development and the performance analysis of a spatio- temporal form of the Gaussian mixture model. On the other hand, the problem of motion detection in moving backgrounds is addressed by accounting for registration errors in the captured images. By adopting a robust optimisation technique that takes into account the uncertainty about the visual measurements, we show that high detection accuracy can be achieved. In the second part of this thesis, we investigate solutions to the problem of acoustic source localisation using a trust region based optimisation technique. The proposed method shows an overall higher accuracy and convergence improvement compared to a linear-search based method. More importantly, we show that through characterising the errors in measurements, which is a common problem for such platforms, higher accuracy in the localisation can be attained. The last part of this work studies the different possibilities of combining visual and acoustic information in a distributed sensors network. In this context, we first propose to include the acoustic information in the visual model. The obtained new augmented model provides promising improvements in the detection and localisation processes. The second investigated solution consists in the fusion of the measurements coming from the different sensors. An evaluation of the accuracy of localisation and tracking using a centralised/decentralised architecture is conducted in various scenarios and experimental conditions. Results have shown the capability of this fusion approach to yield higher accuracy in the localisation and tracking of an active acoustic source than by using a single type of data

Distributed parameter and state estimation for wireless sensor networks

The research in distributed algorithms is linked with the developments of statistical inference in wireless sensor networks (WSNs) applications. Typically, distributed approaches process the collected signals from networked sensor nodes. That is to say, the sensors receive local observations and transmit information between each other. Each sensor is capable of combining the collected information with its own observations to improve performance. In this thesis, we propose novel distributed methods for the inference applications using wireless sensor networks. In particular, the efficient algorithms which are not computationally intensive are investigated. Moreover, we present a number of novel algorithms for processing asynchronous network events and robust state estimation. In the first part of the thesis, a distributed adaptive algorithm based on the component-wise EM method for decentralized sensor networks is investigated. The distributed component-wise Expectation-Maximization (EM) algorithm has been designed for application in a Gaussian density estimation. The proposed algorithm operates a component-wise EM procedure for local parameter estimation and exploit an incremental strategy for network updating, which can provide an improved convergence rate. Numerical simulation results have illustrated the advantages of the proposed distributed component-wise EM algorithm for both well-separated and overlapped mixture densities. The distributed component-wise EM algorithm can outperform other EM-based distributed algorithms in estimating overlapping Gaussian mixtures. In the second part of the thesis, a diffusion based EM gradient algorithm for density estimation in asynchronous wireless sensor networks has been proposed. Specifically, based on the asynchronous adapt-then-combine diffusion strategy, a distributed EM gradient algorithm that can deal with asynchronous network events has been considered. The Bernoulli model has been exploited to approximate the asynchronous behaviour of the network. Compared with existing distributed EM based estimation methods using a consensus strategy, the proposed algorithm can provide more accurate estimates in the presence of asynchronous networks uncertainties, such as random link failures, random data arrival times, and turning on or off sensor nodes for energy conservation. Simulation experiments have been demonstrated that the proposed algorithm significantly outperforms the consensus based strategies in terms of Mean-Square- Deviation (MSD) performance in an asynchronous network setting. Finally, the challenge of distributed state estimation in power systems which requires low complexity and high stability in the presence of bad data for a large scale network is addressed. A gossip based quasi-Newton algorithm has been proposed for solving the power system state estimation problem. In particular, we have applied the quasi-Newton method for distributed state estimation under the gossip protocol. The proposed algorithm exploits the Broyden- Fletcher-Goldfarb-Shanno (BFGS) formula to approximate the Hessian matrix, thus avoiding the computation of inverse Hessian matrices for each control area. The simulation results for IEEE 14 bus system and a large scale 4200 bus system have shown that the distributed quasi-Newton scheme outperforms existing algorithms in terms of Mean-Square-Error (MSE) performance with bad data

Likelihood Consensus and Its Application to Distributed Particle Filtering

We consider distributed state estimation in a wireless sensor network without a fusion center. Each sensor performs a global estimation task---based on the past and current measurements of all sensors---using only local processing and local communications with its neighbors. In this estimation task, the joint (all-sensors) likelihood function (JLF) plays a central role as it epitomizes the measurements of all sensors. We propose a distributed method for computing, at each sensor, an approximation of the JLF by means of consensus algorithms. This "likelihood consensus" method is applicable if the local likelihood functions of the various sensors (viewed as conditional probability density functions of the local measurements) belong to the exponential family of distributions. We then use the likelihood consensus method to implement a distributed particle filter and a distributed Gaussian particle filter. Each sensor runs a local particle filter, or a local Gaussian particle filter, that computes a global state estimate. The weight update in each local (Gaussian) particle filter employs the JLF, which is obtained through the likelihood consensus scheme. For the distributed Gaussian particle filter, the number of particles can be significantly reduced by means of an additional consensus scheme. Simulation results are presented to assess the performance of the proposed distributed particle filters for a multiple target tracking problem

Design and theoretical analysis of advanced power based positioning in RF system

Accurate locating and tracking of people and resources has become a fundamental requirement for many applications. The global navigation satellite systems (GNSS) is widely used. But its accuracy suffers from signal obstruction by buildings, multipath fading, and disruption due to jamming and spoof. Hence, it is required to supplement GPS with inertial sensors and indoor localization schemes that make use of WiFi APs or beacon nodes. In the GPS-challenging or fault scenario, radio-frequency (RF) infrastructure based localization schemes can be a fallback solution for robust navigation. For the indoor/outdoor transition scenario, we propose hypothesis test based fusion method to integrate multi-modal localization sensors. In the first paper, a ubiquitous tracking using motion and location sensor (UTMLS) is proposed. As a fallback approach, power-based schemes are cost-effective when compared with the existing ToA or AoA schemes. However, traditional power-based positioning methods suffer from low accuracy and are vulnerable to environmental fading. Also, the expected accuracy of power-based localization is not well understood but is needed to derive the hypothesis test for the fusion scheme. Hence, in paper 2-5, we focus on developing more accurate power-based localization schemes. The second paper improves the power-based range estimation accuracy by estimating the LoS component. The ranging error model in fading channel is derived. The third paper introduces the LoS-based positioning method with corresponding theoretical limits and error models. In the fourth and fifth paper, a novel antenna radiation-pattern-aware power-based positioning (ARPAP) system and power contour circle fitting (PCCF) algorithm are proposed to address antenna directivity effect on power-based localization. Overall, a complete LoS signal power based positioning system has been developed that can be included in the fusion scheme --Abstract, page iv

Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications, and witnessing a steady transition of this technology to industry. We survey the current state of SLAM. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved

Distributed detection, localization, and estimation in time-critical wireless sensor networks

In this thesis the problem of distributed detection, localization, and estimation (DDLE) of a stationary target in a fusion center (FC) based wireless sensor network (WSN) is considered. The communication process is subject to time-critical operation, restricted power and bandwidth (BW) resources operating over a shared communication channel Buffering from Rayleigh fading and phase noise. A novel algorithm is proposed to solve the DDLE problem consisting of two dependent stages: distributed detection and distributed estimation. The WSN performs distributed detection first and based on the global detection decision the distributed estimation stage is performed. The communication between the SNs and the FC occurs over a shared channel via a slotted Aloha MAC protocol to conserve BW. In distributed detection, hard decision fusion is adopted, using the counting rule (CR), and sensor censoring in order to save power and BW. The effect of Rayleigh fading on distributed detection is also considered and accounted for by using distributed diversity combining techniques where the diversity combining is among the sensor nodes (SNs) in lieu of having the processing done at the FC. Two distributed techniques are proposed: the distributed maximum ratio combining (dMRC) and the distributed Equal Gain Combining (dEGC). Both techniques show superior detection performance when compared to conventional diversity combining procedures that take place at the FC. In distributed estimation, the segmented distributed localization and estimation (SDLE) framework is proposed. The SDLE enables efficient power and BW processing. The SOLE hinges on the idea of introducing intermediate parameters that are estimated locally by the SNs and transmitted to the FC instead of the actual measurements. This concept decouples the main problem into a simpler set of local estimation problems solved at the SNs and a global estimation problem solved at the FC. Two algorithms are proposed for solving the local problem: a nonlinear least squares (NLS) algorithm using the variable projection (VP) method and a simpler gird search (GS) method. Also, Four algorithms are proposed to solve the global problem: NLS, GS, hyperspherical intersection method (HSI), and robust hyperspherical intersection (RHSI) method. Thus, the SDLE can be solved through local and global algorithm combinations. Five combinations are tied: NLS2 (NLS-NLS), NLS-HSI, NLS-RHSI, GS2, and GS-N LS. It turns out that the last algorithm combination delivers the best localization and estimation performance. In fact , the target can be localized with less than one meter error. The SNs send their local estimates to the FC over a shared channel using the slotted-Aloha MAC protocol, which suits WSNs since it requires only one channel. However, Aloha is known for its relatively high medium access or contention delay given the medium access probability is poorly chosen. This fact significantly hinders the time-critical operation of the system. Hence, multi-packet reception (MPR) is used with slotted Aloha protocol, in which several channels are used for contention. The contention delay is analyzed for slotted Aloha with and without MPR. More specifically, the mean and variance have been analytically computed and the contention delay distribution is approximated. Having theoretical expressions for the contention delay statistics enables optimizing both the medium access probability and the number of MPR channels in order to strike a trade-off between delay performance and complexity

Cooperative localisation in underwater robotic swarms for ocean bottom seismic imaging.

Spatial information must be collected alongside the data modality of interest in wide variety of sub-sea applications, such as deep sea exploration, environmental monitoring, geological and ecological research, and samples collection. Ocean-bottom seismic surveys are vital for oil and gas exploration, and for productivity enhancement of an existing production facility. Ocean-bottom seismic sensors are deployed on the seabed to acquire those surveys. Node deployment methods used in industry today are costly, time-consuming and unusable in deep oceans. This study proposes the autonomous deployment of ocean-bottom seismic nodes, implemented by a swarm of Autonomous Underwater Vehicles (AUVs). In autonomous deployment of ocean-bottom seismic nodes, a swarm of sensor-equipped AUVs are deployed to achieve ocean-bottom seismic imaging through collaboration and communication. However, the severely limited bandwidth of underwater acoustic communications and the high cost of maritime assets limit the number of AUVs that can be deployed for experiments. A holistic fuzzy-based localisation framework for large underwater robotic swarms (i.e. with hundreds of AUVs) to dynamically fuse multiple position estimates of an autonomous underwater vehicle is proposed. Simplicity, exibility and scalability are the main three advantages inherent in the proposed localisation framework, when compared to other traditional and commonly adopted underwater localisation methods, such as the Extended Kalman Filter. The proposed fuzzy-based localisation algorithm improves the entire swarm mean localisation error and standard deviation (by 16.53% and 35.17% respectively) at a swarm size of 150 AUVs when compared to the Extended Kalman Filter based localisation with round-robin scheduling. The proposed fuzzy based localisation method requires fuzzy rules and fuzzy set parameters tuning, if the deployment scenario is changed. Therefore a cooperative localisation scheme that relies on a scalar localisation confidence value is proposed. A swarm subset is navigationally aided by ultra-short baseline and a swarm subset (i.e. navigation beacons) is configured to broadcast navigation aids (i.e. range-only), once their confidence values are higher than a predetermined confidence threshold. The confidence value and navigation beacons subset size are two key parameters for the proposed algorithm, so that they are optimised using the evolutionary multi-objective optimisation algorithm NSGA-II to enhance its localisation performance. Confidence value-based localisation is proposed to control the cooperation dynamics among the swarm agents, in terms of aiding acoustic exteroceptive sensors. Given the error characteristics of a commercially available ultra-short baseline system and the covariance matrix of a trilaterated underwater vehicle position, dead reckoning navigation - aided by Extended Kalman Filter-based acoustic exteroceptive sensors - is performed and controlled by the vehicle's confidence value. The proposed confidence-based localisation algorithm has significantly improved the entire swarm mean localisation error when compared to the fuzzy-based and round-robin Extended Kalman Filter-based localisation methods (by 67.10% and 59.28% respectively, at a swarm size of 150 AUVs). The proposed fuzzy-based and confidence-based localisation algorithms for cooperative underwater robotic swarms are validated on a co-simulation platform. A physics-based co-simulation platform that considers an environment's hydrodynamics, industrial grade inertial measurement unit and underwater acoustic communications characteristics is implemented for validation and optimisation purposes