High Accuracy Distributed Target Detection and Classification in Sensor Networks Based on Mobile Agent Framework

High-accuracy distributed information exploitation plays an important role in sensor networks. This dissertation describes a mobile-agent-based framework for target detection and classification in sensor networks. Specifically, we tackle the challenging problems of multiple- target detection, high-fidelity target classification, and unknown-target identification. In this dissertation, we present a progressive multiple-target detection approach to estimate the number of targets sequentially and implement it using a mobile-agent framework. To further improve the performance, we present a cluster-based distributed approach where the estimated results from different clusters are fused. Experimental results show that the distributed scheme with the Bayesian fusion method have better performance in the sense that they have the highest detection probability and the most stable performance. In addition, the progressive intra-cluster estimation can reduce data transmission by 83.22% and conserve energy by 81.64% compared to the centralized scheme. For collaborative target classification, we develop a general purpose multi-modality, multi-sensor fusion hierarchy for information integration in sensor networks. The hierarchy is com- posed of four levels of enabling algorithms: local signal processing, temporal fusion, multi-modality fusion, and multi-sensor fusion using a mobile-agent-based framework. The fusion hierarchy ensures fault tolerance and thus generates robust results. In the meanwhile, it also takes into account energy efficiency. Experimental results based on two field demos show constant improvement of classification accuracy over different levels of the hierarchy. Unknown target identification in sensor networks corresponds to the capability of detecting targets without any a priori information, and of modifying the knowledge base dynamically. In this dissertation, we present a collaborative method to solve this problem among multiple sensors. When applied to the military vehicles data set collected in a field demo, about 80% unknown target samples can be recognized correctly, while the known target classification ac- curacy stays above 95%

Tracking interacting targets in multi-modal sensors

PhDObject tracking is one of the fundamental tasks in various applications such as surveillance, sports, video conferencing and activity recognition. Factors such as occlusions, illumination changes and limited field of observance of the sensor make tracking a challenging task. To overcome these challenges the focus of this thesis is on using multiple modalities such as audio and video for multi-target, multi-modal tracking. Particularly, this thesis presents contributions to four related research topics, namely, pre-processing of input signals to reduce noise, multi-modal tracking, simultaneous detection and tracking, and interaction recognition. To improve the performance of detection algorithms, especially in the presence of noise, this thesis investigate filtering of the input data through spatio-temporal feature analysis as well as through frequency band analysis. The pre-processed data from multiple modalities is then fused within Particle filtering (PF). To further minimise the discrepancy between the real and the estimated positions, we propose a strategy that associates the hypotheses and the measurements with a real target, using a Weighted Probabilistic Data Association (WPDA). Since the filtering involved in the detection process reduces the available information and is inapplicable on low signal-to-noise ratio data, we investigate simultaneous detection and tracking approaches and propose a multi-target track-beforedetect Particle filtering (MT-TBD-PF). The proposed MT-TBD-PF algorithm bypasses the detection step and performs tracking in the raw signal. Finally, we apply the proposed multi-modal tracking to recognise interactions between targets in regions within, as well as outside the cameras’ fields of view. The efficiency of the proposed approaches are demonstrated on large uni-modal, multi-modal and multi-sensor scenarios from real world detections, tracking and event recognition datasets and through participation in evaluation campaigns

Optimal UAS Assignments and Trajectories for Persistent Surveillance and Data Collection from a Wireless Sensor Network

This research developed a method for multiple Unmanned Aircraft Systems (UAS) to efficiently collect data from a Wireless Sensor Networks (WSN). WSN are composed of any number of fixed, ground-based sensors that collect and upload local environmental data to over flying UAS. The three-step method first uniquely assigns aircraft to specific sensors on the ground. Second, an efficient flight path is calculated to minimize the aircraft flight time required to verify their assigned sensors. Finally, sensors reporting relatively higher rates of local environmental activity are re-assigned to dedicated aircraft tasked with concentrating on only those sensors. This work was sponsored by the Air Force Research Laboratory, Control Sciences branch, at Wright Patterson AFB. Based on simulated scenarios and preliminary flight tests, optimal flight paths resulted in a 14 to 32 reduction in flight time and distance when compared to traditional flight planning methods

Interpreting EEG and MEG signal modulation in response to facial features: the influence of top-down task demands on visual processing strategies

The visual processing of faces is a fast and efficient feat that our visual system usually accomplishes many times a day. The N170 (an Event-Related Potential) and the M170 (an Event-Related Magnetic Field) are thought to be prominent markers of the face perception process in the ventral stream of visual processing that occur ~ 170 ms after stimulus onset. The question of whether face processing at the time window of the N170 and M170 is automatically driven by bottom-up visual processing only, or whether it is also modulated by top-down control, is still debated in the literature. However, it is known from research on general visual processing, that top-down control can be exerted much earlier along the visual processing stream than the N170 and M170 take place. I conducted two studies, each consisting of two face categorization tasks. In order to examine the influence of top-down control on the processing of faces, I changed the task demands from one task to the next, while presenting the same set of face stimuli. In the first study, I recorded participants’ EEG signal in response to faces while they performed both a Gender task and an Expression task on a set of expressive face stimuli. Analyses using Bubbles (Gosselin & Schyns, 2001) and Classification Image techniques revealed significant task modulations of the N170 ERPs (peaks and amplitudes) and the peak latency of maximum information sensitivity to key facial features. However, task demands did not change the information processing during the N170 with respect to behaviourally diagnostic information. Rather, the N170 seemed to integrate gender and expression diagnostic information equally in both tasks. In the second study, participants completed the same behavioural tasks as in the first study (Gender and Expression), but this time their MEG signal was recorded in order to allow for precise source localisation. After determining the active sources during the M170 time window, a Mutual Information analysis in connection with Bubbles was used to examine voxel sensitivity to both the task-relevant and the task-irrelevant face category. When a face category was relevant for the task, sensitivity to it was usually higher and peaked in different voxels than sensitivity to the task-irrelevant face category. In addition, voxels predictive of categorization accuracy were shown to be sensitive to task-relevant, behaviourally diagnostic facial features only. I conclude that facial feature integration during both N170 and M170 is subject to top-down control. The results are discussed against the background of known face processing models and current research findings on visual processing

Towards Improving Learning with Consumer-Grade, Closed-Loop, Electroencephalographic Neurofeedback

Learning is an enigmatic process composed of a multitude of cognitive systems that are functionally and neuroanatomically distinct. Nevertheless, two undeniable pillars which underpin learning are attention and memory; to learn, one must attend, and maintain a representation of, an event. Psychological and neuroscientific technologies that permit researchers to “mind-read” have revealed much about the dynamics of these distinct processes that contribute to learning. This investigation first outlines the cognitive pillars which support learning and the technologies that permit such an understanding. It then employs a novel task—the amSMART paradigm—with the goal of building a real-time, closed-loop, electroencephalographic (EEG) neurofeedback paradigm using consumergrade brain-computer interface (BCI) hardware. Data are presented which indicate the current status of consumer-grade BCI for EEG cognition classification and enhancement, and directions are suggested for the developing world of consumer neurofeedback

DESIGN OPTIMIZATION OF EMBEDDED SIGNAL PROCESSING SYSTEMS FOR TARGET DETECTION

Sensor networks for automated detection of targets, such as pedestrians and vehicles, are highly relevant in defense and surveillance applications. For this purpose, a variety of target detection algorithms and systems using different types of sensors have been proposed in the literature. Among them, systems based on non-image sensors are of special interest in many practical deployment scenarios because of their power efficiency and low computational loads. In this thesis, we investigate low power sensor systems for detecting people and vehicles using non-image sensors such as acoustic and seismic sensors. Our investigation is focused on design optimization across trade-offs including real-time performance, energy efficiency, and target detection accuracy, which are key design evaluation metrics for this class of systems. Design and implementation of low power, embedded target detection systems can be decomposed into two major, inter-related subproblems: (a) algorithm development, which encompasses the development or selection of detection algorithms and optimization of their parameters, and (b) system development, which involves the mapping of the algorithms derived from (a) into real-time, energy efficient implementations on the targeted embedded platforms. In this thesis, we address both of these subproblems in an integrated manner. That is, we investigate novel algorithmic techniques for improvement of accuracy without excessive computational complexity, and we develop new design methodologies, tools, and implementations for efficient realization of target detection algorithms on embedded platforms. We focus specifically on target detection systems that employ acoustic and seismic sensing modalities. These selected modalities support the low power design objectives of our work. However, we envision that our developed algorithms and implementation techniques can be extended readily to other types or combinations of relevant sensing modalities. Throughout this research, we have developed prototypes of our new algorithms and design methods on embedded platforms, and we have experimented with these prototypes to demonstrate our findings, and iteratively improve upon the achieved implementation trade-offs. The main contributions of this thesis are summarized in the following. (1). Classification algorithm for acoustic and seismic signals. We have developed a new classification algorithm for discrimination among people, vehicles, and noise. The algorithm is based on a new fusion technique for acoustic and seismic signals. Our new fusion technique was evaluated through experiments using actual measured datasets, which were collected from different sensors installed in different locations and at different times of day. Our proposed classification algorithm was shown to achieve a significant reduction in the number of false alarms compared to a baseline fusion approach. (2). Joint target localization and classification framework using sensor networks. We designed a joint framework for target localization and classification using a single generalized model for non-imaging based multi- modal sensor data. For target localization, we exploited both sensor data and estimated dynamics within a local neighborhood. We validated the capabilities of our framework by using an actual multi-modal dataset, which includes ground truth GPS information (e.g., time and position) and data from co-located seismic and acoustic sensors. Experimental results showed that our framework achieves better classification accuracy compared to state of the art fusion algorithms using temporal accumulation and achieves more accurate target localizations than a baseline target localization approach. (3). Design and optimization of target detection systems on embedded platforms using dataflow methods. We developed a foundation for our system-level design research by introducing a new rapid prototyping methodology and associated software tool. Using this tool, we presented the design and implementation of a novel, multi-mode embedded signal processing system for detection of people and vehicles related to our algorithmic contributions. We applied a strategically-configured suite of single- and dual-modality signal processing techniques together with dataflow-based design optimization for energy-efficient, real-time implementation. Through experiments using a Raspberry Pi platform, we demonstrated the capability of our target detection system to provide efficient operational trade-offs among detection accuracy, energy efficiency, and processing speed. (4). Software synthesis from dataflow schedule graphs on multicore platforms. We developed new software synthesis methods and tools for design and implementation of embedded signal processing systems using dataflow schedule graphs (DSGs). DSGs provide formal representations of dataflow schedules, which encapsulate information about the assignment of computational tasks (signal processing modules) to processing resources and the ordering of tasks that are assigned to the same resource. Building on fundamental DSG modeling concepts from the literature, we developed the first algorithms and supporting software synthesis tools for mapping DSG representations into efficient multi-threaded implementations. Our tools replace ad-hoc multicore signal processing system development processes with a structured process that is rooted in dataflow formalisms and supported with a high degree of automation. We evaluated our new DSG methods and tools through a demonstration involving multi-threaded implementation of our proposed classification algorithm and associated fusion technique for acoustic/seismic signals

Uncertainty Management of Intelligent Feature Selection in Wireless Sensor Networks

Wireless sensor networks (WSN) are envisioned to revolutionize the paradigm of monitoring complex real-world systems at a very high resolution. However, the deployment of a large number of unattended sensor nodes in hostile environments, frequent changes of environment dynamics, and severe resource constraints pose uncertainties and limit the potential use of WSN in complex real-world applications. Although uncertainty management in Artificial Intelligence (AI) is well developed and well investigated, its implications in wireless sensor environments are inadequately addressed. This dissertation addresses uncertainty management issues of spatio-temporal patterns generated from sensor data. It provides a framework for characterizing spatio-temporal pattern in WSN. Using rough set theory and temporal reasoning a novel formalism has been developed to characterize and quantify the uncertainties in predicting spatio-temporal patterns from sensor data. This research also uncovers the trade-off among the uncertainty measures, which can be used to develop a multi-objective optimization model for real-time decision making in sensor data aggregation and samplin

Security, trust and cooperation in wireless sensor networks

Wireless sensor networks are a promising technology for many real-world applications such as critical infrastructure monitoring, scientific data gathering, smart buildings, etc.. However, given the typically unattended and potentially unsecured operation environment, there has been an increased number of security threats to sensor networks. In addition, sensor networks have very constrained resources, such as limited energy, memory, computational power, and communication bandwidth. These unique challenges call for new security mechanisms and algorithms. In this dissertation, we propose novel algorithms and models to address some important and challenging security problems in wireless sensor networks. The first part of the dissertation focuses on data trust in sensor networks. Since sensor networks are mainly deployed to monitor events and report data, the quality of received data must be ensured in order to make meaningful inferences from sensor data. We first study a false data injection attack in the distributed state estimation problem and propose a distributed Bayesian detection algorithm, which could maintain correct estimation results when less than one half of the sensors are compromised. To deal with the situation where more than one half of the sensors may be compromised, we introduce a special class of sensor nodes called \textit{trusted cores}. We then design a secure distributed trust aggregation algorithm that can utilize the trusted cores to improve network robustness. We show that as long as there exist some paths that can connect each regular node to one of these trusted cores, the network can not be subverted by attackers. The second part of the dissertation focuses on sensor network monitoring and anomaly detection. A sensor network may suffer from system failures due to loss of links and nodes, or malicious intrusions. Therefore, it is critical to continuously monitor the overall state of the network and locate performance anomalies. The network monitoring and probe selection problem is formulated as a budgeted coverage problem and a Markov decision process. Efficient probing strategies are designed to achieve a flexible tradeoff between inference accuracy and probing overhead. Based on the probing results on traffic measurements, anomaly detection can be conducted. To capture the highly dynamic network traffic, we develop a detection scheme based on multi-scale analysis of the traffic using wavelet transforms and hidden Markov models. The performance of the probing strategy and of the detection scheme are extensively evaluated in malicious scenarios using the NS-2 network simulator. Lastly, to better understand the role of trust in sensor networks, a game theoretic model is formulated to mathematically analyze the relation between trust and cooperation. Given the trust relations, the interactions among nodes are modeled as a network game on a trust-weighted graph. We then propose an efficient heuristic method that explores network heterogeneity to improve Nash equilibrium efficiency

Automotive Interior Sensing - Imaging Solutions

Esta dissertação recai sobre o problema da deteção de objetos dentro do veículo. Comparando todos os algoritmos do estado da arte, abordagens baseadas em R-CNN destacam-se em termos de "Average Precision". Normalmente, os detectores "two-stage" têm taxas de precisão mais altas enquanto os detectores "one-stage" conseguem alcançar menores tempos de inferência. O Mask R-CNN foi escolhido graças aos altos valores de "Average Precision" obtidos sem comprometer o tempo de inferência, assim como fornecer a segmentação de instância dos objetos. Isto pode ser útil para abordagens como o "Multiview" no qual é importante estabelecer uma ligação entre os pontos da imagem adquirida por uma câmera com os pontos da imagem adquirida por outra câmera noutra posição. Foi necessário testar a adaptabilidade da Mask R-CNN para outros "datasets" alterando o "dataset" do COCO para ter um número de classes diferente do original. No fim, a rede treinada sobre o "dataset" da Bosch, foi a Faster R-CNN em que os pesos da Mask usados foram os pré-treinados sobre o COCO.This dissertation addresses the problem of object detection inside the vehicle. Comparing all the state-of-the-art algorithms, approaches based on R-CNN stand out in terms of Average Precision. Typically two-stage detectors have higher accuracy rates while one-stage detectors reaches lower inference times. Mask R-CNN was chosen thanks to the high values obtained for Average Precision without compromising inference times, as well as providing object instance segmentation. This may be useful for approaches such as Multiview in which it is important to match points of the image acquired from one camera with points of the image acquired by other camera in another position. It was necessary to test the adaptability of Mask R-CNN to other datasets by changing COCO dataset to have different number of classes. At the end, the trained network over Bosch dataset, was Faster R-CNN with Mask weights pre-trained over COCO

An Impulse Detection Methodology and System with Emphasis on Weapon Fire Detection

This dissertation proposes a methodology for detecting impulse signatures. An algorithm with specific emphasis on weapon fire detection is proposed. Multiple systems in which the detection algorithm can operate, are proposed. In order for detection systems to be used in practical application, they must have high detection performance, minimizing false alarms, be cost effective, and utilize available hardware. Most applications require real time processing and increased range performance, and some applications require detection from mobile platforms. This dissertation intends to provide a methodology for impulse detection, demonstrated for the specific application case of weapon fire detection, that is intended for real world application, taking into account acceptable algorithm performance, feasible system design, and practical implementation. The proposed detection algorithm is implemented with multiple sensors, allowing spectral waveband versatility in system design. The proposed algorithm is also shown to operate at a variety of video frame rates, allowing for practical design using available common, commercial off the shelf hardware. Detection, false alarm, and classification performance are provided, given the use of different sensors and associated wavebands. The false alarms are further mitigated through use of an adaptive, multi-layer classification scheme, leading to potential on-the-move application. The algorithm is shown to work in real time. The proposed system, including algorithm and hardware, is provided. Additional systems are proposed which attempt to complement the strengths and alleviate the weaknesses of the hardware and algorithm. Systems are proposed to mitigate saturation clutter signals and increase detection of saturated targets through the use of position, navigation, and timing sensors, acoustic sensors, and imaging sensors. Furthermore, systems are provided which increase target detection and provide increased functionality, improving the cost effectiveness of the system. The resulting algorithm is shown to enable detection of weapon fire targets, while minimizing false alarms, for real-world, fieldable applications. The work presented demonstrates the complexity of detection algorithm and system design for practical applications in complex environments and also emphasizes the complex interactions and considerations when designing a practical system, where system design is the intersection of algorithm performance and design, hardware performance and design, and size, weight, power, cost, and processing