Review of data fusion methods for real-time and multi-sensor traffic flow analysis

Recently, development in intelligent transportation systems (ITS) requires the input of various kinds of data in real-time and from multiple sources, which imposes additional research and application challenges. Ongoing studies on Data Fusion (DF) have produced significant improvement in ITS and manifested an enormous impact on its growth. This paper reviews the implementation of DF methods in ITS to facilitate traffic flow analysis (TFA) and solutions that entail the prediction of various traffic variables such as driving behavior, travel time, speed, density, incident, and traffic flow. It attempts to identify and discuss real-time and multi-sensor data sources that are used for various traffic domains, including road/highway management, traffic states estimation, and traffic controller optimization. Moreover, it attempts to associate abstractions of data level fusion, feature level fusion, and decision level fusion on DF methods to better understand the role of DF in TFA and ITS. Consequently, the main objective of this paper is to review DF methods used for real-time and multi-sensor (heterogeneous) TFA studies. The review outcomes are (i) a guideline of constructing DF methods which involve preprocessing, filtering, decision, and evaluation as core steps, (ii) a description of the recent DF algorithms or methods that adopt real-time and multi-sensor sources data and the impact of these data sources on the improvement of TFA, (iii) an examination of the testing and evaluation methodologies and the popular datasets and (iv) an identification of several research gaps, some current challenges, and new research trends

on the fly integration of soft and sensor data for enhanced situation assessment

Abstract Situation assessment is at the core of many critical tasks in the civilian and military domains: border monitoring, surveillance of areas and facilities, entity tracking and identification, all require accurate and up-to-day descriptions of the course of events. For all those applications, situations to be built are complex, dynamic and uncertain and their assessment is based on the integration of diverse sources, including sensors and their row values, images, observations, tactical information and knowledge expressed by domain experts or synthesized through discovery techniques. This paper presents a method to combine soft and sensor data to create enhanced situation assessment for a track-and-detect application. First we create a situation of entities and relationships by using only hard data provided by sensors and then we enrich this situation thanks to soft data, in the form of succinct or more complex observation reports. The system relies on semantic mediation to combine observations and sensor data by using ontologies as a common ground creating a bridge between two complementary yet incomplete representations of the world. The result is an augmented situation, having more precise, accurate or complete descriptions of entities and which is easier to analyze. This enhanced assessment allows for the situation to be understood and processed in a meaningful way by decision makers

Robust Fusion of LiDAR and Wide-Angle Camera Data for Autonomous Mobile Robots

Autonomous robots that assist humans in day to day living tasks are becoming increasingly popular. Autonomous mobile robots operate by sensing and perceiving their surrounding environment to make accurate driving decisions. A combination of several different sensors such as LiDAR, radar, ultrasound sensors and cameras are utilized to sense the surrounding environment of autonomous vehicles. These heterogeneous sensors simultaneously capture various physical attributes of the environment. Such multimodality and redundancy of sensing need to be positively utilized for reliable and consistent perception of the environment through sensor data fusion. However, these multimodal sensor data streams are different from each other in many ways, such as temporal and spatial resolution, data format, and geometric alignment. For the subsequent perception algorithms to utilize the diversity offered by multimodal sensing, the data streams need to be spatially, geometrically and temporally aligned with each other. In this paper, we address the problem of fusing the outputs of a Light Detection and Ranging (LiDAR) scanner and a wide-angle monocular image sensor for free space detection. The outputs of LiDAR scanner and the image sensor are of different spatial resolutions and need to be aligned with each other. A geometrical model is used to spatially align the two sensor outputs, followed by a Gaussian Process (GP) regression-based resolution matching algorithm to interpolate the missing data with quantifiable uncertainty. The results indicate that the proposed sensor data fusion framework significantly aids the subsequent perception steps, as illustrated by the performance improvement of a uncertainty aware free space detection algorith

City Data Fusion: Sensor Data Fusion in the Internet of Things

Internet of Things (IoT) has gained substantial attention recently and play a significant role in smart city application deployments. A number of such smart city applications depend on sensor fusion capabilities in the cloud from diverse data sources. We introduce the concept of IoT and present in detail ten different parameters that govern our sensor data fusion evaluation framework. We then evaluate the current state-of-the art in sensor data fusion against our sensor data fusion framework. Our main goal is to examine and survey different sensor data fusion research efforts based on our evaluation framework. The major open research issues related to sensor data fusion are also presented.Comment: Accepted to be published in International Journal of Distributed Systems and Technologies (IJDST), 201

Hypothesis Testing and Model Estimation with Dependent Observations in Heterogeneous Sensor Networks

Advances in microelectronics, communication and signal processing have enabled the development of inexpensive sensors that can be networked to collect vital information from their environment to be used in decision-making and inference. The sensors transmit their data to a central processor which integrates the information from the sensors using a so-called fusion algorithm. Many applications of sensor networks (SNs) involve hypothesis testing or the detection of a phenomenon. Many approaches to data fusion for hypothesis testing assume that, given each hypothesis, the sensors\u27 measurements are conditionally independent. However, since the sensors are densely deployed in practice, their field of views overlap and consequently their measurements are dependent. Moreover, a sensor\u27s measurement samples may be correlated over time. Another assumption often used in data fusion algorithms is that the underlying statistical model of sensors\u27 observations is completely known. However, in practice these statistics may not be available prior to deployment and may change over the lifetime of the network due to hardware changes, aging, and environmental conditions. In this dissertation, we consider the problem of data fusion in heterogeneous SNs (SNs in which the sensors are not identical) collecting dependent data. We develop the expectation maximization algorithm for hypothesis testing and model estimation. Copula distributions are used to model the correlation in the data. Moreover, it is assumed that the distribution of the sensors\u27 measurements is not completely known. we consider both parametric and non-parametric model estimation. The proposed approach is developed for both batch and online processing. In batch processing, fusion can only be performed after a block of data samples is received from each sensor, while in online processing, fusion is performed upon arrival of each data sample. Online processing is of great interest since for many applications, the long delay required for the accumulation of data in batch processing is not acceptable. To evaluate the proposed algorithms, both simulation data and real-world datasets are used. Detection performances of the proposed algorithms are compared with well-known supervised and unsupervised learning methods as well as with similar EM-based methods, which either partially or entirely ignore the dependence in the data

Dynamic learning of the environment for eco-citizen behavior

Le développement de villes intelligentes et durables nécessite le déploiement des technologies de l'information et de la communication (ITC) pour garantir de meilleurs services et informations disponibles à tout moment et partout. Comme les dispositifs IoT devenant plus puissants et moins coûteux, la mise en place d'un réseau de capteurs dans un contexte urbain peut être coûteuse. Cette thèse propose une technique pour estimer les informations environnementales manquantes dans des environnements à large échelle. Notre technique permet de fournir des informations alors que les dispositifs ne sont pas disponibles dans une zone de l'environnement non couverte par des capteurs. La contribution de notre proposition est résumée dans les points suivants : - limiter le nombre de dispositifs de détection à déployer dans un environnement urbain ; - l'exploitation de données hétérogènes acquises par des dispositifs intermittents ; - le traitement en temps réel des informations ; - l'auto-calibration du système. Notre proposition utilise l'approche AMAS (Adaptive Multi-Agent System) pour résoudre le problème de l'indisponibilité des informations. Dans cette approche, une exception est considérée comme une situation non coopérative (NCS) qui doit être résolue localement et de manière coopérative. HybridIoT exploite à la fois des informations homogènes (informations du même type) et hétérogènes (informations de différents types ou unités) acquises à partir d'un capteur disponible pour fournir des estimations précises au point de l'environnement où un capteur n'est pas disponible. La technique proposée permet d'estimer des informations environnementales précises dans des conditions de variabilité résultant du contexte d'application urbaine dans lequel le projet est situé, et qui n'ont pas été explorées par les solutions de l'état de l'art : - ouverture : les capteurs peuvent entrer ou sortir du système à tout moment sans qu'aucune configuration particulière soit nécessaire ; - large échelle : le système peut être déployé dans un contexte urbain à large échelle et assurer un fonctionnement correct avec un nombre significatif de dispositifs ; - hétérogénéité : le système traite différents types d'informations sans aucune configuration a priori. Notre proposition ne nécessite aucun paramètre d'entrée ni aucune reconfiguration. Le système peut fonctionner dans des environnements ouverts et dynamiques tels que les villes, où un grand nombre de capteurs peuvent apparaître ou disparaître à tout moment et sans aucun préavis. Nous avons fait différentes expérimentations pour comparer les résultats obtenus à plusieurs techniques standard afin d'évaluer la validité de notre proposition. Nous avons également développé un ensemble de techniques standard pour produire des résultats de base qui seront comparés à ceux obtenus par notre proposition multi-agents.The development of sustainable smart cities requires the deployment of Information and Communication Technology (ICT) to ensure better services and available information at any time and everywhere. As IoT devices become more powerful and low-cost, the implementation of an extensive sensor network for an urban context can be expensive. This thesis proposes a technique for estimating missing environmental information in large scale environments. Our technique enables providing information whereas devices are not available for an area of the environment not covered by sensing devices. The contribution of our proposal is summarized in the following points: * limiting the number of sensing devices to be deployed in an urban environment; * the exploitation of heterogeneous data acquired from intermittent devices; * real-time processing of information; * self-calibration of the system. Our proposal uses the Adaptive Multi-Agent System (AMAS) approach to solve the problem of information unavailability. In this approach, an exception is considered as a Non-Cooperative Situation (NCS) that has to be solved locally and cooperatively. HybridIoT exploits both homogeneous (information of the same type) and heterogeneous information (information of different types or units) acquired from some available sensing device to provide accurate estimates in the point of the environment where a sensing device is not available. The proposed technique enables estimating accurate environmental information under conditions of uncertainty arising from the urban application context in which the project is situated, and which have not been explored by the state-of-the-art solutions: * openness: sensors can enter or leave the system at any time without the need for any reconfiguration; * large scale: the system can be deployed in a large, urban context and ensure correct operation with a significative number of devices; * heterogeneity: the system handles different types of information without any a priori configuration. Our proposal does not require any input parameters or reconfiguration. The system can operate in open, dynamic environments such as cities, where a large number of sensing devices can appear or disappear at any time and without any prior notification. We carried out different experiments to compare the obtained results to various standard techniques to assess the validity of our proposal. We also developed a pipeline of standard techniques to produce baseline results that will be compared to those obtained by our multi-agent proposal