High-Level Information Fusion in Visual Sensor Networks

Information fusion techniques combine data from multiple sensors, along with additional information and knowledge, to obtain better estimates of the observed scenario than could be achieved by the use of single sensors or information sources alone. According to the JDL fusion process model, high-level information fusion is concerned with the computation of a scene representation in terms of abstract entities such as activities and threats, as well as estimating the relationships among these entities. Recent experiences confirm that context knowledge plays a key role in the new-generation high-level fusion systems, especially in those involving complex scenarios that cause the failure of classical statistical techniques –as it happens in visual sensor networks. In this chapter, we study the architectural and functional issues of applying context information to improve high-level fusion procedures, with a particular focus on visual data applications. The use of formal knowledge representations (e.g. ontologies) is a promising advance in this direction, but there are still some unresolved questions that must be more extensively researched.The UC3M Team gratefully acknowledges that this research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02

A Practical Approach to the Development of Ontology-Based Information Fusion Systems

Proceedings of: NATO Advanced Study Institute (ASI) on Prediction and Recognition of Piracy Efforts Using Collaborative Human-Centric Information Systems, Salamanca, 19-30 September, 2011Ontology-based representations are gaining momentum among other alternatives to implement the knowledge model of high-level fusion applications. In this paper, we provide an introduction to the theoretical foundations of ontology-based knowledge representation and reasoning, with a particular focus on the issues that appear in maritime security –where heterogeneous regulations, information sources, users, and systems are involved. We also present some current approaches and existing technologies for high-level fusion based on ontological representations. Unfortunately, current tools for the practical implementation of ontology-based systems are not fully standardized, or even prepared to work together in medium-scale systems. Accordingly, we discuss different alternatives to face problems such as spatial and temporal knowledge representation or uncertainty management. To illustrate the conclusions drawn from this research, an ontology-based semantic tracking system is briefly presented. Results and latent capabilities of this framework are shown at the end of the paper, where we also envision future opportunities for this kind of applications.This research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS 2008-07029-C02-02.Publicad

Strategies and Techniques for Use and Exploitation of Contextual Information in High-Level Fusion Architectures

Proceedings of: 13th Conference on Information Fusion (FUSION 2010): Edinburgh, UK. 26-29 July 2010.Contextual Information is proving to be not only an additional exploitable information source for improving entity and situational estimates in certain Information Fusion systems, but can also be the entire focus of estimation for such systems as those directed to Ambient Intelligence (AI) and Context-Aware(CA) applications. This paper will discuss the role(s) of Contextual Information (CI) in a wide variety of IF applications to include AI, CA, Defense, and Cyber-security among possible others, the issues involved in designing strategies and techniques for CI use and exploitation, provide some exemplars of evolving CI use/exploitation designs on our current projects, and describe some general frameworks that are evolving in various application domains where CI is proving critical.The UC3M Team gratefully acknowledge that this research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732- C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02. UC3M also thanks Prof. James Llinas for his helpful comments during his stay, which has been supported by the collaboration agreement ‘Chairs of Excellence’ between University Carlos III and Banco Santander. The US/UB Team gratefully acknowledge that this research activity is supported by a Multidisciplinary University Research Initiative (MURI) grant (Number W911NF-09-1-0392) for “Unified Research on Networkbased Hard/Soft Information Fusion”, issued by the US Army Research Office (ARO) under the program management of Dr. John LaveryPublicad

A distributed camera system for multi-resolution surveillance

We describe an architecture for a multi-camera, multi-resolution surveillance system. The aim is to support a set of distributed static and pan-tilt-zoom (PTZ) cameras and visual tracking algorithms, together with a central supervisor unit. Each camera (and possibly pan-tilt device) has a dedicated process and processor. Asynchronous interprocess communications and archiving of data are achieved in a simple and effective way via a central repository, implemented using an SQL database. Visual tracking data from static views are stored dynamically into tables in the database via client calls to the SQL server. A supervisor process running on the SQL server determines if active zoom cameras should be dispatched to observe a particular target, and this message is effected via writing demands into another database table. We show results from a real implementation of the system comprising one static camera overviewing the environment under consideration and a PTZ camera operating under closed-loop velocity control, which uses a fast and robust level-set-based region tracker. Experiments demonstrate the effectiveness of our approach and its feasibility to multi-camera systems for intelligent surveillance

Context-based scene recognition from visual data in smart homes: an Information Fusion approach

Ambient Intelligence (AmI) aims at the development of computational systems that process data acquired by sensors embedded in the environment to support users in everyday tasks. Visual sensors, however, have been scarcely used in this kind of applications, even though they provide very valuable information about scene objects: position, speed, color, texture, etc. In this paper, we propose a cognitive framework for the implementation of AmI applications based on visual sensor networks. The framework, inspired by the Information Fusion paradigm, combines a priori context knowledge represented with ontologies with real time single camera data to support logic-based high-level local interpretation of the current situation. In addition, the system is able to automatically generate feedback recommendations to adjust data acquisition procedures. Information about recognized situations is eventually collected by a central node to obtain an overall description of the scene and consequently trigger AmI services. We show the extensible and adaptable nature of the approach with a prototype system in a smart home scenario.This research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008- 06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02.Publicad

Overview of contextual tracking approaches in information fusion

Proceedings of: Geospatial InfoFusion III. 2-3 May 2013 Baltimore, Maryland, United States.Many information fusion solutions work well in the intended scenarios; but the applications, supporting data, and capabilities change over varying contexts. One example is weather data for electro-optical target trackers of which standards have evolved over decades. The operating conditions of: technology changes, sensor/target variations, and the contextual environment can inhibit performance if not included in the initial systems design. In this paper, we seek to define and categorize different types of contextual information. We describe five contextual information categories that support target tracking: (1) domain knowledge from a user to aid the information fusion process through selection, cueing, and analysis, (2) environment-to-hardware processing for sensor management, (3) known distribution of entities for situation/threat assessment, (4) historical traffic behavior for situation awareness patterns of life (POL), and (5) road information for target tracking and identification. Appropriate characterization and representation of contextual information is needed for future high-level information fusion systems design to take advantage of the large data content available for a priori knowledge target tracking algorithm construction, implementation, and application.Publicad

Communication in distributed tracking systems: an ontology-based approach to improve cooperation

Current Computer Vision systems are expected to allow for the management of data acquired by physically distributed cameras. This is especially the case for modern surveillance systems, which require communication between components and a combination of their outputs in order to obtain a complete view of the scene. Information fusion techniques have been successfully applied in this area, but several problems remain unsolved. One of them is the increasing need for coordination and cooperation between independent and heterogeneous cameras. A solution to achieve an understanding between them is to use a common and well-defined message content vocabulary. In this research work, we present a formal ontology aimed at the symbolic representation of visual data, mainly detected tracks corresponding to real-world moving objects. Such an ontological representation provides support for spontaneous communication and component interoperability, increases system scalability and facilitates the development of high-level fusion procedures. The ontology is used by the agents of Cooperative Surveillance Multi-Agent System, our multi-agent framework for multi-camera surveillance systems.This work was supported in part by Projects CICYT TIN2008-06742-C02-02=TSI, CICYT TEC2008-06732-C02-02=TEC, CAM CONTEXTS (S2009=TIC-1485) and DPS2008-07029-C02-02.Publicad

A General Purpose Context Reasoning Environment to Deal with Tracking Problems: An Ontology-based Prototype

Proceedings of: 6th International Conference on Hybrid Artificial Intelligence Systems (HAIS 2011). Wroclaw, Poland, May 23-25, 2011The high complexity of semantics extraction with automatic video analysis has forced the researchers to the generalization of mixed approaches based on perceptual and context data. These mixed approaches do not usually take in account the advantages and benefits of the data fusion discipline. This paper presents a context reasoning environment to deal with general and specific tracking problems. The cornerstone of the environment is a symbolic architecture based on the Joint Directors of Laboratories fusion model. This architecture may build a symbolic data representation from any source, check the data consistency, create new knowledge and refine it through inference obtaining a higher understanding level of the scene and providing feedback to autocorrect the tracking errors. An ontology-based prototype has been developed to carry out experimental tests. The prototype has been proved against tracking analysis occlusion problems.This work was supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/ TIC-1485) and DPS2008-07029-C02-02.Publicad

Context-based Information Fusion: A survey and discussion

This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of \u201ccontext\u201d. It shows how its fortune in the distributed computing world eventually permeated in the world of IF, discussing the current strategies and techniques, and hinting possible future trends. IF processes can represent context at different levels (structural and physical constraints of the scenario, a priori known operational rules between entities and environment, dynamic relationships modelled to interpret the system output, etc.). In addition to the survey, several novel context exploitation dynamics and architectural aspects peculiar to the fusion domain are presented and discussed