Research Opportunities in Contextualized Fusion Systems. The Harbor Surveillance Case

Proceedings of: International Workshop of Intelligent Systems for Context-Based Information Fusion (ISCIF 2011) associated to 11th International Work-Conference on Artificial Neural Networks, IWANN, Torremolinos-Málaga, Spain, June 8-10, 2011.The design of modern Information Fusion (IF) systems involves a complex process to achieve the requirements in the selected applications, especially in domains with a high degree of customization. In general, an advanced fusion system is required to show robust, context-sensitive behavior and efficient performance in real time. It is necessary to exploit all potentially relevant sensor and contextual information in the most appropriate way. Among modern applications for IF technology is the case of surveillance of complex harbor environments that are comprised of large numbers of surface vessels, high-value and dangerous facilities, and many people. The particular conditions and open needs in the harbor scenario are reviewed in this paper, highlighting research opportunities to explore in the development of fusion systems in this area.This work was supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC and CAM CONTEXTS S2009/TIC-1485.Publicad

Contextual Knowledge and Information Fusion for Maritime Piracy Surveillance

Proceedings of: NATO Advanced Study Institute (ASI) on Prediction and Recognition of Piracy Efforts Using Collaborative Human-Centric Information Systems, Salamanca, 19-30 September, 2011Though piracy accounts for only a small fraction of the general losses of the maritime industry it creates a serious threat to the maritime security because of the connections between organized piracy and wider criminal networks and corruption on land. Fighting piracy requires monitoring the waterways, harbors,and criminal networks on the land to increase the ability of the decision makers to predict piracy attracts and manage operations to prevent or contain them. Piracy surveillance involves representing and processing huge amount heterogeneous information often uncertain, unreliable, and irrelevant within a specific context to detect and recognize suspicious activities to alert decision makers on vessel behaviors of interest with minimal false alarm. The paper discusses the role of information fusion, and context representation and utilization in building an piracy surveillance picture.This paper has utilized the results of the research activity supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC and CAM CONTEXTS (S2009/TIC-1485)Publicad

Applying the Dynamic Region Connection Calculus to Exploit Geographic Knowledge in Maritime Surveillance

Proceedings of: 15th International Conference on Information Fusion (FUSION 2012), Singapore, 9-12 July 2012.Concerns about the protection of the global transport network have risen the need of new security and surveillance systems. Ontology-based and fusion systems represent an attractive alternative for practical applications focused on fast and accurate responses. This paper presents an architecture based on a geometric model to efficiently predict and calculate the topological relationships between spatial objects. This model aims to reduce the number of calculations by relying on a spatial data structure. The goal is the detection of threatening behaviors next to points of interest without a noticeable loss of efficiency. The architecture has been embedded in an ontology-based prototype compliant with the Joint Directors of Laboratories (JDL) model for Information Fusion. The prototype capabilities are illustrated by applying international protection rules in maritime scenarios.This work was supported in part by Projects CICYT TIN2011-28620-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029- C02-02.Publicad

A vision system for mobile maritime surveillance platforms

Mobile surveillance systems play an important role to minimise security and safety threats in high-risk or hazardous environments. Providing a mobile marine surveillance platform with situational awareness of its environment is important for mission success. An essential part of situational awareness is the ability to detect and subsequently track potential target objects.Typically, the exact type of target objects is unknown, hence detection is addressed as a problem of finding parts of an image that stand out in relation to their surrounding regions or are atypical to the domain. Contrary to existing saliency methods, this thesis proposes the use of a domain specific visual attention approach for detecting potential regions of interest in maritime imagery. For this, low-level features that are indicative of maritime targets are identified. These features are then evaluated with respect to their local, regional, and global significance. Together with a domain specific background segmentation technique, the features are combined in a Bayesian classifier to direct visual attention to potential target objects.The maritime environment introduces challenges to the camera system: gusts, wind, swell, or waves can cause the platform to move drastically and unpredictably. Pan-tilt-zoom cameras that are often utilised for surveillance tasks can adjusting their orientation to provide a stable view onto the target. However, in rough maritime environments this requires high-speed and precise inputs. In contrast, omnidirectional cameras provide a full spherical view, which allows the acquisition and tracking of multiple targets at the same time. However, the target itself only occupies a small fraction of the overall view. This thesis proposes a novel, target-centric approach for image stabilisation. A virtual camera is extracted from the omnidirectional view for each target and is adjusted based on the measurements of an inertial measurement unit and an image feature tracker. The combination of these two techniques in a probabilistic framework allows for stabilisation of rotational and translational ego-motion. Furthermore, it has the specific advantage of being robust to loosely calibrated and synchronised hardware since the fusion of tracking and stabilisation means that tracking uncertainty can be used to compensate for errors in calibration and synchronisation. This then completely eliminates the need for tedious calibration phases and the adverse effects of assembly slippage over time.Finally, this thesis combines the visual attention and omnidirectional stabilisation frameworks and proposes a multi view tracking system that is capable of detecting potential target objects in the maritime domain. Although the visual attention framework performed well on the benchmark datasets, the evaluation on real-world maritime imagery produced a high number of false positives. An investigation reveals that the problem is that benchmark data sets are unconsciously being influenced by human shot selection, which greatly simplifies the problem of visual attention. Despite the number of false positives, the tracking approach itself is robust even if a high number of false positives are tracked