Attentive monitoring of multiple video streams driven by a Bayesian foraging strategy

In this paper we shall consider the problem of deploying attention to subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multi-stream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g. activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Dataset, a publicly available dataset, are presented to illustrate the utility of the proposed technique.Comment: Accepted to IEEE Transactions on Image Processin

Optimized Packet Scheduling in Multiview Video Navigation Systems

In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems

A Collective Adaptive Approach to Decentralised k-Coverage in Multi-robot Systems

We focus on the online multi-object k-coverage problem (OMOkC), where mobile robots are required to sense a mobile target from k diverse points of view, coordinating themselves in a scalable and possibly decentralised way. There is active research on OMOkC, particularly in the design of decentralised algorithms for solving it. We propose a new take on the issue: Rather than classically developing new algorithms, we apply a macro-level paradigm, called aggregate computing, specifically designed to directly program the global behaviour of a whole ensemble of devices at once. To understand the potential of the application of aggregate computing to OMOkC, we extend the Alchemist simulator (supporting aggregate computing natively) with a novel toolchain component supporting the simulation of mobile robots. This way, we build a software engineering toolchain comprising language and simulation tooling for addressing OMOkC. Finally, we exercise our approach and related toolchain by introducing new algorithms for OMOkC; we show that they can be expressed concisely, reuse existing software components and perform better than the current state-of-the-art in terms of coverage over time and number of objects covered overall

Saliency-directed prioritization of visual data in wireless surveillance networks

YesIn wireless visual sensor networks (WVSNs), streaming all imaging data is impractical due to resource constraints. Moreover, the sheer volume of surveillance videos inhibits the ability of analysts to extract actionable intelligence. In this work, an energy-efficient image prioritization framework is presented to cope with the fragility of traditional WVSNs. The proposed framework selects semantically relevant information before it is transmitted to a sink node. This is based on salient motion detection, which works on the principle of human cognitive processes. Each camera node estimates the background by a bootstrapping procedure, thus increasing the efficiency of salient motion detection. Based on the salient motion, each sensor node is classified as being high or low priority. This classification is dynamic, such that camera nodes toggle between high-priority and low-priority status depending on the coverage of the region of interest. High-priority camera nodes are allowed to access reliable radio channels to ensure the timely and reliable transmission of data. We compare the performance of this framework with other state-of-the-art methods for both single and multi-camera monitoring. The results demonstrate the usefulness of the proposed method in terms of salient event coverage and reduced computational and transmission costs, as well as in helping analysts find semantically relevant visual information.Supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2012904)

Resource-Aware Coverage and Task Assignment in Visual Sensor Networks

visual sensor network (VSN) consists of a large amount of camera nodes which are able to process the captured image data locally and to extract the relevant information. The tight resource limitations in these networks of embedded sensors and processors represent a major challenge for the application development. In this paper we focus on finding optimal VSN configurations which are basically given by (i) the selection of cameras to sufficiently monitor the area of interest, (ii) the setting of the cameras\u2019 frame rate and resolution to fulfill the quality of service (QoS) requirements, and (iii) the assignment of processing tasks to cameras to achieve all required monitoring activities. We formally specify this configuration problem and describe an efficient approximation method based on an evolutionary algorithm. We analyze our approximation method on three different scenarios and compare the predicted results with measurements on real implementations on a VSN platform. We finally combine our approximation method with an expectation-maximization algorithm for optimizing the coverage and resource allocation in VSN with pan-tilt-zoom (PTZ) camera nodes

Cooperative task assignment for multiple vehicles

Multi-vehicle systems have been increasingly exploited to accomplish difficult and complex missions, where effective and efficient coordinations of the vehicles can greatly improve the team's performance. Motivated by need from practice, we study the multi-vehicle task assignment in various challenging environments. We first investigate the task assignment for multiple vehicles in a time-invariant drift field. The objective is to employ the vehicles to visit a set of target locations in the drift field while trying to minimize the vehicles' total travel time. Using optimal control theory, a path planning algorithm is designed to generate the time-optimal path for a vehicle to travel between any two prescribed locations in a drift field. The path planning algorithm provides the cost matrix for the target assignment, and generates routes once the target locations are assigned to the vehicles. Using tools from graph theory, a lower bound on the optimal solution is found, which can be used to measure the proximity of a solution from the optimal. We propose several clustering-based task assignment algorithms in which two of them guarantee that all the target locations will be visited within a computable maximal travel time, which is at most twice of the optimal when the cost matrix is symmetric. In addition, we extend the multi-vehicle task assignment study in a time-invariant drift field with obstacles. The vehicles have different capabilities, and each kind of vehicles need to visit a certain type of target locations; each target location might have the demand to be visited more than once by different kinds of vehicles. A path planning method has been designed to enable the vehicles to move between two prescribed locations in a drift field with the minimal time while avoiding obstacles. This task assignment problem is shown to be NP-hard, and a distributed task assignment algorithm has been designed, which can achieve near-optimal solutions to the task assignment problem. Furthermore, we study the task assignment problem in which multiple dispersed heterogeneous vehicles with limited communication range need to visit a set of target locations while trying to minimize the vehicles' total travel distance. Each vehicle initially has the position information of all the targets and of those vehicles that are within its limited communication range, and each target demands a vehicle with some specified capability to visit it. We design a decentralized auction algorithm which first employs an information consensus procedure to merge the local information carried by each communication-connected vehicle subnetwork. Then, the algorithm constructs conflict-free target assignments for the communication-connected vehicles, and guarantees that the total travel distance of the vehicles is at most twice of the optimal when the communication network is initially connected. In the end we exploit the precedence-constrained task assignment problem for a truck and a micro drone to deliver packages to a set of dispersed customers subject to precedence constraints that specify which customers need to be visited before which other customers. The truck is restricted to travel in a street network and the micro drone, restricted by its loading capacity and operation range, can fly from the truck to perform the last mile package deliveries. The objective is to minimize the time to serve all the customers respecting every precedence constraint. The problem is shown to be NP-hard, and a lower bound on the optimal time to serve all the customers is constructed by using tools from graph theory. Integrating with a topological sorting technique, several heuristic task assignment algorithms are constructed to solve the task assignment problem