High-Dynamic-Range Lighting Estimation From Face Portraits.

We present a CNN-based method for outdoor highdynamic-range (HDR) environment map prediction from low-dynamic-range (LDR) portrait images. Our method relies on two different CNN architectures, one for light encoding and another for face-to-light prediction. Outdoor lighting is characterised by an extremely high dynamic range, and thus our encoding splits the environment map data between low and high-intensity components, and encodes them using tailored representations. The combination of both network architectures constitutes an end-to-end method for accurate HDR light prediction from faces at real-time rates, inaccessible for previous methods which focused on low dynamic range lighting or relied on non-linear optimisation schemes. We train our networks using both real and synthetic images, we compare our light encoding with other methods for light representation, and we analyse our results for light prediction on real images. We show that our predicted HDR environment maps can be used as accurate illumination sources for scene renderings, with potential applications in 3D object insertion for augmented reality

Influence of zoom selection on a Kalman filter

The use of a single camera with a zoom lens for tracking involves a continuous arbitration of accuracy vs. reliability. We address this problem with an information-theoretic approach, where we extend zoom selection based on conditional entropy by incorporating the fixation errors into the observation likelihood. We present a thorough analysis of previous approaches, revealing zoom and speed limits, especially how the ratio of process to measurement noise effectively limits the maximally usable zoom for any system tracking with a Kalman filter. This work finally presents means to circumvent aforementioned limitations.Eric Sommerlade and Ian Rei

Probabilistic surveillance with multiple active cameras

In this work we present a consistent probabilistic approach to control multiple, but diverse pan-tilt-zoom cameras concertedly observing a scene. There are disparate goals to this control: the cameras are not only to react to objects moving about, arbitrating conflicting interests of target resolution and trajectory accuracy, they are also to anticipate the appearance of new targets. We base our control function on maximisation of expected mutual information gain, which to our knowledge is novel to the field of computer vision in the context of multiple pan-tilt-zoom camera control. This information theoretic measure yields a utility for each goal and parameter setting, making the use of physical or computational resources comparable. Weighting this utility allows to prioritise certain objectives or targets in the control. The resulting behaviours in typical situations for multi-camera systems, such as camera hand-off, acquisition of close-ups and scene exploration, are emergent but intuitive. We quantitatively show that without the need for hand crafted rules they address the given objectives.Eric Sommerlade and Ian Rei

Information-theoretic active scene exploration

Studies support the need for high resolution imagery to identify persons in surveillance videos. However, the use of telephoto lenses sacrifices a wider field of view and thereby increases the uncertainty of other, possibly more interesting events in the scene. Using zoom lenses offers the possibility of enjoying the benefits of both wide field of view and high resolution, but not simultaneously. We approach this problem of balancing these finite imaging resources - or of exploration vs exploitation - using an information-theoretic approach. We argue that the camera parameters - pan, tilt and zoom - should be set to maximise information gain, or equivalently minimising conditional entropy of the scene model, comprised of multiple targets and a yet unobserved one. The information content of the former is supplied directly by the uncertainties computed using a Kalman filter tracker, while the latter is modelled using a rdquobackgroundrdquo Poisson process whose parameters are learned from extended scene observations; together these yield an entropy for the scene. We support our argument with quantitative and qualitative analyses in simulated and real-world environments, demonstrating that this approach yields sensible exploration behaviours in which the camera alternates between obtaining close-up views of the targets while paying attention to the background, especially to areas of known high activity.Eric Sommerlade, Ian Rei

Gaze directed camera control for face image acquisition

Face recognition in surveillance situations usually requires high resolution face images to be captured from remote active cameras. Since the recognition accuracy is typically a function of the face direction with frontal faces more likely to lead to reliable recognition we propose a system which optimises the capturing of such images by using coarse gaze estimates from a static camera. By considering the potential information gain from observing each target, our system automatically sets the pan, tilt and zoom values (i.e. the field of view) of multiple cameras observing different tracked targets in order to maximise the likelihood of correct identification. The expected gain in information is influenced by the controllable field of view, and by the false positive and negative rates of the identification process, which are in turn a function of the gaze angle. We validate the approach using a combination of simulated situations and real tracking output to demonstrate superior performance over alternative approaches, notably using no gaze information, or using gaze inferred from direction of travel (i.e. assuming each person is always looking directly ahead).We also show results from a live implementation with a static camera and two pan-tilt-zoom devices, involving real-time tracking, processing and control.Eric Sommerlade, Ben Benfold and Ian Rei

Modelling pedestrian trajectory patterns with Gaussian processes

We propose a non-parametric model for pedestrian motion based on Gaussian Process regression, in which trajectory data are modelled by regressing relative motion against current position. We show how the underlying model can be learned in an unsupervised fashion, demonstrating this on two databases collected from static surveillance cameras. We furthermore exemplify the use of model for prediction, comparing the recently proposed GP-Bayesfilters with a Monte Carlo method. We illustrate the benefit of this approach for long term motion prediction where parametric models such as Kalman Filters would perform poorly.David Ellis, Eric Sommerlade and Ian Rei

Action recognition using shared motion parts

In this paper we analyse the advantages of a joint boosting method, previously known in the object recognition community, to detect and classify action keyframes. The method focuses on sharing object parts among action classes. Instead of sharing parts that only encode shape similarities, we propose to include motion information as an extra clue for sharing. We show that the inclusion of motion information significantly improves the recognition accuracy. The method is tested using a standard action database containing 10 action classes obtaining perfect classification. It also yields promising results on complicated videos including complex background.Alonso Patron-Perez, Eric Sommerlade and Ian Rei

Finding prototypes to estimate trajectory development in outdoor scenarios

The incorporation of qualitative information into tracking systems is a challenging topic of research in the computer vision area. Within the context of an outdoor scenario, it is usual to find some patterns of human motion and, in general there is a reduced set of typical entrance and exit points. In this work we present a method to analyze an existing set of trajectories obtained in a selected outdoor environment. Trajectories are clustered based on their entrance and exit points in order to model prototypes that are used to on–line estimate the development of a new trajectory. Those prototypes are modeled using spline curves to avoid the inaccuracy pulled from the vision system. Interesting applications comprise abnormal behavior detection, spatio–temporal event analysis and nd semantic interpretation of human behavior based on the context.Pau Baiget, Eric Sommerlade, Ian Reid, Jordi Gonzàle

Cognitive active vision for human identification

We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision. © 2012 IEEE

Understanding interactions and guiding visual surveillance by tracking attention

The central tenet of this paper is that by determining where people are looking, other tasks involved with understanding and interrogating a scene are simplified. To this end we describe a fully automatic method to determine a person’s attention based on real-time visual tracking of their head and a coarse classification of their head pose. We estimate the head pose, or coarse gaze, using randomised ferns with decision branches based on both histograms of gradient orientations and colour based features. We use the coarse gaze for three applications to demonstrate its value: (i) we show how by building static and temporally varying maps of areas where people look we are able to identify interesting regions; (ii) we show how by determining the gaze of people in the scene we can more effectively control a multi-camera surveillance system to acquire faces for identification; (iii) we show how by identifying where people are looking we can more effectively classify human interactions.Ian Reid, Ben Benfold, Alonso Patron, and Eric Sommerlad