Learning from Multiple Sources for Video Summarisation

Many visual surveillance tasks, e.g.video summarisation, is conventionally accomplished through analysing imagerybased features. Relying solely on visual cues for public surveillance video understanding is unreliable, since visual observations obtained from public space CCTV video data are often not sufficiently trustworthy and events of interest can be subtle. On the other hand, non-visual data sources such as weather reports and traffic sensory signals are readily accessible but are not explored jointly to complement visual data for video content analysis and summarisation. In this paper, we present a novel unsupervised framework to learn jointly from both visual and independently-drawn non-visual data sources for discovering meaningful latent structure of surveillance video data. In particular, we investigate ways to cope with discrepant dimension and representation whist associating these heterogeneous data sources, and derive effective mechanism to tolerate with missing and incomplete data from different sources. We show that the proposed multi-source learning framework not only achieves better video content clustering than state-of-the-art methods, but also is capable of accurately inferring missing non-visual semantics from previously unseen videos. In addition, a comprehensive user study is conducted to validate the quality of video summarisation generated using the proposed multi-source model

On-the-fly feature importance mining for person re-identification

State-of-the-art person re-identification methods seek robust person matching through combining various feature types. Often, these features are implicitly assigned with generic weights, which are assumed to be universally and equally good for all individuals, independent of people's different appearances. In this study, we show that certain features play more important role than others under different viewing conditions. To explore this characteristic, we propose a novel unsupervised approach to bottom-up feature importance mining on-the-fly specific to each re-identification probe target image, so features extracted from different individuals are weighted adaptively driven by their salient and inherent appearance attributes. Extensive experiments on three public datasets give insights on how feature importance can vary depending on both the viewing condition and specific person's appearance, and demonstrate that unsupervised bottom-up feature importance mining specific to each probe image can facilitate more accurate re-identification especially when it is combined with generic universal weights obtained using existing distance metric learning methods. © 2013 Elsevier Ltd

Constrained Clustering: Effective Constraint Propagation with Imperfect Oracles

Abstract—While spectral clustering is usually an unsuper-vised operation, there are circumstances in which we have prior belief that pairs of samples should (or should not) be assigned with the same cluster. Constrained spectral clustering aims to exploit this prior belief as constraint (or weak supervision) to influence the cluster formation so as to obtain a structure more closely resembling human perception. Two important issues re-main open: (1) how to propagate sparse constraints effectively, (2) how to handle ill-conditioned/noisy constraints generated by imperfect oracles. In this paper we present a unified framework to address the above issues. Specifically, in contrast to existing constrained spectral clustering approaches that blindly rely on all features for constructing the spectral, our approach searches for neighbours driven by discriminative feature selection for more effective constraint diffusion. Crucially, we formulate a novel data-driven filtering approach to handle the noisy constraint problem, which has been unrealistically ignored in constrained spectral clustering literature. Keywords-Constrained clustering, constraint propagation, feature selection, imperfect oracles, spectral clustering. I

Sketch Me That Shoe

This project received support from the European Union’s Horizon 2020 research and innovation programme under grant agreement #640891, the Royal Society and Natural Science Foundation of China (NSFC) joint grant #IE141387 and #61511130081, and the China Scholarship Council (CSC). We gratefully acknowledge the support of NVIDIA Corporation for the donation of the GPUs used for this research

Cumulative Attribute Space for Age and Crowd Density Estimation

A number of computer vision problems such as human age estimation, crowd density estimation and body/face pose (view angle) estimation can be formulated as a regression problem by learning a mapping function between a high dimensional vector-formed feature input and a scalarvalued output. Such a learning problem is made difficult due to sparse and imbalanced training data and large feature variations caused by both uncertain viewing conditions and intrinsic ambiguities between observable visual features and the scalar values to be estimated. Encouraged by the recent success in using attributes for solving classification problems with sparse training data, this paper introduces a novel cumulative attribute concept for learning a regression model when only sparse and imbalanced data are available. More precisely, low-level visual features extracted from sparse and imbalanced image samples are mapped onto a cumulative attribute space where each dimension has clearly defined semantic interpretation (a label) that captures how the scalar output value (e.g. age, people count) changes continuously and cumulatively. Extensive experiments show that our cumulative attribute framework gains notable advantage on accuracy for both age estimation and crowd counting when compared against conventional regression models, especially when the labelled training data is sparse with imbalanced sampling. 1

Person re-identification with soft biometrics through deep learning

Re-identification of persons is usually based on primary biometric features such as their faces, fingerprints, iris or gait. However, in most existing video surveillance systems, it is difficult to obtain these features due to the low resolution of surveillance footages and unconstrained real-world environments. As a result, most of the existing person re-identification techniques only focus on overall visual appearance. Recently, the use of soft biometrics has been proposed to improve the performance of person re-identification. Soft biometrics such as height, gender, age are physical or behavioural features, which can be described by humans. These features can be obtained from low-resolution videos at a distance ideal for person re-identification application. In addition, soft biometrics are traits for describing an individual with human-understandable labels. It allows human verbal descriptions to be used in the person re-identification or person retrieval systems. In some deep learning based person re-identification methods, soft biometrics attributes are integrated into the network to boot the robustness of the feature representation. Biometrics can also be utilised as a domain adaptation bridge for addressing the cross-dataset person re-identification problem. This chapter will review the state-of-the-art deep learning methods involving soft biometrics from three perspectives: supervised, semi-supervised and unsupervised approaches. In the end, we discuss the existing issues that are not addressed by current works

A Single Basis for Developmental Buffering of Drosophila Wing Shape

The nature of developmental buffering processes has been debated extensively, based on both theoretical reasoning and empirical studies. In particular, controversy has focused on the question of whether distinct processes are responsible for canalization, the buffering against environmental or genetic variation, and for developmental stability, the buffering against random variation intrinsic in developmental processes. Here, we address this question for the size and shape of Drosophila melanogaster wings in an experimental design with extensively replicated and fully controlled genotypes. The amounts of variation among individuals and of fluctuating asymmetry differ markedly among genotypes, demonstrating a clear genetic basis for size and shape variability. For wing shape, there is a high correlation between the amounts of variation among individuals and fluctuating asymmetry, which indicates a correspondence between the two types of buffering. Likewise, the multivariate patterns of shape variation among individuals and of fluctuating asymmetry show a close association. For wing size, however, the amounts of individual variation and fluctuating asymmetry are not correlated. There was a significant link between the amounts of variation between wing size and shape, more so for fluctuating asymmetry than for variation among individuals. Overall, these experiments indicate a considerable degree of shared control of individual variation and fluctuating asymmetry, although it appears to differ between traits