Video spatio-temporal filtering based on cameras and target objects trajectories - Videosurveillance forensic framework

International audienceThis paper presents our work about assisting video-surveillance agents in the search for particular video scenes of interest in transit network. This work has been developed based on requirements defined within different projects with the French National Police in a forensic goal. The video-surveillance agent inputs a query in the form of a hybrid trajectory (date, time, locations expressed with regards to different reference systems) and potentially some visual descriptions of the scene. The query processing starts with the interpretation of the hybrid trajectory and continues with a selection of a set of cameras likely to have filmed the spatial trajectory. The main contributions of this paper are: (1) a definition of the hybrid trajectory query concept, trajectory that is constituted of geometrical and symbolic segments represented with regards to different reference systems (e.g., Geodesic system, road network), (2) a spatio-temporal filtering framework based on a spatio-temporal modeling of the transit network and associated cameras

Interrogation des données spatio-temporelles de géolocalisation indoor à partir des trajectoires hybrides

Le GPS (Global Positioning System) basé sur la visibilité directe entre le satellite et le récepteur, s’est imposé pour la localisation outdoor. En l’absence de signal GPS, la localisation d’une cible dans les environnements indoor peut se faire avec un réseau de capteurs. Ces réseaux déployés dans les bâtiments sont de plus en plus nombreux et servent à plusieurs applications basées sur la localisation : surveillance, détection, navigation, etc. Ces capteurs de localisation indoor génèrent une quantité importante d’informations de tracking. Le contexte d’application de ce travail est celui des systèmes de localisation indoor basés sur les cellules Wifi, sur les capteurs ICCARD et sur les caméras de vidéosurveillance. Dans un tel contexte, il n’existe aucun système global de référence similaire au GPS : les informations de localisation sont hétérogènes (positions géométriques et symboliques / multitude de systèmes de référence). Cet article présente un ensemble de démarches qui ont permis de concevoir et implémenter un framework utilisant les informations générées par les réseaux de capteurs de localisation déployés dans un environnement indoor, illustré dans le cadre du forensic

Automatic object classification for surveillance videos.

PhDThe recent popularity of surveillance video systems, specially located in urban scenarios, demands the development of visual techniques for monitoring purposes. A primary step towards intelligent surveillance video systems consists on automatic object classification, which still remains an open research problem and the keystone for the development of more specific applications. Typically, object representation is based on the inherent visual features. However, psychological studies have demonstrated that human beings can routinely categorise objects according to their behaviour. The existing gap in the understanding between the features automatically extracted by a computer, such as appearance-based features, and the concepts unconsciously perceived by human beings but unattainable for machines, or the behaviour features, is most commonly known as semantic gap. Consequently, this thesis proposes to narrow the semantic gap and bring together machine and human understanding towards object classification. Thus, a Surveillance Media Management is proposed to automatically detect and classify objects by analysing the physical properties inherent in their appearance (machine understanding) and the behaviour patterns which require a higher level of understanding (human understanding). Finally, a probabilistic multimodal fusion algorithm bridges the gap performing an automatic classification considering both machine and human understanding. The performance of the proposed Surveillance Media Management framework has been thoroughly evaluated on outdoor surveillance datasets. The experiments conducted demonstrated that the combination of machine and human understanding substantially enhanced the object classification performance. Finally, the inclusion of human reasoning and understanding provides the essential information to bridge the semantic gap towards smart surveillance video systems

SYMMETRY IN HUMAN MOTION ANALYSIS: THEORY AND EXPERIMENTS

Video based human motion analysis has been actively studied over the past decades. We propose novel approaches that are able to analyze human motion under such challenges and apply them to surveillance and security applications. Part I analyses the cyclic property of human motion and presents algorithms to classify humans in videos by their gait patterns. Two approaches are proposed. The first employs the omputationally efficient periodogram, to characterize periodicity. In order to integrate shape and motion, we convert the cyclic pattern into a binary sequence using the angle between two legs when the toe-to-toe distance is maximized during walking. Part II further extends the previous approaches to analyze the symmetry in articulation within a stride. A feature that has been shown in our work to be a particularly strong indicator of the presence of pedestrians is the X-junction generated by bipedal swing of body limbs. The proposed algorithm extracts the patterns in spatio-temporal surfaces. In Part III, we present a compact characterization of human gait and activities. Our approach is based on decomposing an image sequence into x-t slices, which generate twisted patterns defined as the Double Helical Signature (DHS). It is shown that the patterns sufficiently characterize human gait and a class of activities. The features of DHS are: (1) it naturally codes appearance and kinematic parameters of human motion; (2) it reveals an inherent geometric symmetry (Frieze Group); and (3) it is effective and efficient for recovering gait and activity parameters. Finally, we use the DHS to classify activities such as carrying a backpack, briefcase etc. The advantage of using DHS is that we only need a small portion of 3D data to recognize various symmetries

Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics

This paper proposes a method to enhance video object detection for indoor environments in robotics. Concretely, it exploits knowledge about the camera motion between frames to propagate previously detected objects to successive frames. The proposal is rooted in the concepts of planar homography to propose regions of interest where to find objects, and recursive Bayesian filtering to integrate observations over time. The proposal is evaluated on six virtual, indoor environments, accounting for the detection of nine object classes over a total of ∼ 7k frames. Results show that our proposal improves the recall and the F1-score by a factor of 1.41 and 1.27, respectively, as well as it achieves a significant reduction of the object categorization entropy (58.8%) when compared to a two-stage video object detection method used as baseline, at the cost of small time overheads (120 ms) and precision loss (0.92).</p

Applications of Internet of Things

This book introduces the Special Issue entitled “Applications of Internet of Things”, of ISPRS International Journal of Geo-Information. Topics covered in this issue include three main parts: (I) intelligent transportation systems (ITSs), (II) location-based services (LBSs), and (III) sensing techniques and applications. Three papers on ITSs are as follows: (1) “Vehicle positioning and speed estimation based on cellular network signals for urban roads,” by Lai and Kuo; (2) “A method for traffic congestion clustering judgment based on grey relational analysis,” by Zhang et al.; and (3) “Smartphone-based pedestrian’s avoidance behavior recognition towards opportunistic road anomaly detection,” by Ishikawa and Fujinami. Three papers on LBSs are as follows: (1) “A high-efficiency method of mobile positioning based on commercial vehicle operation data,” by Chen et al.; (2) “Efficient location privacy-preserving k-anonymity method based on the credible chain,” by Wang et al.; and (3) “Proximity-based asynchronous messaging platform for location-based Internet of things service,” by Gon Jo et al. Two papers on sensing techniques and applications are as follows: (1) “Detection of electronic anklet wearers’ groupings throughout telematics monitoring,” by Machado et al.; and (2) “Camera coverage estimation based on multistage grid subdivision,” by Wang et al

Evaluation and analysis of hybrid intelligent pattern recognition techniques for speaker identification

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The rapid momentum of the technology progress in the recent years has led to a tremendous rise in the use of biometric authentication systems. The objective of this research is to investigate the problem of identifying a speaker from its voice regardless of the content (i.e. text-independent), and to design efficient methods of combining face and voice in producing a robust authentication system. A novel approach towards speaker identification is developed using wavelet analysis, and multiple neural networks including Probabilistic Neural Network (PNN), General Regressive Neural Network (GRNN)and Radial Basis Function-Neural Network (RBF NN) with the AND voting scheme. This approach is tested on GRID and VidTIMIT cor-pora and comprehensive test results have been validated with state- of-the-art approaches. The system was found to be competitive and it improved the recognition rate by 15% as compared to the classical Mel-frequency Cepstral Coe±cients (MFCC), and reduced the recognition time by 40% compared to Back Propagation Neural Network (BPNN), Gaussian Mixture Models (GMM) and Principal Component Analysis (PCA). Another novel approach using vowel formant analysis is implemented using Linear Discriminant Analysis (LDA). Vowel formant based speaker identification is best suitable for real-time implementation and requires only a few bytes of information to be stored for each speaker, making it both storage and time efficient. Tested on GRID and Vid-TIMIT, the proposed scheme was found to be 85.05% accurate when Linear Predictive Coding (LPC) is used to extract the vowel formants, which is much higher than the accuracy of BPNN and GMM. Since the proposed scheme does not require any training time other than creating a small database of vowel formants, it is faster as well. Furthermore, an increasing number of speakers makes it di±cult for BPNN and GMM to sustain their accuracy, but the proposed score-based methodology stays almost linear. Finally, a novel audio-visual fusion based identification system is implemented using GMM and MFCC for speaker identi¯cation and PCA for face recognition. The results of speaker identification and face recognition are fused at different levels, namely the feature, score and decision levels. Both the score-level and decision-level (with OR voting) fusions were shown to outperform the feature-level fusion in terms of accuracy and error resilience. The result is in line with the distinct nature of the two modalities which lose themselves when combined at the feature-level. The GRID and VidTIMIT test results validate that the proposed scheme is one of the best candidates for the fusion of face and voice due to its low computational time and high recognition accuracy

Gaze-Based Human-Robot Interaction by the Brunswick Model

We present a new paradigm for human-robot interaction based on social signal processing, and in particular on the Brunswick model. Originally, the Brunswick model copes with face-to-face dyadic interaction, assuming that the interactants are communicating through a continuous exchange of non verbal social signals, in addition to the spoken messages. Social signals have to be interpreted, thanks to a proper recognition phase that considers visual and audio information. The Brunswick model allows to quantitatively evaluate the quality of the interaction using statistical tools which measure how effective is the recognition phase. In this paper we cast this theory when one of the interactants is a robot; in this case, the recognition phase performed by the robot and the human have to be revised w.r.t. the original model. The model is applied to Berrick, a recent open-source low-cost robotic head platform, where the gazing is the social signal to be considered

Advances in Image Processing, Analysis and Recognition Technology

For many decades, researchers have been trying to make computers’ analysis of images as effective as the system of human vision is. For this purpose, many algorithms and systems have previously been created. The whole process covers various stages, including image processing, representation and recognition. The results of this work can be applied to many computer-assisted areas of everyday life. They improve particular activities and provide handy tools, which are sometimes only for entertainment, but quite often, they significantly increase our safety. In fact, the practical implementation of image processing algorithms is particularly wide. Moreover, the rapid growth of computational complexity and computer efficiency has allowed for the development of more sophisticated and effective algorithms and tools. Although significant progress has been made so far, many issues still remain, resulting in the need for the development of novel approaches