Surveillance video retrieval: what we have already done?

International audienceWhile many overview papers have been published for information retrieval in general and image retrieval in particular, there is a lack of paper in the literature focusing on retrieval for surveillance video. The aim of this paper is to provide an analysis on what we have ready done for surveillance video retrieval and therefore to point out what are still challenges in this domain. By supposing that there are two main types of information in surveillance video named object and event, we divide the existing approaches in the literature into two sub categories: approaches at object level and approaches at both object and event levels. A quantitative comparison of three approaches of the former category in the same dataset is also given

Surveillance Video Indexing and Retrieval using Object Features and semantic Events

International audienceIn this paper, we propose an approach for surveillance video indexing and retrieval. The objective of this approach is to answer five main challenges we have met in this domain: (1) the lack of means for finding data from the indexed databases, (2) the lack of approaches working at different abstraction levels, (3) imprecise indexing, (4) incomplete indexing, (5) the lack of user-centered search. We propose a new data model containing two main types of extracted video contents: physical objects and events. Based on this data model we present a new rich and flexible query language. This language works at different abstraction levels, provides both exact and approximate matching and takes into account users' interest. In order to work with the imprecise indexing, two new methods respectively for object representation and object matching are proposed. Videos from two projects which have been partially indexed are used to validate the proposed approach. We have analyzed both query language usage and retrieval results. The obtained retrieval results are analyzed by the average normalized ranks are promising. The retrieval results at the object level are compared with another state of the art approach

Automatic detection and indexing of video-event shots for surveillance applications

Increased communication capabilities and automatic scene understanding allow human operators to simultaneously monitor multiple environments. Due to the amount of data to be processed in new surveillance systems, the human operator must be helped by automatic processing tools in the work of inspecting video sequences. In this paper, a novel approach allowing layered content-based retrieval of video-event shots referring to potentially interesting situations is presented. Interpretation of events is used for defining new video-event shot detection and indexing criteria. Interesting events refer to potentially dangerous situations: abandoned objects and predefined human events are considered in this paper. Video-event shot detection and indexing capabilities are used for online and offline content-based retrieval of scenes to be detected

Automatic detection and indexing of video-event shots for surveillance applications

Increased communication capabilities and automatic scene understanding allow human operators to simultaneously monitor multiple environments. Due to the amount of data to be processed in new surveillance systems, the human operator must be helped by automatic processing tools in the work of inspecting video sequences. In this paper, a novel approach allowing layered content-based retrieval of video-event shots referring to potentially interesting situations is presented. Interpretation of events is used for defining new video-event shot detection and indexing criteria. Interesting events refer to potentially dangerous situations: abandoned objects and predefined human events are considered in this paper. Video-event shot detection and indexing capabilities are used for online and offline content-based retrieval of scenes to be detected

CAMBADA@Home: deteção e seguimento de humanos

Mestrado em Engenharia Electrónica e TelecomunicaçõesEste trabalho apresenta uma abordagem ao problema da deteção e seguimento de humanos, usando uma câmara RGB-D. Existem soluções propostas para este tipo de problema, no entanto, algumas são baseadas em técnicas de extração de fundo ou outras e, como tal, necessitam que a câmara se encontre numa posição estacionária. Com o sistema proposto, a deteção e seguimento podem ser desempenhadas enquanto a câmara se move, em tempo real. O objetivo deste projeto é a implementação de um sistema de deteção e seguimento de pessoas para o robô de serviço CAMBADA@Home, permitindo assim o desenvolvimento de futuras aplicações na área da interação humano-robô. O sistema aqui descrito permite realizar deteção, classificação e monitorização de múltiplas pessoas. Na primeira etapa, regiões de interesse (ROIs) são segmentadas através da análise do histograma da imagem de profundidade seguido da utilização de um algoritmo de preenchimento. Na etapa seguinte, cada região é classificada como humana ou não-humana através de uma técnica de correspondência de modelos, baseada no algoritmo de descida de gradiantes RPROP, com suporte para múltiplos modelos. A terceira e última etapa permite a monitorização de várias pessoas, através de um método de atribuição de identificadores únicos baseado em comparação de histogramas, assim como estimação de pose e localização. Os resultados obtidos em ambiente não controlado são encorajadores, com altas taxas de deteção, e, em geral, os algoritmos de estimação de pose e localização são executados como esperado. Para além disto, o projeto CAMBADA@Home foi premiado com o primeiro lugar no Desafio Free Bots, que teve lugar durante o campeonato nacional de robótica, Robótica 2013, onde o robô provou ser capaz de executar rondas autónomas num ambiente desconhecido enquanto detetava e monitorizava pessoas com as quais se cruzava.This work presents an approach to the people detection and tracking problem, using an RGB-D camera. While there are already solutions for this problem, some are based on background extraction techniques or other, which require the camera to be in a stationary position. With the proposed method, detection and tracking can be performed while the camera is moving, in real time. The aim of this project is the implementation of a people detection and tracking system for the CAMBADA@Home service robot, enabling the development of further human-robot interaction applications. The system here described enables object detection, classi cation and multiple person tracking. In the rst stage, regions of interest (ROIs) are segmented through the analysis of the depth image histogram and using a ood ll algorithm. On the next stage, each region is classi ed as human or not-human using a template matching technique, based on the RPROP gradient descent algorithm, with support for multiple templates. The third and last stage enables the tracking for multiple persons, using a unique identi cation assignment method based on histogram comparison, as well as pose and location estimation. The results obtained in unconstrained environments are encouraging, with high detection rates, and, in general, the algorithms for pose and location estimation perform as expected. Furthermore the CAMBADA@Home project has been awarded with the rst place in the Free Bots Challenge, which took place on the Rob otica 2013 robotics national championship, where the robot was proven to be capable of performing autonomous tours in an unknown environment while at the same time detecting and tracking people it came across

Comparaison des documents audiovisuels<br />par Matrice de Similarité

The work of this thesis relates to the comparison of video documents. The field of digital video is in full expansion. Videos are now present in large quantity even for personal use. The video comparison is a basic analysis operation in complement of classification, extraction and structuring of videos.Traditional approaches of comparison are primarily based on the low-level features of the videos to be compared, considered as multidimensional vectors. Other approaches are based on the similarity of frames without taking into account neither the temporal composition of the video nor the audiolayer. The main disadvantage of these methods is that they reduce the comparison role to a simple operator robust to noise effects. Such operators are generally used in order to identify the various specimens of a same document.The originality of our approach lies in the introduction of the of style similarity notion, taking as a starting point the human criteria into the comparison. These criteria are more flexible, and do not impose a strict similarity of all the studied features at the same time.We define an algorithm of extraction of the similarities between the series of values produced bythe analysis of the audiovisual low-level features. The algorithm is inspired by the dynamic programmingand the time series comparison methods.We propose a representation of the data resulting from these processings in the form of a matrixpattern suitable for the visual and immediate comparison of two videos. This matrix is then used topropose a generic similarity measure. The measure is applicable independently to videos of comparableor heterogeneous contents.We developed several applications to demonstrate the behavior of the comparison method and thesimilarity measure. The experiments concern primarily: - the identification of the structure in acollection/sub-collection of documents, - the description of stylistics elements in a movie, and - theanalysis of the grid of programs from a TV stream.Les travaux de cette thèse concernent la comparaison des documents vidéo. Dans le domaine en pleine expansion de la vidéo numérique, les documents disponibles sont maintenant présents en quantité importante même dans les foyers. Opération de base de tout type d'analyse de contenus, en complément de la classification, de l'extraction et de la structuration, la comparaison dans le domaine de l'audiovisuel est d'une utilité qui n'est pas à démontrer.Des approches classiques de comparaison se basent essentiellement sur l'ensemble des caractéristiquesbas niveaux des documents à comparer, en les considérant comme des vecteurs multidimensionnels. D'autres approches se basent sur la similarité des images composant la vidéo sans tenir compte de la composition temporelle du document ni de la bandeson. Le défaut que l'on peut reprocher à ces méthodes est qu'elles restreignent la comparaison à un simple opérateur binaire robuste au bruit. De tels opérateurs sont généralement utilisés afin d'identifier les différents exemplaires d'un même document. L'originalité de notre démarche réside dans le fait que nous introduisons la notion de la similarité de styleen s'inspirant des critères humains dans la comparaison des documents vidéo. Ces critèressont plus souples, et n'imposent pas une similarité stricte de toutes les caractéristiques étudiéesà la fois.En nous inspirant de la programmation dynamique et de la comparaison des séries chronologiques, nous définissons un algorithme d'extraction des similarités entre les séries de valeurs produites par l'analyse de caractéristiques audiovisuelles de bas-niveau. Ensuite, un second traitement générique approxime le résultat de l'algorithme de la longueur de la PlusLongue Sous-Séquence Commune (PLSC) plus rapidement que ce dernier. Nous proposons une représentation des données issues de ces traitements sous la forme d'un schéma matriciel propre à la comparaison visuelle et immédiate de deux contenus. Cette matrice peut être également utilisée pour définir une mesure de similarité générique, applicable à des documents de même genre ou de genres hétérogènes.Plusieurs applications ont été mises en place pour démontrer le comportement de la méthode de comparaison et de la mesure de similarité, ainsi que leur pertinence. Les expérimentations concernent essentiellement : - l'identification d'une structure organisationnelle en collection / sous-collection d'une base de documents, - la mise en évidence d'élémentsstylistiques dans un film de cinéma, - la mise en évidence de la grille de programmes d'unflux de télévision

An effective trajectory-based algorithm for ball detection and tracking with application to the analysis of broadcast sports video

Ph.DDOCTOR OF PHILOSOPH

Unusual event detection in real-world surveillance applications

Given the near-ubiquity of CCTV, there is significant ongoing research effort to apply image and video analysis methods together with machine learning techniques towards autonomous analysis of such data sources. However, traditional approaches to scene understanding remain dependent on training based on human annotations that need to be provided for every camera sensor. In this thesis, we propose an unusual event detection and classification approach which is applicable to real-world visual monitoring applications. The goal is to infer the usual behaviours in the scene and to judge the normality of the scene on the basis on the model created. The first requirement for the system is that it should not demand annotated data to train the system. Annotation of the data is a laborious task, and it is not feasible in practice to annotate video data for each camera as an initial stage of event detection. Furthermore, even obtaining training examples for the unusual event class is challenging due to the rarity of such events in video data. Another requirement for the system is online generation of results. In surveillance applications, it is essential to generate real-time results to allow a swift response by a security operator to prevent harmful consequences of unusual and antisocial events. The online learning capabilities also mean that the model can be continuously updated to accommodate natural changes in the environment. The third requirement for the system is the ability to run the process indefinitely. The mentioned requirements are necessary for real-world surveillance applications and the approaches that conform to these requirements need to be investigated. This thesis investigates unusual event detection methods that conform with real-world requirements and investigates the issue through theoretical and experimental study of machine learning and computer vision algorithms