A semantic-based probabilistic approach for real-time video event recognition

This is the author’s version of a work that was accepted for publication in Journal Computer Vision and Image Understanding. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal Computer Vision and Image Understanding, 116, 9 (2012) DOI: 10.1016/j.cviu.2012.04.005This paper presents an approach for real-time video event recognition that combines the accuracy and descriptive capabilities of, respectively, probabilistic and semantic approaches. Based on a state-of-art knowledge representation, we define a methodology for building recognition strategies from event descriptions that consider the uncertainty of the low-level analysis. Then, we efficiently organize such strategies for performing the recognition according to the temporal characteristics of events. In particular, we use Bayesian Networks and probabilistically-extended Petri Nets for recognizing, respectively, simple and complex events. For demonstrating the proposed approach, a framework has been implemented for recognizing human-object interactions in the video monitoring domain. The experimental results show that our approach improves the event recognition performance as compared to the widely used deterministic approach.This work has been partially supported by the Spanish Administration agency CDTI (CENIT-VISION 2007- 1007), by the Spanish Government (TEC2011-25995 EventVideo), by the Consejería de Educación of the Comunidad de Madrid and by The European Social Fund

Математичні методи розпізнавання надзвичайних ситуацій в умовах невизначеності

It has been shown that emergency events recognition systems exhibit various types of uncertainty: incomplete data streams, data stream errors, and inappropriate patterns of complex events. There were presented an overview of existing approaches for complex event recognition under uncertainty. It was noticed that the field of complex event recognition under uncertainty is relatively new and proposed to adopt methods of targeting activity recognition. It was shown that the streams of time-stamped derived events arriving at a complex event recognition system carry a certain degree of uncertainty and ambiguity. Information sources have to be heterogeneous, with data of different structures schemas and procedures of respond to corrupted data. Even for perfectly accurate sensors, the domain might be difficult to model precisely, thereby leading to another type of uncertainty. Thus, it is noted that it is important to consider methods for recognizing complex events that can be classified as uncertain, for this purpose, appropriate model objects were proposed. The analysis of key moments in the construction of complex recognition systems that are capable of working effectively under uncertainty included stochastic modeling, time representation models, and relational models. There were considered techniques based on automata, probabilistic graphical models, first-order logic, Petri Networks and Hidden Petri Networks. It is specified that the intermediate stage of the work of the corresponding algorithms should be the creation of a hierarchy of complex objects that are not always clearly defined. A number of limitations have been found regarding the syntax, models and performance used, which were compared with the specific variants of their implementation. An approach was proposed for the transition from a deterministic mathematical apparatus to a system of recognition of complex events under uncertainty conditions, through the introduction of the probability function of an event. The developed methodology allowed highlighting directions for investigation and estimating efficiency of the mathematical methods to be used.Показано, що системи розпізнавання надзвичайних подій виявляють різні типи невизначеності: неповні потоки даних, помилки в потоках даних і невідповідні шаблони складних подій. Показано, що потоки подій, що потрапляють на вхід системи розпізнавання складних подій, характеризуються певним ступенем невизначеності. Джерела даних є неоднорідними і характеризуються різною структуризацією даних і відповідними процедурами реагування на пошкоджені блоки даних. Навіть для даних, визначених достатньо точно, система може некоректно моделювати складні події, що призводить до подальшого типу невизначеності. Отже, зазначено, що важливо розглянути методи розпізнавання складних подій, які можна віднести до невизначених. З цією метою було запропоновано відповідні модельні об'єкти. Проведений аналіз ключових моментів побудови систем розпізнавання складних подій, які здатні ефективно працювати в умовах невизначеності, охоплював          методи стохастичного моделювання, моделі часового представлення та реляційні моделі. Розглянуто методики, що базуються на абстрактних автоматах, імовірнісних моделях графів, системах логіки першого порядку, мережах Петрі та прихованих мережах Петрі. Зазначено, що проміжним етапом роботи відповідних алгоритмів має бути створення ієрархії складних об'єктів, що не завжди піддаються чіткому визначенню. Виявлено низку обмежень щодо використовуваного синтаксису, моделей і ефективності, які були зіставлені з конкретними варіантами їх реалізації. Запропоновано підхід щодо переходу від детерміністичного математичного апарату до системи розпізнавання складних подій в умовах невизначеності, через введення функції вірогідності події. Розроблена методологія дала змогу виділити напрями досліджень і оцінити продуктивність використовуваних математичних методів

A Methodology for Extracting Human Bodies from Still Images

Monitoring and surveillance of humans is one of the most prominent applications of today and it is expected to be part of many future aspects of our life, for safety reasons, assisted living and many others. Many efforts have been made towards automatic and robust solutions, but the general problem is very challenging and remains still open. In this PhD dissertation we examine the problem from many perspectives. First, we study the performance of a hardware architecture designed for large-scale surveillance systems. Then, we focus on the general problem of human activity recognition, present an extensive survey of methodologies that deal with this subject and propose a maturity metric to evaluate them. One of the numerous and most popular algorithms for image processing found in the field is image segmentation and we propose a blind metric to evaluate their results regarding the activity at local regions. Finally, we propose a fully automatic system for segmenting and extracting human bodies from challenging single images, which is the main contribution of the dissertation. Our methodology is a novel bottom-up approach relying mostly on anthropometric constraints and is facilitated by our research in the fields of face, skin and hands detection. Experimental results and comparison with state-of-the-art methodologies demonstrate the success of our approach

Measurements, Models, Systems and Design

531 s.

The PSEIKI Report—Version 2. Evidence Accumulation and Flow of Control in a Hierarchical Spatial Reasoning System

A fundamental goal of computer vision is the development of systems capable of carrying out scene interpretation while taking into account all the available knowledge. In this report, we have focused on how the interpretation task may be aided by expected-scene information which, in most cases, would not be in registration with the perceived scene. In this report, we describe PSEIKI, a framework for expectation-driven interpretation of image data. PSEIKI builds abstraction hierarchies in image data using, for cues, supplied abstraction hierarchies in a scene expectation map. Hypothesized abstractions in the image data are geometrically compared with the known abstractions in the expected scene; the metrics used for these comparisons translate into belief values. The Dempster-Shafer formalism is used to accumulate beliefs for the synthesized abstractions in the image data. For accumulating belief values, a computationally efficient variation of Dempster’s rule of combination is developed to enable the system to deal with the overwhelming amount of information present in most images. This variation of Dempster’s rule allows the reasoning process to be embedded into the abstraction hierarchy by allowing for the propagation of belief values between elements at different levels of abstraction. The system has been implemented as a 2- panel, 5-level blackboard in OPS 83. This report also discusses the control aspects of the blackboard, achieved via a distributed monitor using the OPS83 demons and a scheduler. Various knowledge sources for forming groupings in the image data and for labeling such groupings with abstractions from the scene expectation map are also discussed

Activity Recognition and Uncertain Knowledge in Video Scenes

International audienceActivity recognition has been a growing research topic in the last years and its application varies from auto-matic recognition of social interaction such as shaking hands, parking lot surveillance, traffic monitoring and the detection of abandoned luggage. This paper describes a probabilistic framework for uncertainty handling in a description-based event recognition approach. The proposed approach allows the flexible modeling of composite events with complex temporal constraints. It uses probability theory to provide a consistent framework for dealing with uncertain knowledge for the recognition of complex events. We validate the event recognition accuracy of the proposed algorithm on real-world videos. The experimental results show that our system can successfully recognize activities with a high recognition rate. We conclude by comparing our algorithm with the state of the art and showing how the definition of event models and the probabilistic reasoning can influence the results of real-time event recognitio