Program supervision techniques for easy configuration of video understanding systems

Abstract

The context of this thesis is the semantic understanding of video sequences. Semantic video understanding is here defined as the task of recognition of predefined event models in a given application domain (e.g., human activities) starting from a pixel analysis up to a symbolic description of what is happening in the scene viewed by cameras. Although various approaches and techniques have already been proposed for understanding video sequences, this task remains a complex problem regarding the lack of performances when facing challenging environments (e.g., a cluttered scene) and the lack of reusability when a system has to be deployed on a large scale with minimum human interventions. In this context, this thesis proposes a complete framework to conceive supervised video understanding platforms. Such platforms are used to easily create and configure video understanding systems. Several issues need to be addressed to provide a correct configuration: (1) the ability to choose, among a library of programs, those which are best satisfying a given request, (2) the ability to dynamically adapt programs and parameters to situation changes, (3) the ability to evaluate performances and to guarantee continuously a maximum performance rate which is satisfactory regarding end-user requirements. We propose a knowledge-based approach for the supervision of video processing programs in order to externalize both the control and the knowledge of programs. We propose a model for program control which is generic in the sense that it is independent of any application. The need of having a formalism for knowledge representation is demonstrated for each type of knowledge: knowledge of the application domain, knowledge of the scene environment and knowledge of video processing programs. In addition, the framework provides a methodology for the evaluation of system performances. This methodology proposes a video sequence characterization which guides the selection of video sequences which are used for testing a system. In order to perform the evaluation, the methodology also proposes a set of metrics to compare results with reference data. Thanks to this evaluation methodology, video processing experts are able to acquire expertise on the use of programs. Finally, the framework allows the use of learning techniques where knowledge is hardly available. The proposed framework has been validated on a video understanding platform. Thanks to this supervised platform, we have created six systems in a few amount of time. These systems are characterized by four properties: adaptability, reusability, effectiveness and real-time processing.Doctorat en sciences appliquées (FSA 3)--UCL, 200