Human activity recognition for the use in intelligent spaces

The aim of this Graduation Project is to develop a generic biological inspired activity recognition system for the use in intelligent spaces. Intelligent spaces form the context for this project. The goal is to develop a working prototype that can learn and recognize human activities from a limited training set in all kinds of spaces and situations. For testing purposes, the office environment is chosen as subject for the intelligent space. The purpose of the intelligent space, in this case the office, is left out of the scope of the project. The scope is limited to the perceptive system of the intelligent space. The notion is that the prototype should not be bound to a specific space, but it should be a generic perceptive system able to cope in any given space within the build environment. The fact that no space is the same, developing a prototype without any domain knowledge in which it can learn and recognize activities, is the main challenge of this project. In al layers of the prototype, the data processing is kept as abstract and low level as possible to keep it as generic as possible. This is done by using local features, scale invariant descriptors and by using hidden Markov models for pattern recognition. The novel approach of the prototype is that it combines structure as well as motion features in one system making it able to train and recognize a variety of activities in a variety of situations. From rhythmic expressive actions with a simple cyclic pattern to activities where the movement is subtle and complex like typing and reading, can all be trained and recognized. The prototype has been tested on two very different data sets. The first set in which the videos are shot in a controlled environment in which simple actions were performed. The second set in which videos are shot in a normal office where daily office activities are captured and categorized afterwards. The prototype has given some promising results proving it can cope with very different spaces, actions and activities. The aim of this Graduation Project is to develop a generic biological inspired activity recognition system for the use in intelligent spaces. Intelligent spaces form the context for this project. The goal is to develop a working prototype that can learn and recognize human activities from a limited training set in all kinds of spaces and situations. For testing purposes, the office environment is chosen as subject for the intelligent space. The purpose of the intelligent space, in this case the office, is left out of the scope of the project. The scope is limited to the perceptive system of the intelligent space. The notion is that the prototype should not be bound to a specific space, but it should be a generic perceptive system able to cope in any given space within the build environment. The fact that no space is the same, developing a prototype without any domain knowledge in which it can learn and recognize activities, is the main challenge of this project. In al layers of the prototype, the data processing is kept as abstract and low level as possible to keep it as generic as possible. This is done by using local features, scale invariant descriptors and by using hidden Markov models for pattern recognition. The novel approach of the prototype is that it combines structure as well as motion features in one system making it able to train and recognize a variety of activities in a variety of situations. From rhythmic expressive actions with a simple cyclic pattern to activities where the movement is subtle and complex like typing and reading, can all be trained and recognized. The prototype has been tested on two very different data sets. The first set in which the videos are shot in a controlled environment in which simple actions were performed. The second set in which videos are shot in a normal office where daily office activities are captured and categorized afterwards. The prototype has given some promising results proving it can cope with very different spaces, actions and activities

Optic flow from multi-scale dynamic anchor point attributes

Optic flow describes the apparent motion that is present in an image sequence. We show the feasibility of obtaining optic flow from dynamic properties of a sparse set of multi-scale anchor points. Singular points of a Gaussian scale space image are identified as feasible anchor point candidates and analytical expressions describing their dynamic properties are presented. Advantages of approaching the optic flow estimation problem using these anchor points are that (i) in these points the notorious aperture problem does not manifest itself, (ii) it combines the strengths of variational and multi-scale methods, (iii) optic flow definition becomes independent of image resolution, (iv) computations of the components of the optic flow field are decoupled and that (v) the feature set inducing the optic flow field is very sparse (typically of the number of pixels in a frame). A dense optic flow vector field is obtained through projection into a Sobolev space defined by and consistent with the dynamic constraints in the anchor points. As opposed to classical optic flow estimation schemes the proposed method accounts for an explicit scale component of the vector field, which encodes some dynamic differential flow property

Optic flow from multi-scale dynamic anchor point attributes

Optic flow describes the apparent motion that is present in an image sequence. We show the feasibility of obtaining optic flow from dynamic properties of a sparse set of multi-scale anchor points. Singular points of a Gaussian scale space image are identified as feasible anchor point candidates and analytical expressions describing their dynamic properties are presented. Advantages of approaching the optic flow estimation problem using these anchor points are that (i) in these points the notorious aperture problem does not manifest itself, (ii) it combines the strengths of variational and multi-scale methods, (iii) optic flow definition becomes independent of image resolution, (iv) computations of the components of the optic flow field are decoupled and that (v) the feature set inducing the optic flow field is very sparse (typically of the number of pixels in a frame). A dense optic flow vector field is obtained through projection into a Sobolev space defined by and consistent with the dynamic constraints in the anchor points. As opposed to classical optic flow estimation schemes the proposed method accounts for an explicit scale component of the vector field, which encodes some dynamic differential flow property