SpatioTemporal LBP and shape feature for human activity representation and recognition

In this paper, we propose a histogram based feature to represent and recognize human action in video sequences. Motion History Image (MHI) merges a video sequence into a single image. However, in this method, we use Directional Motion History Image (DMHI) to create four directional spatiotemporal templates. We, then, extract the Local Binary Pattern (LBP) from those templates. Then, spatiotemporal LBP histograms are formed to represent the distribution of those patterns which makes the feature vector. We also use shape feature taken from three selective snippets and concatenate them with the LBP histograms. We measure the performance of the proposed representation method along with some variants of it by experimenting on the Weizmann action dataset. Higher recognition rates found in the experiment suggest that, compared to complex representation, the proposed simple and compact representation can achieve robust recognition of human activity for practical use

Descriptive temporal template features for visual motion recognition

In this paper, a human action recognition system is proposed. The system is based on new, descriptive `temporal template' features in order to achieve high-speed recognition in real-time, embedded applications. The limitations of the well known `Motion History Image' (MHI) temporal template are addressed and a new `Motion History Histogram' (MHH) feature is proposed to capture more motion information in the video. MHH not only provides rich motion information, but also remains computationally inexpensive. To further improve classification performance, we combine both MHI and MHH into a low dimensional feature vector which is processed by a support vector machine (SVM). Experimental results show that our new representation can achieve a significant improvement in the performance of human action recognition over existing comparable methods, which use 2D temporal template based representations

Real-time human action recognition on an embedded, reconfigurable video processing architecture

Copyright @ 2008 Springer-Verlag.In recent years, automatic human motion recognition has been widely researched within the computer vision and image processing communities. Here we propose a real-time embedded vision solution for human motion recognition implemented on a ubiquitous device. There are three main contributions in this paper. Firstly, we have developed a fast human motion recognition system with simple motion features and a linear Support Vector Machine (SVM) classifier. The method has been tested on a large, public human action dataset and achieved competitive performance for the temporal template (eg. “motion history image”) class of approaches. Secondly, we have developed a reconfigurable, FPGA based video processing architecture. One advantage of this architecture is that the system processing performance can be reconfiured for a particular application, with the addition of new or replicated processing cores. Finally, we have successfully implemented a human motion recognition system on this reconfigurable architecture. With a small number of human actions (hand gestures), this stand-alone system is performing reliably, with an 80% average recognition rate using limited training data. This type of system has applications in security systems, man-machine communications and intelligent environments.DTI and Broadcom Ltd

High-Speed Human Motion Recognition Based on a Motion History Image and an Eigenspace

This paper proposes an efficient technique for human motion recognition based on motion history images and an eigenspace technique. In recent years, human motion recognition has become one of the most popular research fields. It is expected to be applied in a security system, man-machine communication, and so on. In the proposed technique, we use two feature images and the eigenspace technique to realize high-speed recognition. An experiment was performed on recognizing six human motions and the results showed satisfactory performance of the technique

FPGA implementation of real-time human motion recognition on a reconfigurable video processing architecture

In recent years, automatic human motion recognition has been widely researched within the computer vision and image processing communities. Here we propose a real-time embedded vision solution for human motion recognition implemented on a ubiquitous device. There are three main contributions in this paper. Firstly, we have developed a fast human motion recognition system with simple motion features and a linear Support Vector Machine(SVM) classifier. The method has been tested on a large, public human action dataset and achieved competitive performance for the temporal template (eg. ``motion history image") class of approaches. Secondly, we have developed a reconfigurable, FPGA based video processing architecture. One advantage of this architecture is that the system processing performance can be reconfigured for a particular application, with the addition of new or replicated processing cores. Finally, we have successfully implemented a human motion recognition system on this reconfigurable architecture. With a small number of human actions (hand gestures), this stand-alone system is performing reliably, with an 80% average recognition rate using limited training data. This type of system has applications in security systems, man-machine communications and intelligent environments

A robust fall detection system for the elderly in a smart room

In the paper, we propose a robust fall detection method which combines head tracking and extraction of human shape within a smart home environment equipped with video cameras. A motion history image and an improved code-book background subtraction technique are combined to extract the human shape. An additional motion-based particle filtering head tracker is also used to ensure the robustness of the system. The extracted human shape information and the head tracking results are combined as criteria for judging the occurrence of a fall. The success of the method is confirmed on real video sequences

A dynamic framework based on local Zernike Moment and motion history image for facial expression recognition

A dynamic descriptor facilitates robust recognition of facial expressions in video sequences. The current two main approaches to the recognition are basic emotion recognition and recognition based on facial action coding system (FACS) action units. In this paper we focus on basic emotion recognition and propose a spatio-temporal feature based on local Zernike moment in the spatial domain using motion change frequency. We also design a dynamic feature comprising motion history image and entropy. To recognise a facial expression, a weighting strategy based on the latter feature and sub-division of the image frame is applied to the former to enhance the dynamic information of facial expression, and followed by the application of the classical support vector machine. Experiments on the CK+ and MMI datasets using leave-one-out cross validation scheme demonstrate that the integrated framework achieves a better performance than using individual descriptor separately. Compared with six state-of-arts methods, the proposed framework demonstrates a superior performance

Visualization of perfusion changes with laser speckle contrast imaging using the method of motion history image

Laser speckle contrast imaging (LSCI) is a real-time imaging modality reflecting microvascular perfusion. We report on the application of the motion history image (MHI) method on LSCI data obtained from the two hemispheres of a mouse. Through the generation of a single image, MHI stresses the microvascular perfusion changes. Our experimental results performed during a pinprick-triggered spreading depolarization demonstrate the effectiveness of MHI: MHI allows the visualization of perfusion changes without loss of resolution and definition. Moreover, MHI provides close results to the ones given by the generalized differences (GD) algorithm. However, MHI has the advantage of giving information on the temporal evolution of the perfusion variations, which GD does not

Fall detection in the elderly by head-tracking

In the paper, we propose a fall detection method based on head tracking within a smart home environment equipped with video cameras. A motion history image and code-book background subtraction are combined to determine whether large movement occurs within the scene. Based on the magnitude of the movement information, particle filters with different state models are used to track the head. The head tracking procedure is performed in two video streams taken bytwoseparatecamerasandthree-dimensionalheadposition is calculated based on the tracking results. Finally, the threedimensional horizontal and vertical velocities of the head are used to detect the occurrence of a fall. The success of the method is confirmed on real video sequences

Human action representation and recognition: An approach to histogram od spatiotemporal templates

The motion sequences of human actions have its own discriminating profile that can be represented as a spatiotemporal template like Motion History Image (MHI). A histogram is a popular statistic to present the underlying information in a template. In this paper a histogram oriented action recognition method is presented. In the proposed method, we use the Directional Motion History Images (DMHI), their corresponding Local Binary Pattern (LBP) images and the Motion Energy Image (MEI) as spatiotemporal template. The intensity histogram is then extracted from those images which are concatenated together to form the feature vector for action representation. A linear combination of the histograms taken from DMHIs and LBP images is used in the experiment. We evaluated the performance of the proposed method along with some variants of it using the renowned KTH action dataset and found higher accuracies. The obtained results justify the superiority of the proposed method compared to other approaches for action recognition found in literature