Trajectory-based Human Action Recognition

Human activity recognition has been a hot topic for some time. It has several challenges, which makes this task hard and exciting for research. The sparse representation became more popular during the past decade or so. Sparse representation methods represent a video by a set of independent features. The features used in the literature are usually lowlevel features. Trajectories, as middle-level features, capture the motion of the scene, which is discriminant in most cases. Trajectories have also been proven useful for aligning small neighborhoods, before calculating the traditional descriptors. In fact, the trajectory aligned descriptors show better discriminant power than the trajectory shape descriptors proposed in the literature. However, trajectories have not been investigated thoroughly, and their full potential has not been put to the test before this work. This thesis examines trajectories, defined better trajectory shape descriptors and finally it augmented trajectories with disparity information. This thesis formally define three different trajectory extraction methods, namely interest point trajectories (IP), Lucas-Kanade based trajectories (LK), and Farnback optical flow based trajectories (FB). Their discriminant power for human activity recognition task is evaluated. Our tests reveal that LK and FB can produce similar reliable results, although the FB perform a little better in particular scenarios. These experiments demonstrate which method is suitable for the future tests. The thesis also proposes a better trajectory shape descriptor, which is a superset of existing descriptors in the literature. The examination reveals the superior discriminant power of this newly introduced descriptor. Finally, the thesis proposes a method to augment the trajectories with disparity information. Disparity information is relatively easy to extract from a stereo image, and they can capture the 3D structure of the scene. This is the first time that the disparity information fused with trajectories for human activity recognition. To test these ideas, a dataset of 27 activities performed by eleven actors is recorded and hand labelled. The tests demonstrate the discriminant power of trajectories. Namely, the proposed disparity-augmented trajectories improve the discriminant power of traditional dense trajectories by about 3.11%

Detecção de movimento anormal em videovigilância baseada em rastreamento e agrupamentos uniformes ótimos

Tese (doutorado)- Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2015.A videovigilância compõe-se de um conjunto de câmeras e demais recursos tecnológicos para servir como uma ferramenta que visa a segurança pública ou privada em locais estratégicos da movimentação de pessoas e/ou veículos. Os interesses por estes sistemas, em expansão pelo mundo, estão ligados a sua potencialidade em coibir atos antissociais, apoiar na melhoria da mobilidade urbana ou ainda detectar ou prevenir eventos que demandem ação imediata para evitar colapsos, ou mesmo salvar vidas. A automação na monitoração é uma necessidade irreversível pois, sendo centralizada, depende de um operador humano para fiscalizar muitas câmeras através de um trabalho tedioso, cansativo e sujeito a erros e omissões no acompanhamento de movimentação suspeita. A Detecção de Movimento Anormal (DMA) é uma análise de vídeo útil para fins de videovigilância e, em especial, aquela realizada sobre o rastreamento de objetos em trajetos globais não usuais. Em função das barreiras no tratamento computacional de grandes volumes de dados, mesmo nas modernas arquiteturas de sistemas embarcados, propostas encontradas nas abordagens baseadas em rastreamento são geralmente limitadas em flexibilidade no que diz respeito a cenários, metas, duração do vídeo e realidade e assim, nem sempre viáveis nas aplicações em tempo real. Visando extrair o melhor de um modelo estatístico para esse propósito, como o modelo de misturas gaussianas (GMM - Gaussian Mixture Modeling), o presente trabalho apresenta uma nova abordagem para DMA ancorada sobre um classificador binário ótimo e construída a partir de três processos iterativos durante um treinamento supervisionado: a geração de amostras sobre agrupamentos uniformes formando uma grade de regiões, a aprendizagem por região dos parâmetros de umafunção de distribuição de probabilidade (pdf ) multivariada e por fim, o uso de curvas características de operação do receptor (ROC - Receiver Operating Characteristics) para encontrar o melhor classificador. Como base para avaliar a abordagem foram utilizados dados resultantes de anotações de vídeo do mundo real, elaborados a partir de ferramentas próprias ou de domínio público. Os resultados avaliados demonstraram que cada cenário possui uma área de agrupamento que otimiza o desempenho da DMA mesmo com uma significativa redução de amostras. Neste aspecto, além da tese contribuir com uma metodologia que garante a melhor performance dentro da abordagem da DMA proposta, ela revela que uma análise baseada em região reduz o custo computacional sem afetar significativamente a qualidade das inferências.Abstract : Video surveillance is composed of a set of cameras and other technological resources to serve as a tool to public or private safety in strategic locations of moving people and/or vehicles. The interest by these systems, expanding worldwide, are linked to their potentiality in curbing antisocial acts, to assist in improving urban mobility or also detect or prevent events that require immediate action to prevent collapses, or even save lives. The automation in monitoring these systems is an irreversible necessity, because being centralized, depends on a human operator to monitor many cameras through a tedious, tiresome and prone to errors and omissions job in the monitoring of suspicious motions. The Abnormal Motion Detection (AMD) is a useful video analysis for video surveillance purposes, and in particular, that performed on the objects tracking in unusual global paths. Due to the barriers in computational treating of large amounts of data, even in modern architectures embedded systems, proposals found in tracking based approaches are generally limited in flexibility regarding scenarios, goals, length of video and reality and thus, not always feasible in real-time applications. Aiming to extract the best froma statistical model for this purpose, as a Gaussian Mixture Model (GMM - Gaussian Mixture Modeling), this work presents a new approach to AMD docked on a best binary classifier and built from three iterative processes over a supervised training: The samples generation over uniform clusters forming a grid of regions, learning the parameters per region of a probability distribution function (pdf ) multivariated and finally, the use of curves the receiver operating characteristics (ROC - Receiver Operating Characteristics) to find the best classifier. As a basis to evaluate the approach, data derived from real world video annotations were used, elaborated from own or public domain tools. The evaluated results demonstrated that each scenario has a clustering area that optimizes the AMD performance even with a substantial samples reduction. In this regard, besides the thesis contribute on a methodology that ensures the best AMD approach performance, it reveals that a region-based analysis reduces computational cost without significantly affecting the inferences quality

Human behavior analysis based on a new motion descriptor

Human behavior analysis is an important area of research in computer vision and is also driven by a wide spectrum of applications, such as smart video surveillance and human-computer interface. In this paper, we present a novel approach for human behavior analysis. Two research challenges, motion representation and behavior recognition, are addressed. A novel motion descriptor, which is an improved feature based on optical flow, is proposed for motion representation. Optical flow is improved with a motion filter, and feature fusion with the shape and trajectory information. To recognize the behavior, the support vector machine is employed to train the classifier where the concatenation of histograms is formed as the input features. Experimental results on the Weizmann behavior database and the Institute of Automation, Chinese Academy of Science real-world multiview behavior database demonstrate the robustness and effectiveness of our method