Time-slice analysis of dyadic human activity

La reconnaissance d’activités humaines à partir de données vidéo est utilisée pour la surveillance ainsi que pour des applications d’interaction homme-machine. Le principal objectif est de classer les vidéos dans l’une des k classes d’actions à partir de vidéos entièrement observées. Cependant, de tout temps, les systèmes intelligents sont améliorés afin de prendre des décisions basées sur des incertitudes et ou des informations incomplètes. Ce besoin nous motive à introduire le problème de l’analyse de l’incertitude associée aux activités humaines et de pouvoir passer à un nouveau niveau de généralité lié aux problèmes d’analyse d’actions. Nous allons également présenter le problème de reconnaissance d’activités par intervalle de temps, qui vise à explorer l’activité humaine dans un intervalle de temps court. Il a été démontré que l’analyse par intervalle de temps est utile pour la caractérisation des mouvements et en général pour l’analyse de contenus vidéo. Ces études nous encouragent à utiliser ces intervalles de temps afin d’analyser l’incertitude associée aux activités humaines. Nous allons détailler à quel degré de certitude chaque activité se produit au cours de la vidéo. Dans cette thèse, l’analyse par intervalle de temps d’activités humaines avec incertitudes sera structurée en 3 parties. i) Nous présentons une nouvelle famille de descripteurs spatiotemporels optimisés pour la prédiction précoce avec annotations d’intervalle de temps. Notre représentation prédictive du point d’intérêt spatiotemporel (Predict-STIP) est basée sur l’idée de la contingence entre intervalles de temps. ii) Nous exploitons des techniques de pointe pour extraire des points d’intérêts afin de représenter ces intervalles de temps. iii) Nous utilisons des relations (uniformes et par paires) basées sur les réseaux neuronaux convolutionnels entre les différentes parties du corps de l’individu dans chaque intervalle de temps. Les relations uniformes enregistrent l’apparence locale de la partie du corps tandis que les relations par paires captent les relations contextuelles locales entre les parties du corps. Nous extrayons les spécificités de chaque image dans l’intervalle de temps et examinons différentes façons de les agréger temporellement afin de générer un descripteur pour tout l’intervalle de temps. En outre, nous créons une nouvelle base de données qui est annotée à de multiples intervalles de temps courts, permettant la modélisation de l’incertitude inhérente à la reconnaissance d’activités par intervalle de temps. Les résultats expérimentaux montrent l’efficience de notre stratégie dans l’analyse des mouvements humains avec incertitude.Recognizing human activities from video data is routinely leveraged for surveillance and human-computer interaction applications. The main focus has been classifying videos into one of k action classes from fully observed videos. However, intelligent systems must to make decisions under uncertainty, and based on incomplete information. This need motivates us to introduce the problem of analysing the uncertainty associated with human activities and move to a new level of generality in the action analysis problem. We also present the problem of time-slice activity recognition which aims to explore human activity at a small temporal granularity. Time-slice recognition is able to infer human behaviours from a short temporal window. It has been shown that temporal slice analysis is helpful for motion characterization and for video content representation in general. These studies motivate us to consider timeslices for analysing the uncertainty associated with human activities. We report to what degree of certainty each activity is occurring throughout the video from definitely not occurring to definitely occurring. In this research, we propose three frameworks for time-slice analysis of dyadic human activity under uncertainty. i) We present a new family of spatio-temporal descriptors which are optimized for early prediction with time-slice action annotations. Our predictive spatiotemporal interest point (Predict-STIP) representation is based on the intuition of temporal contingency between time-slices. ii) we exploit state-of-the art techniques to extract interest points in order to represent time-slices. We also present an accumulative uncertainty to depict the uncertainty associated with partially observed videos for the task of early activity recognition. iii) we use Convolutional Neural Networks-based unary and pairwise relations between human body joints in each time-slice. The unary term captures the local appearance of the joints while the pairwise term captures the local contextual relations between the parts. We extract these features from each frame in a time-slice and examine different temporal aggregations to generate a descriptor for the whole time-slice. Furthermore, we create a novel dataset which is annotated at multiple short temporal windows, allowing the modelling of the inherent uncertainty in time-slice activity recognition. All the three methods have been evaluated on TAP dataset. Experimental results demonstrate the effectiveness of our framework in the analysis of dyadic activities under uncertaint

Stimulus configuration and the perceived rigidity of eight-vertex polyhedra

In a series of four experiments, subjects examined the perceived rigidity of rotating eight-vertex polyhedra. Four different categories of polyhedra were observed under parallel projection: (1) line drawings where the initial orientation appeared to be a cube (LN), though the depth components of the eight vertices were randomly positioned (upon rotation, it could be seen that the stimuli were not cubes); (2) line drawings where the vertices were randomly placed (LR); (3) vertex-only drawings where the initial orientation appeared to be a cube (VN), though the depth components of the eight vertices were randomly positioned; and (4) vertex-only drawings with randomly positioned vertices (VR). Preliminary observations indicated that some of the mathematically rigid configurations were perceived as deforming in a nonrigid manner. Given the different stimulus categories, the following questions were addressed: (1) Could subjects identify stimuli that appeared to deform based on a large set of mathematically rigid objects?; and (2) Was it possible to identify gross qualities about the stimulus that control whether or not the human visual system adopts a rigid versus a nonrigid interpretation? Through several deformation-rating tasks, the results indicated that although most of the configurations maintained a rigid appearance throughout their rotations, the LN stimuli appeared to deform more than the LR, VN, and VR categories of stimuli. In addition, based on a signal detection paradigm, when subjects were asked to detect a physical nonrigidity embedded within mathematically rigid rotations, they had a more difficult time doing so when viewing the LN stimuli, compared to the other three stimulus categories. To account for these findings, a theory was formulated based on the behavior of line segments as they are projected onto the two-dimensional image plane. It seems that when the visual system is forced to interpret such images, two conflicting sources of information may exist: local shape cues formed by the intersecting line segments and motion-induced depth information. In order for the visual system to make sense of these images, the conflicting cues need to be driven into agreement with one another, via the adoption of a nonrigid interpretation

Models for Motion Perception

As observers move through the environment or shift their direction of gaze, the world moves past them. In addition, there may be objects that are moving differently from the static background, either rigid-body motions or nonrigid (e.g., turbulent) ones. This dissertation discusses several models for motion perception. The models rely on first measuring motion energy, a multi-resolution representation of motion information extracted from image sequences. The image flow model combines the outputs of a set of spatiotemporal motion-energy filters to estimate image velocity, consonant with current views regarding the neurophysiology and psychophysics of motion perception. A parallel implementation computes a distributed representation of image velocity that encodes both a velocity estimate and the uncertainty in that estimate. In addition, a numerical measure of image-flow uncertainty is derived. The egomotion model poses the detection of moving objects and the recovery of depth from motion as sensor fusion problems that necessitate combining information from different sensors in the presence of noise and uncertainty. Image sequences are segmented by finding image regions corresponding to entire objects that are moving differently from the stationary background. The turbulent flow model utilizes a fractal-based model of turbulence, and estimates the fractal scaling parameter of fractal image sequences from the outputs of motion-energy filters. Some preliminary results demonstrate the model\u27s potential for discriminating image regions based on fractal scaling

Tracking using a local closed-world assumption : tracking in the football domain

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1994.Includes bibliographical references (leaves 81-85).by Stephen Sean Intille.M.S

Modelação biomecânica do corpo humano : aplicação na análise da marcha

Dissertação de mestrado integrado em Engenharia BiomédicaWalking is a complex process, achieved through coordinated movements, which allows the displacement of the human body and therefore has been the subject of study since the beginning of time. Currently, modelling of this movement and the human body realistically, allowing recreating, simulating or analyzing human movement is still a major problem in biomedical engineering. Gait analysis allows the extraction of quantities that characterize human locomotion, allowing the evaluation of the gait pattern of a subject. Accurate measurement of movement is crucial in any technique to characterize the motion. The knowledge provided by this analysis provides geometric characteristics, physical and behavioral traits of the human body that allows the distinction between normal and pathological gait. The aim of this work involves developing an algorithm that allows the estimation of certain spatio-temporal parameters of interest, as are the frequency and period of the gait cycle, stride width, among others. This algorithm is developed in Matlab. It was also developed a model of the human body in Webots, whose function is to present the dynamics and the atual physical body in terms of length and weight. In the process of modeling, approximation and simplification of the form for each segment of the humanoid model is performed in order to meet the basic form of the human body. In the case of the study in question it‘s not necessary a visual result close to reality but a practical result of human locomotion. Thus the modeling of the human body was made using cylinders. In short, the main goal is to make suggestions that may contribute to the analysis of human movement, reproducing the same, using data on the position of the various segments of the human body obtained with the help of the Vicon software.Caminhar é um complexo processo, alcançado através de movimentos coordenados, que permite o deslocamento do corpo humano sendo, portanto, objeto de estudo desde sempre. Atualmente, é, ainda, um dos maiores problemas da engenharia biomédica, a modelação deste movimento e do corpo humano de modo realista, permitindo recriar, simular ou analisar o movimento humano. A análise da marcha possibilita a extração de quantidades que caracterizam a locomoção humana, permitindo a avaliação do padrão de marcha de um sujeito. A medição precisa do movimento é fulcral em qualquer técnica de caracterização da marcha. O conhecimento proporcionado por esta análise fornece características geométricas, físicas e comportamentais do corpo humano tornado possível a distinção entre marcha normal e patológica. O objetivo deste trabalho passa pelo desenvolvimento de um algoritmo que permite a estimação de determinados parâmetros espácio-temporais de interesse, como são a frequência e período do ciclo de marcha, amplitude de passada, entre outros. Este algoritmo é desenvolvido em ambiente Matlab. É ainda desenvolvido um modelo do corpo humano em Webots, cuja função é representar a dinâmica e o corpo humano real em termos de altura e massa. No processo de modelação, a aproximação e simplificação da forma para cada segmento do modelo humanoide é realizada de forma a ir de encontro à forma básica do corpo humano. No caso do estudo em causa, não é necessário um resultado visual aproximado à realidade mas sim um resultado prático de locomoção humana. Assim a modelação do corpo humano foi feita usando cilindros. Em suma, o grande objetivo é a apresentação de sugestões passíveis de contribuírem para a análise do movimento humano, reproduzindo o mesmo, através de dados relativos à posição dos diversos segmentos do corpo humano, obtidos com auxílio do software Vicon

Spatiotemporal analysis of human actions using RGB-D cameras

Markerless human motion analysis has strong potential to provide cost-efficient solution for action recognition and body pose estimation. Many applications including humancomputer interaction, video surveillance, content-based video indexing, and automatic annotation among others will benefit from a robust solution to these problems. Depth sensing technologies in recent years have positively changed the climate of the automated vision-based human action recognition problem, deemed to be very difficult due to the various ambiguities inherent to conventional video. In this work, first a large set of invariant spatiotemporal features is extracted from skeleton joints (retrieved from depth sensor) in motion and evaluated as baseline performance. Next we introduce a discriminative Random Decision Forest-based feature selection framework capable of reaching impressive action recognition performance when combined with a linear SVM classifier. This approach improves upon the baseline performance obtained using the whole feature set with a significantly less number of features (one tenth of the original). The approach can also be used to provide insights on the spatiotemporal dynamics of human actions. A novel therapeutic action recognition dataset (WorkoutSU-10) is presented. We took advantage of this dataset as a benchmark in our tests to evaluate the reliability of our proposed methods. Recently the dataset has been published publically as a contribution to the action recognition community. In addition, an interactive action evaluation application is developed by utilizing the proposed methods to help with real life problems such as 'fall detection' in the elderly people or automated therapy program for patients with motor disabilities