Unsupervised learning of human motion

An unsupervised learning algorithm that can obtain a probabilistic model of an object composed of a collection of parts (a moving human body in our examples) automatically from unlabeled training data is presented. The training data include both useful "foreground" features as well as features that arise from irrelevant background clutter - the correspondence between parts and detected features is unknown. The joint probability density function of the parts is represented by a mixture of decomposable triangulated graphs which allow for fast detection. To learn the model structure as well as model parameters, an EM-like algorithm is developed where the labeling of the data (part assignments) is treated as hidden variables. The unsupervised learning technique is not limited to decomposable triangulated graphs. The efficiency and effectiveness of our algorithm is demonstrated by applying it to generate models of human motion automatically from unlabeled image sequences, and testing the learned models on a variety of sequences

Measuring Bridge Construction Efficiency Using the Wireless Real-Time Video Monitoring System

To enhance the efficiency of bridge construction, the wireless real-time video monitoring system (WRITE) was developed. Utilizing the advanced technologies of computer vision and artificial neural networks, the developed system first wirelessly acquired a sequence of images of work face operations. Then human pose analyzing algorithms processed these images in real time to generate human poses associated with construction workers who performed the operations. Next, a portion of the human poses were manually classified into three categories as effective work, contributory work, and ineffective work and were used to train the built-in artificial neural networks (ANN). Finally, the trained neural networks were employed to decide the ongoing laborers’ working status by comparing the in coming images to the developed human poses. The developed system was tested for accuracy on a bridge construction project. Results of the test showed that efficiency measurements by the system were reasonably accurate when compared to the measurements produced by the manual method. Thus, the success of this research indicates promise for enabling project managers to quickly identify work-face operation problems and to take actions immediately to address these problems

Breaking the Chain: Liberation from the Temporal Markov Assumption for Tracking Human Poses

We present an approach to multi-target tracking that has expressive potential beyond the capabilities of chain-shaped hidden Markov models, yet has significantly reduced complexity. Our framework, which we call tracking-by-selection, is similar to tracking-by-detection in that it sepa-rates the tasks of detection and tracking, but it shifts tempo-ral reasoning from the tracking stage to the detection stage. The core feature of tracking-by-selection is that it reasons about path hypotheses that traverse the entire video instead of a chain of single-frame object hypotheses. A traditional chain-shaped tracking-by-detection model is only able to promote consistency between one frame and the next. In tracking-by-selection, path hypotheses exist across time, and encouraging long-term temporal consistency is as sim-ple as rewarding path hypotheses with consistent image fea-tures. One additional advantage of tracking-by-selection is that it results in a dramatically simplified model that can be solved exactly. We adapt an existing tracking-by-detection model to the tracking-by-selection framework, and show im-proved performance on a challenging dataset (introduced in [18]). 1

Motion capture based on RGBD data from multiple sensors for avatar animation

With recent advances in technology and emergence of affordable RGB-D sensors for a wider range of users, markerless motion capture has become an active field of research both in computer vision and computer graphics. In this thesis, we designed a POC (Proof of Concept) for a new tool that enables us to perform motion capture by using a variable number of commodity RGB-D sensors of different brands and technical specifications on constraint-less layout environments. The main goal of this work is to provide a tool with motion capture capabilities by using a handful of RGB-D sensors, without imposing strong requirements in terms of lighting, background or extension of the motion capture area. Of course, the number of RGB-D sensors needed is inversely proportional to their resolution, and directly proportional to the size of the area to track to. Built on top of the OpenNI 2 library, we made this POC compatible with most of the nonhigh-end RGB-D sensors currently available in the market. Due to the lack of resources on a single computer, in order to support more than a couple of sensors working simultaneously, we need a setup composed of multiple computers. In order to keep data coherency and synchronization across sensors and computers, our tool makes use of a semi-automatic calibration method and a message-oriented network protocol. From color and depth data given by a sensor, we can also obtain a 3D pointcloud representation of the environment. By combining pointclouds from multiple sensors, we can collect a complete and animated 3D pointcloud that can be visualized from any viewpoint. Given a 3D avatar model and its corresponding attached skeleton, we can use an iterative optimization method (e.g. Simplex) to find a fit between each pointcloud frame and a skeleton configuration, resulting in 3D avatar animation when using such skeleton configurations as key frames

Model figging of articulated objects

本稿では人体や手などに代表される多関節物体の三次元姿勢を画像から推定するモデルフィッティングの技術についてサーベイする。画像によるモデルフィッティングの枠組みを,1)推定に利用される画像特徴,2)画像と照合するモデルの表現と照合のパラメータ空間,3)照合時の評価関数と最適解の探索手法,にわけて多関節物体の三次元姿勢推定に特徴的な要素を上記三つの観点から比較整理することを試みる。In this paper, we present a survey report for the model fitting method to estimate3-D posture of articulated objects such as human body and hand. We decompose the model fitting framework into the following threee lements: 1)image feature, 2)model description and parameter space for model-image matching and 3)matching function and its optimization. From the viewpoint of these three issues, we try to compare the various methods of model fitting to each other and summarize them

Applications of Factorization Theorem and Ontologies for Activity ModelingRecognition and Anomaly Detection

In this thesis two approaches for activity modeling and suspicious activity detection are examined. First is application of factorization theorem extension for deformable models in two dierent contexts. First is human activity detection from joint position information, and second is suspicious activity detection for tarmac security. It is shown that the first basis vector from factorization theorem is good enough to dierentiate activities for human data and to distinguish suspicious activities for tarmac security data. Second approach dierentiates individual components of those activities using semantic methodol- ogy. Although currently mainly used for improving search and information retrieval, we show that ontologies are applicable to video surveillance. We evaluate the domain ontologies from Challenge Project on Video Event Taxonomy sponsored by ARDA from the perspective of general ontology design principles. We also focused on the eect of the domain on the granularity of the ontology for suspicious activity detection

Automatic visual detection of human behavior: a review from 2000 to 2014

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviors from video is a very recent research topic. In this paper, we perform a systematic and recent literature review on this topic, from 2000 to 2014, covering a selection of 193 papers that were searched from six major scientific publishers. The selected papers were classified into three main subjects: detection techniques, datasets and applications. The detection techniques were divided into four categories (initialization, tracking, pose estimation and recognition). The list of datasets includes eight examples (e.g., Hollywood action). Finally, several application areas were identified, including human detection, abnormal activity detection, action recognition, player modeling and pedestrian detection. Our analysis provides a road map to guide future research for designing automatic visual human behavior detection systems.This work is funded by the Portuguese Foundation for Science and Technology (FCT - Fundacao para a Ciencia e a Tecnologia) under research Grant SFRH/BD/84939/2012