Inertial Motion Capturing : Rigid Body Pose and Posture Estimation with Inertial Sensors

This dissertation is about estimating poses from inertial sensor data, that is estimating orientations and positions. Both poses of single rigid bodies as well as poses of so called skeletons, i.e. systems of jointed rigid bodies, are covered. The key insight into orientation estimation of a single rigid body is to view it as the fusion of sensor data and its dynamics model with prior information. To this end, three different Kalman Filter variations are presented, which fuse the same sensor data and the same dynamics with three different priors. It turns out that the classical model to correct the inclination in an orientation estimator, namely comparing the accelerometer measurement with (negative) gravity, is equivalent to the assumption that the rigid body does not accelerate on long-term average. Assuming that the velocity is zero on long-term average or that the rigid body stays at the same position on long-term average are alternative assumptions and both priors also yield orientation estimators. Moreover, the orientation estimator resulting from the position assumption also estimates a position, which is locally accurate - it follows the accelerometer measurements - but does not drift unboundedly, which it would if the position were obtained by integrating according to the dynamic model only. The focus here is more on the interplay of inertial sensor data and its dynamic model with prior information than it is on practical applications. For instance, for the integrated position to be a usable quantity, the estimate has to be conditioned on the long-term average of the position being zero instead of the velocity or acceleration being zero. In the second, bigger part of this dissertation the posture of a skeleton, i.e. the poses of all the skeleton's bodies, are estimated, again using inertial sensor data only. Notably, no magnetometers are used to recover the rotations around the vertical. Without magnetometers, the rotation of the skeleton as a whole around the vertical, of course, can not be estimated. However, to asses the skeleton's posture, it is also not important. If inertial sensor data of all bodies is fused with the prior information that a skeleton's bodies are jointed using hinges and spherical joints, the relative orientations of the bodies become observable completely: If two accelerometers of two jointed bodies measure the acceleration of a motion, then the relative orientation of those two bodies can be recovered from the directions of the accelerometer measurements, if effects due to movements of the joints are compensated for. The posture estimator that exploits this insight is developed and used in the sensor suit SIRKA, which is workwear with inertial sensors embedded into the clothing. On computationally very limited hardware, which is completely integrated into the suit, the estimator yields posture estimates in real-time. To make this possible, a technique to decouple the sensor's sampling rate from the estimation rate is introduced. Moreover, the sensor orientations and positions inside the suit are almost arbitrary and do not need adjustment. Instead, they are calibrated automatically. The motion capturing workwear is used in a real-world setting, estimating the posture of a worker welding steel on a shipyard. That would not be possible using a motion capturing suit relying on magnetometers

Responsive Listening Behavior

Humans use their bodies in a highly expressive way during conversation, and animated characters that lack this form of non-verbal expression can seem stiff and unemotional. An important aspect of non-verbal expression is that people respond to each other's behavior and are highly attuned to picking up this type of response. This is particularly important for the feedback given while listening to some one speak. However, automatically generating this type of behavior is difficult as it is highly complex and subtle. This paper takes a data driven approach to generating interactive social behavior. Listening behavior is motion captured, together with the audio being listened to. This data is used to learn an animation model of the responses of one person to the other. This allows us to create characters that respond in real-time during a conversation with a real human

Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis

This paper proposes an arbitrary view gait recognition method where the gait recognition is performed in 3-dimensional (3D) to be robust to variation in speed, inclined plane and clothing, and in the presence of a carried item. 3D parametric gait models in a gait period are reconstructed by an optimized 3D human pose, shape and simulated clothes estimation method using multiview gait silhouettes. The gait estimation involves morphing a new subject with constant semantic constraints using silhouette cost function as observations. Using a clothes-independent 3D parametric gait model reconstruction method, gait models of different subjects with various postures in a cycle are obtained and used as galleries to construct 3D gait dictionary. Using a carrying-items posture synthesized model, virtual gait models with different carrying-items postures are synthesized to further construct an over-complete 3D gait dictionary. A self-occlusion optimized simultaneous sparse representation model is also introduced to achieve high robustness in limited gait frames. Experimental analyses on CASIA B dataset and CMU MoBo dataset show a significant performance gain in terms of accuracy and robustness

Analysis of Electrical Activity of the Core Muscles Exposed to Whole-Body Vibrations While Driving a Tractor Trailer Using Electromyography device

Lower back disorders are observed to be the most significant problem for most of the industrial workers who operate commercial vehicles. A person sitting can have 40% higher load on the lumbar spine than when standing. Human core muscles related to spinal movement respond to rapid potential motions related to external acceleration inputs of vibration in order to control body posture, the increased muscle tension can result in muscle fatigue. The over use of muscles, vibration exposure combined with dehydration of spine and long term seating can lead to lower back pain and axial discomfort to the drivers. Electromyography (EMG) is a technique used to monitor and analyze the electrical activation of the muscle. It has been found that the human spine has a natural frequency of 4-8Hz for natural upper body motion caused by curvature motion for the human spine. Trucks have historically produced the critical frequency which resulted in spinal problems for drivers. The purpose of air-ride seats is to reduce the critical frequency amplitudes. Muscular tension can move the natural frequencies to higher level to evaluate the response of the muscle to the expanded acceleration levels in the range of the spinal natural frequency. A Flex Comp Infinity device, Sonosens® monitor and accelerometers are mounted on the driver of the tractor trailer to collect EMG, ultrasound and acceleration data, respectively. Three testing trials are performed to examine the EMG data, which is correlated with the ultrasonic data and acceleration data collected during the tests. Standard data was collected on the driver using the standard commercial long hall tractor trailer on normal roads. The main purpose of the current research is to collect and analyze the physiological activity of the Erector spinae, Gluteus medius and Rectus abdominis muscles that relate to the lower back disorders. Present thesis work also examines spinal motions of the human body when exposed to whole-body vibrations. The study results show that the Erector spinae muscle activity is higher than that of Rectus abdominis and Gluteus medius muscles. Also the muscle fatigue on all the core muscles is observed to occur after 1hr 40 mins to 2 hr, and lasts for nearly 10 mins.The EMG results are compared to the acceleration and ultrasonic data, which were also collected during the test. It is observed that approximately 75% of the ultrasonic results and 60% of acceleration data correlate with EMG results. More accurate results can be expected if more tests are carried out. This research is highly useful to carry out further investigation in the areas of whole body vibration and the muscular response to reduce the level of excitation

Detection and prediction problems with applications in personalized health care

The United States health-care system is considered to be unsustainable due to its unbearably high cost. Many of the resources are spent on acute conditions rather than aiming at preventing them. Preventive medicine methods, therefore, are viewed as a potential remedy since they can help reduce the occurrence of acute health episodes. The work in this dissertation tackles two distinct problems related to the prevention of acute disease. Specifically, we consider: (1) early detection of incorrect or abnormal postures of the human body and (2) the prediction of hospitalization due to heart related diseases. The solution to the former problem could be used to prevent people from unexpected injuries or alert caregivers in the event of a fall. The latter study could possibly help improve health outcomes and save considerable costs due to preventable hospitalizations. For body posture detection, we place wireless sensor nodes on different parts of the human body and use the pairwise measurements of signal strength corresponding to all sensor transmitter/receiver pairs to estimate body posture. We develop a composite hypothesis testing approach which uses a Generalized Likelihood Test (GLT) as the decision rule. The GLT distinguishes between a set of probability density function (pdf) families constructed using a custom pdf interpolation technique. The GLT is compared with the simple Likelihood Test and Multiple Support Vector Machines. The measurements from the wireless sensor nodes are highly variable and these methods have different degrees of adaptability to this variability. Besides, these methods also handle multiple observations differently. Our analysis and experimental results suggest that GLT is more accurate and suitable for the problem. For hospitalization prediction, our objective is to explore the possibility of effectively predicting heart-related hospitalizations based on the available medical history of the patients. We extensively explored the ways of extracting information from patients' Electronic Health Records (EHRs) and organizing the information in a uniform way across all patients. We applied various machine learning algorithms including Support Vector Machines, AdaBoost with Trees, and Logistic Regression adapted to the problem at hand. We also developed a new classifier based on a variant of the likelihood ratio test. The new classifier has a classification performance competitive with those more complex alternatives, but has the additional advantage of producing results that are more interpretable. Following this direction of increasing interpretability, which is important in the medical setting, we designed a new method that discovers hidden clusters and, at the same time, makes decisions. This new method introduces an alternating clustering and classification approach with guaranteed convergence and explicit performance bounds. Experimental results with actual EHRs from the Boston Medical Center demonstrate prediction rate of 82% under 30% false alarm rate, which could lead to considerable savings when used in practice

Using biomechanical constraints to improve video-based motion capture

In motion capture applications whose aim is to recover human body postures from various input, the high dimensionality of the problem makes it desirable to reduce the size of the search-space by eliminating a priori impossible configurations. This can be carried out by constraining the posture recovery process in various ways. Most recent work in this area has focused on applying camera viewpoint-related constraints to eliminate erroneous solutions. When camera calibration parameters are available, they provide an extremely efficient tool for disambiguating not only posture estimation, but also 3D reconstruction and data segmentation. Increased robustness is indeed to be gained from enforcing such constraints, which we prove in the context of an optical motion capture framework. Our contribution in this respect resides in having applied such constraints consistently to each main step involved in a motion capture process, namely marker reconstruction and segmentation, followed by posture recovery. These steps are made inter-dependent, where each one constrains the other. A more application-independent approach is to encode constraints directly within the human body model, such as limits on the rotational joints. This being an almost unexplored research subject, our efforts were mainly directed at determining a new method for measuring, representing and applying such joint limits. To the present day, the few existing range of motion boundary representations present severe drawbacks that call for an alternative formulation. The joint limits paradigm we propose not only overcomes these drawbacks, but also allows to capture intra- and inter-joint rotation dependencies, these being essential to realistic joint motion representation. The range of motion boundary is defined by an implicit surface, its analytical expression enabling us to readily establish whether a given joint rotation is valid or not. Furthermore, its continuous and differentiable nature provides us with a means of elegantly incorporating such a constraint within an optimisation process for posture recovery. Applying constrained optimisation to our body model and stereo data extracted from video sequence, we demonstrate the clearly resulting decrease in posture estimation errors. As a bonus, we have integrated our joint limits representation in character animation packages to show how motion can be naturally constrained in this manner