Sensor fusion with Gaussian processes

This thesis presents a new approach to multi-rate sensor fusion for (1) user matching and (2) position stabilisation and lag reduction. The Microsoft Kinect sensor and the inertial sensors in a mobile device are fused with a Gaussian Process (GP) prior method. We present a Gaussian Process prior model-based framework for multisensor data fusion and explore the use of this model for fusing mobile inertial sensors and an external position sensing device. The Gaussian Process prior model provides a principled mechanism for incorporating the low-sampling-rate position measurements and the high-sampling-rate derivatives in multi-rate sensor fusion, which takes account of the uncertainty of each sensor type. We explore the complementary properties of the Kinect sensor and the built-in inertial sensors in a mobile device and apply the GP framework for sensor fusion in the mobile human-computer interaction area. The Gaussian Process prior model-based sensor fusion is presented as a principled probabilistic approach to dealing with position uncertainty and the lag of the system, which are critical for indoor augmented reality (AR) and other location-aware sensing applications. The sensor fusion helps increase the stability of the position and reduce the lag. This is of great benefit for improving the usability of a human-computer interaction system. We develop two applications using the novel and improved GP prior model. (1) User matching and identification. We apply the GP model to identify individual users, by matching the observed Kinect skeletons with the sensed inertial data from their mobile devices. (2) Position stabilisation and lag reduction in a spatially aware display application for user performance improvement. We conduct a user study. Experimental results show the improved accuracy of target selection, and reduced delay from the sensor fusion system, allowing the users to acquire the target more rapidly, and with fewer errors in comparison with the Kinect filtered system. They also reported improved performance in subjective questions. The two applications can be combined seamlessly in a proxemic interaction system as identification of people and their positions in a room-sized environment plays a key role in proxemic interactions

Transformations of Gaussian Process priors for user matching

We describe the use of transformations of Gaussian Process (GP) priors to improve the context sensing capability of a system composed of a Kinect sensor and mobile inertial sensors. The Bayesian nonparametric model provides a principled mechanism for incorporating the low-sampling-rate position measurements and the high-sampling-rate derivatives in multi-rate sensor fusion which takes account of the uncertainty of each sensor type. The complementary properties of these sensors enable the GP model to calculate the likelihood of the observed Kinect skeletons and inertial data to identify individual users. We conducted three experiments to test the performance of the proposed GP model: (1) subtle hand movements, (2) walking with a mobile device in the trouser pocket, and (3) walking with a mobile device held in the hand. We compared the GP with the direct acceleration comparison method. Experimental results show that the GP approach can achieve successful matches (with mean accuracy μ>90%) in all 3 contexts, including when there are only subtle hand movements, where the acceleration comparison method performs poorly (μ<20%)