Tracking human motion with multiple cameras using articulated ICP with hard constraints

Questa tesi propone un nuovo algoritmo basato su ICP per il tracking di un modello scheletrico articolato di un corpo umano. L\u2019algoritmo proposto prende in input immagini calibrate di un soggetto, calcola la ricostruzione volumetrica e la linea mediale del corpo e quindi posiziona in modo adeguato il modello, composto di segmenti, in ogni frame usando una versione di ICP modificata (versione che usa una strategia di attraversamento alberi gerarchica che mantiene connessi tutti i segmenti del modello nei giunti relativi). L\u2019approccio proposto usa limiti cinematica per i giunti e un filtro di Kalman esteso per fare il tracking del modello. Il primo contributo originale di questa tesi \ue8 l\u2019algoritmo per trovare i punti sullo scheletro di un volume tridimensionale. L\u2019algoritmo, usando una tecnica di slicing trova l\u2019asse mediale di un volume 3D in modo veloce utilizzando il processore della scheda grafica e le texture units della scheda stessa. Questo algoritmo produce ottimi risultati per quanto riguarda la qualit\ue0 e le prestazioni se comparato con altri algoritmi in letteratura. Un altro contributo originale \ue8 l\u2019introduzione di una nuova strategia di tracking basata su un approccio gerarchico dell\u2019algoritmo ICP, utilizzato per trovare le congruenze tra un modello di corpo umano composto da soli segmenti e un insieme di punti 3D. L\u2019algoritmo usa una versione di ICP dove tutti i punti 3D sono pesati in funzione del segmento del corpo preso in considerazione dall\u2019algoritmo in quel momento. L\u2019applicazione di queste tecniche dimostra la bont\ue0 del metodo e le prestazioni ottenute in termini di qualit\ue0 della stima della posa sono comparabili con altri lavori in letteratura. I risultati presentati nella tesi dimostrano la fattibilit\ue0 dell\u2019approccio generale, che si intende utilizzare in un sistema completo per il tracking di corpi umani senza l\u2019uso di marcatori. In futuro il lavoro pu\uf2 essere esteso ottimizzando l\u2019implementazione e la codifica in modo da poter ottenere prestazioni real-time.This thesis proposed a new ICP-based algorithm for tracking articulated skeletal model of a human body. The proposed algorithm takes as input multiple calibrated views of the subject, computes a volumetric reconstruction and the centerlines of the body and fits the skeletal body model in each frame using a hierarchic tree traversal version of the ICP algorithm that preserves the connection of the segments at the joints. The proposed approach uses the kinematic constraints and an Extended Kalman Filter to track the body pose. The first contribution is a new algorithm to find the skeletal points of a 3D volume. The algorithm using a slicing technique find the medial axis of a volume in a fast way using the graphic card processor and the texture units. This algorithm produce good results in quality and performance compared to other works in literature. Another contribution is the introduction of a new tracking strategy based on a hierarchical application of the ICP standard algorithm to find the match between a stick body model and a set of 3D points. The algorithm use a traversing version of ICP where also all the 3D points are weighted in such a way every limbs of the model can best fit on the right portion of the body. The application of these techniques shown the feasibility of the method and the performances obtained in terms of quality of estimate pose are comparable with other works in literature. The results presented here demonstrate the feasibility of the approach, which is is intended to be used in complete system for vision-based markerless human body tracking. Future work will aimed at optimizing the implementation, in order to achieve real-time performances

Human perception capabilities for socially intelligent domestic service robots

The daily living activities for an increasing number of frail elderly people represent a continuous struggle both for them as well as for their extended families. These people have difficulties coping at home alone but are still sufficiently fit not to need the round-the-clock care provided by a nursing home. Their struggle can be alleviated by the deployment of a mechanical helper in their home, i.e. a service robot that can execute a range of simple object manipulation tasks. Such a robotic application promises to extend the period of independent home living for elderly people, while providing them with a better quality of life. However, despite the recent technological advances in robotics, there are still some remaining challenges, mainly related to the human factors. Arguably, the lack of consistently dependable human detection, localisation, position and pose tracking information and insufficiently refined processing of sensor information makes the close range physical interaction between a robot and a human a high-risk task. The work described in this thesis addresses the deficiencies in the processing of the human information of today’s service robots. This is achieved through proposing a new paradigm for the robot’s situational awareness in regard to people as well as a collection of methods and techniques, operating at the lower levels of the paradigm, i.e. perception of new human information. The collection includes methods for obtaining and processing of information about the presence, location and body pose of the people. In addition to the availability of reliable human perception information, the integration between the separate levels of paradigm is considered to be a critically important factor for achieving the human-aware control of the robot. Improving the cognition, judgment and decision making action links between the paradigm’s layers leads to enhanced capability of the robot to engage in a natural and more meaningful interaction with people and, therefore, to a more enjoyable user experience. Therefore, the proposed paradigm and methodology are envisioned to contribute to making the prolonged assisted living of elderly people at home a more feasible and realistic task. In particular, this thesis proposes a set of methods for human presence detection, localisation and body pose tracking that are operating on the perception level of the paradigm. Also, the problem of having only limited visibility of a person from the on-board sensors of the robot is addressed by the proposed classifier fusion method that combines information from several types of sensors. A method for improved real-time human body pose tracking is also investigated. Additionally, a method for estimation of the multiple human tracks from noisy detections, as well as analysis of the computed human tracks for cognition about the social interactions within the social group, operating at the comprehension level of the robot’s situational awareness paradigm, is proposed. Finally, at the human-aware planning layer, a method that utilises the human related information, generated by the perception and comprehension layers to compute a minimally intrusive navigation path to a target person within a human group, is proposed. This method demonstrates how the improved human perception capabilities of the robot, through its judgement activity, ii ABSTRACT can be utilised by the highest level of the paradigm, i.e. the decision making layer, to achieve user friendly human-robot interactions. Overall, the research presented in this work, drawing on recent innovation in statistical learning, data fusion and optimisation methods, improves the overall situational awareness of the robot in regard to people with the main focus placed on human sensing capabilities of service robots. The improved overall situational awareness of the robot regarding people, as defined by the proposed paradigm, enables more meaningful human-robot interactions