Autonomous Shopping Cart Platform for People with Mobility Impairments

International audienceProviding a platform able to interact with a spe- cific user is a challenging problem for assistance technologies. Among the many platforms accomplishing this task, we address the problem of designing an autonomous shopping cart. We assume that the shopping cart is set-up on a unicycle-like robot endowed with two sensors: an RGB-D camera and a planar laser range finder. To combine the information from these two sensors, a data fusion algorithm has been developed using a particle filter, augmented with a k-clustering step to extract person estimations. The problem of stabilizing the robot's position at a fixed distance from the user has been solved through classical control design. Results on a real mobile platform verify the effectiveness of the approach here proposed

Robot interactive learning through human assistance

Peer ReviewedPostprint (author’s final draft

A multi-modal person perception framework for socially interactive mobile service robots

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot’s user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments

Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities

Abstract — We present an approach to laser-based people tracking using a multi-hypothesis tracker that detects and tracks legs separately with Kalman filters, constant velocity motion models, and a multi-hypothesis data association strategy. People are defined as high-level tracks consisting of two legs that are found with little model knowledge. We extend the data association so that it explicitly handles track occlusions in addition to detections and deletions. Additionally, we adapt the corresponding probabilities in a situation-dependent fashion so as to reflect the fact that legs frequently occlude each other. Experimental results carried out with a mobile robot illustrate that our approach can robustly and efficiently track multiple people even in situations of high levels of occlusion. I

Target tracking using laser range finder with occlusion

Mestrado em Engenharia MecânicaEste trabalho apresenta uma técnica para a detecção e seguimento de múltiplos alvos móveis usando um sensor de distâncias laser em situações de forte oclusão. O processo inicia-se com a aplicação de filtros temporais aos dados em bruto de modo a eliminar o ruído do sensor seguindo-se de uma segmentação em várias fases com o objectivo de contornar o problema da oclusão. Os segmentos obtidos representam objectos presentes no ambiente. Para cada segmento um ponto representativo da sua posição no mundo é calculado, este ponto é definido de modo a ser relativamente invariante à rotação e mudança de forma do objecto. Para fazer o seguimento de alvos uma lista de objectos a seguir é mantida, todos os objectos visíveis são associados a objectos desta lista usando técnicas de procura baseadas na previsão do movimento dos objectos. Uma zona de procura de forma elíptica é definida para cada objecto da lista sendo nesta zona que se dará a associação. A previsão do movimento é feita com base em dois modelos de movimento, um de velocidade constante e um de aceleração constante e com aplicação de filtros de Kalman. O algoritmo foi testado em diversas condições reais e mostrou-se robusto e eficaz no seguimento de pessoas mesmo em situações de extensa oclusão. ABSTRACT: In this work a technique for the detection and tracking of multiple moving targets in situations of strong occlusion using a laser rangefinder is presented. The process starts by the application of temporal filters to the raw data in order to remove noise followed by a multi phase segmentation with the goal of overcoming occlusions. The resulting segments represent objects in the environment. For each segment a representative point is defined; this point is calculated to better represent the object while keeping some invariance to rotation and shape changes. In order to perform the tracking, a list of objects to follow is maintained; all visible objects are associated with objects from this list using search techniques based on the predicted motion of objects. A search zone shaped as an ellipse is defined for each object; it is in this zone that the association is preformed. The motion prediction is based in two motion models, one with constant velocity and the other with constant acceleration and in the application of Kalman filters. The algorithm was tested in diverse real conditions and shown to be robust and effective in the tracking of people even in situations of long occlusions

Contextual Human Trajectory Forecasting within Indoor Environments and Its Applications

A human trajectory is the likely path a human subject would take to get to a destination. Human trajectory forecasting algorithms try to estimate or predict this path. Such algorithms have wide applications in robotics, computer vision and video surveillance. Understanding the human behavior can provide useful information towards the design of these algorithms. Human trajectory forecasting algorithm is an interesting problem because the outcome is influenced by many factors, of which we believe that the destination, geometry of the environment, and the humans in it play a significant role. In addressing this problem, we propose a model to estimate the occupancy behavior of humans based on the geometry and behavioral norms. We also develop a trajectory forecasting algorithm that understands this occupancy and leverages it for trajectory forecasting in previously unseen geometries. The algorithm can be useful in a variety of applications. In this work, we show its utility in three applications, namely person re-identification, camera placement optimization, and human tracking. Experiments were performed with real world data and compared to state-of-the-art methods to assess the quality of the forecasting algorithm and the enhancement in the quality of the applications. Results obtained suggests a significant enhancement in the accuracy of trajectory forecasting and the computer vision applications.Computer Science, Department o

Seguimento ativo de agentes dinâmicos multivariados usando informação vectorial

Doutoramento em Engenharia MecânicaO objeto principal da presente tese é o estudo de sistemas avançados de segurança, no âmbito da segurança automóvel, baseando-se na previsão de movimentos e ações dos agentes externos. Esta tese propõe tratar os agentes como entidades dinâmicas, com motivações e constrangimentos próprios. Apresenta-se, para tal, novas técnicas de seguimento dos referidos agentes levando em linha de conta as suas especificidades. Em decorrência, estuda-se dedicadamente dois tipos de agentes: os veículos automóveis e os peões. Quanto aos veículos automóveis, propõe-se melhorar a capacidade de previsão de movimentos recorrendo a modelos avançados que representam corretamente os constrangimentos presentes nos veículos. Assim, foram desenvolvidos algoritmos avançados de seguimento de agentes com recurso a modelos de movimento não holonómicos. Estes algoritmos fazem uso de dados vectoriais de distância fornecidos por sensores de distância laser. Para os peões, devido à sua complexidade (designadamente a ausência de constrangimentos de movimentos) propõe-se que a análise da sua linguagem corporal permita detetar atempadamente possíveis intenções de movimentos. Assim, foram desenvolvidos algoritmos de perceção de pose de peões adaptados ao campo da segurança automóvel com recurso a uso de dados de distâncias 3D obtidos com uma câmara stereo. De notar que os diversos algoritmos foram testados em experiências realizadas em ambiente real.The main topic of this thesis is the study of advanced safety systems, in the field of automotive safety, based on the prediction of the movement and actions of external agents. This thesis proposes to treat the agents as dynamic entities with their own motivations as constraints. As so, new target tracking techniques are proposed taking into account the targets’ specificities. Therefore, two different types of agents are dedicatedly studied: automobile vehicles and pedestrians. For the automobile vehicles, a technique to improve motion prediction by the use of advanced motion models is proposed, these models will correctly represent the constrains that exist in this kind of vehicle. With this goal, advanced target tracking algorithms coupled with nonholonomic motion models were developed. These algorithms make use of vectorial range data supplied by laser range sensors. Concerning the pedestrians, due to the problem complexity (mainly due to the lack of any specific motion constraint), it is proposed that the analysis of the pedestrians body language will allow to detected early the pedestrian intentions and movements. As so, pedestrian pose estimation algorithms specially adapted to the field of automotive safety were developed; these algorithms use 3D point cloud data obtained with a stereo camera. The various algorithms were tested in experiments conducted in real conditions

Multi-sensor multi-person tracking on a mobile robot platform

Service robots need to be aware of persons in their vicinity in order to interact with them. People tracking enables the robot to perceive persons by fusing the information of several sensors. Most robots rely on laser range scanners and RGB cameras for this task. The thesis focuses on the detection and tracking of heads. This allows the robot to establish eye contact, which makes interactions feel more natural. Developing a fast and reliable pose-invariant head detector is challenging. The head detector that is proposed in this thesis works well on frontal heads, but is not fully pose-invariant. This thesis further explores adaptive tracking to keep track of heads that do not face the robot. Finally, head detector and adaptive tracker are combined within a new people tracking framework and experiments show its effectiveness compared to a state-of-the-art system