A fast and robust hand-driven 3D mouse

The development of new interaction paradigms requires a natural interaction. This means that people should be able to interact with technology with the same models used to interact with everyday real life, that is through gestures, expressions, voice. Following this idea, in this paper we propose a non intrusive vision based tracking system able to capture hand motion and simple hand gestures. The proposed device allows to use the hand as a "natural" 3D mouse, where the forefinger tip or the palm centre are used to identify a 3D marker and the hand gesture can be used to simulate the mouse buttons. The approach is based on a monoscopic tracking algorithm which is computationally fast and robust against noise and cluttered backgrounds. Two image streams are processed in parallel exploiting multi-core architectures, and their results are combined to obtain a constrained stereoscopic problem. The system has been implemented and thoroughly tested in an experimental environment where the 3D hand mouse has been used to interact with objects in a virtual reality application. We also provide results about the performances of the tracker, which demonstrate precision and robustness of the proposed syste

Camera distortion self-calibration using the plumb-line constraint and minimal Hough entropy

In this paper we present a simple and robust method for self-correction of camera distortion using single images of scenes which contain straight lines. Since the most common distortion can be modelled as radial distortion, we illustrate the method using the Harris radial distortion model, but the method is applicable to any distortion model. The method is based on transforming the edgels of the distorted image to a 1-D angular Hough space, and optimizing the distortion correction parameters which minimize the entropy of the corresponding normalized histogram. Properly corrected imagery will have fewer curved lines, and therefore less spread in Hough space. Since the method does not rely on any image structure beyond the existence of edgels sharing some common orientations and does not use edge fitting, it is applicable to a wide variety of image types. For instance, it can be applied equally well to images of texture with weak but dominant orientations, or images with strong vanishing points. Finally, the method is performed on both synthetic and real data revealing that it is particularly robust to noise.Comment: 9 pages, 5 figures Corrected errors in equation 1

COGNITO - captura, reconhecimento e visualização de atividades manuais complexas

Neste artigo curto apresenta-se um sistema capaz de automaticamente capturar, reconhecer e visualizar atividades motoras humanas, em diferentes contextos, mas com aplicação prática, por exemplo, na implementação de manuais virtuais 3D ou em vídeo-jogos de nova geração, passando pelos simuladores de treino. Este trabalho tem vindo a ser desenvolvido em consórcio internacional, no contexto de um projeto apoiado pela comissão europeia (COGNITO), e tem-se centrado na captura, análise, armazenamento e visualização 3D, com recurso a tecnologias de realidade virtual e aumentada, de tarefas manuais complexas, executadas em ambiente industrial. O sistema é composto por quatro módulos principais: uma rede de sensores colocados no corpo, uma unidade de captura dos movimentos e ferramentas utilizadas, uma componente de aprendizagem não supervisionada e uma componente gráfica capaz de fazer a apresentação de informação ao utilizador através de um módulo de realidade aumentada (RA). Este artigo apresenta o sistema global e a sua arquitetura, referindo com mais detalhe os desenvolvimentos efetuados para a componente gráfica

Movement flow-based visual servoing to track moving objects

The purpose of this paper is to describe a new method for tracking trajectories specified in the image space. This method, called movement flow-based visual servoing system, is applied to an eye-in-hand robot and it is shown that it allows the correct tracking of a trajectory, not only in the image but also in the 3-D space. This method is also extended to the case in which the object from which the features are extracted, is in motion. To do so, the estimations obtained, using several Kalman filters, are integrated in the control action

Augmented reality visualization and edition of cognitive workflow capturing

The aim of the COGNITO project is to design a personal assistance system, in which Augmented Reality (AR) is used to support users in task solving and manipulation of objects. Due to its sensing and learning capability, the COGNITO system automatically creates workflow references by observing a shown task in learning mode. After the workflow has been learnt, the system can be run in playback mode, in which it explains the previously learnt task to the operator. The system compares the user activity in real-time with the workflow reference and provides adequate feedback. This system is composed by four main modules. This paper focuses on the last module – the 3D graphics engine – which is the basis to the development of both the augmented and the virtual reality player. Additionally, it also presents the template of actions editor which is an editing tool that enables non-programmers and non-3D-experts to prepare and accompany the composition of visualizations for end-users

Robust extended Kalman filtering for camera pose tracking using 2D to 3D lines correspondences

International audienceIn this paper we present a new robust camera pose estimation approach based on 3D lines tracking. We used an Extended Kalman Filter (EKF) to incrementally update the camera pose in real-time. The principal contributions of our method includes first, the expansion of the RANSAC scheme in order to achieve a robust matching algorithm that associates 2D edges from the image with the 3D line segments from the input model. And second, a new framework for camera pose estimation using 2D-3D straight-lines within an EKF. Experimental results on real image sequences are presented to evaluate the performances and the feasibility of the proposed approach

Real-time model-based slam using line segments

Abstract. Existing monocular vision-based SLAM systems favour interest point features as landmarks, but these are easily occluded and can only be reliably matched over a narrow range of viewpoints. Line segments offer an interesting alternative, as line matching is more stable with respect to viewpoint changes and lines are robust to partial occlusion. In this paper we present a model-based SLAM system that uses 3D line segments as landmarks. Unscented Kalman filters are used to initialise new line segments and generate a 3D wireframe model of the scene that can be tracked with a robust model-based tracking algorithm. Uncertainties in the camera position are fed into the initialisation of new model edges. Results show the system operating in real-time with resilience to partial occlusion. The maps of line segments generated during the SLAM process are physically meaningful and their structure is measured against the true 3D structure of the scene.