Direct model based visual tracking and pose estimation using mutual information

International audienceThis paper deals with model-based pose estimation (or camera localization). The model is rendered as a virtual image and we propose a direct approach that takes into account the image as a whole. For this, we consider a sim- ilarity measure, the mutual information. Mutual information is a measure of the quantity of information shared by two signals (or two images in our case). Exploiting this measure allows our method to deal with different image modalities (real and synthetic). Furthermore, it handles occlusions and illu- mination changes. Results with synthetic (benchmark) and real image sequences, with static or mobile camera, demonstrate the robustness of the method and its ability to produce stable and precise pose estimations

An Intelligent Architecture for Legged Robot Terrain Classification Using Proprioceptive and Exteroceptive Data

In this thesis, we introduce a novel architecture called Intelligent Architecture for Legged Robot Terrain Classification Using Proprioceptive and Exteroceptive Data (iARTEC ) . The proposed architecture integrates different terrain characterization and classification with other robotic system components. Within iARTEC , we consider the problem of having a legged robot autonomously learn to identify different terrains. Robust terrain identification can be used to enhance the capabilities of legged robot systems, both in terms of locomotion and navigation. For example, a robot that has learned to differentiate sand from gravel can autonomously modify (or even select a different) path in favor of traversing over a better terrain. The same knowledge of the terrain type can also be used to guide a robot in order to avoid specific terrains. To tackle this problem, we developed four approaches for terrain characterization, classification, path planning, and control for a mobile legged robot. We developed a particle system inspired approach to estimate the robot footâ ground contact interaction forces. The approach is derived from the well known Bekkerâ s theory to estimate the contact forces based on its point contact model concepts. It is realistically model real-time 3-dimensional contact behaviors between rigid body objects and the soil. For a real-time capable implementation of this approach, its reformulated to use a lookup table generated from simple contact experiments of the robot foot with the terrain. Also, we introduced a short-range terrain classifier using the robot embodied data. The classifier is based on a supervised machine learning approach to optimize the classifier parameters and terrain it using proprioceptive sensor measurements. The learning framework preprocesses sensor data through channel reduction and filtering such that the classifier is trained on the feature vectors that are closely associated with terrain class. For the long-range terrain type prediction using the robot exteroceptive data, we present an online visual terrain classification system. It uses only a monocular camera with a feature-based terrain classification algorithm which is robust to changes in illumination and view points. For this algorithm, we extract local features of terrains using Speed Up Robust Feature (SURF). We encode the features using the Bag of Words (BoW) technique, and then classify the words using Support Vector Machines (SVMs). In addition, we described a terrain dependent navigation and path planning approach that is based on E* planer and employs a proposed metric that specifies the navigation costs associated terrain types. This generated path naturally avoids obstacles and favors terrains with lower values of the metric. At the low level, a proportional input-scaling controller is designed and implemented to autonomously steer the robot to follow the desired path in a stable manner. iARTEC performance was tested and validated experimentally using several different sensing modalities (proprioceptive and exteroceptive) and on the six legged robotic platform CREX. The results show that the proposed architecture integrating the aforementioned approaches with the robotic system allowed the robot to learn both robot-terrain interaction and remote terrain perception models, as well as the relations linking those models. This learning mechanism is performed according to the robot own embodied data. Based on the knowledge available, the approach makes use of the detected remote terrain classes to predict the most probable navigation behavior. With the assigned metric, the performance of the robot on a given terrain is predicted. This allows the navigation of the robot to be influenced by the learned models. Finally, we believe that iARTEC and the methods proposed in this thesis can likely also be implemented on other robot types (such as wheeled robots), although we did not test this option in our work

Dense RGB-D SLAM and object localisation for robotics and industrial applications

Dense reconstruction and object localisation are two critical steps in robotic and industrial applications. The former entails a joint estimation of camera egomotion and the structure of the surrounding environment, also known as Simultaneous Localisation and Mapping (SLAM), and the latter aims to locate the object in the reconstructed scenes. This thesis addresses the challenges of dense SLAM with RGB-D cameras and object localisation towards robotic and industrial applications. Camera drift is an essential issue in camera egomotion estimation. Due to the accumulated error in camera pose estimation, the estimated camera trajectory is inaccurate, and the reconstruction of the environment is inconsistent. This thesis analyses camera drift in SLAM under the probabilistic inference framework and proposes an online map fusion strategy with standard deviation estimation based on frame-to-model camera tracking. The camera pose is estimated by aligning the input image with the global map model, and the global map merges the information in the images by weighted fusion with standard deviation modelling. In addition, a pre-screening step is applied before map fusion to preclude the adverse effect of accumulated errors and noises on camera egomotion estimation. Experimental results indicated that the proposed method mitigates camera drift and improves the global consistency of camera trajectories. Another critical challenge for dense RGB-D SLAM in industrial scenarios is to handle mechanical and plastic components that usually have reflective and shiny surfaces. Photometric alignment in frame-to-model camera tracking tends to fail on such objects due to the inconsistency in intensity patterns of the images and the global map model. This thesis addresses this problem and proposes RSO-SLAM, namely a SLAM approach to reflective and shiny object reconstruction. RSO-SLAM adopts frame-to-model camera tracking and combines local photometric alignment and global geometric registration. This study revealed the effectiveness and excellent performance of the proposed RSO-SLAM on both plastic and metallic objects. In addition, a case study involving the cover of a electric vehicle battery with metallic surface demonstrated the superior performance of the RSO-SLAM approach in the reconstruction of a common industrial product. With the reconstructed point cloud model of the object, the problem of object localisation is tackled as point cloud registration in the thesis. Iterative Closest Point (ICP) is arguably the best-known method for point cloud registration, but it is susceptible to sub-optimal convergence due to the multimodal solution space. This thesis proposes the Bees Algorithm (BA) enhanced with the Singular Value Decomposition (SVD) procedure for point cloud registration. SVD accelerates the speed of the local search of the BA, helping the algorithm to rapidly identify the local optima. It also enhances the precision of the obtained solutions. At the same time, the global outlook of the BA ensures adequate exploration of the whole solution space. Experimental results demonstrated the remarkable performance of the SVD-enhanced BA in terms of consistency and precision. Additional tests on noisy datasets demonstrated the robustness of the proposed procedure to imprecision in the models

Navigation référencée multi-capteurs d'un robot mobile en environnement encombré

Dans ce travail, nous nous intéressons à la navigation référencée vision d'un robot mobile équipé d'une caméra dans un environnement encombré d'obstacles possiblement occultants. Pour réaliser cette tâche, nous nous sommes appuyés sur l'asservissement visuel 2D. Cette technique consiste à synthétiser une loi de commande basée sur les informations visuelles renvoyées par la caméra embarquée. Le robot atteint la situation désirée lorsque les projections dans l'image de l'amer d'intérêt, appelés indices visuels, atteignent des valeurs de consigne prédéfinies. La navigation par asservissement visuel 2D nécessite de s'intéresser à trois problèmes : garantir l'intégrité du robot vis-à-vis des obstacles, gérer les occultations des amers d'intérêts et réaliser de longs déplacements. Nos contributions portent sur les deux derniers problèmes mentionnés. Dans un premier temps nous nous sommes intéressés à l'estimation des indices visuels lorsque ceux-ci ne sont plus disponibles à cause d'une occultation. La profondeur étant un paramètre déterminant dans ce processus, nous avons développé une méthode permettant de l'estimer. Celle-ci est basée sur une paire prédicteur/correcteur et permet d'obtenir des résultats exploitables malgré la présence de bruits dans les mesures. Dans un second temps, nous nous sommes attachés à la réalisation de longs déplacements par asservissement visuel. Cette technique nécessitant de percevoir l'amer d'intérêt dès le début de la tâche, la zone de navigation est limitée par la portée de la caméra. Afin de relaxer cette contrainte, nous avons élaboré un superviseur que nous avons ensuite couplé à une carte topologique intégrant un ensemble d'amers caractéristiques de l'environnement. La tâche de navigation globale peut alors être décomposée sous la forme d'une séquence d'amers à atteindre successivement, la sélection et l'enchainement des mouvements nécessaires étant effectués au sein du superviseur. Les travaux ont été validés par le biais de simulations et d'expérimentations, démontrant la pertinence et l'efficacité de l'approche retenue.This work focuses on the navigation of a mobile robot equipped with a camera in a cluttered environment. To perform such a task, we propose to use the image based visual servoing (IBVS). This method consists in designing a control law using visual features provided by the camera. These features are defined by the projection of a characteristic landmark on the image plane. The IBVS based navigation requires to address three issues : the robot security with respect to the obstacles, the management of the occlusions and the long range navigation realization. Our contributions are mainly focused on the two last mentioned problems. First, we have dealt with the visual features estimation problem during occlusions. As the visual features depth is an important parameter in this process, we have developed a predictor/corrector pair able to estimate its value on-line. This method has provided nice results, even when the used measures are noisy. Second, we have considered the problem of performing a long range navigation with an IBVS. However, classically, using this kind of controller greatly limits the realizable displacement because the reference landmark must be seen from the beginning to the end of the mission. To relax this constraint, we have developed a topological map and a supervision algorithm which have then been coupled. The first one contains the most characteristic landmarks of the environment. Using this information, it is possible to divide the global navigation task into a sequence of landmarks which must be successively reached. The supervision algorithm then allows to select the right task at the right instant and to guarantee a smooth switch between the different motions. Our works have been validated by simulations and experimentations, demonstrating the efficiency of our approach

Special Issue on Robot Vision

International audienceThe International Journal of Robotics Research (IJRR) has a long history of publishing the state-of-the-art in the field of robotic vision. This is the fourth special issue devoted to the topic. Previous special issues were published in. In a closely related field was the special issue on Visual Servoing published in IJRR, 2003 (Volume 22, Nos 10–11). These issues nicely summarize the highlights and progress of the past 12 years of research devoted to the use of visual perception for robotics. Looking back across these issues we see perennial topics such as calibration; feature detection, description and matching; multi-view geometry; and filtering and prediction. Of course for robotic vision we have also seen many papers with a strong control focus and also a focus on high-speed operation. Perennial challenges over that period, perhaps still open problems, include robustness and vision-guided manipulation. Happily, many techniques have matured over this period and become an integral part of many robotic vision systems, for example visual odometry, visual Simultaneous Localization and Mapping (SLAM), visual place recognition and the fusion of vision with other sensors, most notably inertial sensors. This period has truly seen amazing technological change, not just the constant progress due to Moore's law but major innovations such as field-programmable gate arrays (FPGAs) and graphics processing units (GPUs), mobile computing architectures, low-cost high-performance inertial sensors and RGB-D sensors. Many of these have been driven by demand for consumer products such as smartphones and games, but have also provided a rich bounty for roboticists. The ready availability of capable low-cost off-the-shelf robotic platforms for domains such as underwater autonomous unmanned vehicles (AUVs), flying unmanned aerial vehicles (UAVs) and humanoid robots, all of which could usefully use vision sensors, is also helping to advance the field. Finally, the staple of all robotic vision systems, the camera, is evolving in very interesting directions. We now have cameras that are small, cheap and lightweight, that have progressive scan and global shutters, high dynamic range, high frame rate and wide fields of view obtained by catadioptrics or by multiple cameras with stitched imagery