Sensor Fusion for Intuitive Robot Programming

Abstract-Fusion of information from multiple sensors can greatly enhance the performance of human-machine interaction, especially in the intuitive robot programming. The methods aim to allow rapid teaching of robotic tasks in a safe and efficient manner. The techniques can reduce the setup time of a robotic system. This is crucial for SMEs (Small and Medium Enterprize) where the products in the manufacturing area are in small lot size but with high batch mix. The objective of this research is to fuse the information from a range sensor and a camera. An unique method using the surface constraint has been adopted for the calibration of the sensor fusion system. By taking the surface normal of a calibration board as the common feature, the transformation between the two coordinate systems can be formulated. The end result is a fused scene with both range and texture (color in this case) information. The range information will be used for the path generation for robotic tasks. On the other hand, the images captured by the camera together with the graphical user interface provide an user friendly interface platform for the user. As the two images have been fused, the operator can program a path for a robot to execute by 'point-and-click' on the user interface screen. Experimental results have shown that the new method of robot programming, with sensor fusion information, has improved the robotic teaching process by at least 90% as compared to the manual programming method using teaching pendant

Expressing Bayesian Fusion as a Product of Distributions: Application in Robotics

More and more fields of applied computer science involve fusion of multiple data sources, such as sensor readings or model decision. However incompleteness of the models prevent the programmer from having an absolute precision over their variables. Therefore bayesian framework can be adequate for such a process as it allows handling of uncertainty.We will be interested in the ability to express any fusion process as a product, for it can lead to reduction of complexity in time and space. We study in this paper various fusion schemes and propose to add a consistency variable to justify the use of a product to compute distribution over the fused variable. We will then show application of this new fusion process to localization of a mobile robot and obstacle avoidance

Expressing Bayesian Fusion as a Product of Distributions: Application to Randomized Hough Transform

Data fusion is a common issue of mobile robotics, computer assisted medical diagnosis or behavioral control of simulated character for instance. However data sources are often noisy, opinion for experts are not known with absolute precision, and motor commands do not act in the same exact manner on the environment. In these cases, classic logic fails to manage efficiently the fusion process. Confronting different knowledge in an uncertain environment can therefore be adequately formalized in the bayesian framework. Besides, bayesian fusion can be expensive in terms of memory usage and processing time. This paper precisely aims at expressing any bayesian fusion process as a product of probability distributions in order to reduce its complexity. We first study both direct and inverse fusion schemes. We show that contrary to direct models, inverse local models need a specific prior in order to allow the fusion to be computed as a product. We therefore propose to add a consistency variable to each local model and we show that these additional variables allow the use of a product of the local distributions in order to compute the global probability distribution over the fused variable. Finally, we take the example of the Randomized Hough Transform. We rewrite it in the bayesian framework, considering that it is a fusion process to extract lines from couples of dots in a picture. As expected, we can find back the expression of the Randomized Hough Transform from the literature with the appropriate assumptions

Enhanced online programming for industrial robots

The use of robots and automation levels in the industrial sector is expected to grow, and is driven by the on-going need for lower costs and enhanced productivity. The manufacturing industry continues to seek ways of realizing enhanced production, and the programming of articulated production robots has been identified as a major area for improvement. However, realizing this automation level increase requires capable programming and control technologies. Many industries employ offline-programming which operates within a manually controlled and specific work environment. This is especially true within the high-volume automotive industry, particularly in high-speed assembly and component handling. For small-batch manufacturing and small to medium-sized enterprises, online programming continues to play an important role, but the complexity of programming remains a major obstacle for automation using industrial robots. Scenarios that rely on manual data input based on real world obstructions require that entire production systems cease for significant time periods while data is being manipulated, leading to financial losses. The application of simulation tools generate discrete portions of the total robot trajectories, while requiring manual inputs to link paths associated with different activities. Human input is also required to correct inaccuracies and errors resulting from unknowns and falsehoods in the environment. This study developed a new supported online robot programming approach, which is implemented as a robot control program. By applying online and offline programming in addition to appropriate manual robot control techniques, disadvantages such as manual pre-processing times and production downtimes have been either reduced or completely eliminated. The industrial requirements were evaluated considering modern manufacturing aspects. A cell-based Voronoi generation algorithm within a probabilistic world model has been introduced, together with a trajectory planner and an appropriate human machine interface. The robot programs so achieved are comparable to manually programmed robot programs and the results for a Mitsubishi RV-2AJ five-axis industrial robot are presented. Automated workspace analysis techniques and trajectory smoothing are used to accomplish this. The new robot control program considers the working production environment as a single and complete workspace. Non-productive time is required, but unlike previously reported approaches, this is achieved automatically and in a timely manner. As such, the actual cell-learning time is minimal

An embedded implementation of Bayesian network robot programming methods

A wide variety of approaches exist for dealing with uncertainty in robotic reasoning, but relatively few consider the propagation of statistical information throughout an entire robotic system. The concept of Bayesian Robot Programming (BRP) involves making decisions based on inference into probability distributions, but can be complex and difficult to implement due to the number of priors and random variables involved. In this work, we apply Bayesian network structures to a modified BRP paradigm to provide intuitive structure and simplify the programming process. The use of discrete random variables in the network can allow high inference speeds, and an efficient programming toolkit suitable for use on embedded platforms has been developed for use on mobile robots. A simple example of navigational reasoning for a small mobile robot is provided as an example of how such a network can be used for probabilistic decisional programming

Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

A major challenge for the realization of intelligent robots is to supply them with cognitive abilities in order to allow ordinary users to program them easily and intuitively. One way of such programming is teaching work tasks by interactive demonstration. To make this effective and convenient for the user, the machine must be capable to establish a common focus of attention and be able to use and integrate spoken instructions, visual perceptions, and non-verbal clues like gestural commands. We report progress in building a hybrid architecture that combines statistical methods, neural networks, and finite state machines into an integrated system for instructing grasping tasks by man-machine interaction. The system combines the GRAVIS-robot for visual attention and gestural instruction with an intelligent interface for speech recognition and linguistic interpretation, and an modality fusion module to allow multi-modal task-oriented man-machine communication with respect to dextrous robot manipulation of objects.Comment: 7 pages, 8 figure

Bayesian Programming Multi-Target Tracking: an Automotive Application

A prerequisite to the design of future Advanced Driver Assistance Systems for cars is a sensing system providing all the information required for high-level driving assistance tasks. In particular, target tracking is still challenging in urban trafc situations, because of the large number of rapidly maneuvering targets. The goal of this paper is to present an original way to perform target position and velocity, based on the occupancy grid framework. The main interest of this method is to avoid the decision problem of classical multi-target tracking algorithms. Obtained occupancy grids are combined with danger estimation to perform an elementary task of obstacle avoidance with an electric car

Accurate 3D maps from depth images and motion sensors via nonlinear Kalman filtering

This paper investigates the use of depth images as localisation sensors for 3D map building. The localisation information is derived from the 3D data thanks to the ICP (Iterative Closest Point) algorithm. The covariance of the ICP, and thus of the localization error, is analysed, and described by a Fisher Information Matrix. It is advocated this error can be much reduced if the data is fused with measurements from other motion sensors, or even with prior knowledge on the motion. The data fusion is performed by a recently introduced specific extended Kalman filter, the so-called Invariant EKF, and is directly based on the estimated covariance of the ICP. The resulting filter is very natural, and is proved to possess strong properties. Experiments with a Kinect sensor and a three-axis gyroscope prove clear improvement in the accuracy of the localization, and thus in the accuracy of the built 3D map.Comment: Submitted to IROS 2012. 8 page