An active stereo vision-based learning approach for robotic tracking, fixating and grasping control

In this paper, an active stereo vision-based learning approach is proposed for a robot to track, fixate and grasp an object in unknown environments. First, the functional mapping relationships between the joint angles of the active stereo vision system and the spatial representations of the object are derived and expressed in a three-dimensional workspace frame. Next, the self-adaptive resonance theory-based neural networks and the feedforward neural networks are used to learn the mapping relationships in a self-organized way. Then, the approach is verified by simulation using the models of an active stereo vision system which is installed in the end-effector of a robot. Finally, the simulation results confirm the effectiveness of the present approach. <br /

Automating Active Stereo Vision Calibration Process with Cobots

Collaborative robots help the academia and industry to accelerate the work by introducing a new concept of cooperation between human and robot. In this paper, a calibration process for an active stereo vision rig has been automated to accelerate the task and improve the quality of the calibration. As illustrated in this paper by using Baxter Robot, the calibration process has been done faster by three times in comparison to the manual calibration that depends on the human. The quality of the calibration was improved by 120% when the Baxter robot was used

Object tracking with stereo vision

A real-time active stereo vision system incorporating gaze control and task directed vision is described. Emphasis is placed on object tracking and object size and shape determination. Techniques include motion-centroid tracking, depth tracking, and contour tracking

3D image acquisition and processing with high continuous data throughput for human-machine-interaction and adaptive manufacturing

Many applications in industrial environment are able to detect structures in measurement volumes from macroscopic to microscopic range. One way to process the resulting image data and to calculate three-dimensional (3D) images is the use of active stereo vision technology. In this context, one of the main challenges is to deal with the permanently increasing amount of data. This paper aims to describes methods for handling the required data throughput for 3D image acquisition in active stereo vision systems. Thus, the main focus is on implementing the steps of the image processing chain on re-configurable hardware. Among other things, this includes the pre-processing step with the correction of distortion and rectification of incoming image data. Therefore, the approach uses the offline pre-calculation of rectification maps. Furthermore, with the aid of the rectified maps, each image is directly rectified during the image acquisition. Afterwards, an FPGA and GPU-based approach is selected for optimal performance of stereo matching and 3D point calculation

Implementation of Fuzzy Decision Based Mobile Robot Navigation Using Stereo Vision

AbstractIn this article, we discuss implementation phases for an autonomous navigation of a mobile robotic system using SLAM data, while relying on the features of learned navigation maps. The adopted SLAM based learned maps, was relying entirely on an active stereo vision for observing features of the navigation environment. We show the framework for the adopted lower-level software coding, that was necessary once a vision is used for multiple purposes, distance measurements, and obstacle discovery. In addition, the article describes the adopted upper-level of system intelligence using fuzzy based decision system. The proposed map based fuzzy autonomous navigation was trained from data patterns gathered during numerous navigation tasks. Autonomous navigation was further validated and verified on a mobile robot platform

Design of an Active Stereo Vision 3D Scene Reconstruction System Based on the Linear Position Sensor Module

Active vision systems and passive vision systems currently exist for three-dimensional (3D) scene reconstruction. Active systems use a laser that interacts with the scene. Passive systems implement stereo vision, using two cameras and geometry to reconstruct the scene. Each type of system has advantages and disadvantages in resolution, speed, and scene depth. It may be possible to combine the advantages of both systems as well as new hardware technologies such as position sensitive devices (PSDs) and field programmable gate arrays (FPGAs) to create a real-time, mid-range 3D scene reconstruction system. Active systems usually reconstruct long-range scenes so that a measurable amount of time can pass for the laser to travel to the scene and back. Passive systems usually reconstruct close-range scenes but must overcome the correspondence problem. If PSDs are placed in a stereo vision configuration and a laser is directed at the scene, the correspondence problem can be eliminated. The laser can scan the entire scene as the PSDs continually pick up points, and the scene can be reconstructed. By eliminating the correspondence problem, much of the computation time of stereo vision is removed, allowing larger scenes, possibly at mid-range, to be modeled. To give good resolution at a real-time frame rate, points would have to be recorded very quickly. PSDs are analog devices that give the position of a light spot and have very fast response times. The cameras in the system can be replaced by PSDs to help achieve real- time refresh rates and better resolution. A contribution of this thesis is to design a 3D scene reconstruction system by placing two PSDs in a stereo vision configuration and to use FPGAs to perform calculations to achieve real-time frame rates of mid-range scenes. The linear position sensor module (LPSM) made by Noah Corp is based on a PSD and outputs a position in terms of voltage. The LPSM is characterized for this application by testing it with different power lasers while also varying environment variables such as background light, scene type, and scene distance. It is determined that the LPSM is sensitive to red wavelength lasers. When the laser is reflected off of diffuse surfaces, the laser must output at least 500 mW to be picked up by the LPSM and the scene must be within 15 inches, or the power intensity will not meet the intensity requirements of the LPSM. The establishment of these performance boundaries is a contribution of the thesis along with characterizing and testing the LPSM as a vision sensor in the proposed scene reconstruction system. Once performance boundaries are set, the LPSM is used to model calibrated objects. LPSM sensitivity to power intensity changes seems to cause considerable error. The change in power appears to be a function of depth due to the dispersion of the laser beam. The model is improved by using a correction factor to find the position of the light spot. Using a better-focused laser may improve the results. Another option is to place two PSDs in the same configuration and test to see whether the intensity problem is intrinsic to all PSDs or if the problem is unique to the LPSM

A Low Cost Virtual Reality Human Computer Interface for CAD Model Manipulation

Interactions with high volume complex three-dimensional data using traditional two-dimensional computer interfaces have, historically, been inefficient and restrictive. However, during the past decade, virtual reality (VR) has presented a new paradigm for human-computer interaction. This paper presents a VR human-computer interface system, which aims at providing a solution to the human-computer interaction problems present in today’s computer-aided design (CAD) software applications. A data glove device is used as a 3D interface for CAD model manipulation in a virtual design space. To make the visualization more realistic, real-time active stereo vision is provided using LCD shutter glasses. To determine the ease of use and intuitiveness of the interface, a human subject study was conducted for performing standard CAD manipulation tasks. Analysis results and technical issues are also presented and discussed

Fast heuristic method to detect people in frontal depth images

This paper presents a new method for detecting people using only depth images captured by a camera in a frontal position. The approach is based on first detecting all the objects present in the scene and determining their average depth (distance to the camera). Next, for each object, a 3D Region of Interest (ROI) is processed around it in order to determine if the characteristics of the object correspond to the biometric characteristics of a human head. The results obtained using three public datasets captured by three depth sensors with different spatial resolutions and different operation principle (structured light, active stereo vision and Time of Flight) are presented. These results demonstrate that our method can run in realtime using a low-cost CPU platform with a high accuracy, being the processing times smaller than 1 ms per frame for a 512 × 424 image resolution with a precision of 99.26% and smaller than 4 ms per frame for a 1280 × 720 image resolution with a precision of 99.77%