Registration Combining Wide and Narrow Baseline Feature Tracking Techniques for Markerless AR Systems

Augmented reality (AR) is a field of computer research which deals with the combination of real world and computer generated data. Registration is one of the most difficult problems currently limiting the usability of AR systems. In this paper, we propose a novel natural feature tracking based registration method for AR applications. The proposed method has following advantages: (1) it is simple and efficient, as no man-made markers are needed for both indoor and outdoor AR applications; moreover, it can work with arbitrary geometric shapes including planar, near planar and non planar structures which really enhance the usability of AR systems. (2) Thanks to the reduced SIFT based augmented optical flow tracker, the virtual scene can still be augmented on the specified areas even under the circumstances of occlusion and large changes in viewpoint during the entire process. (3) It is easy to use, because the adaptive classification tree based matching strategy can give us fast and accurate initialization, even when the initial camera is different from the reference image to a large degree. Experimental evaluations validate the performance of the proposed method for online pose tracking and augmentation

Fast Scene Recognition and Camera Relocalisation for Wide Area Augmented Reality Systems

This paper focuses on online scene learning and fast camera relocalisation which are two key problems currently limiting the performance of wide area augmented reality systems. Firstly, we propose to use adaptive random trees to deal with the online scene learning problem. The algorithm can provide more accurate recognition rates than traditional methods, especially with large scale workspaces. Secondly, we use the enhanced PROSAC algorithm to obtain a fast camera relocalisation method. Compared with traditional algorithms, our method can significantly reduce the computation complexity, which facilitates to a large degree the process of online camera relocalisation. Finally, we implement our algorithms in a multithreaded manner by using a parallel-computing scheme. Camera tracking, scene mapping, scene learning and relocalisation are separated into four threads by using multi-CPU hardware architecture. While providing real-time tracking performance, the resulting system also possesses the ability to track multiple maps simultaneously. Some experiments have been conducted to demonstrate the validity of our methods

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Real-Time Occlusion Handling in Augmented Reality Based on an Object Tracking Approach

To produce a realistic augmentation in Augmented Reality, the correct relative positions of real objects and virtual objects are very important. In this paper, we propose a novel real-time occlusion handling method based on an object tracking approach. Our method is divided into three steps: selection of the occluding object, object tracking and occlusion handling. The user selects the occluding object using an interactive segmentation method. The contour of the selected object is then tracked in the subsequent frames in real-time. In the occlusion handling step, all the pixels on the tracked object are redrawn on the unprocessed augmented image to produce a new synthesized image in which the relative position between the real and virtual object is correct. The proposed method has several advantages. First, it is robust and stable, since it remains effective when the camera is moved through large changes of viewing angles and volumes or when the object and the background have similar colors. Second, it is fast, since the real object can be tracked in real-time. Last, a smoothing technique provides seamless merging between the augmented and virtual object. Several experiments are provided to validate the performance of the proposed method

Mobile Augmented Reality: Applications and Spe-cific Technical Issues

DOI: 10.1007/978-3-319-04702-7 Print ISBN: 978-3-319-04701-0 Online ISBN: 978-3-319-04702-7Although human's sedentary nature over time, his wish to travel the world remains as strong as ever. This paper discusses how imagery and Augmented Reality (AR) techniques can be of great help not only when discovering a new urban environment but also when observ-ing the evolution of the natural environment. The study is applied on Smartphone which is currently our most familiar device. Smart phone is utilized in our daily lives because it is low weight, ease of communications, and other valuable applications. In this chapter, we discuss technical issues of augmented reality especially with building recognition. Our building recog-nition method is based on an efficient hybrid approach, which combines the potentials of Speeded Up Robust Features (SURF) features points and lines. Our method relies on Approxi-mate Nearest Neighbors Search approach (ANNS). Although ANNS approaches are high speed, they are less accurate than linear algorithms. To assure an optimal trade-off between speed and accuracy, the proposed method performs a filtering step on the top of the ANNS. Finally, our method calls Hausdorff measure [15] with line models

Robust Estimation of Trifocal Tensors Using Natural Features for Augmented Reality Systems

Augmented reality deals with the problem of dynamically augmenting or enhancing the real world with computer generated virtual scenes. Registration is one of the most pivotal problems currently limiting AR applications. In this paper, a novel registration method using natural features based on online estimation of trifocal tensors is proposed. This method consists of two stages: offline initialization and online registration. Initialization involves specifying four points in two reference images respectively to build the world coordinate system on which a virtual object will be augmented. In online registration, the natural feature correspondences detected from the reference views are tracked in the current frame to build the feature triples. Then these triples are used to estimate the corresponding trifocal tensors in the image sequence by which the four specified points are transferred to compute the registration matrix for augmentation. The estimated registration matrix will be used as an initial estimate for a nonlinear optimization method that minimizes the actual residual errors based on the Levenberg-Marquardt (LM) minimization method, thus making the results more robust and stable. This paper also proposes a robust method for estimating the trifocal tensors, where a modified RANSAC algorithm is used to remove outliers. Compared with standard RANSAC, our method can significantly reduce computation complexity, while overcoming the disturbance of mismatches. Some experiments have been carried out to demonstrate the validity of the proposed approach

Improving Camera Pose Estimation for Indoor Marker-less Augmented Reality

Vision-based Augmented Reality (AR) techniques rely heavily on Computer Vision algorithms for most of their tasks. It is understood that these algorithms require numerous parameters to function and their values can affect their outputs. Oftentimes the results vary greatly when different parameters were used and as a result, the performance of the AR technique that utilises them varies accordingly as well. This paper aims at improving the performance of AR techniques by employing a novel algorithm to adjust the parameters automatically during runtime. More specifically, the proposed technique works on improving the camera pose estimation stage, arguably one of the most crucial stages, of such AR systems