Registration Combining Wide and Narrow Baseline Feature Tracking Techniques for Markerless AR Systems

Augmented reality (AR) is a field of computer research which deals with the combination of real world and computer generated data. Registration is one of the most difficult problems currently limiting the usability of AR systems. In this paper, we propose a novel natural feature tracking based registration method for AR applications. The proposed method has following advantages: (1) it is simple and efficient, as no man-made markers are needed for both indoor and outdoor AR applications; moreover, it can work with arbitrary geometric shapes including planar, near planar and non planar structures which really enhance the usability of AR systems. (2) Thanks to the reduced SIFT based augmented optical flow tracker, the virtual scene can still be augmented on the specified areas even under the circumstances of occlusion and large changes in viewpoint during the entire process. (3) It is easy to use, because the adaptive classification tree based matching strategy can give us fast and accurate initialization, even when the initial camera is different from the reference image to a large degree. Experimental evaluations validate the performance of the proposed method for online pose tracking and augmentation

Online Estimation of Trifocal Tensors for Augmenting Live Video

We propose a method to augment live video based on the tracking of natural features, and the online estimation of the trinocular geometry. Previous without-marker approaches require the computation of camera pose to render virtual objects. The strength of our proposed method is that it doesn't require tracking of camera pose, and exploits the usual advantages of marker-based approaches for a fast implementation. A 3-view AR system is used to demonstrate our approach. It consists of an uncalibrated camera that moves freely inside the scene of interest, and of three reference frames taken at the time of system initialization. As the camera is moving, image features taken from an initial triplet set are tracked throughout the video sequence. And the trifocal tensor associated with each frame is estimated online. With this tensor, the square pattern that was visible in the reference frames is transferred to the video. This invisible pattern is then used by the ARToolkit to embed virtual objects.Nous proposons une m\ue9thode d'augmentation de la vid\ue9o directe, bas\ue9e sur le suivi de caract\ue9ristiques naturelles et l'estimation en direct de la g\ue9om\ue9trie trinoculaire. Les m\ue9thodes ant\ue9rieures sans marqueurs obligent \ue0 calculer la pose de la cam\ue9ra pour le rendu d'objets virtuels. Le point fort de notre m\ue9thode est qu'elle n'exige pas le suivi de la pose de la cam\ue9ra et exploite les avantages usuels des m\ue9thodes \ue0 marqueurs pour acc\ue9l\ue9rer la mise en oeuvre. Un syst\ue8me AR \ue0 trois vues est utilis\ue9 pour faire la d\ue9monstration de notre m\ue9thode. Il se compose d'une cam\ue9ra non \ue9talonn\ue9e qui se d\ue9place librement dans la zone d'int\ue9r\ueat, et de trois cadres de r\ue9f\ue9rence saisis lors de l'initialisation du syst\ue8me. Lors du d\ue9placement de la cam\ue9ra, les caract\ue9ristiques d'image saisies \ue0 partir d'un ensemble de triplets initial sont suivies pendant toute la dur\ue9e de la s\ue9quence vid\ue9o. De plus, le tenseur trifocal associ\ue9 \ue0 chaque cadre est estim\ue9 en direct. \uc0 l'aide de ce tenseur, le motif carr\ue9 qui \ue9tait visible dans les cadres de r\ue9f\ue9rence est transf\ue9r\ue9 \ue0 la vid\ue9o. Ce motif invisible est ensuite utilis\ue9 par l'ARToolkit pour incorporer des objets virtuels.NRC publication: Ye

Remote Collaborative BIM-based Mixed Reality Approach for Supporting Facilities Management Field Tasks

Facilities Management (FM) day-to-day tasks require suitable methods to facilitate work orders and improve performance by better collaboration between the office and the field. Building Information Modeling (BIM) provides opportunities to support collaboration and to improve the efficiency of Computerized Maintenance Management Systems (CMMSs) by sharing building information between different applications/users throughout the lifecycle of the facility. However, manual retrieval of building element information can be challenging and time consuming for field workers during FM operations. Mixed Reality (MR) is a visualization technique that can be used to improve the visual perception of the facility by superimposing 3D virtual objects and textual information on top of the view of real-world building objects. The objectives of this research are: (1) investigating an automated method to capture and record task-related data (e.g., defects) with respect to a georeferenced BIM model and share them directly with the remote office based on the field worker point of view in mobile situations; (2) investigating the potential of using MR, BIM, and sensory data for FM tasks to provide improved visualization and perception that satisfy the needs of the facility manager at the office and the field workers with less visual and mental disturbance; and (3) developing an effective method for interactive visual collaboration to improve FM field tasks. This research discusses the development of a collaborative BIM-based MR approach to support facilities field tasks. The research framework integrates multisource facilities information, BIM models, and hybrid tracking in an MR-based setting to retrieve information based on time (e.g., inspection schedule) and the location of the field worker, visualize inspection and maintenance operations, and support remote collaboration and visual communication between the field worker and the manager at the office. The field worker uses an Augmented Reality (AR) application installed on his/her tablet. The manager at the office uses an Immersive Augmented Virtuality (IAV) application installed on a desktop computer. Based on the field worker location, as well as the inspection or maintenance schedule, the field worker is assigned work orders and instructions from the office. Other sensory data (e.g., infrared thermography) can provide additional layers of information by augmenting the actual view of the field worker and supporting him/her in making effective decisions about existing and potential problems while communicating with the office in an Interactive Virtual Collaboration (IVC) mode. The contributions of this research are (1) developing a MR framework for facilities management which has a field AR module and an office IAV module. These modules can be used independently or combined using remote IVC, (2) developing visualization methods for MR including the virtual hatch and multilayer views to enhance visual depth and context perception, (3) developing methods for AR and IAV modeling including BIM-based data integration and customization suitable for each MR method, and (4) enhancing indoor tracking for AR FM systems by developing a hybrid tracking method. To investigate the applicability of the research method, a prototype system called Collaborative BIM-based Markerless Mixed Reality Facility Management System (CBIM3R-FMS) is developed and tested in a case study. The usability testing and validation show that the proposed methods have high potential to improve FM field tasks