Robust tracking for augmented reality

In this paper a method for improving a tracking algorithm in an augmented reality application is presented. This method addresses several issues to this particular application, like marker-less tracking and color constancy with low quality cameras, or precise tracking with real-time constraints. Due to size restrictions some of the objects are tracked using color information. To improve the quality of the detection, a color selection scheme is proposed to increase color distance between different objects in the scene. Moreover, a new color constancy method based in a diagonal-offset model and k-means is presented. Finally, some real images are used to show the improvement with this new method.Universidad de Málaga, Campus de Excelencia Internacional Andalucía Tech. Ministry of Education of Spain (TIN2013-42253P), Junta de Andalucía of Spain (TIC-1692

Recognition of Human Periodic Movements From Unstructured Information Using A Motion-based Frequency Domain Approach

Feature-based motion cues play an important role in biological visual perception. We present a motion-based frequency-domain scheme for human periodic motion recognition. As a baseline study of feature based recognition we use unstructured feature-point kinematic data obtained directly from a marker-based optical motion capture (MoCap) system, rather than accommodate bootstrapping from the low-level image processing of feature detection. Motion power spectral analysis is applied to a set of unidentified trajectories of feature points representing whole body kinematics. Feature power vectors are extracted from motion power spectra and mapped to a low dimensionality of feature space as motion templates that offer frequency domain signatures to characterise different periodic motions. Recognition of a new instance of periodic motion against pre-stored motion templates is carried out by seeking best motion power spectral similarity. We test this method through nine examples of human periodic motion using MoCap data. The recognition results demonstrate that feature-based spectral analysis allows classification of periodic motions from low-level, un-structured interpretation without recovering underlying kinematics. Contrasting with common structure-based spatio-temporal approaches, this motion-based frequency-domain method avoids a time-consuming recovery of underlying kinematic structures in visual analysis and largely reduces the parameter domain in the presence of human motion irregularities

Enhancing awareness on green consumption of electronic devices: the application of Augmented Reality

During the past few decades, the adoption of electronic devices (e.g. television sets, computers, etc.) in both households and businesses has brought immense benefits to human beings. However, their increasing utilization adversely contribute to a rise in energy consumption, electronic waste and is even detrimental to human health and to the environment. As such, it is essential that human beings undertake actions to reduce the risks and damages caused by these devices to the environment. Concurrently, studies have revealed that a lack of knowledge is considered amongst the important barriers to the adoption of environmentally sustainable actions. One emerging technology that has shown to foster engagement, improve understanding and provide a unique learning experience is Augmented Reality (AR). However, this technology has not been exploited to improve awareness on environmentally sustainable use of electronic devices. As such, this paper investigates the application of AR to improve awareness on green consumption of electronic devices by delving into key research questions pertaining to technology acceptance, knowledge gains and associated barriers to adoption. In this endeavor, an AR-based mobile application was implemented to enable individuals learn energy consumption of electronic devices being used at home and offices. The research questions were answered following application of the Technology Acceptance Model and by conducting multiple-choice based tests. Results principally indicated a positive correlation between AR and learning enhancement on green consumption of electronic devices

Dynamic Pose Tracking Performance Evaluation of HTC Vive Virtual Reality System

Virtual reality tracking devices are rapidly becoming the go-to system for cost-effective motion tracking solutions across different communities such as robotics, biomechanics, sports, rehabilitation, motion simulators, etc. This article focuses on the spatial tracking performance of HTC Vive's lighthouse tracking system (VLTS) devices (tracker, controller, and head mount display). A comprehensive literature survey on the performance analysis of VLTS on the various aspects is presented along with its shortcomings in terms of spatial tracking evaluation. The two key limitations have been identified: in static cases, there is a lack of standard procedures and criteria, and in dynamic cases, the entire study of spatial tracking. We address the first by assessing VLTS using the optical tracking system standard specified by ASTM International, and the latter by revising the standards to determine the upper-velocity limit for reliable tracking. The findings are substantiated with the trajectories of human wrist motion. Each evaluation's results are systematically analyzed with statistical hypothesis tests and criteria fulfillment. Comau NS16, an industrial serial robot, was used as the ground truth motion generator due to its repeatability and 6 degrees of workspace freedom. One of the major reasons for not having more generalized spatial tracking studies is that the tracking performance heavily depends on the configurations of the setup, work volume, environment, etc. Thus, the guidelines for configuring VLTS and the approach adapted from ASTM standards for evaluating VLTS for custom applications using our reported findings for both static and dynamic cases are included in the appendix

Sviluppo di un sistema ottico per il controllo di posizione di uno scanner 3D

L’ambito scientifico in cui si inserisce questo lavoro è il processo di rilievo di forme tridimensionali, meglio conosciuto con il termine di Reverse Engineering. Scopo principale di questa tesi è stato lo studio e lo sviluppo di un sistema per il controllo di posizione di uno scanner ottico 3D con il fine di ottenere un riallineamento automatizzato delle nuvole di punti provenienti da diversi punti di osservazione. Un sistema del genere sarà molto utile nella digitalizzazione di oggetti di notevoli dimensioni privi di particolari caratteristiche geometriche. Dopo una prima parte introduttiva sull' analisi delle varie tipologie di tracking si è deciso di usare un sistema ottico indipendente per visionare e controllare la posizione dello scanner. La scelta del tipo di tracking è ricaduta su una sistema ottico principalmente per i vantaggi chetale tecnologia ha in applicazioni di tipo industriale e quindi nell'utilizzo in ambienti non controllati. Il sistema ottico ideato e realizzato presso i laboratori del “Dipartimento di Ingegneria Meccanica, Nucleare e della Produzione dell'Università di Pisa” è costituito da un sistema di visione 3D basato sul principio di visione stereo; in particolare esso è costituito da due telecamere che munite di opportuni filtri IR inferiscono la posizione di alcuni punti fiduciali (Marker IR) solidali allo scanner. La conoscenza della posizione dei Marker IR permette il calcolo delle matrici di rototraslazione utili per il riallineamento delle nuvole di punti

Keyframe Tagging: Unambiguous Content Delivery for Augmented Reality Environments

Context: When considering the use of Augmented Reality to provide navigation cues in a completely unknown environment, the content must be delivered into the environment with a repeatable level of accuracy such that the navigation cues can be understood and interpreted correctly by the user. Aims: This thesis aims to investigate whether a still image based reconstruction of an Augmented Reality environment can be used to develop a content delivery system that providers a repeatable level of accuracy for content placement. It will also investigate whether manipulation of the properties of a Spatial Marker object is sufficient to reduce object selection ambiguity in an Augmented Reality environment. Methods: A series of experiments were conducted to test the separate aspects of these aims. Participants were required to use the developed Keyframe Tagging tool to introduce virtual navigation markers into an Augmented Reality environment, and also to identify objects within an Augmented Reality environment that was signposted using different Virtual Spatial Markers. This tested the accuracy and repeatability of content placement of the approach, while also testing participants’ ability to reliably interpret virtual signposts within an Augmented Reality environment. Finally the Keyframe Tagging tool was tested by an expert user against a pre-existing solution to evaluate the time savings offered by this approach against the overall accuracy of content placement. Results: The average accuracy score for content placement across 20 participants was 64%, categorised as “Good” when compared with an expert benchmark result, while no tags were considered “incorrect” and only 8 from 200 tags were considered to have “Poor” accuracy, supporting the Keyframe Tagging approach. In terms of object identification from virtual cues, some of the predicted cognitive links between virtual marker property and target object did not surface, though participants reliably identified the correct objects across several trials. Conclusions: This thesis has demonstrated that accurate content delivery can be achieved through the use of a still image based reconstruction of an Augmented Reality environment. By using the Keyframe Tagging approach, content can be placed quickly and with a sufficient level of accuracy to demonstrate its utility in the scenarios outlined within this thesis. There are some observable limitations to the approach, which are discussed with the proposals for further work in this area

A Programmable Display-Layer Architecture for Virtual-Reality Applications

Two important technical objectives of virtual-reality systems are to provide compelling visuals and effective 3D user interaction. In this respect, modern virtual reality system architectures suffer from a number of short-comings. The reduction of end-to-end latency, crosstalk and judder are especially difficult challenges, each of which negatively affects visual quality or user interaction. In order to provide higher quality visuals, complex scenes consisting of large models are often used. Rendering such a complex scene is a time-consuming process resulting in high end-to-end latency, thereby hampering user interaction. Classic virtual-reality architectures can not adequately address these challenges due to their inherent design principles. In particular, the tight coupling between input devices, the rendering loop and the display system inhibits these systems from addressing all the aforementioned challenges simultaneously. In this thesis, a virtual-reality architecture design is introduced that is based on the addition of a new logical layer: the Programmable Display Layer (PDL). The governing idea is that an extra layer is inserted between the rendering system and the display. In this way, the display can be updated at a fast rate and in a custom manner independent of the other components in the architecture, including the rendering system. To generate intermediate display updates at a fast rate, the PDL performs per-pixel depth-image warping by utilizing the application data. Image warping is the process of computing a new image by transforming individual depth-pixels from a closely matching previous image to their updated locations. The PDL architecture can be used for a range of algorithms and to solve problems that are not easily solved using classic architectures. In particular, techniques to reduce crosstalk, judder and latency are examined using algorithms implemented on top of the PDL. Concerning user interaction techniques, several six-degrees-of-freedom input methods exists, of which optical tracking is a popular option. However, optical tracking methods also introduce several constraints that depend on the camera setup, such as line-of-sight requirements, the volume of the interaction space and the achieved tracking accuracy. These constraints generally cause a decline in the effectiveness of user interaction. To investigate the effectiveness of optical tracking methods, an optical tracker simulation framework has been developed, including a novel optical tracker to test this framework. In this way, different optical tracking algorithms can be simulated and quantitatively evaluated under a wide range of conditions. A common approach in virtual reality is to implement an algorithm and then to evaluate the efficacy of that algorithm by either subjective, qualitative metrics or quantitative user experiments, after which an updated version of the algorithm may be implemented and the cycle repeated. A different approach is followed here. Throughout this thesis, an attempt is made to automatically detect and quantify errors using completely objective and automated quantitative methods and to subsequently attempt to resolve these errors dynamically

Configurable Input Devices for 3D Interaction using Optical Tracking

Three-dimensional interaction with virtual objects is one of the aspects that needs to be addressed in order to increase the usability and usefulness of virtual reality. Human beings have difficulties understanding 3D spatial relationships and manipulating 3D user interfaces, which require the control of multiple degrees of freedom simultaneously. Conventional interaction paradigms known from the desktop computer, such as the use of interaction devices as the mouse and keyboard, may be insufficient or even inappropriate for 3D spatial interaction tasks. The aim of the research in this thesis is to develop the technology required to improve 3D user interaction. This can be accomplished by allowing interaction devices to be constructed such that their use is apparent from their structure, and by enabling efficient development of new input devices for 3D interaction. The driving vision in this thesis is that for effective and natural direct 3D interaction the structure of an interaction device should be specifically tuned to the interaction task. Two aspects play an important role in this vision. First, interaction devices should be structured such that interaction techniques are as direct and transparent as possible. Interaction techniques define the mapping between interaction task parameters and the degrees of freedom of interaction devices. Second, the underlying technology should enable developers to rapidly construct and evaluate new interaction devices. The thesis is organized as follows. In Chapter 2, a review of the optical tracking field is given. The tracking pipeline is discussed, existing methods are reviewed, and improvement opportunities are identified. In Chapters 3 and 4 the focus is on the development of optical tracking techniques of rigid objects. The goal of the tracking method presented in Chapter 3 is to reduce the occlusion problem. The method exploits projection invariant properties of line pencil markers, and the fact that line features only need to be partially visible. In Chapter 4, the aim is to develop a tracking system that supports devices of arbitrary shapes, and allows for rapid development of new interaction devices. The method is based on subgraph isomorphism to identify point clouds. To support the development of new devices in the virtual environment an automatic model estimation method is used. Chapter 5 provides an analysis of three optical tracking systems based on different principles. The first system is based on an optimization procedure that matches the 3D device model points to the 2D data points that are detected in the camera images. The other systems are the tracking methods as discussed in Chapters 3 and 4. In Chapter 6 an analysis of various filtering and prediction methods is given. These techniques can be used to make the tracking system more robust against noise, and to reduce the latency problem. Chapter 7 focusses on optical tracking of composite input devices, i.e., input devices 197 198 Summary that consist of multiple rigid parts that can have combinations of rotational and translational degrees of freedom with respect to each other. Techniques are developed to automatically generate a 3D model of a segmented input device from motion data, and to use this model to track the device. In Chapter 8, the presented techniques are combined to create a configurable input device, which supports direct and natural co-located interaction. In this chapter, the goal of the thesis is realized. The device can be configured such that its structure reflects the parameters of the interaction task. In Chapter 9, the configurable interaction device is used to study the influence of spatial device structure with respect to the interaction task at hand. The driving vision of this thesis, that the spatial structure of an interaction device should match that of the task, is analyzed and evaluated by performing a user study. The concepts and techniques developed in this thesis allow researchers to rapidly construct and apply new interaction devices for 3D interaction in virtual environments. Devices can be constructed such that their spatial structure reflects the 3D parameters of the interaction task at hand. The interaction technique then becomes a transparent one-to-one mapping that directly mediates the functions of the device to the task. The developed configurable interaction devices can be used to construct intuitive spatial interfaces, and allow researchers to rapidly evaluate new device configurations and to efficiently perform studies on the relation between the spatial structure of devices and the interaction task