The Augmented User: a Wearable Augmented Reality Interface

In this paper, 'The Augmented User' project is presented, a new Augmented Reality (AR) system based on the ARToolKit and MRXToolKit technologies. We have developed an innovative system in which the user is visible in the AR scenario and indispensable part of the final result: he/she is in fact the carrier of the markers that are recognized by the system allowing his/her own transformation; this opens the possibilities of the users that are immersed in an AR system, being at the same time the augmented matter and the spectator. We also introduce two interesting applications for our system: the 'changing personalities' game and the 'image carrousel' performance, as simple examples of the multiple applications that our system actually have. The collaborative part is also proposed and presented, which makes the applications even more interesting. Finally, some conclusions are presented and further work is proposed

El túnel mágico

“Realidad aumentada”, se puede definir como la amplificación de la capacidad sensorial de la percepción del mundo que nos rodea mediante el uso de dispositivos que superponen elementos virtuales a la imagen real. El Túnel Mágico es una aplicación de realidad aumentada basada en las librerías de ARToolKit [1]; estas librerías son de libre distribución y se han utilizado en numerosas aplicaciones. La mayoría de estas aplicaciones tratan de mezclar mundos y/o escenarios nuevos, insertando elementos virtuales dentro de la realidad que nos rodea. En [3] se puede encontrar una descripción y clasificación de diferentes sistemas de realidad aumentada y su evolución en el tiempo. El Túnel Mágico consta de una serie de pantallas a modo de espejo con las que diversos usuarios pueden interactuar al verse transformados en un personaje pseudovirtual. Estas pantallas están dispuestas linealmente, a modo de túnel, y el usuario se puede mover por el escenario y así verse reflejado en las distintas pantallas, cada una de ellas aportándole una nueva identidad

Methods for Augmented Reality E-commerce

A new type of e-commerce system and related techniques are presented in this dissertation that customers of this type of e-commerce could visually bring product into their physical environment for interaction. The development and user study of this e-commerce system are provided. A new modeling method, which recovers 3D model directly from 2D photos without knowing camera information, is also presented to reduce the modeling cost of this new type of e-commerce. Also an immersive AR environment with GPU based occlusion is also presented to improve the rendering and usability of AR applications. Experiment results and data show the validity of these new technologies

Real-time spatial modeling to detect and track resources on construction sites

For more than 10 years the U.S. construction industry has experienced over 1,000 fatalities annually. Many fatalities may have been prevented had the individuals and equipment involved been more aware of and alert to the physical state of the environment around them. Awareness may be improved by automatic 3D (three-dimensional) sensing and modeling of the job site environment in real-time. Existing 3D modeling approaches based on range scanning techniques are capable of modeling static objects only, and thus cannot model in real-time dynamic objects in an environment comprised of moving humans, equipment, and materials. Emerging prototype 3D video range cameras offer another alternative by facilitating affordable, wide field of view, automated static and dynamic object detection and tracking at frame rates better than 1Hz (real-time). This dissertation presents an imperical work and methodology to rapidly create a spatial model of construction sites and in particular to detect, model, and track the position, dimension, direction, and velocity of static and moving project resources in real-time, based on range data obtained from a three-dimensional video range camera in a static or moving position. Existing construction site 3D modeling approaches based on optical range sensing technologies (laser scanners, rangefinders, etc.) and 3D modeling approaches (dense, sparse, etc.) that offered potential solutions for this research are reviewed. The choice of an emerging sensing tool and preliminary experiments with this prototype sensing technology are discussed. These findings led to the development of a range data processing algorithm based on three-dimensional occupancy grids which is demonstrated in detail. Testing and validation of the proposed algorithms have been conducted to quantify the performance of sensor and algorithm through extensive experimentation involving static and moving objects. Experiments in indoor laboratory and outdoor construction environments have been conducted with construction resources such as humans, equipment, materials, or structures to verify the accuracy of the occupancy grid modeling approach. Results show that modeling objects and measuring their position, dimension, direction, and speed had an accuracy level compatible to the requirements of active safety features for construction. Results demonstrate that video rate 3D data acquisition and analysis of construction environments can support effective detection, tracking, and convex hull modeling of objects. Exploiting rapidly generated three-dimensional models for improved visualization, communications, and process control has inherent value, broad application, and potential impact, e.g. as-built vs. as-planned comparison, condition assessment, maintenance, operations, and construction activities control. In combination with effective management practices, this sensing approach has the potential to assist equipment operators to avoid incidents that result in reduce human injury, death, or collateral damage on construction sites.Civil, Architectural, and Environmental Engineerin

Simbologia em realidade aumentada

Tese de mestrado em Engenharia Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011A representação de símbolos gráficos para transmitir informação levanta questões cognitivas que devem ser tidas em conta para obter resultados compreensíveis em termos de visualização. Quando estes símbolos são representados sobre imagens reais, como acontece na Realidade Aumentada, estas questões tornam-se mais agudas. Sobretudo quando a visualização ocorre em ambientes exteriores: não existindo controlo sobre as características da imagem real do fundo, é necessário ir adaptando dinamicamente os símbolos gráficos que vão sendo sobrepostos. A Realidade Aumentada é uma área de estudo que surgiu no final da década de 1960, contudo, só muito recentemente tem vindo a ser aplicada em larga escala. Tal facto deve-se ao aparecimento também recente, de dispositivos móveis dotados de capacidades que suportam esta tecnologia e têm custos acessíveis ao público em geral. Por seu turno, o estudo de aspectos cognitivos relativos ao uso da cor e de símbolos tem sido tema de variados estudos. No entanto, não há ainda estudos feitos sobre a utilização de símbolos gráficos sobre imagens reais, estáticas ou em movimento, de modo a analisar como melhorar a sua visibilidade sobre o fundo real. O objectivo deste estudo foi, depois da análise do trabalho desenvolvido nesta área, implementar um protótipo que permite criar vários tipos de adaptações em símbolos gráficos e depois testar estas soluções com utilizadores. Com os resultados dos testes, tiraram-se ilações sobre as adaptações mais adequadas para desenvolver aplicações de Realidade Aumentada que vão ao encontro da satisfação do utilizador final.The representation of graphical symbols to convey information raises cognitive issues that must be taken into account to obtain comprehensible results in terms of visualization. When these symbols are drawn over real images, as in Augmented Reality, these issues become more acute. Especially when visualization occurs in outdoor environments: with the absence of control over the characteristics of the real background image, it is necessary to dynamically adapt the graphical symbols that are being overlaid. Augmented Reality is a research area that emerged in the end of the 1960s, however, only recently has been widely applied. This is due to the recent appearance of mobile devices, equipped with capabilities that support this technology, and affordable to the general public. The study of cognition aspects related to the use of color and symbols has been the subject of several studies. However, there are still no studies about the use of graphic symbols over real images, still or in motion, to examine how to improve their visibility over the background. The purpose of this study was, after studying the work developed in this area, the implementation of a prototype that allows the creation of different approaches to symbol adaptation, and then test them, with users. Taking into account the results of the tests, it was possible to identify the most important characteristics that best fulfill the needs of the users of Augmented Reality applications

Image processing techniques for mixed reality and biometry

2013 - 2014This thesis work is focused on two applicative fields of image processing research, which, for different reasons, have become particularly active in the last decade: Mixed Reality and Biometry. Though the image processing techniques involved in these two research areas are often different, they share the key objective of recognizing salient features typically captured through imaging devices. Enabling technologies for augmented/mixed reality have been improved and refined throughout the last years and more recently they seems to have finally passed the demo stage to becoming ready for practical industrial and commercial applications. To this regard, a crucial role will likely be played by the new generation of smartphones and tablets, equipped with an arsenal of sensors connections and enough processing power for becoming the most portable and affordable AR platform ever. Within this context, techniques like gesture recognition by means of simple, light and robust capturing hardware and advanced computer vision techniques may play an important role in providing a natural and robust way to control software applications and to enhance onthe- field operational capabilities. The research described in this thesis is targeted toward advanced visualization and interaction strategies aimed to improve the operative range and robustness of mixed reality applications, particularly for demanding industrial environments... [edited by Author]XIII n.s

Augmented reality for non-rigid surfaces

Augmented Reality (AR) is the process of integrating virtual elements in reality, often by mixing computer graphics into a live video stream of a real scene. It requires registration of the target object with respect to the cameras. To this end, some approaches rely on dedicated hardware, such as magnetic trackers or infra-red cameras, but they are too expensive and cumbersome to reach a large public. Others are based on specifically designed markers which usually look like bar-codes. However, they alter the look of objects to be augmented, thereby hindering their use in application for which visual design matters. Recent advances in Computer Vision have made it possible to track and detect objects by relying on natural features. However, no such method is commonly used in the AR community, because the maturity of available packages is not sufficient yet. As far as deformable surfaces are concerned, the choice is even more limited, mainly because initialization is so difficult. Our main contribution is therefore a new AR framework that can properly augment deforming surfaces in real-time. Its target platform is a standard PC and a single webcam. It does not require any complex calibration procedure, making it perfectly suitable for novice end-users. To satisfy to the most demanding application designers, our framework does not require any scene engineering, renders virtual objects illuminated by real light, and let real elements occlude virtual ones. To meet this challenge, we developed several innovative techniques. Our approach to real-time registration of a deforming surface is based on wide-baseline feature matching. However, traditional outlier elimination techniques such as RANSAC are unable to handle the non-rigid surface's large number of degrees of freedom. We therefore proposed a new robust estimation scheme that allows both 2–D and 3–D non-rigid surface registration. Another issue of critical importance in AR to achieve realism is illumination handling, for which existing techniques often require setup procedures or devices such as reflective spheres. By contrast, our framework includes methods to estimate illumination for rendering purposes without sacrificing ease of use. Finally, several existing approaches to handling occlusions in AR rely on multiple cameras or can only deal with occluding objects modeled beforehand. Our requires only one camera and models occluding objects at runtime. We incorporated these components in a consistent and flexible framework. We used it to augment many different objects such as a deforming T-shirt or a sheet of paper, under challenging conditions, in real-time, and with correct handling of illumination and occlusions. We also used our non-rigid surface registration technique to measure the shape of deformed sails. We validated the ease of deployment of our framework by distributing a software package and letting an artist use it to create two AR applications

Entornos multimedia de realidad aumentada en el campo del arte

La relación ente Ciencia y Arte ha mantenido a lo largo de la historia momentos de proximidad o distanciamiento, llegando a entenderse como dos culturas diferentes, pero también se han producido situaciones interdisciplinares de colaboración e intercambio que en nuestros días mantienen como nexo común la cultura digital y el uso del ordenador. Según Berenguer (2002) desde la aparición del ordenador, científicos y artistas están encontrando un espacio común de trabajo y entendimiento. Mediante el empleo de las nuevas tecnologías, la distancia que separa ambas disciplinas es cada vez más corta. En esta tesis, cuyo título es "Entornos Multimedia de Realidad Aumentada en el Campo del Arte", se presenta una investigación teórico-práctica de la tecnología de realidad aumentada aplicada al arte y campos afines, como el edutainment (educación + entretenimiento). La investigación se ha realizado en dos bloques: en el primer bloque se trata la tecnología desde distintos factores que se han considerado relevantes para su entendimiento y funcionamiento; en el segundo se presentan un total de seis ensayos que constituyen la parte práctica de esta tesis.Portalés Ricart, C. (2008). Entornos multimedia de realidad aumentada en el campo del arte [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/3402Palanci

Desenvolvimento e avaliação de um sistema de realidade aumentada para visualização cartográfica

Orientador : Prof. Dr. Jorge Antonio Silva CentenoTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências da Terra, Programa de Pós-Graduação em Ciências Geodésicas. Defesa: Curitiba, 16/02/2017Inclui referências : f. 174-95Resumo: Nas últimas décadas é notável o aparecimento de sistemas computacionais que favorecem os usuários a executar suas tarefas. Dentre os sistemas que prometem auxiliar os usuários a executar suas tarefas - inclusive as tarefas de leitura de mapa - os baseados em Realidade Aumentada (RA) mostram-se bastante promissores. Estas interfaces computacionais mais naturais podem possibilitar novas formas de interação com a informação e com o ambiente ao redor do usuário. Entretanto, pouco deste tema tem sido explorado, principalmente, no Brasil. Por esse motivo, elaborou-se a presente pesquisa com a intenção de desenvolver e analisar alguns protótipos de Realidade Aumentada para a Cartografia, para interação com mapas e dados geográficos. Neste trabalho abordou-se e utilizou-se de diferentes metodologias para RA, como com e sem marcadores fiduciais especiais, em versão desktop ou para dispositivos móveis. Foram utilizados dados geoespaciais referentes à características físicas, políticas, sociais e demográficas, econômicas e ambientais, integradas sobre diferentes tipos de mapas impressos, de temáticos a topográficos. Explorou-se também a utilização vários formatos de dados, de vetorial a raster, 2D, 3D, animações, etc. É possível destacar que os sistemas de Realidade Aumentada desenvolvidos trazem novas possibilidades de manipulação de dados geográficos, trazendo novas oportunidades de interação com o mapa impresso ou dados geoespaciais digitais. Porém, destaca-se que o desenvolvimento para dispositivos móveis e sem a utilização de marcadores fiduciais externos apresenta-se altamente atrativo, uma vez que este sistema de RA pode ser considerado como uma "lente mágica" ou uma "janela" para o mundo virtual, onde o usuário pode ter acesso a outras informações geoespaciais digitais sobrepostas ao ambiente real através destes dispositivos móveis. Ainda, com a execução de testes com usuários verificou-se que os sistemas de RA desenvolvidos trouxeram significativas melhorias para seus usuários na execução das tarefas, assim como ganhos em quesitos como motivação, confiança, diminuição de demandas físicas e mentais, etc. Com relação à preferencia dos usuários, a grande maioria dos usuários entrevistados preferiu trabalhar com estes sistemas de RA desenvolvidos do que apenas com os mapas impressos. Estudos também sobre se o conhecimento prévio do usuário poderia influenciar na execução das tarefas indicaram que, por mais que os sistemas de RA tenham auxiliado a grande maioria dos usuários, os sistemas de RA possuíram maior aceite e trouxeram maiores contribuições aos usuários não especialistas. Também, verificou-se que, por várias vezes, os usuários tendiam a "brincar" mais com o sistema de RA do que com o mapa impresso, passando mais tempo analisando determinada área ou fenômeno geográfico, demonstrando assim maior interação com os dados e espaço geográfico. Em todos os testes executados foi possível verificar que os usuários se mostravam mais interessados em executar suas tarefas quando apoiados pelos sistemas de RA, por vezes executando as tarefas sorrindo ou fazendo exclamações e comentários positivos sobre o sistema de RA. Desta forma, aplicações de Realidade Aumentada, como a descrita neste trabalho, se tornam importantes ferramentas no auxílio às questões profissionais como planejamento e gestão do espaço geográfico, ou até à área da educação, ensino e aprendizagem, uma vez que a integração dos dados em um ambiente mais interativo pode auxiliar na melhoria do processo de entendimento e compreensão do espaço geográfico por parte dos seus usuários. Palavras-chave: Realidade Aumentada, dados geoespaciais, interatividade, teste com usuários.Abstract: In the last decades it is notable the increase of computational systems that help users to perform their tasks. Among the systems that promise to help users to perform their tasks - including map-reading tasks - those based on Augmented Reality (AR) are quite promising. These natural computational interfaces may allow new forms of interaction with the information and the real world around the users. However, just a little has been explored about it in the world - and especially in Brazil. For this reason, we elaborated this research to development and analyze some prototypes of Augmented Reality for Cartography, for interaction with maps and geographic data, addressing and using different methodologies, such as with or without special fiducial markers, for desktop computers or mobile devices. We have used geospatial data referring to physical, political, social, demographic, economic and environmental characteristics, integrated over different types of printed maps, from thematic to topographic maps. It is possible to highlight that the Augmented Reality systems developed in this work brought new opportunities of interaction with printed maps or digital geospatial data. Moreover, the AR systems brought the possibility of exploration of several formats, as vectors and rasters, 2D, 3D, animations, etc. It was verified that - independently of the methodologies used - with AR systems it was possible to interact with a printed physical map and virtual geographic data simultaneously. However, it is possible to affirm that the AR system developed for mobile devices and using the markerless approach is highly attractive. With this approach the AR system can be considered as a "magic lens" or a "window" to the virtual world, where users could access digital geospatial information superimposed on the real environment through these mobile AR systems. Also, with users' tests, it was verified that the AR systems developed have brought significant improvements to the execution of the tasks, such as gains in motivation, confidence, decrease of physical and mental demands, etc. About the user's preference, a large majority of users interviewed preferred to work with these AR systems than just the printed maps. Also, it has been found that sometimes users tended to "play" more with the AR system than with the printed maps, spending more time analyzing a particular area or geographic phenomenon, demonstrating more interaction with geographic data. Also, it has been verified that users were more interested in performing their tasks when supported by the AR systems - performing their tasks smiling or making exclamations and positive comments about the AR system. By this way, applications of Augmented Reality as a described in this work become an important tool for assistance to professional questions like geographic space planning and management, or to the educational area, since the integration of the geospatial data in an interactive environment can help to improve the process of understanding and comprehension of the geographical space by its users. Keywords: Augmented Reality, geospatial data, interactivity, users' tests

The Use of Dense Stereo Range Data in Augmented Reality

This paper describes an augmented reality system that incorporates a real-time dense stereo vision system. Analysis of range and intensity data is used to perform two functions: 1) 3D detection and tracking of the user's fingertip or a pen to provide natural 3D pointing gestures, and 2) computation of the 3D position and orientation of the user's viewpoint without the need for fiducial marks calibration procedures, or manual initialization. The paper describes the stereo depth camera, the algorithms developed for pointer tracking and camera pose tracking, and demonstrates their use within an application in the field of oil and gas exploration