A 360 VR and Wi-Fi Tracking Based Autonomous Telepresence Robot for Virtual Tour

This study proposes a novel mobile robot teleoperation interface that demonstrates the applicability of a robot-aided remote telepresence system with a virtual reality (VR) device to a virtual tour scenario. To improve realism and provide an intuitive replica of the remote environment for the user interface, the implemented system automatically moves a mobile robot (viewpoint) while displaying a 360-degree live video streamed from the robot to a VR device (Oculus Rift). Upon the user choosing a destination location from a given set of options, the robot generates a route based on a shortest path graph and travels along that the route using a wireless signal tracking method that depends on measuring the direction of arrival (DOA) of radio signals. This paper presents an overview of the system and architecture, and discusses its implementation aspects. Experimental results show that the proposed system is able to move to the destination stably using the signal tracking method, and that at the same time, the user can remotely control the robot through the VR interface

Application of augmented reality and robotic technology in broadcasting: A survey

As an innovation technique, Augmented Reality (AR) has been gradually deployed in the broadcast, videography and cinematography industries. Virtual graphics generated by AR are dynamic and overlap on the surface of the environment so that the original appearance can be greatly enhanced in comparison with traditional broadcasting. In addition, AR enables broadcasters to interact with augmented virtual 3D models on a broadcasting scene in order to enhance the performance of broadcasting. Recently, advanced robotic technologies have been deployed in a camera shooting system to create a robotic cameraman so that the performance of AR broadcasting could be further improved, which is highlighted in the paper

Development and usability analysis of a mixed reality GPS navigator application for the microsoft hololens

The present work aims to perform a comparative usability analysis between two Human- Computer Interaction systems (HCI) for global geolocation (GPS) navigators. The intent is to compare the conventional use of a navigation application on a mobile device, such as a smartphone attached to the dashboard of a vehicle, to an implementation in Mixed Reality (MR) powered by the Head Mounted Display (HMD) Microsoft HoloLens. By connecting the MR device to a local network routed by an ordinary cellular phone, which is connected to a mobile data network, it is possible to ubiquitously acquire the phone’s geolocation data, its magnetometer deviation and a route graph of a navigation Application Programming Interface (API) from its current location to a destination entered by the user. Thus, a series of three-dimensional holograms are created at runtime, geolocated and placed around the user, guiding him through a path indicated on the floor, pertinent to the streets around him that lead to the desired destination. Apart from that, arrows are projected on the way at each crucial point of the path, where some maneuver must be performed, e.g., turning right or taking an exit at a roundabout. In a user experiment, performance and usability were assessed. Results show that users deemed the MR solution to offer a higher visibility both to the oncoming traffic and the suggested route, when compared to the conventional interface, being less attention demanding. EEG readings for most participants also exposed a significantly more demanding focus level for the handheld device. Additionally, an easiness to learn and use was indicated for our system, being almost on par with the already known and highly used application tested.O presente trabalho visa realizar uma análise comparativa de usabilidade entre dois sistemas de interação humano-computador para navegadores de geolocalização global (GPS). Foi almejado comparar o uso convencional do sistema, através de um dispositivo móvel tal qual um smartphone afixado ao painel de um veículo, com uma nova implementação em Realidade Mista potencializada pelo HMD Microsoft HoloLens. Conectando o dispositivo de realidade mista (MR) a uma rede local roteada por um aparelho celular convencional, este conectado a uma rede de dados móvel, foi possível receber ubiquamente os dados de sua geolocalização, de seu magnetômetro e um grafo de rota de uma API de navegação de alta disponibilidade partindo do presente local até um destino inserido pelo usuário. Com isso, é criada em tempo de execução uma série de hologramas tridimensionais geolocalizados ao redor do usuário, guiando-o através de um caminho indicado em seu chão, pertinente às ruas a sua volta que o levarão ao destino desejado. Também são projetadas flechas em seu caminho em cada ponto crucial de seu trajeto, onde deve-se realizar alguma manobra, e.g., dobrar à direita ou tomar uma saída de uma rotatória. Em um experimento realizado com usuários reais, seu desempenho e usabilidade foram aferidos. Resultados mostram que os usuários estimaram que a solução em MR oferecia uma visibilidade maior tanto ao tráfego passante quanto à rota sugerida, em comparação à interface convencional, requerindo menos atenção. Leituras de eletroencefalografia (EEG) na maioria dos participantes indicaram uma demanda significativamente maior de atenção focada no uso do dispositivo móvel. Uma grande facilidade de aprendizado e de uso também foi apontada para nosso sistema, estando quase a par da aplicação móvel altamente conhecida e usada

Integração de localização baseada em movimento na aplicação móvel EduPARK

More and more, mobile applications require precise localization solutions in a variety of environments. Although GPS is widely used as localization solution, it may present some accuracy problems in special conditions such as unfavorable weather or spaces with multiple obstructions such as public parks. For these scenarios, alternative solutions to GPS are of extreme relevance and are widely studied recently. This dissertation studies the case of EduPARK application, which is an augmented reality application that is implemented in the Infante D. Pedro park in Aveiro. Due to the poor accuracy of GPS in this park, the implementation of positioning and marker-less augmented reality functionalities presents difficulties. Existing relevant systems are analyzed, and an architecture based on pedestrian dead reckoning is proposed. The corresponding implementation is presented, which consists of a positioning solution using the sensors available in the smartphones, a step detection algorithm, a distance traveled estimator, an orientation estimator and a position estimator. For the validation of this solution, functionalities were implemented in the EduPARK application for testing purposes and usability tests performed. The results obtained show that the proposed solution can be an alternative to provide accurate positioning within the Infante D. Pedro park, thus enabling the implementation of functionalities of geocaching and marker-less augmented reality.Cada vez mais, as aplicações móveis requerem soluções de localização precisa nos mais variados ambientes. Apesar de o GPS ser amplamente usado como solução para localização, pode apresentar alguns problemas de precisão em condições especiais, como mau tempo, ou espaços com várias obstruções, como parques públicos. Para estes casos, soluções alternativas ao GPS são de extrema relevância e veem sendo desenvolvidas. A presente dissertação estuda o caso do projeto EduPARK, que é uma aplicação móvel de realidade aumentada para o parque Infante D. Pedro em Aveiro. Devido à fraca precisão do GPS nesse parque, a implementação de funcionalidades baseadas no posionamento e de realidade aumentada sem marcadores apresenta dificuldades. São analisados sistemas relevantes existentes e é proposta uma arquitetura baseada em localização de pedestres. Em seguida é apresentada a correspondente implementação, que consiste numa solução de posicionamento usando os sensores disponiveis nos smartphones, um algoritmo de deteção de passos, um estimador de distância percorrida, um estimador de orientação e um estimador de posicionamento. Para a validação desta solução, foram implementadas funcionalidades na aplicação EduPARK para fins de teste, e realizados testes com utilizadores e testes de usabilidade. Os resultados obtidos demostram que a solução proposta pode ser uma alternativa para a localização no interior do parque Infante D. Pedro, viabilizando desta forma a implementação de funcionalidades baseadas no posicionamento e de realidade aumenta sem marcadores.EduPARK é um projeto financiado por Fundos FEDER através do Programa Operacional Competitividade e Internacionalização - COMPETE 2020 e por Fundos Nacionais através da FCT - Fundação para a Ciência e a Tecnologia no âmbito do projeto POCI-01-0145-FEDER-016542.Mestrado em Engenharia Informátic

A virtual 3D mobile guide in the INTERMEDIA project

In this paper, we introduce a European research project, interactive media with personal networked devices (INTERMEDIA) in which we seek to progress beyond home and device-centric convergence toward truly user-centric convergence of multimedia. Our vision is to make the user the multimedia center: the user as the point at which multimedia services and the means for interacting with them converge. This paper proposes the main research goals in providing users with a personalized interface and content independent of physical networked devices, and space and time. As a case study, we describe an indoors, mobile mixed reality guide system: Chloe@University. With a see-through head-mounted display (HMD) connected to a small wearable computing device, Chloe@University provides users with an efficient way to guide someone in a building. A 3D virtual character in front of the user guides him/her to the required destinatio

IMMERSIVE WAYFINDING: VIRTUAL RECONSTRUCTION AND EYE-TRACKING FOR ORIENTATION STUDIES INSIDE COMPLEX ARCHITECTURE

Abstract. The study presents a process of digital simulation that aims to investigate the legibility of multiple spaces in a complex architecture through architectural survey, virtual reconstruction and 3D visualization in immersive environment. The collaboration between two research institutions, one Italian and the other Chinese, developed a reconstruction of a building in the campus of the Tsinghua University of Beijing using digital tools, in order to understand the behaviours during the fruition of that space. Digital simulation in the visibility theme guarantees the possibility of testing multiple configurations and showing the impacts of the different environmental hypothesis.</p