Interaction Techniques using Head Mounted Displays and Handheld Devices for Outdoor Augmented Reality

Depending upon their nature, Outdoor AR applications can be deployed on head mounted displays (HMD) like Google glass or handheld Displays (HHD) like smartphones. This master’s thesis investigates novel gesture-based interaction techniques and applications for a HMD-HHD hybrid system that account for advantages presented by each platform. Prior research in HMD-HHD hybrid systems and gestures used in VR and surface computing were taken into account while designing the applications and interaction techniques. A prototype system combining a HMD and HHD was developed and four applications were created for the system. For evaluating the gestures, an application that compared four of the proposed gestures for selection tasks was developed. The results showed a significant difference between the different gestures and that the choice of gesture for selection tasks using a hybrid system depended upon application requirements like speed and accuracy

Realidade aumentada para produção assistida em ambiente industrial

Smart factories are becoming more and more common and Augmented Reality (AR) is a pillar of the transition to Industry 4.0 and smart manufacturing. AR can improve many industrial processes such as training, maintenance, assembly, quality control, remote collaboration and others. AR has the potential to revolutionize the way information is accessed, used and exchanged, extending user’s perception and improving their performance. This work proposes a Pervasive AR tool, created in collaboration with industrial partners, to support the training of operators on industrial shop floors while performing production operations. A Human-Centered Design (HCD) methodology was used to identify operators’ difficulties, challenges, and define requirements. After initial meetings with stakeholders, an AR prototype was designed and developed to allow the configuration and visualization of AR content on the shop floor. Several meetings and user studies were conducted to evaluate the developed tools and improve their usability and features. Comparisons between the proposed Head Mounted Display (HMD) solution, the method currently being used in the shopfloor and alternative AR solutions (mobile based) were conducted. The results of user studies suggest that the proposed AR system can significantly improve the performance (up to 70% when compared with the method currently used in the shop floor) of novice operators.Fábricas inteligentes estão a tornar-se cada vez mais comuns e a Realidade Aumentada (Augmented Reality) é essencial para a transição para a Indústria 4.0 e para a produção inteligente. A AR pode ser usada para melhorar muitos processos industriais, tais como treino, assistência, montagem, controlo de qualidade, colaboração remota, entre outros. A AR tem potencial para revolucionar a maneira como a informação é acedida, usada e partilhada, expandindo a perceção do utilizador e melhorando a sua performance. Este trabalho propõe uma ferramenta de AR Pervasiva, criada em colaboração com parceiros da indústria, para ajudar no treino de operadores de chão de fábrica em tarefas de produção fabril. Para identificar as dificuldades, desafios e definir requisitos, foi seguida uma metodologia de Desenho Centrada no Utilizador (HCD). Depois de vários encontros com o público-alvo, um protótipo de AR foi desenhado e desenvolvido para permitir a configuração e visualização de conteúdo em AR na linha de montagem de uma fábrica. Diversas reuniões e testes com utilizadores foram realizados de modo a avaliar as ferramentas desenvolvidas e melhorar a usabilidade e as suas funcionalidades. Foram também realizadas comparações entre a solução de AR proposta, o método atualmente utilizado na linha de produção e uma solução alternativa de AR para dispositivos móveis. Os resultados dos testes de utilizador realizados sugerem que a solução proposta pode melhorar substancialmente a eficiência (até 70% quando comparado com método atualmente utilizado na linha de produção) de novos operadores.Mestrado em Engenharia de Computadores e Telemátic

A systematic review of augmented reality applications in maintenance

Augmented Reality (AR) technologies for supporting maintenance operations have been an academic research topic for around 50 years now. In the last decade, major progresses have been made and the AR technology is getting closer to being implemented in industry. In this paper, the advantages and disadvantages of AR have been explored and quantified in terms of Key Performance Indicators (KPI) for industrial maintenance. Unfortunately, some technical issues still prevent AR from being suitable for industrial applications. This paper aims to show, through the results of a systematic literature review, the current state of the art of AR in maintenance and the most relevant technical limitations. The analysis included filtering from a large number of publications to 30 primary studies published between 1997 and 2017. The results indicate a high fragmentation among hardware, software and AR solutions which lead to a high complexity for selecting and developing AR systems. The results of the study show the areas where AR technology still lacks maturity. Future research directions are also proposed encompassing hardware, tracking and user-AR interaction in industrial maintenance is proposed

Study of Augmented Reality based manufacturing for further integration of quality control 4.0: a systematic literature review

Augmented Reality (AR) has gradually become a mainstream technology enabling Industry 4.0 and its maturity has also grown over time. AR has been applied to support different processes on the shop-floor level, such as assembly, maintenance, etc. As various processes in manufacturing require high quality and near-zero error rates to ensure the demands and safety of end-users, AR can also equip operators with immersive interfaces to enhance productivity, accuracy and autonomy in the quality sector. However, there is currently no systematic review paper about AR technology enhancing the quality sector. The purpose of this paper is to conduct a systematic literature review (SLR) to conclude about the emerging interest in using AR as an assisting technology for the quality sector in an industry 4.0 context. Five research questions (RQs), with a set of selection criteria, are predefined to support the objectives of this SLR. In addition, different research databases are used for the paper identification phase following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) methodology to find the answers for the predefined RQs. It is found that, in spite of staying behind the assembly and maintenance sector in terms of AR-based solutions, there is a tendency towards interest in developing and implementing AR-assisted quality applications. There are three main categories of current AR-based solutions for quality sector, which are AR-based apps as a virtual Lean tool, AR-assisted metrology and AR-based solutions for in-line quality control. In this SLR, an AR architecture layer framework has been improved to classify articles into different layers which are finally integrated into a systematic design and development methodology for the development of long-term AR-based solutions for the quality sector in the future

A roadmap for semantically-enabled human device interactions

With the evolving Internet of Things (IoT), the number of smart devices we interact in our day-to-day life has significantly increased. The nature of human interaction with these devices must be perceived, because too much complexity could lead to the risk of IoT appearing unattractive for users or the system losing its efficiency, regardless of its potential. Therefore, it is important to address problems in Human Device Interactions to provide a high-quality User Experience (UX) in the emerging IoT. This paper proposes a roadmap to address the complexities associated with human smart device (sensor or actuator) interaction through a methodology that incorporates context awareness in Augmented Reality (AR) using semantic Web technologies. Further, we analyse the use of Natural User Interfaces (NUI), such as hand gestures and gaze, to provide noninvasive and intuitive interaction in optimising the user experience

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

Augmented reality in architecture and construction education: state of the field and opportunities

Over the past decade, the architecture and construction (AC) industries have been evolving from traditional practices into more current, interdisciplinary and technology integrated methods. Complex and intricate digital technologies and mobile computing such as simulation, computational design and immersive technologies, have been exploited for different purposes such as reducing cost and time, improving design and enhancing overall project efficiency. Immersive technologies and augmented reality (AR), in particular, have proven to be extremely beneficial in this field. However, the application and usage of these technologies and devices in higher education teaching and learning environments are yet to be fully explored and still scarce. More importantly, there is still a significant gap in developing pedagogies and teaching methods that embrace the usage of such technologies in the AC curricula. This study, therefore, aims to critically analyse the current state-of-the-art and present the developed and improved AR approaches in teaching and learning methods of AC, addressing the identified gap in the extant literature, while developing transformational frameworks to link the gaps to their future research agenda. The conducted analysis incorporates the critical role of the AR implications on the AC students’ skillsets, pedagogical philosophies in AC curricula, techno-educational aspects and content domains in the design and implementation of AR environments for AC learning. The outcomes of this comprehensive study prepare trainers, instructors, and the future generation of AC workers for the rapid advancements in this industry

Pilvipalveluiden käyttö paikantamiseen perustuvissa AR-sovelluksissa

Tiivistelmä. Tutkimus perustui paikantamiseen perustuvan AR-sovelluksen (Augmented Reality, AR) kehittämiseen mobiililaitteelle. Tällaisen sovelluksen toiminta perustuu laitteen sijaintiin sekä sitä ympäröivästä alueesta saatavilla olevan tiedon vertaamiseen toisiinsa. Paikannukseen perustuvan sovelluksen täytyy saada informaatio käyttäjän sijainnista sekä sovelluksen haluaman informaation liittyen käyttäjän sijaintiin. Kerättyään tarvittavan informaation, AR-sovellus osaa näyttää käyttäjälleen ympäriltä kerätyn tiedon lisättynä todellisuutena reaalikuvan päälle. Tutkimuksessa kehitetyn sovelluksen alustana päätettiin käyttää Android-käyttöjärjestelmää, jolle mobiilisovelluksen kehittäminen onnistuu helpoiten sekä tähän työhön sopivimmin. Sovelluksen kehittämiseen käytettiin Android Studio -kehitystyökalua, jonka käyttöön löytyy ohjeita developer.android.com nettisivustolta. Työssä hyödynnettiin pilvipalvelu Google Servicen kategorioittain tarjoamaa informaatiota eri paikoista. Sovelluksessa käytettävä informaatio rajattiin yhteen Google Places-API:n kategoriaan, jossa arvioitiin olevan sopiva määrä kohteita sovelluksen kehittämistä varten. Informaation piirtäminen mobiililaitteen näytölle toteutettiin vertaamalla osuuko laitteen ja kohteen koordinaattien välinen suora linja kamerakuvan rajaamalle sektorille. Tässä työssä onnistuttiin kehittämään AR-sovellus, joka paikantaa laitteen käyttäjää ympäröivät kohteet sekä esittää niiden sijainnin mobiililaitteen näytölle kameran luoman reaalikuvan päälle. Sovelluksen kehitys onnistui Android-alustalle vaivattomasti Googlen tarjoamien palveluiden avulla. Mobiililaitteet sopivat hyvin AR-tyyppisten sovellusten käyttämiseen niiden käytettävyyden ja riittävän tehokkuuden ansiosta.Using cloud services in a location based augmented reality application. Abstract. This research is based on developing a location-based AR-application (Augmented Reality, AR) for mobile devices. This type of application requires information about the device’s location, orientation and it must be aware of its surroundings. A location-based application must get information about the user’s location and necessary information relative to that position. After gathering this data, the AR-app can display the surrounding’s information on top of the real-time camera picture as an augmented reality. Android-operating system was used as a platform for the application. It is an easy development platform and it was appropriate for this study. AndroidStudio was used as the application development environment. It has a full online documentation in the following internet page: developer.android.com. Categorial information about nearby places from Google Services was used in this project. Data used in the application was limited to one of Google Places API category, that was estimated to produce an appropriate amount of results to develop the application. Drawing the information to the mobile device’s screen was accomplished by comparing if the line between the device and the place is on the camera’s current view. This paper includes a successfully developed application that can fetch the nearby places for the user and show their location in the camera’s real-time view. The development of the application for the Android-platform was an ease with the services offered by Google. Mobile devices are suitable for using ARtype applications because of their usability and adequate performance

Augmented reality applications in the automotive industry

This research paper examines the implications and transformative capabilities of Augmented Reality (AR) within the automotive landscape. We examine how AR catalyses radical changes across various automotive functions, from design and manufacturing to customer engagement and vehicle operation. While AR is poised to elevate operational efficiencies significantly, it also presents challenges, such as AR content authoring complexity and hardware constraints that restrict mass consumer adoption. This paper surveys the current AR state-of-the-art technology, systems and pivotal automotive applications. The primary objective is to offer a thorough comprehension of the underlying concepts, methodologies, and applications that underpin the integration of AR within the automotive industry. It concludes by discussing potential barriers to AR implementation and outlines future avenues for research, including software scalability and the integration of cloud and edge computing to alleviate device limitations