Applying Augmented Reality to Outdoors Industrial Use

Augmented Reality (AR) is currently gaining popularity in multiple different fields. However, the technology for AR still requires development in both hardware and software when considering industrial use. In order to create immersive AR applications, more accurate pose estimation techniques to define virtual camera location are required. The algorithms for pose estimation often require a lot of processing power, which makes robust pose estimation a difficult task when using mobile devices or designated AR tools. The difficulties are even larger in outdoor scenarios where the environment can vary a lot and is often unprepared for AR. This thesis aims to research different possibilities for creating AR applications for outdoor environments. Both hardware and software solutions are considered, but the focus is more on software. The majority of the thesis focuses on different visual pose estimation and tracking techniques for natural features. During the thesis, multiple different solutions were tested for outdoor AR. One commercial AR SDK was tested, and three different custom software solutions were developed for an Android tablet. The custom software solutions were an algorithm for combining data from magnetometer and a gyroscope, a natural feature tracker and a tracker based on panorama images. The tracker based on panorama images was implemented based on an existing scientific publication, and the presented tracker was further developed by integrating it to Unity 3D and adding a possibility for augmenting content. This thesis concludes that AR is very close to becoming a usable tool for professional use. The commercial solutions currently available are not yet ready for creating tools for professional use, but especially for different visualization tasks some custom solutions are capable of achieving a required robustness. The panorama tracker implemented in this thesis seems like a promising tool for robust pose estimation in unprepared outdoor environments.Lisätyn todellisuuden suosio on tällä hetkellä kasvamassa usealla eri alalla. Saatavilla olevat ohjelmistot sekä laitteet eivät vielä riitä lisätyn todellisuuden soveltamiseen ammattimaisessa käytössä. Erityisesti posen estimointi vaatii tarkempia menetelmiä, jotta immersiivisten lisätyn todellisuuden sovellusten kehittäminen olisi mahdollista. Posen estimointiin (laitteen asennon- sekä paikan arviointiin) käytetyt algoritmit ovat usein monimutkaisia, joten ne vaativat merkittävästi laskentatehoa. Laskentatehon vaatimukset ovat usein haasteellisia varsinkin mobiililaitteita sekä lisätyn todellisuuden laitteita käytettäessä. Lisäongelmia tuottaa myös ulkotilat, jossa ympäristö voi muuttua usein ja ympäristöä ei ole valmisteltu lisätyn todellisuuden sovelluksille. Diplomityön tarkoituksena on tutkia mahdollisuuksia lisätyn todellisuuden sovellusten kehittämiseen ulkotiloihin. Sekä laitteisto- että ohjelmistopohjaisia ratkaisuja käsitellään. Ohjelmistopohjaisia ratkaisuja käsitellään työssä laitteistopohjaisia ratkaisuja laajemmin. Suurin osa diplomityöstä keskittyy erilaisiin visuaalisiin posen estimointi tekniikoihin, jotka perustuvat kuvasta tunnistettujen luonnollisten piirteiden seurantaan. Työn aikana testattiin useita ratkaisuja ulkotiloihin soveltuvaan lisättyyn todellisuuteen. Yhtä kaupallista työkalua testattiin, jonka lisäksi toteutettiin kolme omaa sovellusta Android tableteille. Työn aikana kehitetyt sovellukset olivat yksinkertainen algoritmi gyroskoopin ja magnetometrin datan yhdistämiseen, luonnollisen piirteiden seuranta-algoritmi sekä panoraamakuvaan perustuva seuranta-algoritmi. Panoraamakuvaan perustuva seuranta-algoritmi on toteuteutettu toisen tieteellisen julkaisun pohjalta, ja algoritmia jatkokehitettiin integroimalla se Unity 3D:hen. Unity 3D-integrointi mahdollisti myös sisällön esittämisen lisätyn todellisuuden avulla. Työn lopputuloksena todetaan, että lisätyn todellisuuden teknologia on lähellä pistettä, jossa lisätyn todellisuuden työkaluja voitaisiin käyttää ammattimaisessa käytössä. Tällä hetkellä saatavilla olevat kaupalliset työkalut eivät vielä pääse ammattikäytön vaatimalle tasolle, mutta erityisesti visualisointitehtäviin soveltuvia ei-kaupallisia ratkaisuja on jo olemassa. Lisäksi työn aikana toteutetun panoraamakuviin perustuvan seuranta-algoritmin todetaan olevan lupaava työkalu posen estimointiin ulkotiloissa.Siirretty Doriast

An Efficient and Robust Mobile Augmented Reality Application

AR technology is perceived to be evolved from the bases of Virtual Reality (VR) technology. The ultimate goal of AR is to provide better management and ubiquitous access to information by using seamless techniques in which the interactive real world is combined with an interactive computer-generated world in one coherent environment. The direction of research in the field of AR has been shifted from traditional Desktop based mediums to the mobile devices such as the smartphones. However, image recognition on smartphones enforces many restrictions and challenges in the form of efficiency and robustness which are the general performance measurement of image recognition. Smart phones have limited processing capabilities as compared to the PC platform, hence the process of mobile AR application development and use of image recognition algorithm need to be emphasised. The processes of mobile AR application development include detection, description and matching. All the processes and algorithms need to be carefully selected in order to create an efficient and robust mobile AR application. The algorithm used in this work for detection, description and matching are AGAST, FREAK and Hamming distance respectively. The computation time, robustness towards rotation, scale and brightness are evaluated. The dataset used to evaluate the mobile AR application is the benchmark dataset; Mikolajczyk. The results showed that the mobile AR application is efficient with a computation time of 29.1ms. The robustness towards scale, rotation and brightness changes of the mobile AR application also obtained high accuracy which is 89.76%, 87.71% and 83.87% respectively. Hence, combination of algorithm AGAST, FREAK and Hamming distance are suitable to create an efficient and robust mobile AR application

An Augmented Reality Multimedia Learning Platform Assisting Online Lecture Delivery of Engineering Classes: an HVAC Course

Background: The ongoing COVID-19 pandemic has recently resulted in an emergent shift of traditional instruction to (presumably temporary) online delivery, and many students were not able to participate in practical activities (e.g., laboratory experiments) due to inaccessible or unavailable “brick and mortar” laboratories. Purpose: This works-in-progress paper developed a multimedia learning platform with virtual laboratory modules through an Augmented Reality (AR) environment, where virtual objects (augmented components) are superimposed onto a real learning setting during online lecture instruction. Specifically, to facilitate students’ gaining practical skills, a library of virtual objects were established for the main physical components or systems related to the undergraduate “Heating, Ventilating, and Air-conditioning (HVAC)” class to allow students to be immersed in an augmented learning reality representing the real physical world. Design: This proceeds in the following steps: 1) selecting a set of common HVAC components in the HVAC course; 2) developing an AR method to recognize each component’s figures or pictures from any learning documents (e.g., printed lecture ppt notes and textbook, and documents shown on computer or mobile screens); 3) establishing components’ 3D models with learning materials (e.g., concept and evaluation); 4) developing an AR app sequencing the learning materials upon request once an component is recognized; and 5) evaluating the app’s effectiveness. Results: This works-in-progress developed an initial AR app for an air handling unit (one main HAVC component), and the AR-based supplementary learning tool was tested and validated by graduate students who have already taken this HVAC class before. Their feedback showed that the AR tool would allow them to learn at their own pace while the instructor is not face-to-face with them, and the results revealed that the tool enhanced their practical skills especially when they are sheltered at their homes without accessing a lab or field trip. Conclusion: A well-designed AR learning app will effectively guide students to perform hands-on experiments related to the HVAC course and other STEM laboratory courses. The alternative pedagogy through AR technology also provides an efficient way to deliver practical experience online, especially when on-campus lab resources are limited or people are sheltered at home during natural disasters like the COVID-19 pandemic

Augmented reality using features accelerated segment test for learning tajweed

Currently, education forms students to think creatively and critically, this can be supported by the multimedia technology for education, including Islamic religious education. Islam requires all of its Muslim to read the Qur'an. Tajweed is an important because it is related to the articulation of reading the Qur'an properly and correctly. This article discusses the application of augmented reality (AR) as one of the multimedia technologies that can be used as an interactive educational medium to help study the tajweed of Qur'an. The method used in this research is Features from accelerated segment test (FAST) corner detection. The testing result with 31 tajweed objects show that FAST is able to recognize all Tajweed objects and display their AR. Besides, based on a survey with questionnaires to several students, the result shows that 88.2% of students responded very well and judged that it was sufficient to help study the tajweed

Augmented reality over maps

Dissertação de mestrado integrado em Engenharia InformáticaMaps and Geographic Information System (GIS) play a major role in modern society, particularly on tourism, navigation and personal guidance. However, providing geographical information of interest related to individual queries remains a strenuous task. The main constraints are (1) the several information scales available, (2) the large amount of information available on each scale, and (3) difficulty in directly infer a meaningful geographical context from text, pictures, or diagrams that are used by most user-aiding systems. To that extent, and to overcome the aforementioned difficulties, we develop a solution which allows the overlap of visual information over the maps being queried — a method commonly referred to as Augmented Reality (AR). With that in mind, the object of this dissertation is the research and implementation of a method for the delivery of visual cartographic information over physical (analogue) and digital two-dimensional (2D) maps utilizing AR. We review existing state-of-art solutions and outline their limitations across different use cases. Afterwards, we provide a generic modular solution for a multitude of real-life applications, to name a few: museums, fairs, expositions, and public street maps. During the development phase, we take into consideration the trade-off between speed and accuracy in order to develop an accurate and real-time solution. Finally, we demonstrate the feasibility of our methods with an application on a real use case based on a map of the city of Oporto, in Portugal.Mapas e Sistema de Informação Geográfica (GIS) desempenham um papel importante na sociedade, particularmente no turismo, navegação e orientação pessoal. No entanto, fornecer informações geográficas de interesse a consultas dos utilizadores é uma tarefa árdua. Os principais dificuldades são (1) as várias escalas de informações disponíveis, (2) a grande quantidade de informação disponível em cada escala e (3) dificuldade em inferir diretamente um contexto geográfico significativo a partir dos textos, figuras ou diagramas usados. Assim, e para superar as dificuldades mencionadas, desenvolvemos uma solução que permite a sobreposição de informações visuais sobre os mapas que estão a ser consultados - um método geralmente conhecido como Realidade Aumentada (AR). Neste sentido, o objetivo desta dissertação é a pesquisa e implementação de um método para a visualização de informações cartográficas sobre mapas 2D físicos (analógicos) e digitais utilizando AR. Em primeiro lugar, analisamos o estado da arte juntamente com as soluções existentes e também as suas limitações nas diversas utilizações possíveis. Posteriormente, fornecemos uma solução modular genérica para uma várias aplicações reais tais como: museus, feiras, exposições e mapas públicos de ruas. Durante a fase de desenvolvimento, tivemos em consideração o compromisso entre velocidade e precisão, a fim de desenvolver uma solução precisa que funciona em tempo real. Por fim, demonstramos a viabilidade de nossos métodos com uma aplicação num caso de uso real baseado num mapa da cidade do Porto (Portugal)

Augmented reality using features accelerated segment test for property catalogue

Promotional media used in the marketing of housing using catalogs that display 2D images of houses from one side of the house make potential customers unable to imagine the design of all parts of the house. Augmented Reality can be used as an interactive marketing media so that it can be used to display homes in 3D so that they appear more real from all sides so that prospective customers can consider the type of house to be chosen. Development of this application using the Multimedia Development Life Cycle. Application development uses the FAST algorithm as detection of home catalog markers to define how well images can be detected and tracked. The FAST algorithm will calculate every pixel on the target image in determining the corner when scanning the home catalog then it will produce a 3D object home to see the real shape design of the house

Object Tracking Using Local Binary Descriptors

Visual tracking has become an increasingly important topic of research in the field of Computer Vision (CV). There are currently many tracking methods based on the Detect-then-Track paradigm. This type of approach may allow for a system to track a random object with just one initialization phase, but may often rely on constructing models to follow the object. Another limitation of these methods is that they are computationally and memory intensive, which hinders their application to resource constrained platforms such as mobile devices. Under these conditions, the implementation of Augmented Reality (AR) or complex multi-part systems is not possible. In this thesis, we explore a variety of interest point descriptors for generic object tracking. The SIFT descriptor is considered a benchmark and will be compared with binary descriptors such as BRIEF, ORB, BRISK, and FREAK. The accuracy of these descriptors is benchmarked against the ground truth of the object\u27s location. We use dictionaries of descriptors to track regions with small error under variations due to occlusions, illumination changes, scaling, and rotation. This is accomplished by using Dense-to-Sparse Search Pattern, Locality Constraints, and Scale Adaptation. A benchmarking system is created to test the descriptors\u27 accuracy, speed, robustness, and distinctness. This data offers a comparison of the tracking system to current state of the art systems such as Multiple Instance Learning Tracker (MILTrack), Tracker Learned Detection (TLD), and Continuously Adaptive MeanShift (CAMSHIFT)