Augmented reality X-ray vision on optical see-through head mounted displays

Abstract. In this thesis, we present the development and evaluation of an augmented reality X-ray system on optical see-through head-mounted displays. Augmented reality X-ray vision allows users to see through solid surfaces such as walls and facades, by augmenting the real view with virtual images representing the hidden objects. Our system is developed based on the optical see-through mixed reality headset Microsoft Hololens. We have developed an X-ray cutout algorithm that uses the geometric data of the environment and enables seeing through surfaces. We have developed four different visualizations as well based on the algorithm. The first visualization renders simply the X-ray cutout without displaying any information about the occluding surface. The other three visualizations display features extracted from the occluder surface to help the user to get better depth perception of the virtual objects. We have used Sobel edge detection to extract the information. The three visualizations differ in the way to render the extracted features. A subjective experiment is conducted to test and evaluate the visualizations and to compare them with each other. The experiment consists of two parts; depth estimation task and a questionnaire. Both the experiment and its results are presented in the thesis

Mobile 2D and 3D Spatial Query Techniques for the Geospatial Web

The increasing availability of abundant geographically referenced information in the Geospatial Web provides a variety of opportunities for developing value-added LBS applications. However, large data volumes of the Geospatial Web and small mobile device displays impose a data visualization problem, as the amount of searchable information overwhelms the display when too many query results are returned. Excessive returned results clutter the mobile display, making it harder for users to prioritize information and causes confusion and usability problems. Mobile Spatial Interaction (MSI) research into this “information overload” problem is ongoing where map personalization and other semantic based filtering mechanisms are essential to de-clutter and adapt the exploration of the real-world to the processing/display limitations of mobile devices. In this thesis, we propose that another way to filter this information is to intelligently refine the search space. 3DQ (3-Dimensional Query) is our novel MSI prototype for information discovery on today’s location and orientation-aware smartphones within 3D Geospatial Web environments. Our application incorporates human interactions (interpreted from embedded sensors) in the geospatial query process by determining the shape of their actual visibility space as a query “window” in a spatial database, e.g. Isovist in 2D and Threat Dome in 3D. This effectively applies hidden query removal (HQR) functionality in 360º 3D that takes into account both the horizontal and vertical dimensions when calculating the 3D search space, significantly reducing display clutter and information overload on mobile devices. The effect is a more accurate and expected search result for mobile LBS applications by returning information on only those objects visible within a user’s 3D field-of-view. ii

Mobile 2D and 3D Spatial Query Techniques for the Geospatial Web

The increasing availability of abundant geographically referenced information in the Geospatial Web provides a variety of opportunities for developing value-added LBS applications. However, large data volumes of the Geospatial Web and small mobile device displays impose a data visualization problem, as the amount of searchable information overwhelms the display when too many query results are returned. Excessive returned results clutter the mobile display, making it harder for users to prioritize information and causes confusion and usability problems. Mobile Spatial Interaction (MSI) research into this “information overload” problem is ongoing where map personalization and other semantic based filtering mechanisms are essential to de-clutter and adapt the exploration of the real-world to the processing/display limitations of mobile devices. In this thesis, we propose that another way to filter this information is to intelligently refine the search space. 3DQ (3-Dimensional Query) is our novel MSI prototype for information discovery on today’s location and orientation-aware smartphones within 3D Geospatial Web environments. Our application incorporates human interactions (interpreted from embedded sensors) in the geospatial query process by determining the shape of their actual visibility space as a query “window” in a spatial database, e.g. Isovist in 2D and Threat Dome in 3D. This effectively applies hidden query removal (HQR) functionality in 360º 3D that takes into account both the horizontal and vertical dimensions when calculating the 3D search space, significantly reducing display clutter and information overload on mobile devices. The effect is a more accurate and expected search result for mobile LBS applications by returning information on only those objects visible within a user’s 3D field-of-view

Playful AR - Playfulness and augmented reality in a route planning application

This thesis explores implementing playfulness into a route planning application using augmented reality. While AR in itself has been around since the 1960s, the use of it on smartphones is a quite new territory. The potential for AR as an everyday tool grows with the development of technology, and the use of the cameras in smartphones. Because of its newness there are still issues with the user experience and different opinions on how to best utilise it. The preliminary idea for this design concept was of a route planning application with AR, which helps the user map themselves in 3D space. Looking at past and present applications, many of them are games or have a lighthearted way of using AR. Implementing playfulness in design can make the user experience more pleasurable for the user. Adding playfulness as a means of introducing users to a new way of navigating, not only new applications and technology, but in the real world as well. The combining of the AR app and playfulness was achieved in this thesis with the help of the Playful Experience Framework (PLEX). A video prototype was made based on the design concept derived from the PLEX cards. The video prototype was then used in interviews where the design concept was explained. The aim of the interviews was to gauge the overall reaction to this type of application and answer questions such as: who would use it, how would they use it, and how they found the playful experience? The reception of playful aspect of the application played a part in analysing the responses in order to see how to proceed with the concept and whether it was a viable idea or not. The overall opinion of the design concept was positive with many wanting to test it out, which would be the next step in development after new iteration is complete, fixing the issues found in the current one.Denna avhandling utforskar implementationen av lekfullhet i en ruttplaneringsapplikation med hjälp av förstärkt verklighet (augmented reality / AR). Medan AR i sig har funnits sedan 1960-talet är användningen av det i smartmobiler ett helt nytt område. Potentialen för AR som ett vardagsverktyg växer i takt med utvecklingen av teknik och användningen av kamerorna i smartmobiler. Eftersom AR är en ny användarupplevelse, finns det fortfarande problem med den och åsikterna om hur den bäst ska utnyttjas varierar. Den preliminära idén för avhandlingens koncept var en ruttplaneringsapplikation med AR som hjälper användaren orientera sig själv i en 3D-värld. Många nu existerande AR-applikationer är spel eller utnyttjar AR på ett lekfullt sätt vilket kan göra användarupplevelsen mer njutbar för användaren. Lekfullhet kan användas som ett sätt att introducera användare till ett nytt sätt att navigera, inte bara nya applikationer och teknik, utan också i den verkliga världen. I denna avhandling förverkligades kombinationen av ar-appen och lekfullhet med hjälp av ett så kallat “Playful Experience Framework” (PLEX). En videoprototyp gjordes baserat på PLEX kortens koncept. Videoprototypen användes sedan i in-tervjuer där designkonceptet förklarades. Syftet med intervjuerna var att mäta den övergripande reaktionen på denna typ av applikation och svara på frågor som: vem skulle använda den, hur skulle de använda den, och hur den lekfulla upplevelsen upplevdes. För att se hur man skulle kunna fortsätta med konceptet och huruvida det var en genomförbar idé eller inte, var motta-gandet av den lekfulla aspekten av applikationen speciellt viktig i analysen av svaren. Det allmänna mottagandet av konceptet var positiv och många sade sig vilja testa prototypen, vilket skulle vara nästa steg i utvecklingen efter det att en ny iteration är färdig, med korrigeringar av de problem som identifierats i den nuvarande versionen

Vu: Integrating AR Technology and Interaction into an Event Planning App

Planning a social event can be expensive and time consuming. To minimalize the risk of event problems, the organizer can consult professional event planners. However, a consultant can also be costly. Therefore, purchasing decor, food, and other items, without knowing if they look right or fit the venue, is a guessing game, and the game could be an expensive one. If the original plan cannot be completed efficiently, then modifying or improving these works are likely to cost extra time and funds. However, testing the revised plan may also increase the likelihood of risk in future. By integrating Augmented Reality (AR) into an event planning App, the App will allow users to arrange virtual items onto environment captured by the device. Thus, users can envision their plan and make changes before actually making purchases, calling in construction teams and doing the decorations. The goal of this thesis is to integrate AR into an App design that allows users to design, view, and make budgets for their event plan in advance, optimizing their design beforehand

Reviving the Euston Arch: A Mixed Reality Approach to Cultural Heritage Tours

Augmented Reality (AR) and Virtual Reality (VR) users have distinct capabilities and experiences during Extended Reality (XR) collaborations: while AR users benefit from real-time contextual information due to physical presence, VR users enjoy the flexibility to transition between locations rapidly, unconstrained by physical space.Our research aims to utilize these spatial differences to facilitate engaging, shared XR experiences. Using Google Geospatial Creator, we enable large-scale outdoor authoring and precise localization to create a unified environment. We integrated Ubiq to allow simultaneous voice communication, avatar-based interaction and shared object manipulation across platforms.We apply AR and VR technologies in cultural heritage exploration. We selected the Euston Arch as our case study due to its dramatic architectural transformations over time. We enriched the co-exploration experience by integrating historical photos, a 3D model of the Euston Arch, and immersive audio narratives into the shared AR/VR environment