Augmented reality application assessment for disseminating rock art

[EN] Currently, marker-based tracking is the most used method to develop augmented reality (AR) applications (apps). However, this method cannot be applied in some complex and outdoor settings such as prehistoric rock art sites owing to the fact that the usage of markers is restricted on site. Thus, natural feature tracking methods have to be used. There is a wide range of libraries to develop AR apps based on natural feature tracking. In this paper, a comparative study of Vuforia and ARToolKit libraries is carried out, analysing factors such as distance, occlusion and lighting conditions that affect user experience in both indoor and outdoor environments, and eventually the app developer. Our analysis confirms that Vuforia¿s user experience indoor is better, faster and flicker-free whether the images are properly enhanced, but it does not work properly on site. Therefore, the development of AR apps for complex outdoor environments such as rock art sites should be performed with ARToolKit.The authors gratefully acknowledge the support from the Spanish Ministerio de Economia y Competitividad to the project HAR2014-59873-R. Similarly, the authors want to express their gratitude to the General Directorate of Culture and Heritage, Conselleria d'Educacio, Investigacio, Cultura i Esport, Generalitat Valenciana for letting us access and carry out research at the archaeological site.Blanco-Pons, S.; Carrión-Ruiz, B.; Lerma, JL. (2018). Augmented reality application assessment for disseminating rock art. Multimedia Tools and Applications. 78(8):10265-10286. https://doi.org/10.1007/s11042-018-6609-xS1026510286788Alahi A., Ortiz R., Vandergheynst P (2012) FREAK: fast retina keypoint. Comput Vis Pattern Recognit 510–517 . doi: https://doi.org/10.1109/CVPR.2012.6247715Amin D, Govilkar S (2015) Comparative study of augmented reality Sdk’S. Int J Comput Sci Appl 5:11–26. https://doi.org/10.1227/01.NEU.0000297044.82035.57ARCore ARCore - Google Developer | ARCore | Google Developers. https://developers.google.com/ar/ . Accessed 26 Jun 2018ARKit ARKit - Apple Developer. https://developer.apple.com/arkit/ . Accessed 26 Jun 2018ARToolkit (2017) ARToolkit. https://archive.artoolkit.org/ . Accessed 2 Oct 2017ARToolkit (2017) About. https://artoolkit.org/about-artoolkit . Accessed 11 Apr 2017ARToolkit (2017) Documentation. https://artoolkit.org/documentation/ . Accessed 12 Apr 2017ArUco ArUco: A minimal library for Augmented Reality applications based on OpenCV | Aplicaciones de la Visión Artificial. https://www.uco.es/investiga/grupos/ava/node/26 . Accessed 19 Apr 2018Azuma R (1997) A survey of augmented reality. Presence Teleoperators Virt Environ 6:355–385 . doi: 10.1.1.30.4999Azuma R, Baillot Y, Feiner S et al (2001) Recent advances in augmented reality. Ieee Comput Graph Appl 34–47. doi: https://doi.org/10.4061/2011/908468Blanco-Novoa O, Fernandez-Carames TM, Fraga-Lamas P, Vilar-Montesinos M (2018) A practical evaluation of commercial industrial augmented reality systems in an industry 4.0 shipyard. IEEE Access 6:1–1. https://doi.org/10.1109/ACCESS.2018.2802699Blanco-Pons S, Carrión-Ruiz B, Lerma JL (2016) Review of augmented reality and virtual reality techniques in rock art. Proc 8th Int Congress Archaeol Comput Graph Cult Herit Innov ‘ARQUEOLÓGICA 2.0L: 176–183Brancati N, Caggianese G, Frucci M et al (2017) Experiencing touchless interaction with augmented content on wearable head-mounted displays in cultural heritage applications. Pers Ubiquitous Comput 21:203–217. https://doi.org/10.1007/s00779-016-0987-8Cagalaban G, Kim S (2010) Multiple object tracking in unprepared environments using combined feature for augmented reality applications. Springer, Berlin, HeidelbergCamera-Calibration Camera Calibration App for Android [ARToolkit]. https://archive.artoolkit.org/documentation/doku.php?id=4_Android:android_camera_calibration . Accessed 16 Oct 2017Carmigniani J, Furht B, Anisetti M et al (2011) Augmented reality technologies, systems and applications. Multimed Tools Appl 51:341–377. https://doi.org/10.1007/s11042-010-0660-6Carrión-Ruiz B, Blanco-Pons S, Lerma JL (2016) Digital image analysis of the visible region through simulation of rock art paintings. Proc 8th Int Congress Archaeol Comput Graph, Cult Heritage Innov ‘ARQUEOLÓGICA 2.0.’: 169–175Chen CY, Chang BR, Sen HP (2014) Multimedia augmented reality information system for museum guidance. Pers Ubiquitous Comput 18:315–322. https://doi.org/10.1007/s00779-013-0647-1CRYENGINE CRYENGINE | The complete solution for next generation game development by Crytek. https://www.cryengine.com/ . Accessed 7 Jun 2017Domingo I, Carrión B, Blanco S, Lerma JL (2015) Evaluating conventional and advanced visible image enhancement solutions to produce digital tracings at el Carche rock art shelter. Digit Appl Archaeol Cult Herit 2:79–88. https://doi.org/10.1016/j.daach.2015.01.001Dos Santos AB, Dourado JB, Bezerra A (2016) ARToolkit and Qualcomm Vuforia: An Analytical Collation. Proc - 18th Symp Virt Augment Real SVR 2016:229–233. https://doi.org/10.1109/SVR.2016.46DroidAR (2017) DroidAR by bitstars. https://bitstars.github.io/droidar/ . Accessed 10 Dec 2017Engine U (2017) Unreal Engine. https://www.unrealengine.com/ . Accessed 10 Oct 2017Fiala M (2005) ARTag, a fiducial marker system using digital techniques. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:590–596. https://doi.org/10.1109/CVPR.2005.74Fischer J, Eichler M, Bartz D, Straßer W (2007) A hybrid tracking method for surgical augmented reality. Comput Graph 31:39–52. https://doi.org/10.1016/j.cag.2006.09.007González C, Vallejo D, Albusac J, Castro J (2011) Realidad Aumentada. Un enfoque práctico con ARToolKit y Blender. 2–120Gutierrez JM, Molinero MA, Soto-Martín O, Medina CR (2015) Augmented reality technology spreads information about historical graffiti in temple of Debod. Procedia Comput Sci 75:390–397. https://doi.org/10.1016/j.procs.2015.12.262Haladová ZB, Szemzö R, Kovačovský T, Žižka J (2015) Utilizing Multispectral Scanning and Augmented Reality for Enhancement and Visualization of the Wooden Sculpture Restoration Process. Procedia Comput Sci 67:340–347. https://doi.org/10.1016/j.procs.2015.09.278Jamali SS, Shiratuddin MF, Wong KW, Oskam CL (2015) Utilising mobile-augmented reality for learning human anatomy. Procedia - Soc Behav Sci 197:659–668. https://doi.org/10.1016/j.sbspro.2015.07.054Khan D, Ullah S, Rabbi I (2015) Factors affecting the design and tracking of ARToolKit markers. Comput Stand Interf 41:56–66. https://doi.org/10.1016/j.csi.2015.02.006Khan D, Ullah S, Yan D et al (2018) Robust tracking through the design of high quality fiducial markers: an optimization tool for ARToolKit. IEEE Access 4:22421–22433. https://doi.org/10.1109/ACCESS.2018.2801028Kim SL, Suk HJ, Kang JH, et al (2014) Using unity 3D to facilitate mobile augmented reality game development. Internet things (WF-IoT), 2014 IEEE World Forum 21–26 . doi: https://doi.org/10.1109/WF-IoT.2014.6803110Kounavis CD, Kasimati AE, Zamani ED (2012) Enhancing the tourism experience through mobile augmented reality: challenges and prospects. Int J Eng Bus Manag 4:1–6. https://doi.org/10.5772/51644La Delfa GC, Monteleone S, Catania V et al (2016) Performance analysis of visualmarkers for indoor navigation systems. Front Inf Technol Electron Eng 17:730–740. https://doi.org/10.1631/FITEE.1500324Liu S, Ge S, Yu H (2016) Research on Robustness recognition algorithms in augmented reality. 3rd Int Conf Inf Sci Control Eng: 547–552. doi: https://doi.org/10.1109/ICISCE.2016.123Lowe DG (2004) Distinctive image features from scale invariant keypoints. Int J Comput Vis 60:91–11020042. https://doi.org/10.1023/B:VISI.0000029664.99615.94Lytridis C, Tsinakos A, Kazanidis I (2018) ARTutor—an augmented reality platform for interactive distance learning. Educ Sci 8:6. https://doi.org/10.3390/educsci8010006Marchand E, Uchiyama H, Spindler F et al (2016) Pose estimation for augmented reality : a hands-on survey. IEEE Trans Vis Comput Graph 22:2633–2651. https://doi.org/10.1109/TVCG.2015.2513408Martínez R, Villaverde V (2002) La cova dels cavalls en el Barranc de la ValltortaMarto AGR, Sousa AA, de Gonçalves A (2017) DinofelisAR demo augmented reality based on natural features. 12a Conferência Ibérica Sist e Tecnol Informação, Lisboa 64:852–861. https://doi.org/10.1016/j.procs.2015.08.638Moreels P, Perona P (2007) Evaluation of feature detectors and descriptors based on 3D objects. Int J Comput Vis 73:263–284. https://doi.org/10.1007/s11263-006-9967-1Pierdicca R, Frontoni E, Zingaretti P et al (2015) Making visible the invisible. augmented reality visualization for 3D reconstructions of archaeological sites. Augment Virt Real Sec Int Conf AVR 2015 9254:25–37. https://doi.org/10.1007/978-3-319-22888-4Rabbi I, Ullah S, Javed M, Zen K (2014) Analysis of ARToolKit fiducial markers attributes for robust tracking. 1st Int Conf Recent Trends Inf Commun Technol Anal 281–290Radkowski R, Oliver J (2013) Natural feature tracking augmented reality for on-site assembly assistance systems. In: Shumaker R (ed) Virtual, Augmented and Mixed Reality. Systems and Applications. VAMR 2013. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg, pp 281–290Ridel B, Reuter P, Laviole J et al (2014) The revealing flashlight: interactive spatial augmented reality for detail exploration of cultural heritage artifacts. J Comput Cult Herit 7(6):1–6:18. https://doi.org/10.1145/2611376Seo J, Shim J, Choi JH, et al (2011) Enhancing marker-based AR technology. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 6773 LNCS:97–104 . doi: https://doi.org/10.1007/978-3-642-22021-0_12Seo J, Shim J, Choi JH et al (2011) Enhancing marker-based AR technology. In: International conference on virtual and mixed reality. virtual and mixed reality - new trends. Springer, Berlin, Heidelberg, pp 97–104Siltanen S (2015) Diminished reality for augmented reality interior design. Vis Comput 33:1–16. https://doi.org/10.1007/s00371-015-1174-zSörös G, Seichter H, Rautek P, Gröller E (2011) Augmented visualization with natural feature tracking. Proc 10th Int Conf Mob Ubiquitous Multimed 4–12. doi: https://doi.org/10.1145/2107596.2107597Uchiyama H, Marchand E (2012) Object detection and pose tracking for augmented reality: recent approaches. 18th Korea-Japan Jt Work Front Comput Vis 1–8Unity Unity. https://unity3d.com/es . Accessed 12 Oct 2017Vuforia (2017) Vuforia. https://www.vuforia.com/ . Accessed 2 Oct 2017Vuforia (2017) Vuforia-VuMark. https://library.vuforia.com/articles/Training/VuMark . Accessed 4 Apr 2017Vuforia (2017) Image targets. https://library.vuforia.com/articles/Training/Image-Target-Guide . Accessed 11 Apr 2017Wang H, Qin J, Zhang F (2015) A new interaction method for augmented reality based on ARToolKit. 2015 8th Int Congr Image Signal Process 578–583. doi: https://doi.org/10.1109/CISP.2015.7407945Wang G, Wang B, Zhong F et al (2015) Global optimal searching for textureless 3D object tracking. Vis Comput 31:979–988. https://doi.org/10.1007/s00371-015-1098-7Wu S, Oerlemans A, Bakker EM, Lew MS (2017) A comprehensive evaluation of local detectors and descriptors. Signal Process Image Commun 59:150–167. https://doi.org/10.1016/J.IMAGE.2017.06.010Xu Y, Wu Y, Zhou H, View M (2018) Multi-scale Voxel Hashing and Efficient 3D Representation for Mobile Augmented Reality. Cvpr 1618–1625 . doi: https://doi.org/10.1109/CVPRW.2018.0020

When reality backfires: Product evaluation context and the effectiveness of augmented reality in e‐commerce

Augmented reality (AR) enables consumers to project product holograms into their surrounding real-world context in real time using their mobile devices. Although AR may improve online consumers' product evaluation, AR-deploying retailers give up control over the context in which their products are evaluated. As a result, AR retailers' products might end up being evaluated in unfavorable contexts, such as disorganized rooms. Negative spillover effects from such unfavorable AR contexts onto the perceptions of evaluated products may lead consumers to refrain from purchasing the products. In two online experiments and a controlled field study with a total of 1000 participants, we find that unfavorable AR contexts negatively affect consumers' product-related purchase intention. This relationship is serially mediated by processing disfluency and deteriorating product quality perceptions of consumers. The negative contextual effects are mitigated if the product under evaluation is of unique design and thus more conceptually fluent or if the AR context becomes less perceptually salient and thus the product more perceptually fluent. We discuss diminished reality and facilitated product comparisons via AR as potential countermeasures for AR retailers and provide suggestions for future research

Augmented Reality: Emergent Applications and Opportunities for Industry 4.0

As the industry evolves further into digitalization, companies grow interested in new solutions made possible by the technologies of the Industry 4.0. One such paradigm is that of Augmented Reality (AR), which has seen significant growth in recent years, but still with great room for improvement, and still with many people unaware of its potential benefits. In its implementations, AR has generated value across a wide range of applications that aim at reducing the frequency of human error, decreasing the training time of professionals, and improving workflow. The purpose of this study is to understand the existing uses of AR across different fields, including its current state-of-the-art applications in the industrial sector, and find ways in which its systems in production lines could be improved, with a focus on the interaction between the user and the system, presenting a series of proposed solutions. To conclude, possible opportunities that could aid industry professionals and operators of manufacturing systems supported by AR are presented and discussed

Augmented reality: emergent applications and opportunities for industry 4.0

As the industry evolves further into digitalization, companies grow interested in new solutions made possible by the technologies of the Industry 4.0. One such paradigm is that of Augmented Reality (AR), which has seen significant growth in recent years, but still with great room for improvement, and still with many people unaware of its potential benefits. In its implementations, AR has generated value across a wide range of applications that aim at reducing the frequency of human error, decreasing the training time of professionals, and improving workflow. The purpose of this study is to understand the existing uses of AR across different fields, including its current state-of-the-art applications in the industrial sector, and find ways in which its systems in production lines could be improved, with a focus on the interaction between the user and the system, presenting a series of proposed solutions. To conclude, possible opportunities that could aid industry professionals and operators of manufacturing systems supported by AR are presented and discussed.publishe

DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality

Diminished reality (DR) refers to the removal of real objects from the environment by virtually replacing them with their background. Modern DR frameworks use inpainting to hallucinate unobserved regions. While recent deep learning-based inpainting is promising, the DR use case is complicated by the need to generate coherent structure and 3D geometry (i.e., depth), in particular for advanced applications, such as 3D scene editing. In this paper, we propose DeepDR, a first RGB-D inpainting framework fulfilling all requirements of DR: Plausible image and geometry inpainting with coherent structure, running at real-time frame rates, with minimal temporal artifacts. Our structure-aware generative network allows us to explicitly condition color and depth outputs on the scene semantics, overcoming the difficulty of reconstructing sharp and consistent boundaries in regions with complex backgrounds. Experimental results show that the proposed framework can outperform related work qualitatively and quantitatively.Comment: 11 pages, 8 figures + 13 pages, 10 figures supplementary. Accepted at 3DV 202

Beyond the Screen: Reshaping the Workplace with Virtual and Augmented Reality

Although extended reality technologies have enjoyed an explosion in popularity in recent years, few applications are effectively used outside the entertainment or academic contexts. This work consists of a literature review regarding the effective integration of such technologies in the workplace. It aims to provide an updated view of how they are being used in that context. First, we examine existing research concerning virtual, augmented, and mixed-reality applications. We also analyze which have made their way to the workflows of companies and institutions. Furthermore, we circumscribe the aspects of extended reality technologies that determined this applicability

Towards a Digital Window: Interpenetrations, challenges and potential of augmented reality in architecture

The present work has as its starting point and inspiration in the observation of the habit of staying at the window, mostly performed by older adults, in Lisbon, Portugal. Beginning from this habit, we seek to substantiate, develop, and record speculative and artistic visual experiments that propose a digital reinterpretation of the architectural element window. Such experiments deal with the intertwining of diverse concepts as hybrid architecture, material, digital, virtual, and augmented reality (AR). The experiment, entitled Projected Windows, consists of three different installations where we visually simulate, through image capture and projection, various possibilities of visual reinterpretations of the window in the context of the interior of dwellings. The experiment is based on digital imaging. The first two parts of the experiment are non-interactive AR experience, while the third one is a visual interactive AR experience. The project is in the initial phases of development, indicating the potential of correlating concepts, which allow to fundament experiences and visual narratives that can instigate greater advances in terms of interactivity as the work progresses. &nbsp

Real-Time Object Removal in Augmented Reality

Diminished reality, as a sub-topic of augmented reality where digital information is overlaid on an environment, is the perceived removal of an object from an environment. Previous approaches to diminished reality used digital replacement techniques, inpainting, and multi-view homographies. However, few used a virtual representation of the real environment, limiting their domains to planar environments. This thesis provides a framework to achieve real-time diminished reality on an augmented reality headset. Using state-of-the-art hardware, we combine a virtual representation of the real environment with inpainting to remove existing objects from complex environments. Our work is found to be competitive with previous results, with a similar qualitative outcome under the limitations of available technology. Additionally, by implementing new texturing algorithms, a more detailed representation of the real environment is achieved