Protecting Visual Information in Augmented Reality from Malicious Application Developers

abstract: Visual applications – those that use camera frames as part of the application – provide a rich, context-aware experience. The continued development of mixed and augmented reality (MR/AR) computing environments furthers the richness of this experience by providing applications a continuous vision experience, where visual information continuously provides context for applications and the real world is augmented by the virtual. To understand user privacy concerns in continuous vision computing environments, this work studies three MR/AR applications (augmented markers, augmented faces, and text capture) to show that in a modern mobile system, the typical user is exposed to potential mass collection of sensitive information, posing privacy and security deficiencies to be addressed in future systems. To address such deficiencies, a development framework is proposed that provides resource isolation between user information contained in camera frames and application access to the network. The design is implemented using existing system utilities as a proof of concept on the Android operating system and demonstrates its viability with a modern state-of-the-art augmented reality library and several augmented reality applications. Evaluation is conducted on the design on a Samsung Galaxy S8 phone by comparing the applications from the case study with modified versions which better protect user privacy. Early results show that the new design efficiently protects users against data collection in MR/AR applications with less than 0.7% performance overhead.Dissertation/ThesisMasters Thesis Computer Engineering 201

TouristicAR: A Smart Glass Augmented Reality Application for UNESCO World Heritage Sites in Malaysia.

Modern day technologies including wearable devices such as smart glasses have changed the way people interact with their surroundings. Such developments have resulted in the increased popularity of Augmented Reality (AR) applications to project context-aware information on objects or users’ immediate surroundings. In order to enhance the overall tourism experience, a number of recent works have highlighted the opportunities for using outdoor AR or navigation-based AR systems. Due to the development of context-aware AR, tourists using such technology can acquire valuable knowledge and experience. Recent context-aware AR research lack empirical studies and works that integrate dimensions which are specific to cultural heritage tourism and smart glass specific context. This work presents the mapping of requirements identified during the affinity mapping and focuses group experiments into a smart glass-based AR application called “TouristicAR” to identifying the factors of user acceptance and specific tourist requirements. This works highlights the technical requirement, design conceptualisation, smart glass User Interface (UI) design and smart glass application development using a Rapid Application Development (RAD) methodology. The smart glass AR application is developed in the context of cultural heritage tourism and provides context-aware and visually appealing tourism contents to the visitors at the UNESCO World Heritage sites in Malaysia

Smartphone Augmented Reality Applications for Tourism

Invisible, attentive and adaptive technologies that provide tourists with relevant services and information anytime and anywhere may no longer be a vision from the future. The new display paradigm, stemming from the synergy of new mobile devices, context-awareness and AR, has the potential to enhance tourists’ experiences and make them exceptional. However, effective and usable design is still in its infancy. In this publication we present an overview of current smartphone AR applications outlining tourism-related domain-specific design challenges. This study is part of an ongoing research project aiming at developing a better understanding of the design space for smartphone context-aware AR applications for tourists

Location-based and contextual mobile learning. A STELLAR Small-Scale Study

This study starts from several inputs that the partners have collected from previous and current running research projects and a workshop organised at the STELLAR Alpine Rendevous 2010. In the study, several steps have been taken, firstly a literature review and analysis of existing systems; secondly, mobile learning experts have been involved in a concept mapping study to identify the main challenges that can be solved via mobile learning; and thirdly, an identification of educational patterns based on these examples has been done. Out of this study the partners aim to develop an educational framework for contextual learning as a unifying approach in the field. Therefore one of our central research questions is: how can we investigate, theorise, model and support contextual learning

Teaching Cultural Heritage using Mobile Augmented Reality

open2noThe relationship between augmented reality, mobile learning, gamification and non-formal education methods provide a great potential. The AR-CIMUVE Augmented Reality for the Walled Cities of the Veneto is an original project in collaboration with Italia Nostra and other associations which deal with transmitting our cultural heritage and which teach primary and middle school children the cultural and historical importance of the Veneto’s and the surrounding territories’ walled cities. In this learning experience students will explore how our environment has developed across the ages using the mobile devices with the technical back-up of the AR App. This will allow them to see maps, examine data, 3D models and will enable them to judge and improve their skills. From a pedagogical and educational point of view the emphasis is on a constructivist social-cultural approach which helps students to become active citizens more aware of their historical identity.openPetrucco, Corrado; Agostini, DanielePetrucco, Corrado; Agostini, Daniel

Adding generic contextual capabilities to wearable computers

Context-awareness has an increasingly important role to play in the development of wearable computing systems. In order to better define this role we have identified four generic contextual capabilities: sensing, adaptation, resource discovery, and augmentation. A prototype application has been constructed to explore how some of these capabilities could be deployed in a wearable system designed to aid an ecologist's observations of giraffe in a Kenyan game reserve. However, despite the benefits of context-awareness demonstrated in this prototype, widespread innovation of these capabilities is currently stifled by the difficulty in obtaining the contextual data. To remedy this situation the Contextual Information Service (CIS) is introduced. Installed on the user's wearable computer, the CIS provides a common point of access for clients to obtain, manipulate and model contextual information independently of the underlying plethora of data formats and sensor interface mechanisms

From Sensor to Observation Web with Environmental Enablers in the Future Internet

This paper outlines the grand challenges in global sustainability research and the objectives of the FP7 Future Internet PPP program within the Digital Agenda for Europe. Large user communities are generating significant amounts of valuable environmental observations at local and regional scales using the devices and services of the Future Internet. These communities’ environmental observations represent a wealth of information which is currently hardly used or used only in isolation and therefore in need of integration with other information sources. Indeed, this very integration will lead to a paradigm shift from a mere Sensor Web to an Observation Web with semantically enriched content emanating from sensors, environmental simulations and citizens. The paper also describes the research challenges to realize the Observation Web and the associated environmental enablers for the Future Internet. Such an environmental enabler could for instance be an electronic sensing device, a web-service application, or even a social networking group affording or facilitating the capability of the Future Internet applications to consume, produce, and use environmental observations in cross-domain applications. The term ?envirofied? Future Internet is coined to describe this overall target that forms a cornerstone of work in the Environmental Usage Area within the Future Internet PPP program. Relevant trends described in the paper are the usage of ubiquitous sensors (anywhere), the provision and generation of information by citizens, and the convergence of real and virtual realities to convey understanding of environmental observations. The paper addresses the technical challenges in the Environmental Usage Area and the need for designing multi-style service oriented architecture. Key topics are the mapping of requirements to capabilities, providing scalability and robustness with implementing context aware information retrieval. Another essential research topic is handling data fusion and model based computation, and the related propagation of information uncertainty. Approaches to security, standardization and harmonization, all essential for sustainable solutions, are summarized from the perspective of the Environmental Usage Area. The paper concludes with an overview of emerging, high impact applications in the environmental areas concerning land ecosystems (biodiversity), air quality (atmospheric conditions) and water ecosystems (marine asset management)