Technological framework for ubiquitous interactions using context–aware mobile devices

This report presents research and development of dedicated system architecture, designed to enable its users to interact with each other as well as to access information on Points of Interest that exist in their immediate environment. This is accomplished through managing personal preferences and contextual information in a distributed manner and in real-time. The advantage of this system architecture is that it uses mobile devices, heterogeneous sensors and a selection of user interface paradigms to produce a sociotechnical framework to enhance the perception of the environment and promote intuitive interactions. The thrust of the work has been on software development and component integration. Iterative prototyping was adopted as a development method in order to effectively implement the users’ feedback and establish a platform for collaboration that closely meets the requirements and aids their decision-making process. The requirement acquisition was followed by the system-modelling phase in order to produce a robust software prototype. The implementation includes component-based development and extensive use of design patterns over native programming. Conclusively, the software product has become the means to evaluate differences in the use of mixed reality technologies in a ubiquitous scenario. The prototype can query a number of context sources such as sensors, or details of the personal profile, to acquire relevant data. The data (and metadata) is stored in opensource structures, so that they are accessible at every layer of the system architecture and at any time. By proactively processing the acquired context, the system can assist the users in their tasks (e.g. navigation) without explicit input – e.g. by simply creating a gesture with the device. However, advanced interaction with the application via the user interface is available for requests that are more complex. Representations of the real world objects, their spatial relations and other captured features of interest are visualised on scalable interfaces, ranging from 2D to 3D models and from photorealism to stylised clues and symbols. Two principal modes of operation have been implemented; one, using geo-referenced virtual reality models of the environment, updated in real time, and second, using the overlay of descriptive annotations and graphics on the video images of the surroundings, captured by a video camera. The latter is referred to as augmented reality. The continuous feed of the device position and orientation data, from the GPS receiver and the digital compass, into the application, makes the framework fit for use in unknown environments and therefore suitable for ubiquitous operation. This is one of the novelties of the proposed framework, because it enables a whole range of social, peer-to-peer interactions to take place. The scenarios of how the system could be employed to pursue these remote interactions and collaborative efforts on mobile devices are addressed in the context of urban navigation. The conceptual design and implementation of the novel location and orientation based algorithm for mobile AR are presented in detail. The system is, however, multifaceted and capable of supporting peer-to-peer exchange of information in a pervasive fashion, usable in various contexts. The modalities of these interactions are explored and laid out in several scenarios, but particularly in the context of user adoption. Two evaluation tasks took place. The preliminary evaluation examined certain aspects that influence user interaction while being immersed in a virtual environment, whereas the second summative evaluation compared the utility and certain usability aspects of the AR and VR interfaces

3D digital modelling, fabrication and installation for understanding space and place

Traditionally the teaching of history or theory on art and design courses often takes place in a lecture theatre. Space and place theory is integral to informing the practice led and practice-based experiences in architecture, interior and the built environment. The research team has investigated how digital modeling, fabrication and population tools can enhance the understanding of current theoretical debates surrounding space and place. The aim is to integrate inter-disciplinary practice allowing us to address key research questions relating to the emergence of digital fabrication and its potential impact upon art and design education. The purpose is to provide an engaging and informative situated display, offering an experiential and intuitive frame of reference for constructing and placing objects, activities or events into their spatial context. The research has potential to act as an integrative experiential framework through which we can learn more about different contexts or connections between themes or theories which provides a deeper understanding of space or place. In this new work with Taylor, Benincasa, and Unver evolve their practice through translating 3D research data for a series of new digital and physical experiments intended for enhancing or informing teaching and learning in art, design & architecture. The researchers experimented with a range of 3D software and the functionality of different tool parameters. Fabrication apps and 3D crowd simulation animation tools were used for the first time in this research to explore digital fabrication using cardboard in order to compose and construct 2D and 3D physical simulations of this well-known built environment in the landscape. The fabricated physical cardboard models we produced were located in studio spaces and 3D visual projection live drawing experiences were tested with students and staff working together. The 2D and 3D simulations that the team envisioned are both digital and real; and when installed facilitate a more kinesthetic experience of learning as students are able to create together, and interact with fabricated structures. This evolving research demonstrates how these 3D models, animations and fabrications have the potential to be used together as a catalyst to explore multiple projections of space, place identities, historical and cultural built environment concepts for art, design and architecture students at undergraduate and postgraduate level

A survey of pervasive displays for information presentation

Weiser’s seminal vision of ubiquitous computing had calm information presentation at its heart and identified an important challenge in providing pervasive yet unobtrusive information display while avoiding problems of information overload. Since this vision was first articulated, a range of approaches have emerged for presenting information on pervasive displays and digital screens of varying sizes are now an everyday feature of our environments. Such displays provide significant opportunities for presenting information in-situ to support users in a range of activities, and the growing expectation is that there is constant peripheral access to digital information. In this article we review three different pervasive display technologies used for information presentation: traditional 2D display media, urban media facades, and novel display hardware. Our survey identifies five emerging trends that cross all three technologies: an increasing focus on situatedness, a movement towards non-expert users, growing demand for accessible interaction, a potential for new applications of data, and a difficulty in balancing ‘calm’ computing against presentation of data at an appropriate granularity and complexity