Negotiating Reality

Our understanding of research through design is demonstrated by a close examination of the methods used in the project lifeClipper2. This design research project investigates the applicability of immersive outdoor Augmented Reality (AR). lifeClipper2 offers an audiovisual walking experience in a virtually extended public space and focuses on audiovisual perception as well as on the development of the appropriate technology. The project involves contributions of partners from different fields of research. Thus, lifeClipper2 is able to test the potential of AR for visualizing architecture and archaeological information and to challenge our understanding of perception and interaction. Using examples from our research, the paper reflects on how scenario design contributes to the production of design knowledge and explores the possibilities and variations of AR. Finally, the paper drafts our approach to design research. The three tenets of our work are: the use of scenarios as a tool of interdisciplinary research, the experimental exploration of media and the intention to make design knowledge explicit. Keywords: augmented reality; locative media; hybrid environment; immersion; perception; experience design; research through design; scenario design</p

'Breaking the glass': preserving social history in virtual environments

New media technologies play an important role in the evolution of our society. Traditional museums and heritage sites have evolved from the ‘cabinets of curiosity’ that focused mainly on the authority of the voice organising content, to the places that offer interactivity as a means to experience historical and cultural events of the past. They attempt to break down the division between visitors and historical artefacts, employing modern technologies that allow the audience to perceive a range of perspectives of the historical event. In this paper, we discuss virtual reconstruction and interactive storytelling techniques as a research methodology and educational and presentation practices for cultural heritage sites. We present the Narrating the Past project as a case study, in order to illustrate recent changes in the preservation of social history and guided tourist trails that aim to make the visitor’s experience more than just an architectural walk through

3D (embodied) projection mapping and sensing bodies : a study in interactive dance performance

This dissertation identifies the synergies between physical and virtual environments when designing for immersive experiences in interactive dance performances. The integration of virtual information in physical space is transforming our interactions and experiences with the world. By using the body and creative expression as the interface between real and virtual worlds, dance performance creates a privileged framework to research and design interactive mixed reality environments and immersive augmented architectures. The research is primarily situated in the fields of visual art and interaction design. It combines performance with transdisciplinary fields and intertwines practice with theory. The theoretical and conceptual implications involved in designing and experiencing immersive hybrid environments are analyzed using the reality–virtuality continuum. These theories helped frame the ways augmented reality architectures are achieved through the integration of dance performance with digital software and reception displays. They also helped identify the main artistic affordances and restrictions in the design of augmented reality and augmented virtuality environments for live performance. These pervasive media architectures were materialized in three field experiments, the live dance performances. Each performance was created in three different stages of conception, design and production. The first stage was to “digitize” the performer’s movement and brain activity to the virtual environment and our system. This was accomplished through the use of depth sensor cameras, 3D motion capture, and brain computer interfaces. The second stage was the creation of the computational architecture and software that aggregates the connections and mapping between the physical body and the spatial dynamics of the virtual environment. This process created real-time interactions between the performer’s behavior and motion and the real-time generative computer 3D graphics. Finally, the third stage consisted of the output modality: 3D projector based augmentation techniques were adopted in order to overlay the virtual environment onto physical space. This thesis proposes and lays out theoretical, technical, and artistic frameworks between 3D digital environments and moving bodies in dance performance. By sensing the body and the brain with the 3D virtual environments, new layers of augmentation and interactions are established, and ultimately this generates mixed reality environments for embodied improvisational self-expression.Radio-Television-Fil

Enabling Audiovisual User Interfaces

“Enabling Audiovisual User Interfaces” was a 2-year project, supported by a Marie Curie EU fellowship, conducted at EAVI, Goldsmiths. During the project, I investigated how human-computer interactions can be audiovisualized in order to improve user experience and usability. To address this issue, a new UI paradigm was proposed – AVUI (AudioVisual User Interface). AVUI links interaction, sound and image, building upon the concept of Graphical User Interface (GUI) by adding interconnected sound and image. The research hypothesis was: the introduction of AVUI, integrating interrelated sonic and visual feedback, reacting to user interactions, would lead to more usable, accessible, playful and engaging UIs, as compared to a traditional GUI – particularly in use cases where accessibility and/or engagement were determinant. I applied AVUIs to case studies, which was the object of user testing. After reaching conclusions from these, I proposed an AVUI framework, including software modules and a set of best practices. Dissemination activities were also implemented

Understanding user experience of mobile video: Framework, measurement, and optimization

Since users have become the focus of product/service design in last decade, the term User eXperience (UX) has been frequently used in the field of Human-Computer-Interaction (HCI). Research on UX facilitates a better understanding of the various aspects of the user’s interaction with the product or service. Mobile video, as a new and promising service and research field, has attracted great attention. Due to the significance of UX in the success of mobile video (Jordan, 2002), many researchers have centered on this area, examining users’ expectations, motivations, requirements, and usage context. As a result, many influencing factors have been explored (Buchinger, Kriglstein, Brandt &amp; Hlavacs, 2011; Buchinger, Kriglstein &amp; Hlavacs, 2009). However, a general framework for specific mobile video service is lacking for structuring such a great number of factors. To measure user experience of multimedia services such as mobile video, quality of experience (QoE) has recently become a prominent concept. In contrast to the traditionally used concept quality of service (QoS), QoE not only involves objectively measuring the delivered service but also takes into account user’s needs and desires when using the service, emphasizing the user’s overall acceptability on the service. Many QoE metrics are able to estimate the user perceived quality or acceptability of mobile video, but may be not enough accurate for the overall UX prediction due to the complexity of UX. Only a few frameworks of QoE have addressed more aspects of UX for mobile multimedia applications but need be transformed into practical measures. The challenge of optimizing UX remains adaptations to the resource constrains (e.g., network conditions, mobile device capabilities, and heterogeneous usage contexts) as well as meeting complicated user requirements (e.g., usage purposes and personal preferences). In this chapter, we investigate the existing important UX frameworks, compare their similarities and discuss some important features that fit in the mobile video service. Based on the previous research, we propose a simple UX framework for mobile video application by mapping a variety of influencing factors of UX upon a typical mobile video delivery system. Each component and its factors are explored with comprehensive literature reviews. The proposed framework may benefit in user-centred design of mobile video through taking a complete consideration of UX influences and in improvement of mobile videoservice quality by adjusting the values of certain factors to produce a positive user experience. It may also facilitate relative research in the way of locating important issues to study, clarifying research scopes, and setting up proper study procedures. We then review a great deal of research on UX measurement, including QoE metrics and QoE frameworks of mobile multimedia. Finally, we discuss how to achieve an optimal quality of user experience by focusing on the issues of various aspects of UX of mobile video. In the conclusion, we suggest some open issues for future study

AVUI: Designing a toolkit for audiovisual interfaces

The combined use of sound and image has a rich history, from audiovisual artworks to research exploring the potential of data visualization and sonification. However, we lack standard tools or guidelines for audiovisual (AV) interaction design, particularly for live performance. We propose the AVUI (AudioVisual User Interface), where sound and image are used together in a cohesive way in the interface; and an enabling technology, the ofxAVUI toolkit. AVUI guidelines and ofxAVUI were developed in a three-stage process, together with AV producers: 1) participatory design activities; 2) prototype development; 3) encapsulation of prototype as a plug-in, evaluation, and roll out. Best practices identified include: reconfigurable interfaces and mappings; object-oriented packaging of AV and UI; diverse sound visualization; flexible media manipulation and management. The toolkit and a mobile app developed using it have been released as open-source. Guidelines and toolkit demonstrate the potential of AVUI and offer designers a convenient framework for AV interaction design

Framework for the implementation of urban big screens in the public space

In the last decade, big urban screens have appeared in town squares and on building facades across the UK. The use of these screens brings new potentials and challenges for city regulators, artists, architects, urban designers, producers, broadcasters and advertisers. Dynamic moving images form new architectural material, affecting our perception and the experience of the space around us. A new form of urban space is emerging that is fundamentally different from what we have known, and it seems that we are ill-equipped to deal with and analyse it. We are just beginning to understand the opportunities for public information, art and community engagement. Most of screens at present serve mainly commercial purposes, they do not broadcast information aimed at sharing community content nor do they support public social interactions. We need to see more negotiation between commercial, public and cultural interests. The SCREAM project addresses these new challenges by looking at the physical urban spaces and the potential spaces created by the new technologies