Pervasive Displays Research: What's Next?

Reports on the 7th ACM International Symposium on Pervasive Displays that took place from June 6-8 in Munich, Germany

Viewing the Future? Virtual Reality In Journalism

Journalism underwent a flurry of virtual reality content creation, production and distribution starting in the final months of 2015. The New York Times distributed more than 1 million cardboard virtual reality viewers and released an app showing a spherical video short about displaced refugees. The Los Angeles Times landed people next to a crater on Mars. USA TODAY took visitors on a ride-along in the "Back to the Future" car on the Universal Studios lot and on a spin through Old Havana in a bright pink '57 Ford. ABC News went to North Korea for a spherical view of a military parade and to Syria to see artifacts threatened by war. The Emblematic Group, a company that creates virtual reality content, followed a woman navigating a gauntlet of anti- abortion demonstrators at a family planning clinic and allowed people to witness a murder-suicide stemming from domestic violence.In short, the period from October 2015 through February 2016 was one of significant experimentation with virtual reality (VR) storytelling. These efforts are part of an initial foray into determining whether VR is a feasible way to present news. The year 2016 is shaping up as a period of further testing and careful monitoring of potential growth in the use of virtual reality among consumers

Teaching complex theoretical multi-step problems in ICT networking through 3D printing and augmented reality

This paper presents a pilot study rationale and research methodology using a mixed media visualisation (3D printing and Augmented Reality simulation) learning intervention to help students in an ICT degree represent theoretical complex multi-step problems without a corresponding real world physical analog model. This is important because these concepts are difficult to visualise without a corresponding mental model. The proposed intervention uses an augmented reality application programmed with free commercially available tools, tested through an action research methodology, to evaluate the effectiveness of the mixed media visualisation techniques to teach ICT students networking. Specifically, 3D models of network equipment will be placed in a field and then the augmented reality app can be used to observe packet traversal and routing between the different devices as data travels from the source to the destination. Outcomes are expected to be an overall improvement in final skill level for all students

Substitutional reality:using the physical environment to design virtual reality experiences

Experiencing Virtual Reality in domestic and other uncontrolled settings is challenging due to the presence of physical objects and furniture that are not usually defined in the Virtual Environment. To address this challenge, we explore the concept of Substitutional Reality in the context of Virtual Reality: a class of Virtual Environments where every physical object surrounding a user is paired, with some degree of discrepancy, to a virtual counterpart. We present a model of potential substitutions and validate it in two user studies. In the first study we investigated factors that affect participants' suspension of disbelief and ease of use. We systematically altered the virtual representation of a physical object and recorded responses from 20 participants. The second study investigated users' levels of engagement as the physical proxy for a virtual object varied. From the results, we derive a set of guidelines for the design of future Substitutional Reality experiences

Promoting New Patterns in Household Energy Consumption with Pervasive Learning Games

Engaging computer games can be used to change energy consumption patterns in the home. PowerAgent is a pervasive game for Java-enabled mobile phones that is designed to influence everyday activities and use of electricity in the domestic setting. PowerAgent is connected to the household’s automatic electricity meter reading equipment via the cell network, and this setup makes it possible to use actual consumption data in the game. In this paper, we present a two-level model for cognitive and behavior learning, and we discuss the properties of PowerAgent in relation to the underlying situated learning, social learning, and persuasive technology components that we have included in the game

Reconfiguring experimental archaeology using 3D reconstruction

Experimental archaeology has long yielded valuable insights into the tools and techniques that featured in past peoples’ relationship with the material world around them. We can determine, for example, how many trees would need to be felled to construct a large round-house of the southern British Iron Age (over one hundred), infer the exact angle needed to strike a flint core in order to knap an arrowhead in the manner of a Neolithic hunter-gatherer, or recreate the precise environmental conditions needed to store grain in underground silos over the winter months, with only the technologies and materials available to Romano-Briton villagers (see Coles 1973; Reynolds 1993). However, experimental archaeology has, hitherto, confined itself to rather rigid, empirical and quantitative questions such as those posed in these examples. This is quite understandable, and in line with good scientific practice, which stipulates that any ‘experiment’ must be based on replicable data, and be reproducible. Despite their potential in this area however, it is notable that digital reconstruction technologies have yet to play a significant role in experimental archaeology. Whilst many excellent examples of digital 3D reconstruction of heritage sites exist (for example the Digital Roman Forum project: http://dlib.etc.ucla.edu/projects/Forum) most, if not all, of these are characterized by a drive to establish a photorealistic re-creation of physical features. This paper will discuss possibilities that lie beyond straightforward positivist re-creation of heritage sites, in the experimental reconstruction of intangible heritage. Between 2010 and 2012, the authors led the Motion in Place Platform project (MiPP: http://www.motioninplace.org/), a capital grant under the AHRC's DEDEFI scheme developing motion capture and analysis tools for exploring how people move through spaces. In the course of MiPP, a series of experiments were conducted using motion capture hardware and software at the Silchester Roman town archaeological excavation in Hampshire, and at the Butser Ancient Farm facility, where Romano-British and Iron Age dwellings have been constructed according to the best experimental practice. As well as reconstructing such Roman and early British dwellings in 3D, the authors were able to use motion capture to reconstruct the kind of activities that – according to the material evidence – are likely to have been carried out by the occupants who used them. Bespoke motion capture suits developed for the project were employed, and the traces captured and rendered with a combination of Autodesk and Unity3D software. This sheds new light on how the reconstructed spaces - and, by inference, their ancient counterparts - were most likely to have been used. In particular the exercises allowed the evaluation and visualisation of changes in behaviour which occur as a result of familiarity with an environment and the acquisition of expertise over time; and to assess how interaction between different actors affects how everyday tasks are carried out

Smart Kitchens for People with Cognitive Impairments: A Qualitative Study of Design Requirements

Individuals with cognitive impairments currently leverage extensive human resources during their transitions from assisted living to independent living. In Western Europe, many government-supported volunteer organizations provide sheltered living facilities; supervised environments in which people with cognitive impairments collaboratively learn daily living skills. In this paper, we describe communal cooking practices in sheltered living facilities and identify opportunities for supporting these with interactive technology to reduce volunteer workload. We conducted two contextual observations of twelve people with cognitive impairments cooking in sheltered living facilities and supplemented this data through interviews with four employees and volunteers who supervise them. Through thematic analysis, we identified four themes to inform design requirements for communal cooking activities: Work organization, community, supervision, and practicalities. Based on these, we present five design implications for assistive systems in kitchens for people with cognitive deficiencies

Enhancing interaction in mixed reality

With continuous technological innovation, we observe mixed reality emerging from research labs into the mainstream. The arrival of capable mixed reality devices transforms how we are entertained, consume information, and interact with computing systems, with the most recent being able to present synthesized stimuli to any of the human senses and substantially blur the boundaries between the real and virtual worlds. In order to build expressive and practical mixed reality experiences, designers, developers, and stakeholders need to understand and meet its upcoming challenges. This research contributes a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. We present the results of seven studies examining the challenges and opportunities of mixed reality experiences, the impact of modalities and interaction techniques on the user experience, and how to enhance the experiences. We begin with a study determining user attitudes towards mixed reality in domestic and educational environments, followed by six research probes that each investigate an aspect of reality or virtuality. In the first, a levitating steerable projector enables us to investigate how the real world can be enhanced without instrumenting the user. We show that the presentation of in-situ instructions for navigational tasks leads to a significantly higher ability to observe and recall real-world landmarks. With the second probe, we enhance the perception of reality by superimposing information usually not visible to the human eye. In amplifying the human vision, we enable users to perceive thermal radiation visually. Further, we examine the effect of substituting physical components with non-functional tangible proxies or entirely virtual representations. With the third research probe, we explore how to enhance virtuality to enable a user to input text on a physical keyboard while being immersed in the virtual world. Our prototype tracked the user’s hands and keyboard to enable generic text input. Our analysis of text entry performance showed the importance and effect of different hand representations. We then investigate how to touch virtuality by simulating generic haptic feedback for virtual reality and show how tactile feedback through quadcopters can significantly increase the sense of presence. Our final research probe investigates the usability and input space of smartphones within mixed reality environments, pairing the user’s smartphone as an input device with a secondary physical screen. Based on our learnings from these individual research probes, we developed a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. The taxonomy is based on the human sensory system and human capabilities of articulation. We showcased its versatility and set our research probes into perspective by organizing them inside the taxonomic space. The design guidelines are divided into user-centered and technology-centered. It is our hope that these will contribute to the bright future of mixed reality systems while emphasizing the new underlining interaction paradigm.Mixed Reality (vermischte Realitäten) gehen aufgrund kontinuierlicher technologischer Innovationen langsam von der reinen Forschung in den Massenmarkt über. Mit der Einführung von leistungsfähigen Mixed-Reality-Geräten verändert sich die Art und Weise, wie wir Unterhaltungsmedien und Informationen konsumieren und wie wir mit Computersystemen interagieren. Verschiedene existierende Geräte sind in der Lage, jeden der menschlichen Sinne mit synthetischen Reizen zu stimulieren. Hierdurch verschwimmt zunehmend die Grenze zwischen der realen und der virtuellen Welt. Um eindrucksstarke und praktische Mixed-Reality-Erfahrungen zu kreieren, müssen Designer und Entwicklerinnen die künftigen Herausforderungen und neuen Möglichkeiten verstehen. In dieser Dissertation präsentieren wir eine neue Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen. Wir stellen die Ergebnisse von sieben Studien vor, in denen die Herausforderungen und Chancen von Mixed-Reality-Erfahrungen, die Auswirkungen von Modalitäten und Interaktionstechniken auf die Benutzererfahrung und die Möglichkeiten zur Verbesserung dieser Erfahrungen untersucht werden. Wir beginnen mit einer Studie, in der die Haltung der nutzenden Person gegenüber Mixed Reality in häuslichen und Bildungsumgebungen analysiert wird. In sechs weiteren Fallstudien wird jeweils ein Aspekt der Realität oder Virtualität untersucht. In der ersten Fallstudie wird mithilfe eines schwebenden und steuerbaren Projektors untersucht, wie die Wahrnehmung der realen Welt erweitert werden kann, ohne dabei die Person mit Technologie auszustatten. Wir zeigen, dass die Darstellung von in-situ-Anweisungen für Navigationsaufgaben zu einer deutlich höheren Fähigkeit führt, Sehenswürdigkeiten der realen Welt zu beobachten und wiederzufinden. In der zweiten Fallstudie erweitern wir die Wahrnehmung der Realität durch Überlagerung von Echtzeitinformationen, die für das menschliche Auge normalerweise unsichtbar sind. Durch die Erweiterung des menschlichen Sehvermögens ermöglichen wir den Anwender:innen, Wärmestrahlung visuell wahrzunehmen. Darüber hinaus untersuchen wir, wie sich das Ersetzen von physischen Komponenten durch nicht funktionale, aber greifbare Replikate oder durch die vollständig virtuelle Darstellung auswirkt. In der dritten Fallstudie untersuchen wir, wie virtuelle Realitäten verbessert werden können, damit eine Person, die in der virtuellen Welt verweilt, Text auf einer physischen Tastatur eingeben kann. Unser Versuchsdemonstrator detektiert die Hände und die Tastatur, zeigt diese in der vermischen Realität an und ermöglicht somit die verbesserte Texteingaben. Unsere Analyse der Texteingabequalität zeigte die Wichtigkeit und Wirkung verschiedener Handdarstellungen. Anschließend untersuchen wir, wie man Virtualität berühren kann, indem wir generisches haptisches Feedback für virtuelle Realitäten simulieren. Wir zeigen, wie Quadrokopter taktiles Feedback ermöglichen und dadurch das Präsenzgefühl deutlich steigern können. Unsere letzte Fallstudie untersucht die Benutzerfreundlichkeit und den Eingaberaum von Smartphones in Mixed-Reality-Umgebungen. Hierbei wird das Smartphone der Person als Eingabegerät mit einem sekundären physischen Bildschirm verbunden, um die Ein- und Ausgabemodalitäten zu erweitern. Basierend auf unseren Erkenntnissen aus den einzelnen Fallstudien haben wir eine neuartige Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen entwickelt. Die Taxonomie basiert auf dem menschlichen Sinnessystem und den Artikulationsfähigkeiten. Wir stellen die vielseitige Verwendbarkeit vor und setzen unsere Fallstudien in Kontext, indem wir sie innerhalb des taxonomischen Raums einordnen. Die Gestaltungsrichtlinien sind in nutzerzentrierte und technologiezentrierte Richtlinien unterteilt. Es ist unsere Anliegen, dass diese Gestaltungsrichtlinien zu einer erfolgreichen Zukunft von Mixed-Reality-Systemen beitragen und gleichzeitig die neuen Interaktionsparadigmen hervorheben

Their memory:exploring veterans’ voices, virtual reality and collective memory

This paper focuses on the virtual reality (VR) project Their Memory and details the development and evaluation of virtual reality environments and experiences with respect to its impact on young people (14-35 demographic) with the narratives of veterans in Scotland. As part of the AHRC Immersive Experiences program, Their Memory was created to explore how game design techniques and immersive technology could be used to enhance existing historical research and enrich narratives to bring expansive experiences to hard-to-reach audiences. The project worked directly with the veterans’ charity, Poppyscotland, to create an environment and experience that would resonate with new audiences, and explore documentary and storytelling techniques for the commemoration of war and conflict. The design of the project evolved through co-design sessions with veterans and young people and culminated in the creation of a short, thought-provoking, narrative-driven experience. The VR experience enabled players to connect with the memories of veterans in Scotland and exploring the different conflicts or situations they experienced and how they make sense of them. The project brought together cross-sector expertise to research how immersive experiences can help memory-based organizations in engaging with wider audiences, raise awareness, and diversify current learning outputs. The paper details the design and development of the Virtual Reality project, through co-design, and how this engaged the audience and evolved the experience created. The paper includes a summative evaluation of events conducted with schoolchildren to assess the project and concludes with how the project evidences impact upon audiences and the potential for both technology and the experience

Sonic souvenirs: exploring the paradoxes of recorded sound for family remembering

Many studies have explored social processes and technologies associated with sharing photos. In contrast, we explore the role of sound as a medium for social reminiscing. We involved 10 families in recording 'sonic souvenirs' of their holidays. They shared and discussed their collections on their return. We compared these sounds with their photo taking activities and reminiscences. Both sounds and pictures triggered active collaborative reminiscing, and attempts to capture iconic representations of events. However sounds differed from photos in that they were more varied, familial and creative. Further, they often expressed the negative or mundane in order to be 'true to life', and were harder to interpret than photos. Finally we saw little use of pure explanatory narrative. We reflect on the relations between sound and family memory and propose new designs on the basis of our findings, to better support the sharing and manipulation of social sounds