Ambient Intelligence for Next-Generation AR

Next-generation augmented reality (AR) promises a high degree of context-awareness - a detailed knowledge of the environmental, user, social and system conditions in which an AR experience takes place. This will facilitate both the closer integration of the real and virtual worlds, and the provision of context-specific content or adaptations. However, environmental awareness in particular is challenging to achieve using AR devices alone; not only are these mobile devices' view of an environment spatially and temporally limited, but the data obtained by onboard sensors is frequently inaccurate and incomplete. This, combined with the fact that many aspects of core AR functionality and user experiences are impacted by properties of the real environment, motivates the use of ambient IoT devices, wireless sensors and actuators placed in the surrounding environment, for the measurement and optimization of environment properties. In this book chapter we categorize and examine the wide variety of ways in which these IoT sensors and actuators can support or enhance AR experiences, including quantitative insights and proof-of-concept systems that will inform the development of future solutions. We outline the challenges and opportunities associated with several important research directions which must be addressed to realize the full potential of next-generation AR.Comment: This is a preprint of a book chapter which will appear in the Springer Handbook of the Metavers

Smart Kitchens for People with Cognitive Impairments: A Qualitative Study of Design Requirements

Individuals with cognitive impairments currently leverage extensive human resources during their transitions from assisted living to independent living. In Western Europe, many government-supported volunteer organizations provide sheltered living facilities; supervised environments in which people with cognitive impairments collaboratively learn daily living skills. In this paper, we describe communal cooking practices in sheltered living facilities and identify opportunities for supporting these with interactive technology to reduce volunteer workload. We conducted two contextual observations of twelve people with cognitive impairments cooking in sheltered living facilities and supplemented this data through interviews with four employees and volunteers who supervise them. Through thematic analysis, we identified four themes to inform design requirements for communal cooking activities: Work organization, community, supervision, and practicalities. Based on these, we present five design implications for assistive systems in kitchens for people with cognitive deficiencies

A prototype to study cognitive and aesthetic aspects of mixed reality technologies

Mixed reality systems integrate virtual reality with real-world perception and cognition to offer enhanced interaction possibilities with the environment. Our aim is to demonstrate that mixed reality technologies strongly affect our aesthetic sense and mental models. So, in designing such technologies, we need to incorporate perspectives from different disciplines. We present different approaches and implementations of cognitive enhancement and cognitive technologies, consider some practical applications of mixed reality systems and discuss how they impact the user perception, body image and aesthetic sense

Designing Sound for Social Robots: Advancing Professional Practice through Design Principles

Sound is one of the core modalities social robots can use to communicate with the humans around them in rich, engaging, and effective ways. While a robot's auditory communication happens predominantly through speech, a growing body of work demonstrates the various ways non-verbal robot sound can affect humans, and researchers have begun to formulate design recommendations that encourage using the medium to its full potential. However, formal strategies for successful robot sound design have so far not emerged, current frameworks and principles are largely untested and no effort has been made to survey creative robot sound design practice. In this dissertation, I combine creative practice, expert interviews, and human-robot interaction studies to advance our understanding of how designers can best ideate, create, and implement robot sound. In a first step, I map out a design space that combines established sound design frameworks with insights from interviews with robot sound design experts. I then systematically traverse this space across three robot sound design explorations, investigating (i) the effect of artificial movement sound on how robots are perceived, (ii) the benefits of applying compositional theory to robot sound design, and (iii) the role and potential of spatially distributed robot sound. Finally, I implement the designs from prior chapters into humanoid robot Diamandini, and deploy it as a case study. Based on a synthesis of the data collection and design practice conducted across the thesis, I argue that the creation of robot sound is best guided by four design perspectives: fiction (sound as a means to convey a narrative), composition (sound as its own separate listening experience), plasticity (sound as something that can vary and adapt over time), and space (spatial distribution of sound as a separate communication channel). The conclusion of the thesis presents these four perspectives and proposes eleven design principles across them which are supported by detailed examples. This work contributes an extensive body of design principles, process models, and techniques providing researchers and designers with new tools to enrich the way robots communicate with humans

RTST Trend Report: lead theme Contextualisation

Specht, M., Börner, D., Tabuenca, B., Ternier, S., De Vries, F., Kalz, M., Drachsler, H., & Schmitz, B. (2012). RTST Trend Report: lead theme Contextualisation. Deliverable 1.7 of STELLAR network of excellence. Heerlen, The Netherlands.In summary this trend-scouting report highlights different design dimensions of contextualizing learning. On the one hand designing educational context: the components and constituents of the educational setting, which also have to be orchestrated in an instructional design or the process of orchestration (Luckin, 2010, Specht, 2009) on the other hand bridging and linking learning contexts for seamless learning support: Wong et al. define design dimensions of seamless learning experiences and which gaps they identify and what challenges must be tackled to create seamless learning experiences (Wong, 2011).STELLAR Network of Excellence, Grant 23191

"Tap it again, Sam": Harmonizing the frontiers between digital and real worlds in education

Lifelong leaners are intrinsically motivated to embed learning activities into daily life activities. Finding a suitable combination of the two is not trivial since lifelong learners have to face conflicts of time and location. Hence, lifelong learners normally build personal learning ecologies in those moments they set aside to learn making use of their available resources. On the other hand, the advent of Near Field Communication (NFC) technology facilitates the harmonization in the interactions between the digital world and daily physical spaces. Likewise, NFC enabled phones are becoming more and more popular. The contribution of this manuscript is threefold: first, scientific literature where NFC has been used with a direct or indirect purpose to learn is reviewed, and potential uses for lifelong learners are identified; based on these findings the Ecology of Resources for Lifelong Learning is presented as suitable setup for the scaffolding of learning activities with NFC augmented physical spaces; finally, this ecology is piloted and different learning scenarios are proposed for further extension