Personalized Interaction with High-Resolution Wall Displays

Fallende Hardwarepreise sowie eine zunehmende Offenheit gegenüber neuartigen Interaktionsmodalitäten haben in den vergangen Jahren den Einsatz von wandgroßen interaktiven Displays möglich gemacht, und in der Folge ist ihre Anwendung, unter anderem in den Bereichen Visualisierung, Bildung, und der Unterstützung von Meetings, erfolgreich demonstriert worden. Aufgrund ihrer Größe sind Wanddisplays für die Interaktion mit mehreren Benutzern prädestiniert. Gleichzeitig kann angenommen werden, dass Zugang zu persönlichen Daten und Einstellungen — mithin personalisierte Interaktion — weiterhin essentieller Bestandteil der meisten Anwendungsfälle sein wird. Aktuelle Benutzerschnittstellen im Desktop- und Mobilbereich steuern Zugriffe über ein initiales Login. Die Annahme, dass es nur einen Benutzer pro Bildschirm gibt, zieht sich durch das gesamte System, und ermöglicht unter anderem den Zugriff auf persönliche Daten und Kommunikation sowie persönliche Einstellungen. Gibt es hingegen mehrere Benutzer an einem großen Bildschirm, müssen hierfür Alternativen gefunden werden. Die daraus folgende Forschungsfrage dieser Dissertation lautet: Wie können wir im Kontext von Mehrbenutzerinteraktion mit wandgroßen Displays personalisierte Schnittstellen zur Verfügung stellen? Die Dissertation befasst sich sowohl mit personalisierter Interaktion in der Nähe (mit Touch als Eingabemodalität) als auch in etwas weiterer Entfernung (unter Nutzung zusätzlicher mobiler Geräte). Grundlage für personalisierte Mehrbenutzerinteraktion sind technische Lösungen für die Zuordnung von Benutzern zu einzelnen Interaktionen. Hierzu werden zwei Alternativen untersucht: In der ersten werden Nutzer via Kamera verfolgt, und in der zweiten werden Mobilgeräte anhand von Ultraschallsignalen geortet. Darauf aufbauend werden Interaktionstechniken vorgestellt, die personalisierte Interaktion unterstützen. Diese nutzen zusätzliche Mobilgeräte, die den Zugriff auf persönliche Daten sowie Interaktion in einigem Abstand von der Displaywand ermöglichen. Einen weiteren Teil der Arbeit bildet die Untersuchung der praktischen Auswirkungen der Ausgabe- und Interaktionsmodalitäten für personalisierte Interaktion. Hierzu wird eine qualitative Studie vorgestellt, die Nutzerverhalten anhand des kooperativen Mehrbenutzerspiels Miners analysiert. Der abschließende Beitrag beschäftigt sich mit dem Analyseprozess selber: Es wird das Analysetoolkit für Wandinteraktionen GIAnT vorgestellt, das Nutzerbewegungen, Interaktionen, und Blickrichtungen visualisiert und dadurch die Untersuchung der Interaktionen stark vereinfacht.An increasing openness for more diverse interaction modalities as well as falling hardware prices have made very large interactive vertical displays more feasible, and consequently, applications in settings such as visualization, education, and meeting support have been demonstrated successfully. Their size makes wall displays inherently usable for multi-user interaction. At the same time, we can assume that access to personal data and settings, and thus personalized interaction, will still be essential in most use-cases. In most current desktop and mobile user interfaces, access is regulated via an initial login and the complete user interface is then personalized to this user: Access to personal data, configurations and communications all assume a single user per screen. In the case of multiple people using one screen, this is not a feasible solution and we must find alternatives. Therefore, this thesis addresses the research question: How can we provide personalized interfaces in the context of multi-user interaction with wall displays? The scope spans personalized interaction both close to the wall (using touch as input modality) and further away (using mobile devices). Technical solutions that identify users at each interaction can replace logins and enable personalized interaction for multiple users at once. This thesis explores two alternative means of user identification: Tracking using RGB+depth-based cameras and leveraging ultrasound positioning of the users' mobile devices. Building on this, techniques that support personalized interaction using personal mobile devices are proposed. In the first contribution on interaction, HyDAP, we examine pointing from the perspective of moving users, and in the second, SleeD, we propose using an arm-worn device to facilitate access to private data and personalized interface elements. Additionally, the work contributes insights on practical implications of personalized interaction at wall displays: We present a qualitative study that analyses interaction using a multi-user cooperative game as application case, finding awareness and occlusion issues. The final contribution is a corresponding analysis toolkit that visualizes users' movements, touch interactions and gaze points when interacting with wall displays and thus allows fine-grained investigation of the interactions

Gsi demo: Multiuser gesture/speech interaction over digital tables by wrapping single user applications

Most commercial software applications are designed for a single user using a keyboard/mouse over an upright monitor. Our interest is exploiting these systems so they work over a digital table. Mirroring what people do when working over traditional tables, we want to allow multiple people to interact naturally with the tabletop application and with each other via rich speech and hand gesture and speech interaction on a digital table for geospatial applications- Google Earth, Warcraft III and The Sims. In this paper, we describe our underlying architecture: GSI Demo. First, GSI Demo creates a run-time wrapper around existing single user applications: it accepts and translates speech and gestures from multiple people into a single stream of keyboard and mouse inputs recognized by the application. Second, it lets people use multimodal demonstration- instead of programming- to quickly map their own speech and gestures to these keyboard/mouse inputs. For example, continuous gestures are trained by saying ¨Computer, when I do (one finger gesture), you do (mouse drag) ¨. Similarly, discrete speech commands can be trained by saying ¨Computer, when I say (layer bars), you do (keyboard and mouse macro) ¨. The end result is that end users can rapidly transform single user commercial applications into a multi-user, multimodal digital tabletop system

Smart Table Top

Dungeons and Dragons 5th Edition (D&D) is a tabletop role playing game (TTRPG) with complicated mechanics and a seemingly overwhelming amount of information. D&D players are often looking for ways to more effectively track information and products that enhance their game experience. One such product is a custom miniature; players use these to track their character’s location and for the enjoyment of creating a physical representation of their imagination. Virtual Tabletops (VTT’s) were developed to make the entire experience digital. The digital nature of VTT’s make all of the game’s rules and mechanics accessible at the click of the mouse; however, it also alienates players from a physical connection to the game. The smart game board offers digital convenience while preserving the physical connection to the game. The game board wirelessly communicates with the pieces on the board to provide quick access to their specific rules and mechanics while also determining their location on the board. The board utilizes a digital display to swap between different game boards/maps. The smart game board saves time and simplifies the experience for players by displaying only the most relevant information and it enhances their experience by utilizing their custom miniatures. This makes the game more accessible to beginners, reduces risk of error for the game master and allows everyone to focus more on enjoying the game. This product could easily be adapted to accommodate any board game with pieces large enough to contain wireless communication technology. Future developments of this product could also include an automatic board setup feature if the piece has a known board layout such as chess, checkers, or monopoly

Information technology for active ageing: A review of theory and practice

Active Ageing aims to foster a physically, mentally and socially active lifestyle as a person ages. It is a complex, multi-faceted problem that involves a variety of different actors, such as policy makers, doctors, care givers, family members, friends and, of course, older adults. This review aims to understand the role of a new actor, which increasingly plays the role of enabler and facilitator, i.e., that of the technology provider. The review specifically focuses on Information Technology (IT), with a particular emphasis on software applications, and on how IT can prevent decline, compensate for lost capabilities, aid care, and enhance existing capabilities. The analysis confirms the crucial role of IT in Active Ageing, shows that Active Ageing requires a multidisciplinary approach, and identifies the need for better integration of hardware, software, the environment and the involved actors

Modeling Urban Spaces with Cubes: Building analogue serious games for collaborative planning

Games are popular as ever. Professionals from every field are trying to build their serious games, combining engaging playability with simulation and learning outcomes. Urban planning is no exception. However, materializing these games is no easy task. We propose a serious game development process to combine modern board game mechanisms with realistic urban maps, profiting from the simplicity, flexibility, and collaboration dynamics analogue games provide. For this, we tested two collaborative games with architecture students. Although different, the games have similar core mechanical and economic systems, modelling urban zones with hexagons and squares. The experience revealed some pitfalls to avoid in game-based planning practice and helped to define a development process for serious games for urban planning

The Irrefutable History of You: Distributed Ledgers and Semantics for Ubiquitous Personal Ratings

A recurring theme in the science-fiction series Black Mirror is the consequence for society of an over-focus on social networking. The episode Nosedive imagines a future in which every public interaction a person has is rated by the other parties, and every aspect of ones life depends on the overall rating computed from these. In this paper, we show how such a scenario is already technically possible using existing technologies such as distributed ledgers, and discuss means by which the negative possibilities may be ameliorated using semantic approaches

Development platform for elderly-oriented tabletop games

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

Development and preliminary evaluation of a novel low cost VR-based upper limb stroke rehabilitation platform using Wii technology.

Abstract Purpose: This paper proposes a novel system (using the Nintendo Wii remote) that offers customised, non-immersive, virtual reality-based, upper-limb stroke rehabilitation and reports on promising preliminary findings with stroke survivors. Method: The system novelty lies in the high accuracy of the full kinematic tracking of the upper limb movement in real-time, offering strong personal connection between the stroke survivor and a virtual character when executing therapist prescribed adjustable exercises/games. It allows the therapist to monitor patient performance and to individually calibrate the system in terms of range of movement, speed and duration. Results: The system was tested for acceptability with three stroke survivors with differing levels of disability. Participants reported an overwhelming connection with the system and avatar. A two-week, single case study with a long-term stroke survivor showed positive changes in all four outcome measures employed, with the participant reporting better wrist control and greater functional use. Activities, which were deemed too challenging or too easy were associated with lower scores of enjoyment/motivation, highlighting the need for activities to be individually calibrated. Conclusions: Given the preliminary findings, it would be beneficial to extend the case study in terms of duration and participants and to conduct an acceptability and feasibility study with community dwelling survivors. Implications for Rehabilitation Low-cost, off-the-shelf game sensors, such as the Nintendo Wii remote, are acceptable by stroke survivors as an add-on to upper limb stroke rehabilitation but have to be bespoked to provide high-fidelity and real-time kinematic tracking of the arm movement. Providing therapists with real-time and remote monitoring of the quality of the movement and not just the amount of practice, is imperative and most critical for getting a better understanding of each patient and administering the right amount and type of exercise. The ability to translate therapeutic arm movement into individually calibrated exercises and games, allows accommodation of the wide range of movement difficulties seen after stroke and the ability to adjust these activities (in terms of speed, range of movement and duration) will aid motivation and adherence - key issues in rehabilitation. With increasing pressures on resources and the move to more community-based rehabilitation, the proposed system has the potential for promoting the intensity of practice necessary for recovery in both community and acute settings.The National Health Service (NHS) London Regional Innovation Fund