Augmented reality as a telemedicine platform for remote procedural training

Traditionally, rural areas in many countries are limited by a lack of access to health care due to the inherent challenges associated with recruitment and retention of healthcare professionals. Telemedicine, which uses communication technology to deliver medical services over distance, is an economical and potentially effective way to address this problem. In this research, we develop a new telepresence application using an augmented reality (AR) system. We explore the use of the Microsoft HoloLens to facilitate and enhance remote medical training. Intrinsic advantages of AR systems enable remote learners to perform complex medical procedures such as Point of Care Ultrasound (PoCUS) without visual interference. This research uses the HoloLens to capture the first-person view of a simulated rural emergency room (ER) through mixed reality capture (MRC) and serves as a novel telemedicine platform with remote pointing capabilities. The mentor's hand gestures are captured using a Leap Motion and virtually displayed in the AR space of the HoloLens. To explore the feasibility of the developed platform, twelve novice medical trainees were guided by a mentor through a simulated ultrasound exploration in a trauma scenario, as part of a pilot user study. The study explores the utility of the system from the trainees, mentor, and objective observers' perspectives and compares the findings to that of a more traditional multi-camera telemedicine solution. The results obtained provide valuable insight and guidance for the development of an AR-supported telemedicine platform

Wearable Tools for Affective Remote Collaboration

Affective computing is the study and development of systems that can recognize human emotions and feelings. Emotions are always an interesting topic of research and these days researchers are trying to develop systems which can recognize, interpret and process emotions based on human physiological and neural changes for the development of well-being. As the market for wearable devices is expanding, it provides more opportunity of research in emotion sharing with remote person. This Master’s thesis investigates the possibility of using wearable devices for affective remote collaboration. Previous research about affective computing, affective communication and remote collaboration using wearable devices is reviewed before starting the design process. Three wearable devices were developed, evaluated and discussed, two for emotion sharing between remote people, and the third for preliminary research to explore if eye gaze information can increase co-presence in remote collaboration. Conclusions and Future work are discussed based on the results from the research evaluation

Enhanced Virtuality: Increasing the Usability and Productivity of Virtual Environments

Mit stetig steigender Bildschirmauflösung, genauerem Tracking und fallenden Preisen stehen Virtual Reality (VR) Systeme kurz davor sich erfolgreich am Markt zu etablieren. Verschiedene Werkzeuge helfen Entwicklern bei der Erstellung komplexer Interaktionen mit mehreren Benutzern innerhalb adaptiver virtueller Umgebungen. Allerdings entstehen mit der Verbreitung der VR-Systeme auch zusätzliche Herausforderungen: Diverse Eingabegeräte mit ungewohnten Formen und Tastenlayouts verhindern eine intuitive Interaktion. Darüber hinaus zwingt der eingeschränkte Funktionsumfang bestehender Software die Nutzer dazu, auf herkömmliche PC- oder Touch-basierte Systeme zurückzugreifen. Außerdem birgt die Zusammenarbeit mit anderen Anwendern am gleichen Standort Herausforderungen hinsichtlich der Kalibrierung unterschiedlicher Trackingsysteme und der Kollisionsvermeidung. Beim entfernten Zusammenarbeiten wird die Interaktion durch Latenzzeiten und Verbindungsverluste zusätzlich beeinflusst. Schließlich haben die Benutzer unterschiedliche Anforderungen an die Visualisierung von Inhalten, z.B. Größe, Ausrichtung, Farbe oder Kontrast, innerhalb der virtuellen Welten. Eine strikte Nachbildung von realen Umgebungen in VR verschenkt Potential und wird es nicht ermöglichen, die individuellen Bedürfnisse der Benutzer zu berücksichtigen. Um diese Probleme anzugehen, werden in der vorliegenden Arbeit Lösungen in den Bereichen Eingabe, Zusammenarbeit und Erweiterung von virtuellen Welten und Benutzern vorgestellt, die darauf abzielen, die Benutzerfreundlichkeit und Produktivität von VR zu erhöhen. Zunächst werden PC-basierte Hardware und Software in die virtuelle Welt übertragen, um die Vertrautheit und den Funktionsumfang bestehender Anwendungen in VR zu erhalten. Virtuelle Stellvertreter von physischen Geräten, z.B. Tastatur und Tablet, und ein VR-Modus für Anwendungen ermöglichen es dem Benutzer reale Fähigkeiten in die virtuelle Welt zu übertragen. Des Weiteren wird ein Algorithmus vorgestellt, der die Kalibrierung mehrerer ko-lokaler VR-Geräte mit hoher Genauigkeit und geringen Hardwareanforderungen und geringem Aufwand ermöglicht. Da VR-Headsets die reale Umgebung der Benutzer ausblenden, wird die Relevanz einer Ganzkörper-Avatar-Visualisierung für die Kollisionsvermeidung und das entfernte Zusammenarbeiten nachgewiesen. Darüber hinaus werden personalisierte räumliche oder zeitliche Modifikationen vorgestellt, die es erlauben, die Benutzerfreundlichkeit, Arbeitsleistung und soziale Präsenz von Benutzern zu erhöhen. Diskrepanzen zwischen den virtuellen Welten, die durch persönliche Anpassungen entstehen, werden durch Methoden der Avatar-Umlenkung (engl. redirection) kompensiert. Abschließend werden einige der Methoden und Erkenntnisse in eine beispielhafte Anwendung integriert, um deren praktische Anwendbarkeit zu verdeutlichen. Die vorliegende Arbeit zeigt, dass virtuelle Umgebungen auf realen Fähigkeiten und Erfahrungen aufbauen können, um eine vertraute und einfache Interaktion und Zusammenarbeit von Benutzern zu gewährleisten. Darüber hinaus ermöglichen individuelle Erweiterungen des virtuellen Inhalts und der Avatare Einschränkungen der realen Welt zu überwinden und das Erlebnis von VR-Umgebungen zu steigern

Exploring context-sensitive collaborative augmented reality applications

In smart spaces limited amount of physical resources are available. Also, system should be able to offer relevant information according to user’s personal preferences. At the same time smart environments could serve many users with same requirement of relevancy and operate on limited resources. Sometimes it may not be possible to share resource in a way that respects all users without compromising collaboration. This thesis is focused on solving the problem of shared resource from the perspective of augmented reality. Selected standpoint is on mobile collaborative augmented reality and context-awareness. A small user study has been arranged as part of thesis to bring out information about user’s thoughts and emotions while using a simple prototype application. In addition, a small literature review about main concepts is conducted. There is a short analysis of some collaborative augmented reality applications presented based on recent literature. Results of the thesis show that even with small experiments it is possible to discover new information from users. Results also provide tentative answers to presented research questions. Main findings are that users have high expectations towards context-awareness and augmented reality technologies. They also expect applications to offer relevant, validated and also surprising information in each situation. This thesis has some evidence about suitability of augmented reality in context-aware applications that are targeted to support human-to human collaboration. With augmented reality it is possible to offer individual standpoints for users while they are inspecting limited, shared resources. Endorsement of user’s ability to monitor their environment is one challenge in large smart environments. Finally, software engineer can take user’s expectations into account when designing context-aware systems for smart environments. Also, developer could implement system that takes advantage of different human sensory modalities

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

A Sensing Platform to Monitor Sleep Efficiency

Sleep plays a fundamental role in the human life. Sleep research is mainly focused on the understanding of the sleep patterns, stages and duration. An accurate sleep monitoring can detect early signs of sleep deprivation and insomnia consequentially implementing mechanisms for preventing and overcoming these problems. Recently, sleep monitoring has been achieved using wearable technologies, able to analyse also the body movements, but old people can encounter some difficulties in using and maintaining these devices. In this paper, we propose an unobtrusive sensing platform able to analyze body movements, infer sleep duration and awakenings occurred along the night, and evaluating the sleep efficiency index. To prove the feasibility of the suggested method we did a pilot trial in which several healthy users have been involved. The sensors were installed within the bed and, on each day, each user was administered with the Groningen Sleep Quality Scale questionnaire to evaluate the user’s perceived sleep quality. Finally, we show potential correlation between a perceived evaluation with an objective index as the sleep efficiency.</p

Interactive mixed reality media with real time 3D human capture

Master'sMASTER OF ENGINEERIN

Removing spatial boundaries in immersive mobile communications

Despite a worldwide trend towards mobile computing, current telepresence experiences focus on stationary desktop computers, limiting how, when, and where researched solutions can be used. In this thesis I demonstrate that mobile phones are a capable platform for future research, showing the effectiveness of the communications possible through their inherent portability and ubiquity. I first describe a framework upon which future systems can be built, which allows two distant users to explore one of several panoramic representations of the local environment by reorienting their device. User experiments demonstrate this framework's ability to induce a sense of presence within the space and between users, and show that capturing this environment live provides no significant benefits over constructing it incrementally. This discovery enables a second application that allows users to explore a three-dimensional representation of their environment. Each user's position is shown as an avatar, with live facial capture to facilitate natural communication. Either may also see the full environment by occupying the same virtual space. This application is also evaluated and shown to provide efficient communications to its users, providing a novel untethered experience not possible on stationary hardware despite the inherent lack of computational ability available on mobile devices

Assessing patient and caregiver intent to use mobile device videoconferencing for remote mechanically-ventilated patient management

The Michigan Medicine adult Assisted Ventilation Clinic (AVC) supports patients with neuromuscular disorders and spinal cord injuries and their caregivers at home, helping them avoid expensive emergency department visits, hospitalization, and unnecessary or excessive treatments. Mobile device videoconferencing provides an effective capability for remote mechanically-ventilated patient management but must rely upon an unknown infrastructure comprising patient and caregiver mobile device ownership, connectivity, and experience—and intent to use the service if provided. The purpose of this study was to measure the extent of this infrastructure and the perceived ease of use, perceived usefulness, and intent to use this mobile device capability using a questionnaire based on the technology acceptance model (TAM). Of 188 patients and caregivers asked, 153 (n = 153) respondents completed a questionnaire comprised of 14 demographic and 24 Likert-type questions. Inferential results indicated a significant correlation between perceived ease of use (PEU) and perceived usefulness (PU) of mobile devices in remote care and their intent to use them (sig. \u3c .001). Also, mobile device own/access significantly correlated with PEU and PU (p = .003 & .004, respectively), but not intent to use. No single demographic variable (age, distance to AVC, diagnoses, mobile device experience, tracheostomy, etc.) significantly correlated with intent to use. Descriptive results indicated a significant patient/caregiver provided infrastructure: 96% have cellular/WiFi/Internet access, 91% own or have access to mobile devices, 77% have downloaded apps, 68% have used videoconferencing, and 80% own between two and five ICT devices

Telemedicine

Telemedicine is a rapidly evolving field as new technologies are implemented for example for the development of wireless sensors, quality data transmission. Using the Internet applications such as counseling, clinical consultation support and home care monitoring and management are more and more realized, which improves access to high level medical care in underserved areas. The 23 chapters of this book present manifold examples of telemedicine treating both theoretical and practical foundations and application scenarios