Usability engineering of interactive voice responsive (IVR) systems in oral users of Southern Africa

Includes bibliographical references (p. 96-109).This research study focuses on the feasibility of using the telephone as a tool for information access in the oral communities of Southern Africa. The OpenPhone and BGR systems are used as case studies and their designs have been influenced by field studies with the targeted users. The OpenPhone project aims to design an Interactive Voice Response (IVR) health information system that enables people who are caregivers for HIV/AIDS infected children to access relevant care-giving information by using a telephone in their native language of Setswana in Botswana, Southern Africa. The BGR system allows soccer fans to access results of recently played matches in Premier Soccer League (PSL) of South Africa

Coping with the inheritance of COVID-19: the role of new interactive technologies to enhance user experience in different contexts of use

The COVID-19 pandemic has upset the habits of people and various sectors of society, including training, entertainment, and retail. These sectors have been forced to adapt to abnormal situations such as social distancing, remote work, and online entertainment. The pandemic has significantly transformed the training field, leading to the closure of many in-person instruction centers and a shift toward online education courses, which can be less effective. In addition, the entertainment industry has been heavily transformed by social distancing, resulting in the cancellation of many live events and the closure of several cinemas. This has increased demand for online entertainment options, such as streaming services and virtual events. Finally, the restrictions imposed by the COVID-19 pandemic substantially impacted physical stores and fairs, suspending exhibitions for more than two years. This has further driven consumers to rely on e-commerce to fulfill their purchasing and companies to increasingly take advantage of new technologies such as augmented reality. In this suddenly disrupted scenario, new technologies have the potential to fill the gap generated by the pandemic, functioning as an interactive bridge to connect people. This Ph.D. thesis explored the potential of interactive technologies in mitigating the challenges posed by the COVID-19 pandemic in various contexts of use in the above-mentioned areas. Specifically, three lines of research were investigated by conducting different studies using a mixed approach in the Human-Computer Interaction field. The first research line focused on the study of immersive virtual reality training, with a particular interest in flood emergencies, a growing phenomenon. The goal was to implement engaging and efficient training for citizens that live near rivers through a human-centric design approach. The second line of research explored innovative ways to improve social interaction and collaboration in the entertainment sector, highlighting guidelines for the design of shared streaming experiences. In particular, three different communication modalities were studied during group viewing of an interactive film on a streaming platform. Finally, the third research line focused on the retail sector. On the one hand, the focus consisted of understanding which aspects of the 3D web and AR technology are helpful for supporting small businesses and trade fairs. On the other hand, the focus was to investigate how to support consumers during an AR shopping experience when interacting with 3D virtual products of different sizes. Overall, this project provides suggestions and guidelines for designing systems that can both increasingly connect people at a distance and offer new hybrid worlds. In addition, this project expands state-of-the-art related to interactive technologies and offers generalizable results outside the crisis created by COVID-19. These technologies, now increasingly integrated into everyday life, can be a tool for empowerment and resilience, improving people's lives.The COVID-19 pandemic has upset the habits of people and various sectors of society, including training, entertainment, and retail. These sectors have been forced to adapt to abnormal situations such as social distancing, remote work, and online entertainment. The pandemic has significantly transformed the training field, leading to the closure of many in-person instruction centers and a shift toward online education courses, which can be less effective. In addition, the entertainment industry has been heavily transformed by social distancing, resulting in the cancellation of many live events and the closure of several cinemas. This has increased demand for online entertainment options, such as streaming services and virtual events. Finally, the restrictions imposed by the COVID-19 pandemic substantially impacted physical stores and fairs, suspending exhibitions for more than two years. This has further driven consumers to rely on e-commerce to fulfill their purchasing and companies to increasingly take advantage of new technologies such as augmented reality. In this suddenly disrupted scenario, new technologies have the potential to fill the gap generated by the pandemic, functioning as an interactive bridge to connect people. This Ph.D. thesis explored the potential of interactive technologies in mitigating the challenges posed by the COVID-19 pandemic in various contexts of use in the above-mentioned areas. Specifically, three lines of research were investigated by conducting different studies using a mixed approach in the Human-Computer Interaction field. The first research line focused on the study of immersive virtual reality training, with a particular interest in flood emergencies, a growing phenomenon. The goal was to implement engaging and efficient training for citizens that live near rivers through a human-centric design approach. The second line of research explored innovative ways to improve social interaction and collaboration in the entertainment sector, highlighting guidelines for the design of shared streaming experiences. In particular, three different communication modalities were studied during group viewing of an interactive film on a streaming platform. Finally, the third research line focused on the retail sector. On the one hand, the focus consisted of understanding which aspects of the 3D web and AR technology are helpful for supporting small businesses and trade fairs. On the other hand, the focus was to investigate how to support consumers during an AR shopping experience when interacting with 3D virtual products of different sizes. Overall, this project provides suggestions and guidelines for designing systems that can both increasingly connect people at a distance and offer new hybrid worlds. In addition, this project expands state-of-the-art related to interactive technologies and offers generalizable results outside the crisis created by COVID-19. These technologies, now increasingly integrated into everyday life, can be a tool for empowerment and resilience, improving people's lives

Health Literacy Changes in a Technology-Enhanced Diabetes Prevention Program

Background: In 2001, the Diabetes Prevention Program was published evaluating the efficacy of a pharmaceutical intervention, Metformin, and a behavioral lifestyle intervention (LI) to reduce incidence of Type 2 diabetes mellitus. The LI was observed to reduce the incidence of the disease by 58% relative to 31% in the medication treatment. Amongst technology based LIs, little has been done to address different health literacy populations. Objectives: This dissertation evaluated how teach-back and teach-to-goal can influence the uptake of information obtained in each health education lesson, behaviors and its influence on engagement and weight loss. Methods: Four hundred forty-two participants were analyzed in study #1, and only 425 were maintained for study #2 and #3. General regression modeling with White’s Standard Error heteroskedacity adjustments was performed assessing the differences in engagement and comprehension performance by health literacy level and modality. Results: In a teach-back/teach-to-goal call, differences in reverse score performance (DVD-15.4±2.5; Class-14.8±2.6; F(3, 425)= 13.72, p\u3c0.001), number of teach-back rounds (DVD-1.9±0.7; Class-2.1±0.7; F(3, 425)=5.98, p\u3c0.001) and number of round 1 questions (DVD-4.2±1.6; Class-3.4±1.8; F(3,425)=20.95, p\u3c0.001) was observed. While not significant, 38.7% of LHL participant completed all 22 lessons vs. 28.7% of HHL. Mean overall comprehension average scores improved 0.8±1.1 to 1.2±0.3 and 0.7±1.0 to 1.5±1.1 for those LHL and HHL participants completing only 1 call versus all 22 calls, respectively, as did physical activity and muscle strengthening minutes per week. Models evaluating IVR-reported weight change against engagement and overall comprehension average revealed engagement had an indirect relationship (β= -0.59, p\u3c0.01) with magnitude of weight change (R²=0.13, F(3, 420)=20.8, p\u3c0.001), and a direct relationship with aerobic physical activity, muscle strengthening and fruit and vegetable intake. Conclusions: Amongst high and low health literacy groups, both groups benefited from teach-back and teach-to-goal health literacy techniques to improve patient comprehension, which in turn, improved engagement rates, especially in the low health literacy population. Reinforcement strategies to promote information uptake is necessary to allow for behavior uptake lending to greater weight loss

Virtual reality interfaces for seamless interaction with the physical reality

In recent years head-mounted displays (HMDs) for virtual reality (VR) have made the transition from research to consumer product, and are increasingly used for productive purposes such as 3D modeling in the automotive industry and teleconferencing. VR allows users to create and experience real-world like models of products; and enables users to have an immersive social interaction with distant colleagues. These solutions are a promising alternative to physical prototypes and meetings, as they require less investment in time and material. VR uses our visual dominance to deliver these experiences, making users believe that they are in another reality. However, while their mind is present in VR their body is in the physical reality. From the user’s perspective, this brings considerable uncertainty to the interaction. Currently, they are forced to take off their HMD in order to, for example, see who is observing them and to understand whether their physical integrity is at risk. This disrupts their interaction in VR, leading to a loss of presence – a main quality measure for the success of VR experiences. In this thesis, I address this uncertainty by developing interfaces that enable users to stay in VR while supporting their awareness of the physical reality. They maintain this awareness without having to take off the headset – which I refer to as seamless interaction with the physical reality. The overarching research vision that guides this thesis is, therefore, to reduce this disconnect between the virtual and physical reality. My research is motivated by a preliminary exploration of user uncertainty towards using VR in co-located, public places. This exploration revealed three main foci: (a) security and privacy, (b) communication with physical collaborators, and (c) managing presence in both the physical and virtual reality. Each theme represents a section in my dissertation, in which I identify central challenges and give directions towards overcoming them as have emerged from the work presented here. First, I investigate security and privacy in co-located situations by revealing to what extent bystanders are able to observe general tasks. In this context, I explicitly investigate the security considerations of authentication mechanisms. I review how existing authentication mechanisms can be transferred to VR and present novel approaches that are more usable and secure than existing solutions from prior work. Second, to support communication between VR users and physical collaborators, I add to the field design implications for VR interactions that enable observers to choose opportune moments to interrupt HMD users. Moreover, I contribute methods for displaying interruptions in VR and discuss their effect on presence and performance. I also found that different virtual presentations of co-located collaborators have an effect on social presence, performance and trust. Third, I close my thesis by investigating methods to manage presence in both the physical and virtual realities. I propose systems and interfaces for transitioning between them that empower users to decide how much they want to be aware of the other reality. Finally, I discuss the opportunity to systematically allocate senses to these two realities: the visual one for VR and the auditory and haptic one for the physical reality. Moreover, I provide specific design guidelines on how to use these findings to alert VR users about physical borders and obstacles.In den letzten Jahren haben Head-Mounted-Displays (HMDs) für virtuelle Realität (VR) den Übergang von der Forschung zum Konsumprodukt vollzogen und werden zunehmend für produktive Zwecke, wie 3D-Modellierung in der Automobilindustrie oder Telekonferenzen, eingesetzt. VR ermöglicht es den Benutzern, schnell und kostengünstig, Prototypen zu erstellen und erlaubt eine immersive soziale Interaktion mit entfernten Kollegen. VR nutzt unsere visuelle Dominanz, um diese Erfahrungen zu vermitteln und gibt Benutzern das Gefühl sich in einer anderen Realität zu befinden. Während der Nutzer jedoch in der virtuellen Realität mental präsent ist, befindet sich der Körper weiterhin in der physischen Realität. Aus der Perspektive des Benutzers bringt dies erhebliche Unsicherheit in die Nutzung von HMDs. Aktuell sind Nutzer gezwungen, ihr HMD abzunehmen, um zu sehen, wer sie beobachtet und zu verstehen, ob ihr körperliches Wohlbefinden gefährdet ist. Dadurch wird ihre Interaktion in der VR gestört, was zu einem Verlust der Präsenz führt - ein Hauptqualitätsmaß für den Erfolg von VR-Erfahrungen. In dieser Arbeit befasse ich mich mit dieser Unsicherheit, indem ich Schnittstellen entwickle, die es den Nutzern ermöglichen, in VR zu bleiben und gleichzeitig unterstützen sie die Wahrnehmung für die physische Realität. Sie behalten diese Wahrnehmung für die physische Realität bei, ohne das Headset abnehmen zu müssen - was ich als nahtlose Interaktion mit der physischen Realität bezeichne. Daher ist eine übergeordenete Vision von meiner Forschung diese Trennung von virtueller und physicher Realität zu reduzieren. Meine Forschung basiert auf einer einleitenden Untersuchung, die sich mit der Unsicherheit der Nutzer gegenüber der Verwendung von VR an öffentlichen, geteilten Orten befasst. Im Kontext meiner Arbeit werden Räume oder Flächen, die mit anderen ortsgleichen Menschen geteilt werden, als geteilte Orte bezeichnet. Diese Untersuchung ergab drei Hauptschwerpunkte: (1) Sicherheit und Privatsphäre, (2) Kommunikation mit physischen Kollaborateuren, und (3) Umgang mit der Präsenz, sowohl in der physischen als auch in der virtuellen Realität. Jedes Thema stellt einen Fokus in meiner Dissertation dar, in dem ich zentrale Herausforderungen identifiziere und Lösungsansätze vorstelle. Erstens, untersuche ich Sicherheit und Privatsphäre an öffentlichen, geteilten Orten, indem ich aufdecke, inwieweit Umstehende in der Lage sind, allgemeine Aufgaben zu beobachten. In diesem Zusammenhang untersuche ich explizit die Gestaltung von Authentifizierungsmechanismen. Ich untersuche, wie bestehende Authentifizierungsmechanismen auf VR übertragen werden können, und stelle neue Ansätze vor, die nutzbar und sicher sind. Zweitens, um die Kommunikation zwischen HMD-Nutzern und Umstehenden zu unterstützen, erweitere ich das Forschungsfeld um VR-Interaktionen, die es Beobachtern ermöglichen, günstige Momente für die Unterbrechung von HMD-Nutzern zu wählen. Darüber hinaus steuere ich Methoden zur Darstellung von Unterbrechungen in VR bei und diskutiere ihre Auswirkungen auf Präsenz und Leistung von Nutzern. Meine Arbeit brachte auch hervor, dass verschiedene virtuelle Präsentationen von ortsgleichen Kollaborateuren einen Effekt auf die soziale Präsenz, Leistung und Vertrauen haben. Drittens, schließe ich meine Dissertation mit der Untersuchung von Methoden zur Verwaltung der Präsenz, sowohl in der physischen als auch in der virtuellen Realität ab. Ich schlage Systeme und Schnittstellen für den Übergang zwischen den Realitäten vor, die die Benutzer in die Lage versetzen zu entscheiden, inwieweit sie sich der anderen Realität bewusst sein wollen. Schließlich diskutiere ich die Möglichkeit, diesen beiden Realitäten systematisch Sinne zuzuordnen: die visuelle für VR und die auditive und haptische für die physische Realität. Darüber hinaus stelle ich spezifische Design-Richtlinien zur Verfügung, wie diese Erkenntnisse genutzt werden können, um VR-Anwender auf physische Grenzen und Hindernisse aufmerksam zu machen

Examining the sense of agency in human-computer interaction

Humans are agents, we feel that we control the course of events on our everyday life. This refers to the Sense of Agency (SoA). This experience is not only crucial in our daily life, but also in our interaction with technology. When we manipulate a user interface (e.g., computer, smartphone, etc.), we expect that the system responds to our input commands with feedback, as we desire to feel that we are in charge of the interaction. If this interplay elicits a SoA, then the user will perceive an instinctive feeling of “I am controlling this”. Although research in Human-Computer Interaction (HCI) pursuits the design of intuitive and responsive systems, most of the current studies have been focussed mainly on interaction techniques (e.g., software-hardware) and User Experience (UX) (e.g., comfort, usability, etc.), and very little has been investigated in terms of the SoA i.e., the conscious experience of being in control regarding the interaction. In this thesis, we present an experimental exploration of the role of the SoA in interaction paradigms typical of HCI. After two chapters of introduction and related work, we describe a series of studies that explore agency implication in interaction with systems through human senses such as vision, audio, touch and smell. Chapter 3 explores the SoA in mid-air haptic interaction through touchless actions. Then, Chapter 4 examines agency modulation through smell and its application for olfactory interfaces. Chapter 5 describes two novel timing techniques based on auditory and haptic cues that provide alternative timing methods to the traditional Libet clock. Finally, we conclude with a discussion chapter that highlights the importance of our SoA during interactions with technology as well as the implications of the results found, in the design of user interfaces

Virtual reality interfaces for seamless interaction with the physical reality

In recent years head-mounted displays (HMDs) for virtual reality (VR) have made the transition from research to consumer product, and are increasingly used for productive purposes such as 3D modeling in the automotive industry and teleconferencing. VR allows users to create and experience real-world like models of products; and enables users to have an immersive social interaction with distant colleagues. These solutions are a promising alternative to physical prototypes and meetings, as they require less investment in time and material. VR uses our visual dominance to deliver these experiences, making users believe that they are in another reality. However, while their mind is present in VR their body is in the physical reality. From the user’s perspective, this brings considerable uncertainty to the interaction. Currently, they are forced to take off their HMD in order to, for example, see who is observing them and to understand whether their physical integrity is at risk. This disrupts their interaction in VR, leading to a loss of presence – a main quality measure for the success of VR experiences. In this thesis, I address this uncertainty by developing interfaces that enable users to stay in VR while supporting their awareness of the physical reality. They maintain this awareness without having to take off the headset – which I refer to as seamless interaction with the physical reality. The overarching research vision that guides this thesis is, therefore, to reduce this disconnect between the virtual and physical reality. My research is motivated by a preliminary exploration of user uncertainty towards using VR in co-located, public places. This exploration revealed three main foci: (a) security and privacy, (b) communication with physical collaborators, and (c) managing presence in both the physical and virtual reality. Each theme represents a section in my dissertation, in which I identify central challenges and give directions towards overcoming them as have emerged from the work presented here. First, I investigate security and privacy in co-located situations by revealing to what extent bystanders are able to observe general tasks. In this context, I explicitly investigate the security considerations of authentication mechanisms. I review how existing authentication mechanisms can be transferred to VR and present novel approaches that are more usable and secure than existing solutions from prior work. Second, to support communication between VR users and physical collaborators, I add to the field design implications for VR interactions that enable observers to choose opportune moments to interrupt HMD users. Moreover, I contribute methods for displaying interruptions in VR and discuss their effect on presence and performance. I also found that different virtual presentations of co-located collaborators have an effect on social presence, performance and trust. Third, I close my thesis by investigating methods to manage presence in both the physical and virtual realities. I propose systems and interfaces for transitioning between them that empower users to decide how much they want to be aware of the other reality. Finally, I discuss the opportunity to systematically allocate senses to these two realities: the visual one for VR and the auditory and haptic one for the physical reality. Moreover, I provide specific design guidelines on how to use these findings to alert VR users about physical borders and obstacles.In den letzten Jahren haben Head-Mounted-Displays (HMDs) für virtuelle Realität (VR) den Übergang von der Forschung zum Konsumprodukt vollzogen und werden zunehmend für produktive Zwecke, wie 3D-Modellierung in der Automobilindustrie oder Telekonferenzen, eingesetzt. VR ermöglicht es den Benutzern, schnell und kostengünstig, Prototypen zu erstellen und erlaubt eine immersive soziale Interaktion mit entfernten Kollegen. VR nutzt unsere visuelle Dominanz, um diese Erfahrungen zu vermitteln und gibt Benutzern das Gefühl sich in einer anderen Realität zu befinden. Während der Nutzer jedoch in der virtuellen Realität mental präsent ist, befindet sich der Körper weiterhin in der physischen Realität. Aus der Perspektive des Benutzers bringt dies erhebliche Unsicherheit in die Nutzung von HMDs. Aktuell sind Nutzer gezwungen, ihr HMD abzunehmen, um zu sehen, wer sie beobachtet und zu verstehen, ob ihr körperliches Wohlbefinden gefährdet ist. Dadurch wird ihre Interaktion in der VR gestört, was zu einem Verlust der Präsenz führt - ein Hauptqualitätsmaß für den Erfolg von VR-Erfahrungen. In dieser Arbeit befasse ich mich mit dieser Unsicherheit, indem ich Schnittstellen entwickle, die es den Nutzern ermöglichen, in VR zu bleiben und gleichzeitig unterstützen sie die Wahrnehmung für die physische Realität. Sie behalten diese Wahrnehmung für die physische Realität bei, ohne das Headset abnehmen zu müssen - was ich als nahtlose Interaktion mit der physischen Realität bezeichne. Daher ist eine übergeordenete Vision von meiner Forschung diese Trennung von virtueller und physicher Realität zu reduzieren. Meine Forschung basiert auf einer einleitenden Untersuchung, die sich mit der Unsicherheit der Nutzer gegenüber der Verwendung von VR an öffentlichen, geteilten Orten befasst. Im Kontext meiner Arbeit werden Räume oder Flächen, die mit anderen ortsgleichen Menschen geteilt werden, als geteilte Orte bezeichnet. Diese Untersuchung ergab drei Hauptschwerpunkte: (1) Sicherheit und Privatsphäre, (2) Kommunikation mit physischen Kollaborateuren, und (3) Umgang mit der Präsenz, sowohl in der physischen als auch in der virtuellen Realität. Jedes Thema stellt einen Fokus in meiner Dissertation dar, in dem ich zentrale Herausforderungen identifiziere und Lösungsansätze vorstelle. Erstens, untersuche ich Sicherheit und Privatsphäre an öffentlichen, geteilten Orten, indem ich aufdecke, inwieweit Umstehende in der Lage sind, allgemeine Aufgaben zu beobachten. In diesem Zusammenhang untersuche ich explizit die Gestaltung von Authentifizierungsmechanismen. Ich untersuche, wie bestehende Authentifizierungsmechanismen auf VR übertragen werden können, und stelle neue Ansätze vor, die nutzbar und sicher sind. Zweitens, um die Kommunikation zwischen HMD-Nutzern und Umstehenden zu unterstützen, erweitere ich das Forschungsfeld um VR-Interaktionen, die es Beobachtern ermöglichen, günstige Momente für die Unterbrechung von HMD-Nutzern zu wählen. Darüber hinaus steuere ich Methoden zur Darstellung von Unterbrechungen in VR bei und diskutiere ihre Auswirkungen auf Präsenz und Leistung von Nutzern. Meine Arbeit brachte auch hervor, dass verschiedene virtuelle Präsentationen von ortsgleichen Kollaborateuren einen Effekt auf die soziale Präsenz, Leistung und Vertrauen haben. Drittens, schließe ich meine Dissertation mit der Untersuchung von Methoden zur Verwaltung der Präsenz, sowohl in der physischen als auch in der virtuellen Realität ab. Ich schlage Systeme und Schnittstellen für den Übergang zwischen den Realitäten vor, die die Benutzer in die Lage versetzen zu entscheiden, inwieweit sie sich der anderen Realität bewusst sein wollen. Schließlich diskutiere ich die Möglichkeit, diesen beiden Realitäten systematisch Sinne zuzuordnen: die visuelle für VR und die auditive und haptische für die physische Realität. Darüber hinaus stelle ich spezifische Design-Richtlinien zur Verfügung, wie diese Erkenntnisse genutzt werden können, um VR-Anwender auf physische Grenzen und Hindernisse aufmerksam zu machen

Countering the Novelty Effect: A Tutorial for Immersive Virtual Reality Learning Environments

Immersive Virtual Reality (iVR) is a new technology, the novelty effect of which can reduce the enjoyment of iVR experiences and, especially, learning achievements when presented in the classroom; an effect that the interactive tutorial proposed in this research can help overcome. Its increasingly complex levels are designed on the basis of Mayer’s Cognitive Theory of Multimedia Learning, so that users can quickly gain familiarity with the iVR environment. The tutorial was included in an iVR learning experience for its validation with 65 users. It was a success, according to the user satisfaction and tutorial usability survey. First, it gained very high ratings for satisfaction, engagement, and immersion. Second, high skill rates suggested that it helped users to gain familiarity with controllers. Finally, a medium-high value for flow pointed to major concerns related to skill and challenges with this sort of iVR experience. A few cases of cybersickness also arose. The survey showed that only intense cybersickness levels significantly limited performance and enjoyment; low levels had no influence on flow and immersion and little influence on skill, presence, and engagement, greatly reducing the benefits of the tutorial, despite which it remained useful.This work was partially supported by the ACIS project (Reference Number INVESTUN/21/BU/0002) of the Consejeria de Empleo e Industria of the Junta de Castilla y León (Spain), the Erasmus+ RISKREAL Project (Reference Number 2020-1-ES01-KA204-081847) of the European Commission and the HumanAid Project (Reference Number TED2021-129485B-C43) of the Proyectos Estratégicos Orientados a la Transición Ecológica y a la Transición Digital of the Spanish Ministry of Science and Innovation

Ono: an open platform for social robotics

In recent times, the focal point of research in robotics has shifted from industrial ro- bots toward robots that interact with humans in an intuitive and safe manner. This evolution has resulted in the subfield of social robotics, which pertains to robots that function in a human environment and that can communicate with humans in an int- uitive way, e.g. with facial expressions. Social robots have the potential to impact many different aspects of our lives, but one particularly promising application is the use of robots in therapy, such as the treatment of children with autism. Unfortunately, many of the existing social robots are neither suited for practical use in therapy nor for large scale studies, mainly because they are expensive, one-of-a-kind robots that are hard to modify to suit a specific need. We created Ono, a social robotics platform, to tackle these issues. Ono is composed entirely from off-the-shelf components and cheap materials, and can be built at a local FabLab at the fraction of the cost of other robots. Ono is also entirely open source and the modular design further encourages modification and reuse of parts of the platform