Conceptualization and Design of a Workflow Management System Front End for Augmented Reality Headsets

A currently discussed approach to increase efficiency during task execution, inter alia to reduce error rates and execution times, is the utilization of headset-based augmented reality systems (HARS). Additional to direct task support, HARSes can offer workflow management and control functions. However, these are only covered very limitedly by existing design-oriented approaches. Thus, users have to resort to additional devices, decreasing efficiency, and usability. Based on a three-step systematic literature analysis and two focus groups, we present a novel tentative design theory for HARSes supporting the full range of workflow management and control functions. Our design theory consists of four design requirements and nine design principles and is the basis for a software artifact prototype. Both our tentative design theory and software artifact are formatively evaluated by a third focus group. Our contributions add to the prescriptive knowledge base of the community and may be adapted by researchers and practitioners

EMPOWERING THE FRONT-SEAT PASSENGER: DESIGN AND EXPERIENCE PROTOTYPING OF LUXURY INFOTAINMENT SYSTEMS THROUGH VR SIMULATION

Automotive user interfaces have been designed within the limitations of driving activity. Therefore, there has been a lack of infotainment solutions that target the front-seat passenger as another car occupant with his/her own needs, interests and capabilities. This research is built on the motivation of empowering the front-seat passengers in luxury car journeys through infotainment systems. It handles front-seat passenger’s empowerment through the investigation of how a pleasant and luxury infotainment experience is manifested via new functionalities and interactions. This research tackles the challenge of understanding how these unprecedented infotainment solutions will add to front-seat passenger’s travel experience with experience prototyping through VR simulation. It follows the ‘research through design approach’ by i) presenting a design proposal for the front-seat passenger infotainment system, ii) developing a VR simulation to communicate the infotainment system interactions within a travel scenario in an immersive way, and iii) conducting experience prototyping study where participants reflect on the design proposal (VR simulation) through administration of mixed data collection methods including semantic differential questionnaires and semi-structured interviews. To achieve the above-mentioned objectives, the research makes use of an extensive literature on User Experience (UX)-Human Computer Interaction (HCI), Automotive UX, Luxury Marketing, and Simulation. The synthesis of the UX and marketing literature enables deconstruction of pleasant and luxury user experience into a set of qualities/metrics to be referred in design and design evaluation. The synthesis of aesthetics of interaction studies in the UX literature helps to categorize the diverse aspects of the infotainment system. The analysis of the academic and industrial efforts to empower front-seat passengers through automotive user interfaces is used for identification of promising technologies and trends for the infotainment system. The literature review on experience prototyping with VR constitutes a reference in prototyping-related decisions and using VR as part of the user study. The thesis finally presents the analysis of the experience prototyping study through i) the quantitative representation and discussion of the diverse aspects of the infotainment system (functionalities and interaction aesthetics) that play role in delivery of the various qualities of luxury experience, ii) structured analysis of the participants’ suggestions for the system with specification of the underlying motivations and iii) development of a framework that conceptualizes the front-seat passengers’ changing role and relations with the infotainment system. Based on these investigations of the link between the infotainment system aspects and the participants’ expectations/concerns, the research concludes with key design considerations and recommendations for the future (luxury) front-seat passenger-oriented infotainment system solutions. It also presents recommendations for integration of VR simulation into future car HMI appraisals by reflecting on the experience prototyping study conducted as part of the PhD research

Is the driver ready to receive just car information in the windshield during manual and autonomous driving?

A automação está a mudar o mundo. Como na aeronáutica, as empresas da indústria automóvel estão atualmente a desenvolver veículos autónomos. No entanto a autonomia do veículo não é completa, necessitando por vezes das ações do condutor. A forma como é feita a transição entre condução manual e autónoma e como mostrar esta informação de transição para o condutor constitui um desafio para a ergonomia. Novos ecrãs estão a ser estudados para facilitar estas transições. Este estudo usou um simulador de condução para investigar, se a informação em realidade aumentada pode influenciar positivamente a experiência do condutor durante a condução manual e autónoma. Compararam-se duas formas de apresentar a comunicação ao condutor. Um “conceito AR” mostrou toda a informação no para-brisas para ser mais fácil o condutor aceder à informação. O “conceito IC” mostrou a informação que aparece atualmente nos carros, usando o painel de instrumentos e o e-HUD. Os resultados indicam que a experiência do utilizador (UX) é influenciada pelos conceitos, sendo que o “conceito AR” teve uma melhor UX em todos os estados de transição. Em termos de confiança, os resultados revelaram também valores mais elevado para o “conceito AR”. O tipo de conceito não influenciou nem o tempo nem o comportamento de retomar o controlo do carro. Em termos de situação consciente, o “conceito AR” deixa os condutores mais conscientes durante a disponibilidade e ativação da função. Este estudo traz implicações para as empresas que desenvolvem a próxima geração de ecrãs no mundo automóvel.Automation is changing the world. As in aviation, the car manufacturers are currently developing autonomous vehicles. However, the autonomy of that vehicles isn’t complete, still being needed in certain moments the driver on ride. The way how is done this transition between manual and autonomous driving and how show this information to the driver is a challenge for Ergonomics. New displays are being studied to facilitate these transitions. This study used a driving simulator to investigates, whether augmented reality information can positively influence the user experience during manual and autonomous driving. Therefore, we compared two ways of present the communicate to the driver. The “AR concept” displays all the information in windshield to be easier to the driver access to the information. The “IC concept” displays the information that appears nowadays in the cars, where they use the Instrument Cluster and the e-HUD to display information. Results indicate that the user experience (UX) is influence by concepts, where “AR concept” had better UX in all the states. In terms of confidence, the results revealed higher scores in “AR concept” too. The type of concept does not influence the takeover times or the behavior of take control. In terms of situational awareness (SA), “AR concept” leave the drivers more aware during availability and activation. This study provides implications for automotive companies developing the next generation of car displays

Driving Towards Inclusion: Revisiting In-Vehicle Interaction in Autonomous Vehicles

This paper presents a comprehensive literature review of the current state of in-vehicle human-computer interaction (HCI) in the context of self-driving vehicles, with a specific focus on inclusion and accessibility. This study's aim is to examine the user-centered design principles for inclusive HCI in self-driving vehicles, evaluate existing HCI systems, and identify emerging technologies that have the potential to enhance the passenger experience. The paper begins by providing an overview of the current state of self-driving vehicle technology, followed by an examination of the importance of HCI in this context. Next, the paper reviews the existing literature on inclusive HCI design principles and evaluates the effectiveness of current HCI systems in self-driving vehicles. The paper also identifies emerging technologies that have the potential to enhance the passenger experience, such as voice-activated interfaces, haptic feedback systems, and augmented reality displays. Finally, the paper proposes an end-to-end design framework for the development of an inclusive in-vehicle experience, which takes into consideration the needs of all passengers, including those with disabilities, or other accessibility requirements. This literature review highlights the importance of user-centered design principles in the development of HCI systems for self-driving vehicles and emphasizes the need for inclusive design to ensure that all passengers can safely and comfortably use these vehicles. The proposed end-to-end design framework provides a practical approach to achieving this goal and can serve as a valuable resource for designers, researchers, and policymakers in this field

A sound idea: An investigation into accessible video game design for the deaf and hard of hearing

A widely accepted, and incorrect, assumption towards hearing accessibility in video games is that deaf and hard of hearing (DHH) users are those who encounter the least barriers and are generally well catered for. Rapid advancement in video game technology has seen video game sound evolve from simple blips generated by internal circuitry to fully realised digital audio used to convey critical information. To accommodate the DHH, this information needs to be conveyed in an alternative manner. However, evidence suggests existing accessible design solutions for the DHH lack specificity and are insufficient. Thus, the inability to hear, or hear well, has historically resulted in DHH users left with impeded experience and gameplay. This thesis describes an investigation to address the primary research question: How might accessible video game design practices be facilitated to better accommodate the deaf and hard of hearing? To examine this question, an action research method as part of a transformative mixed methods methodology was used. Data collection procedures included critical analysis of literature, observations, and a cross-sectional self-administered survey for triangulation. The critical analysis of literature exposed issues relating to accessible video game design, particularly in relation to the identification of solutions and technical implementation. Further, issues related to the classification of video game software were identified. This posed potential problems with identification of game design methods and led to the development of a new video game classification model. The new model informed an analysis on the methods used for the design of video games, and results were visually represented and mapped to the different approaches to accessible design. Subsequent analysis determined that a game assessment framework is a suitable approach to facilitating accessible design. Further investigation identified visual feedback as the most suitable form of complementary feedback to game audio. This led to the development of a new model to classify visual feedback elements used in video games, and identification of audio feedback categories based on diegetic film theory. Through triangulation of results, a new game feedback model (GFM) was developed. The GFM was used for observational experimentation to identify and classify individual visual feedback elements used in video games. Each element was analysed and mapped to categories of game sound. The resulting model, with populated data, was used to determine what visual feedback elements may be used to complement specific categories of critical game audio. A survey was subsequently used for triangulation, and resulted in amendments to the final model. Through iterative development, and interpretation of findings, the research culminated in the development of a game assessment framework. The three-step framework aids in the classification of game sounds; assesses the impact of those game sounds; and provides recommendations for complementary visual feedback elements for sounds identified as having an adverse impact on user experience and gameplay if they were to be removed. The framework is innovative and has the potential to provide practical guidance for developers of video games. In addition, this research provides the foundation for future research, with the potential to influence accessible game design for the DHH

Stereoscopic 3D user interfaces : exploring the potentials and risks of 3D displays in cars

During recent years, rapid advancements in stereoscopic digital display technology has led to acceptance of high-quality 3D in the entertainment sector and even created enthusiasm towards the technology. The advent of autostereoscopic displays (i.e., glasses-free 3D) allows for introducing 3D technology into other application domains, including but not limited to mobile devices, public displays, and automotive user interfaces - the latter of which is at the focus of this work. Prior research demonstrates that 3D improves the visualization of complex structures and augments virtual environments. We envision its use to enhance the in-car user interface by structuring the presented information via depth. Thus, content that requires attention can be shown close to the user and distances, for example to other traffic participants, gain a direct mapping in 3D space

Exploration of smart infrastructure for drivers of autonomous vehicles

The connection between vehicles and infrastructure is an integral part of providing autonomous vehicles information about the environment. Autonomous vehicles need to be safe and users need to trust their driving decision. When smart infrastructure information is integrated into the vehicle, the driver needs to be informed in an understandable manner what the smart infrastructure detected. Nevertheless, interactions that benefit from smart infrastructure have not been the focus of research, leading to knowledge gaps in the integration of smart infrastructure information in the vehicle. For example, it is unclear, how the information from two complex systems can be presented, and if decisions are made, how these can be explained. Enriching the data of vehicles with information from the infrastructure opens unexplored opportunities. Smart infrastructure provides vehicles with information to predict traffic flow and traffic events. Additionally, it has information about traffic events in several kilometers distance and thus enables a look ahead on a traffic situation, which is not in the immediate view of drivers. We argue that this smart infrastructure information can be used to enhance the driving experience. To achieve this, we explore designing novel interactions, providing warnings and visualizations about information that is out of the view of the driver, and offering explanations for the cause of changed driving behavior of the vehicle. This thesis focuses on exploring the possibilities of smart infrastructure information with a focus on the highway. The first part establishes a design space for 3D in-car augmented reality applications that profit from smart infrastructure information. Through the input of two focus groups and a literature review, use cases are investigated that can be introduced in the vehicle's interaction interface which, among others, rely on environment information. From those, a design space that can be used to design novel in-car applications is derived. The second part explores out-of-view visualizations before and during take over requests to increase situation awareness. With three studies, different visualizations for out-of-view information are implemented in 2D, stereoscopic 3D, and augmented reality. Our results show that visualizations improve the situation awareness about critical events in larger distances during take over request situations. In the third part, explanations are designed for situations in which the vehicle drives unexpectedly due to unknown reasons. Since smart infrastructure could provide connected vehicles with out-of-view or cloud information, the driving maneuver of the vehicle might remain unclear to the driver. Therefore, we explore the needs of drivers in those situations and derive design recommendations for an interface which displays the cause for the unexpected driving behavior. This thesis answers questions about the integration of environment information in vehicles'. Three important aspects are explored, which are essential to consider when implementing use cases with smart infrastructure in mind. It enables to design novel interactions, provides insights on how out-of-view visualizations can improve the drivers' situation awareness and explores unexpected driving situations and the design of explanations for them. Overall, we have shown how infrastructure and connected vehicle information can be introduced in vehicles' user interface and how new technology such as augmented reality glasses can be used to improve the driver's perception of the environment.Autonome Fahrzeuge werden immer mehr in den alltäglichen Verkehr integriert. Die Verbindung von Fahrzeugen mit der Infrastruktur ist ein wesentlicher Bestandteil der Bereitstellung von Umgebungsinformationen in autonome Fahrzeugen. Die Erweiterung der Fahrzeugdaten mit Informationen der Infrastruktur eröffnet ungeahnte Möglichkeiten. Intelligente Infrastruktur übermittelt verbundenen Fahrzeugen Informationen über den prädizierten Verkehrsfluss und Verkehrsereignisse. Zusätzlich können Verkehrsgeschehen in mehreren Kilometern Entfernung übermittelt werden, wodurch ein Vorausblick auf einen Bereich ermöglicht wird, der für den Fahrer nicht unmittelbar sichtbar ist. Mit dieser Dissertation wird gezeigt, dass Informationen der intelligenten Infrastruktur benutzt werden können, um das Fahrerlebnis zu verbessern. Dies kann erreicht werden, indem innovative Interaktionen gestaltet werden, Warnungen und Visualisierungen über Geschehnisse außerhalb des Sichtfelds des Fahrers vermittelt werden und indem Erklärungen über den Grund eines veränderten Fahrzeugverhaltens untersucht werden. Interaktionen, welche von intelligenter Infrastruktur profitieren, waren jedoch bisher nicht im Fokus der Forschung. Dies führt zu Wissenslücken bezüglich der Integration von intelligenter Infrastruktur in das Fahrzeug. Diese Dissertation exploriert die Möglichkeiten intelligenter Infrastruktur, mit einem Fokus auf die Autobahn. Der erste Teil erstellt einen Design Space für Anwendungen von augmentierter Realität (AR) in 3D innerhalb des Autos, die unter anderem von Informationen intelligenter Infrastruktur profitieren. Durch das Ergebnis mehrerer Studien werden Anwendungsfälle in einem Katalog gesammelt, welche in die Interaktionsschnittstelle des Autos einfließen können. Diese Anwendungsfälle bauen unter anderem auf Umgebungsinformationen. Aufgrund dieser Anwendungen wird der Design Space entwickelt, mit Hilfe dessen neuartige Anwendungen für den Fahrzeuginnenraum entwickelt werden können. Der zweite Teil exploriert Visualisierungen für Verkehrssituationen, die außerhalb des Sichtfelds des Fahrers sind. Es wird untersucht, ob durch diese Visualisierungen der Fahrer besser auf ein potentielles Übernahmeszenario vorbereitet wird. Durch mehrere Studien wurden verschiedene Visualisierungen in 2D, stereoskopisches 3D und augmentierter Realität implementiert, die Szenen außerhalb des Sichtfelds des Fahrers darstellen. Diese Visualisierungen verbessern das Situationsbewusstsein über kritische Szenarien in einiger Entfernung während eines Übernahmeszenarios. Im dritten Teil werden Erklärungen für Situationen gestaltet, in welchen das Fahrzeug ein unerwartetes Fahrmanöver ausführt. Der Grund des Fahrmanövers ist dem Fahrer dabei unbekannt. Mit intelligenter Infrastruktur verbundene Fahrzeuge erhalten Informationen, die außerhalb des Sichtfelds des Fahrers liegen oder von der Cloud bereit gestellt werden. Dadurch könnte der Grund für das unerwartete Fahrverhalten unklar für den Fahrer sein. Daher werden die Bedürfnisse des Fahrers in diesen Situationen erforscht und Empfehlungen für die Gestaltung einer Schnittstelle, die Erklärungen für das unerwartete Fahrverhalten zur Verfügung stellt, abgeleitet. Zusammenfassend wird gezeigt wie Daten der Infrastruktur und Informationen von verbundenen Fahrzeugen in die Nutzerschnittstelle des Fahrzeugs implementiert werden können. Zudem wird aufgezeigt, wie innovative Technologien wie AR Brillen, die Wahrnehmung der Umgebung des Fahrers verbessern können. Durch diese Dissertation werden Fragen über Anwendungsfälle für die Integration von Umgebungsinformationen in Fahrzeugen beantwortet. Drei wichtige Themengebiete wurden untersucht, welche bei der Betrachtung von Anwendungsfällen der intelligenten Infrastruktur essentiell sind. Durch diese Arbeit wird die Gestaltung innovativer Interaktionen ermöglicht, Einblicke in Visualisierungen von Informationen außerhalb des Sichtfelds des Fahrers gegeben und es wird untersucht, wie Erklärungen für unerwartete Fahrsituationen gestaltet werden können