Queuing Network Modeling of Human Multitask Performance and its Application to Usability Testing of In-Vehicle Infotainment Systems.

Human performance of a primary continuous task (e.g., steering a vehicle) and a secondary discrete task (e.g., tuning radio stations) simultaneously is a common scenario in many domains. It is of great importance to have a good understanding of the mechanisms of human multitasking behavior in order to design the task environments and user interfaces (UIs) that facilitate human performance and minimize potential safety hazards. In this dissertation I investigated and modeled human multitask performance with a vehicle-steering task and several typical in-vehicle secondary tasks. Two experiments were conducted to investigate how various display designs and control modules affect the driver's eye glance behavior and performance. A computational model based on the cognitive architecture of Queuing Network-Model Human Processor (QN-MHP) was built to account for the experiment findings. In contrast to most existing studies that focus on visual search in single task situations, this dissertation employed experimental work that investigates visual search in multitask situations. A modeling mechanism for flexible task activation (rather than strict serial activations) was developed to allow the activation of a task component to be based on the completion status of other task components. A task switching scheme was built to model the time-sharing nature of multitasking. These extensions offer new theoretical insights into visual search in multitask situations and enable the model to simulate parallel processing both within one task and among multiple tasks. The validation results show that the model could account for the observed performance differences from the empirical data. Based on this model, a computer-aided engineering toolkit was developed that allows the UI designers to make quantitative prediction of the usability of design concepts and prototypes. Scientifically, the results of this dissertation research offer additional insights into the mechanisms of human multitask performance. From the engineering application and practical value perspective, the new modeling mechanism and the new toolkit have advantages over the traditional usability testing methods with human subjects by enabling the UI designers to explore a larger design space and address usability issues at the early design stages with lower cost both in time and manpower.PHDIndustrial and Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113590/1/fredfeng_1.pd

A Methodological Process for the Design of Frameworks Oriented to Infotainment User Interfaces

The objective of this paper was to propose a methodological process for the design of frameworks oriented to infotainment user interfaces. Four stages comprise the proposed process, conceptualization, structuring, documentation, and evaluation; in addition, these stages include activities, tasks, and deliverables to guide a work team during the design of a framework. To determine the stages and their components, an analysis of 42 papers was carried out through a systematic literature review in search of similarities during the design process of frameworks related to user interfaces. The evaluation method by a panel of experts was used to determine the validity of the proposal; the conceptual proposal was provided to a panel of 10 experts for their analysis and later a questionnaire in the form of a Likert scale was used to collect the information on the validation of the proposal. The results of the evaluation indicated that the methodological process is valid to meet the objective of designing a framework oriented to infotainment user interfaces

A Methodological Process for the Design of Frameworks Oriented to Infotainment User Interfaces

The objective of this paper was to propose a methodological process for the design of frameworks oriented to infotainment user interfaces. Four stages comprise the proposed process, conceptualization, structuring, documentation, and evaluation; in addition, these stages include activities, tasks, and deliverables to guide a work team during the design of a framework. To determine the stages and their components, an analysis of 42 papers was carried out through a systematic literature review in search of similarities during the design process of frameworks related to user interfaces. The evaluation method by a panel of experts was used to determine the validity of the proposal; the conceptual proposal was provided to a panel of 10 experts for their analysis and later a questionnaire in the form of a Likert scale was used to collect the information on the validation of the proposal. The results of the evaluation indicated that the methodological process is valid to meet the objective of designing a framework oriented to infotainment user interfaces

A novel Big Data analytics and intelligent technique to predict driver's intent

Modern age offers a great potential for automatically predicting the driver's intent through the increasing miniaturization of computing technologies, rapid advancements in communication technologies and continuous connectivity of heterogeneous smart objects. Inside the cabin and engine of modern cars, dedicated computer systems need to possess the ability to exploit the wealth of information generated by heterogeneous data sources with different contextual and conceptual representations. Processing and utilizing this diverse and voluminous data, involves many challenges concerning the design of the computational technique used to perform this task. In this paper, we investigate the various data sources available in the car and the surrounding environment, which can be utilized as inputs in order to predict driver's intent and behavior. As part of investigating these potential data sources, we conducted experiments on e-calendars for a large number of employees, and have reviewed a number of available geo referencing systems. Through the results of a statistical analysis and by computing location recognition accuracy results, we explored in detail the potential utilization of calendar location data to detect the driver's intentions. In order to exploit the numerous diverse data inputs available in modern vehicles, we investigate the suitability of different Computational Intelligence (CI) techniques, and propose a novel fuzzy computational modelling methodology. Finally, we outline the impact of applying advanced CI and Big Data analytics techniques in modern vehicles on the driver and society in general, and discuss ethical and legal issues arising from the deployment of intelligent self-learning cars

Computational Modeling and Experimental Research on Touchscreen Gestures, Audio/Speech Interaction, and Driving

As humans are exposed to rapidly evolving complex systems, there are growing needs for humans and systems to use multiple communication modalities such as auditory, vocal (or speech), gesture, or visual channels; thus, it is important to evaluate multimodal human-machine interactions in multitasking conditions so as to improve human performance and safety. However, traditional methods of evaluating human performance and safety rely on experimental settings using human subjects which require costly and time-consuming efforts to conduct. To minimize the limitations from the use of traditional usability tests, digital human models are often developed and used, and they also help us better understand underlying human mental processes to effectively improve safety and avoid mental overload. In this regard, I have combined computational cognitive modeling and experimental methods to study mental processes and identify differences in human performance/workload in various conditions, through this dissertation research. The computational cognitive models were implemented by extending the Queuing Network-Model Human Processor (QN-MHP) Architecture that enables simulation of human multi-task behaviors and multimodal interactions in human-machine systems. Three experiments were conducted to investigate human behaviors in multimodal and multitasking scenarios, combining the following three specific research aims that are to understand: (1) how humans use their finger movements to input information on touchscreen devices (i.e., touchscreen gestures), (2) how humans use auditory/vocal signals to interact with the machines (i.e., audio/speech interaction), and (3) how humans drive vehicles (i.e., driving controls). Future research applications of computational modeling and experimental research are also discussed. Scientifically, the results of this dissertation research make significant contributions to our better understanding of the nature of touchscreen gestures, audio/speech interaction, and driving controls in human-machine systems and whether they benefit or jeopardize human performance and safety in the multimodal and concurrent task environments. Moreover, in contrast to the previous models for multitasking scenarios mainly focusing on the visual processes, this study develops quantitative models of the combined effects of auditory, tactile, and visual factors on multitasking performance. From the practical impact perspective, the modeling work conducted in this research may help multimodal interface designers minimize the limitations of traditional usability tests and make quick design comparisons, less constrained by other time-consuming factors, such as developing prototypes and running human subjects. Furthermore, the research conducted in this dissertation may help identify which elements in the multimodal and multitasking scenarios increase workload and completion time, which can be used to reduce the number of accidents and injuries caused by distraction.PHDIndustrial & Operations EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/143903/1/heejinj_1.pd

A Hybrid Data-Driven Web-Based UI-UX Assessment Model

Today, a large proportion of end user information systems have their Graphical User Interfaces (GUI) built with web-based technology (JavaScript, CSS, and HTML). Some of these web-based systems include: Internet of Things (IOT), Infotainment (in vehicles), Interactive Display Screens (for digital menu boards, information kiosks, digital signage displays at bus stops or airports, bank ATMs, etc.), and web applications/services (on smart devices). As such, web-based UI must be evaluated in order to improve upon its ability to perform the technical task for which it was designed. This study develops a framework and a processes for evaluating and improving the quality of web-based user interface (UI) as well as at a stratified level. The study develops a comprehensive framework which is a conglomeration of algorithms such as the multi-criteria decision making method of analytical hierarchy process (AHP) in coefficient generation, sentiment analysis, K-means clustering algorithms and explainable AI (XAI)

The Application of Physiological Metrics in Validating User Experience Evaluation on Automotive Human Machine Interface Systems

Automotive in-vehicle information systems have seen an era of continuous development within the industry and are recognised as a key differentiator for prospective customers. This presents a significant challenge for designers and engineers in producing effective next generation systems which are helpful, novel, exciting, safe and easy to use. The usability of any new human machine interface (HMI) has an implicit cost in terms of the perceived aesthetic perception and associated user experience. Achieving the next engaging automotive interface, not only has to address the user requirements but also has to incorporate established safety standards whilst considering new interaction technologies. An automotive (HMI) evaluation may combine a triad of physiological, subjective and performance-based measurements which are employed to provide relevant and valuable data for product evaluation. However, there is also a growing interest and appreciation that determining real-time quantitative metrics to drivers’ affective responses provide valuable user affective feedback. The aim of this research was to explore to what extent physiological metrics such as heart rate variability could be used to quantify or validate subjective testing of automotive HMIs. This research employed both objective and subjective metrics to assess user engagement during interactions with an automotive infotainment system. The mapping of both physiological and self-report scales was examined over a series of studies in order to provide a greater understanding of users’ responses. By analysing the data collected it may provide guidance within the early stages of in-vehicle design evaluation in terms of usability and user satisfaction. This research explored these metrics as an objective, quantitative, diagnostic measure of affective response, in the assessment of HMIs. Development of a robust methodology was constructed for the application and understanding of these metrics. Findings from the three studies point towards the value of using a combination of methods when examining user interaction with an in-car HMI. For the next generation of interface systems, physiological measures, such as heart rate variability may offer an additional dimension of validity when examining the complexities of the driving task that drivers perform every day. There appears to be no boundaries on technology advancements and with this, comes extra pressure for car manufacturers to produce similar interactive and connective devices to those that are already in use in homes. A successful in-car HMI system will be intuitive to use, aesthetically pleasing and possess an element of pleasure however, the design components that are needed for a highly usable HMI have to be considered within the context of the constraints of the manufacturing process and the risks associated with interacting with an in-car HMI whilst driving. The findings from the studies conducted in this research are discussed in relation to the usability and benefits of incorporating physiological measures that can assist in our understanding of driver interaction with different automotive HMIs

A study of cultural models in automotive HMI : framework for accommodating cultural influence.

This doctoral innovation report presents a research study examining the importance of understanding automotive users’ cultural values and their individual preferences for HMI features and functionalities. The goal of this research was to explore how a cultural model can be applied in the development of automotive HMI solutions and future design localisation. To meet this goal, it was necessary to (1) identify the characteristics of the Hofstede cultural model; (2) identify the differences in cultural values using the model; (3) identify differences in HMI design preferences, usability and task performances across automotive user groups; (4) identify the potential success of a culturally adapted automotive HMI solution in automotive user acceptance and satisfaction. To explore the differences between users from two cultural regions, India and the UK, a series of user-centered HMI evaluation studies are conducted in which participants from each cultural region evaluate representative HMI samples. The outcomes of the user studies generate good quality data about automotive users’ cultural values and its relationship with vehicle user interface usability, task performances, and their feature preferences. The results are used in the development of a conceptual culturally adapted HMI design solution. This conceptual design is evaluated during the application phase of the research in order to explore whether such a design solution has a greater level of learnability and usability compared to the conventional solution when evaluated by Indian drivers. The results are also analysed to identify specific cultural traits that may influence the intention to use such solution in emerging markets like India. The outcome of the study shows different cultural groups have different behavioural tendencies and performances while using vehicle HMI solutions and have differences in expectations in design, suggesting an influence of culture on the perception of vehicle user interface technology. The analysis also highlights a preference for the culturally adapted automotive HMI solution when Indian drivers are provided with a choice between this and a non-adapted conventional solution. This leads to the conclusion that an understanding of cultural biases can influence design localisation and, as such, culturally-generated theories and recommendations can be applied as a basis for future automotive HMI design and development

The Digital Transformation of Automotive Businesses: THREE ARTEFACTS TO SUPPORT DIGITAL SERVICE PROVISION AND INNOVATION

Digitalisation and increasing competitive pressure drive original equipment manufacturers (OEMs) to switch their focus towards the provision of digital services and open-up towards increased collaboration and customer integration. This shift implies a significant transformational change from product to product-service providers, where OEMs realign themselves within strategic, business and procedural dimensions. Thus, OEMs must manage digital transformation (DT) processes in order to stay competitive and remain adaptable to changing customer demands. However, OEMs aspiring to become participants or leaders in their domain, struggle to initiate activities as there is a lack of applicable instruments that can guide and support them during this process. Compared to the practical importance of DT, empirical studies are not comprehensive. This study proposes three artefacts, validated within case companies that intend to support automotive OEMs in digital service provisioning. Artefact one, a layered conceptual model for a digital automotive ecosystem, was developed by means of 26 expert interviews. It can serve as a useful instrument for decision makers to strategically plan and outline digital ecosystems. Artefact two is a conceptual reference framework for automotive service systems. The artefact was developed based on an extensive literature review, and the mapping of the business model canvas to the service system domain. The artefact intends to assist OEMs in the efficient conception of digital services under consideration of relevant stakeholders and the necessary infrastructures. Finally, artefact three proposes a methodology by which to transform software readiness assessment processes to fit into the agile software development approach with consideration of the existing operational infrastructure. Overall, the findings contribute to the empirical body of knowledge about the digital transformation of manufacturing industries. The results suggest value creation for digital automotive services occurs in networks among interdependent stakeholders in which customers play an integral role during the services’ life-cycle. The findings further indicate the artefacts as being useful instruments, however, success is dependent on the integration and collaboration of all contributing departments.:Table of Contents Bibliographic Description II Acknowledgment III Table of Contents IV List of Figures VI List of Tables VII List of Abbreviations VIII 1 Introduction 1 1.1 Motivation and Problem Statement 1 1.2 Objective and Research Questions 6 1.3 Research Methodology 7 1.4 Contributions 10 1.5 Outline 12 2 Background 13 2.1 From Interdependent Value Creation to Digital Ecosystems 13 2.1.1 Digitalisation Drives Collaboration 13 2.1.2 Pursuing an Ecosystem Strategy 13 2.1.3 Research Gaps and Strategy Formulation Obstacles 20 2.2 From Products to Product-Service Solutions 22 2.2.1 Digital Service Fulfilment Requires Co-Creational Networks 22 2.2.2 Enhancing Business Models with Digital Services 28 2.2.3 Research Gaps and Service Conception Obstacles 30 2.3 From Linear Development to Continuous Innovation 32 2.3.1 Digital Innovation Demands Digital Transformation 32 2.3.2 Assessing Digital Products 36 2.3.3 Research Gaps and Implementation Obstacles 38 3 Artefact 1: Digital Automotive Ecosystems 41 3.1 Meta Data 41 3.2 Summary 42 3.3 Designing a Layered Conceptual Model of a Digital Ecosystem 45 4 Artefact 2: Conceptual Reference Framework 79 4.1 Meta Data 79 4.2 Summary 80 4.3 On the Move Towards Customer-Centric Automotive Business Models 83 5 Artefact 3: Agile Software Readiness Assessment Procedures 121 5.1 Meta Data 121 5.2 Meta Data 122 5.3 Summary 123 5.4 Adding Agility to Software Readiness Assessment Procedures 126 5.5 Continuous Software Readiness Assessments for Agile Development 147 6 Conclusion and Future Work 158 6.1 Contributions 158 6.1.1 Strategic Dimension: Artefact 1 158 6.1.2 Business Dimension: Artefact 2 159 6.1.3 Process Dimension: Artefact 3 161 6.1.4 Synthesis of Contributions 163 6.2 Implications 167 6.2.1 Scientific Implications 167 6.2.2 Managerial Implications 168 6.2.3 Intelligent Parking Service Example (ParkSpotHelp) 171 6.3 Concluding Remarks 174 6.3.1 Threats to Validity 174 6.3.2 Outlook and Future Research Recommendations 174 Appendix VII Bibliography XX Wissenschaftlicher Werdegang XXXVII Selbständigkeitserklärung XXXVII

Experience Prototyping for Automotive Applications

In recent years, we started to define our life through experiences we make instead of objectswe buy. To attend a concert of our favorite musician may be more important for us thanowning an expensive stereo system. Similarly, we define interactive systems not only by thequality of the display or its usability, but rather by the experiences we can make when usingthe device. A cell phone is primarily built for making calls and receiving text messages,but on an emotional level it might provide a way to be close to our loved ones, even thoughthey are far away sometimes. When designing interactive technology, we do not only haveto answer the question how people use our systems, but also why they use them. Thus,we need to concentrate on experiences, feelings and emotions arising during interaction.Experience Design is an approach focusing on the story that a product communicates beforeimplementing the system. In an interdisciplinary team of psychologists, industrial designers, product developers andspecialists in human-computer interaction, we applied an Experience Design process to theautomotive domain. A major challenge for car manufacturers is the preservation of theseexperiences throughout the development process. When implementing interactive systemsengineers rely on technical requirements and a set of constraints (e.g., safety) oftentimescontradicting aspects of the designed experience. To resolve this conflict, Experience Prototypingis an important tool translating experience stories to an actual interactive product. With this thesis I investigate the Experience Design process focusing on Experience Prototyping.Within the automotive context, I report on three case studies implementing threekinds of interactive systems, forming and following our approach. I implemented (1) anelectric vehicle information system called Heartbeat, communicating the state of the electricdrive and the batteries to the driver in an unobtrusive and ensuring way. I integrated Heartbeatinto the dashboard of a car mock-up with respect to safety and space requirements butat the same time holding on to the story in order to achieve a consistent experience. With (2)the Periscope I implemented a mobile navigation device enhancing the social and relatednessexperiences of the passengers in the car. I built and evaluated several experience prototypesin different stages of the design process and showed that they transported the designed experiencethroughout the implementation of the system. Focusing on (3) the experience offreehand gestures, GestShare explored this interaction style for in-car and car-to-car socialexperiences. We designed and implemented a gestural prototypes for small but effectivesocial interactions between drivers and evaluated the system in the lab and and in-situ study. The contributions of this thesis are (1) a definition of Experience Prototyping in the automotivedomain resulting from a literature review and my own work, showing the importanceand feasibility of Experience Prototyping for Experience Design. I (2) contribute three casestudies and describe the details of several prototypes as milestones on the way from a anexperience story to an interactive system. I (3) derive best practices for Experience Prototypingconcerning their characteristics such as fidelity, resolution and interactivity as well asthe evaluation in the lab an in situ in different stages of the process.Wir definieren unser Leben zunehmend durch Dinge, die wir erleben und weniger durchProdukte, die wir kaufen. Ein Konzert unseres Lieblingsmusikers zu besuchen kann dabeiwichtiger sein, als eine teure Stereoanlage zu besitzen. Auch interaktive Systeme bewertenwir nicht mehr nur nach der Qualität des Displays oder der Benutzerfreundlichkeit, sondernauch nach Erlebnissen, die durch die Benutzung möglich werden. Das Smartphone wurdehauptsächlich zum Telefonieren und Schreiben von Nachrichten entwickelt. Auf einer emotionalenEbene bietet es uns aber auch eine Möglichkeit, wichtigen Personen sehr nah zusein, auch wenn sie manchmal weit weg sind. Bei der Entwicklung interaktiver Systememüssen wir uns daher nicht nur fragen wie, sondern auch warum diese benutzt werden. Erlebnisse,Gefühle und Emotionen, die während der Interaktion entstehen, spielen dabei einewichtige Rolle. Experience Design ist eine Disziplin, die sich auf Geschichten konzentriert,die ein Produkt erzählt, bevor es tatsächlich implementiert wird. In einem interdisziplinären Team aus Psychologen, Industrie-Designern, Produktentwicklernund Spezialisten der Mensch-Maschine-Interaktion wurde ein Prozess zur Erlebnis-Gestaltung im automobilen Kontext angewandt. Die Beibehaltung von Erlebnissen über dengesamten Entwicklungsprozess hinweg ist eine große Herausforderung für Automobilhersteller.Ingenieure hängen bei der Implementierung interaktiver Systeme von technischen,sicherheitsrelevanten und ergonomischen Anforderungen ab, die oftmals dem gestaltetenErlebnis widersprechen. Die Bereitstellung von Erlebnis-Prototypen ermöglicht die Übersetzungvon Geschichten in interaktive Produkte und wirkt daher diesem Konflikt entgegen. Im Rahmen dieser Dissertation untersuche ich den Prozess zur Erlebnis-Gestaltung hinsichtlichder Bedeutung von Erlebnis-Prototypen. Ich berichte von drei Fallbeispielen im automobilenBereich, die die Gestaltung und Implementierung verschiedener interaktiver Systemenumfassen. (1) Ein Informationssystem für Elektrofahrzeuge, der Heartbeat, macht den Zustanddes elektrischen Antriebs und den Ladestand der Batterien für den Fahrer visuell undhaptisch erlebbar. Nach der Implementierung mehrerer Prototypen wurde Heartbeat unterBerücksichtigung verschiedener technischer und sicherheitsrelevanter Anforderungen in dieArmaturen eines Fahrzeugmodells integriert, ohne dass dabei das gestaltete Erlebnis verlorengegangen ist. (2) Das Periscope ist ein mobiles Navigationsgerät, das den Insassensoziale Erlebnisse ermöglicht und das Verbundenheitsgefühl stärkt. Durch die Implementierungmehrere Erlebnis-Prototypen und deren Evaluation in verschiedenen Phasen des Entwicklungsprozesseskonnten die gestalteten Erlebnisse konsistent erhalten werden. (3) ImProjekt GestShare wurde das Potential der Interaktion durch Freiraumgesten im Fahrzeuguntersucht. Dabei standen ein Verbundenheitserlebnis des Fahrers und soziale Interaktionenmit Fahrern anderer Fahrzeuge im Fokus. Es wurden mehrere Prototypen implementiert undauch in einer Verkehrssituation evaluiert. Die wichtigsten Beiträge dieser Dissertation sind (1) eine intensive Betrachtung und Anwendungvon Erlebnis-Prototypen im Auto und deren Relevanz bei der Erlebnis-Gestaltung,beruhend auf einer Literaturauswertung und der eigenen Erfahrung innerhalb des Projekts; (2) drei Fallstudien und eine detaillierte Beschreibung mehrere Prototypen in verschiedenenPhasen des Prozesses und (3) Empfehlungen zu Vorgehensweisen bei der Erstellung vonErlebnis-Prototypen hinsichtlich der Eigenschaften wie Nähe zum finalen Produkt, Anzahlder implementierten Details und Interaktivität sowie zur Evaluation im Labor und in tatsächlichenVerkehrssituationen in verschiedenen Phasen des Entwicklungsprozesses