Voice-controlled in-vehicle infotainment system

Abstract. Speech is a form of a human to human communication that can convey information in a context-rich way that is natural to humans. The naturalness enables us to speak while doing other things, such as driving a vehicle. With the advancement of computing technologies, more and more personal services are introduced for the in-vehicle environment. A limiting factor for these advancements is the impact they cause towards driver distraction with the increased cognitive stress load. This has led to developing in-vehicle devices and applications with a heightened focus on lessening distraction. Amazon Alexa is a natural language processing system that enables its users to receive information and operate smart devices with their voices. This Master’s thesis aims to demonstrate how Alexa could be utilized when operating the in-vehicle infotainment (IVI) systems. This research was conducted by utilizing the design science research methodology. The feasibility of voice-based interaction was assessed by implementing the system as a demonstrable use-case in collaboration with the APPSTACLE project. Prior research was gathered by conducting a literature review on voice-based interaction and its integration to the vehicular domain. The system was designed by applying existing theories together with the requirements of the application domain. The designed system utilized the Amazon Alexa ecosystem and AWS services to provide the vehicular environment with new functionalities. Access to cloud-based speech processing and decision-making makes it possible to design an extendable speech interface where the driver can carry out secondary tasks by using their voice, such as requesting navigation information. The evaluation was done by comparing the system’s performance against the derived requirements. With the results of the evaluation process, the feasibility of the system could be assessed against the objectives of the study: The resulting artefact enables the user to operate the in-vehicle infotainment system while focusing on a separate task. The research proved that speech interfaces with modern technology can improve the handling of secondary tasks while driving, and the resulting system was operable without introducing additional distractions to the driver. The resulting artefact can be integrated into similar systems and used as a base tool for future research on voice-controlled interfaces

Proposal of an adaptive infotainment system depending on driving scenario complexity

Tesi en modalitat Doctorat industrialPla de Doctorat industrial de la Generalitat de CatalunyaThe PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios.Postprint (published version

Proposal of an adaptive infotainment system depending on driving scenario complexity

The PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios

Project BeARCAT : Baselining, Automation and Response for CAV Testbed Cyber Security : Connected Vehicle & Infrastructure Security Assessment

Connected, software-based systems are a driver in advancing the technology of transportation systems. Advanced automated and autonomous vehicles, together with electrification, will help reduce congestion, accidents and emissions. Meanwhile, vehicle manufacturers see advanced technology as enhancing their products in a competitive market. However, as many decades of using home and enterprise computer systems have shown, connectivity allows a system to become a target for criminal intentions. Cyber-based threats to any system are a problem; in transportation, there is the added safety implication of dealing with moving vehicles and the passengers within

A Methodological Process for the Design of Frameworks Oriented to Infotainment User Interfaces

The objective of this paper was to propose a methodological process for the design of frameworks oriented to infotainment user interfaces. Four stages comprise the proposed process, conceptualization, structuring, documentation, and evaluation; in addition, these stages include activities, tasks, and deliverables to guide a work team during the design of a framework. To determine the stages and their components, an analysis of 42 papers was carried out through a systematic literature review in search of similarities during the design process of frameworks related to user interfaces. The evaluation method by a panel of experts was used to determine the validity of the proposal; the conceptual proposal was provided to a panel of 10 experts for their analysis and later a questionnaire in the form of a Likert scale was used to collect the information on the validation of the proposal. The results of the evaluation indicated that the methodological process is valid to meet the objective of designing a framework oriented to infotainment user interfaces

Is Europe in the Driver's Seat? The Competitiveness of the European Automotive Embedded Systems Industry

This report is one of a series resulting from a project entitled ¿Competitiveness by Leveraging Emerging Technologies Economically¿ (COMPLETE), carried out by JRC-IPTS. Each of the COMPLETE studies illustrates in its own right that European companies are active on many fronts of emerging and disruptive ICT technologies and are supplying the market with relevant products and services. Nevertheless, the studies also show that the creation and growth of high tech companies is still very complex and difficult in Europe, and too many economic opportunities seem to escape European initiatives and ownership. COMPLETE helps to illustrate some of the difficulties experienced in different segments of the ICT industry and by growing potential global players. This report reflects the findings of a study conducted by Egil Juliussen and Richard Robinson, two senior experts from iSuppli Corporation on the Competitiveness of the European Automotive Embedded Software industry. The report starts by introducing the market, its trends, the technologies, their characteristics and their potential economic impact, before moving to an analysis of the competitiveness of the corresponding European industry. It concludes by suggesting policy options. The research, initially based on internal expertise and literature reviews, was complemented with further desk research, expert interviews, expert workshops and company visits. The results were ultimately reviewed by experts and also in a dedicated workshop. The report concludes that currently ICT innovation in the automotive industry is a key competence in Europe, with very little ICT innovation from outside the EU finding its way into EU automotive companies. A major benefit of a strong automotive ICT industry is the resulting large and valuable employment base. But future maintenance of automotive ICT jobs within the EU will only be possible if the EU continues to have high levels of product innovation.JRC.DDG.J.4-Information Societ

A Methodological Process for the Design of Frameworks Oriented to Infotainment User Interfaces

The objective of this paper was to propose a methodological process for the design of frameworks oriented to infotainment user interfaces. Four stages comprise the proposed process, conceptualization, structuring, documentation, and evaluation; in addition, these stages include activities, tasks, and deliverables to guide a work team during the design of a framework. To determine the stages and their components, an analysis of 42 papers was carried out through a systematic literature review in search of similarities during the design process of frameworks related to user interfaces. The evaluation method by a panel of experts was used to determine the validity of the proposal; the conceptual proposal was provided to a panel of 10 experts for their analysis and later a questionnaire in the form of a Likert scale was used to collect the information on the validation of the proposal. The results of the evaluation indicated that the methodological process is valid to meet the objective of designing a framework oriented to infotainment user interfaces

The Application of Physiological Metrics in Validating User Experience Evaluation on Automotive Human Machine Interface Systems

Automotive in-vehicle information systems have seen an era of continuous development within the industry and are recognised as a key differentiator for prospective customers. This presents a significant challenge for designers and engineers in producing effective next generation systems which are helpful, novel, exciting, safe and easy to use. The usability of any new human machine interface (HMI) has an implicit cost in terms of the perceived aesthetic perception and associated user experience. Achieving the next engaging automotive interface, not only has to address the user requirements but also has to incorporate established safety standards whilst considering new interaction technologies. An automotive (HMI) evaluation may combine a triad of physiological, subjective and performance-based measurements which are employed to provide relevant and valuable data for product evaluation. However, there is also a growing interest and appreciation that determining real-time quantitative metrics to drivers’ affective responses provide valuable user affective feedback. The aim of this research was to explore to what extent physiological metrics such as heart rate variability could be used to quantify or validate subjective testing of automotive HMIs. This research employed both objective and subjective metrics to assess user engagement during interactions with an automotive infotainment system. The mapping of both physiological and self-report scales was examined over a series of studies in order to provide a greater understanding of users’ responses. By analysing the data collected it may provide guidance within the early stages of in-vehicle design evaluation in terms of usability and user satisfaction. This research explored these metrics as an objective, quantitative, diagnostic measure of affective response, in the assessment of HMIs. Development of a robust methodology was constructed for the application and understanding of these metrics. Findings from the three studies point towards the value of using a combination of methods when examining user interaction with an in-car HMI. For the next generation of interface systems, physiological measures, such as heart rate variability may offer an additional dimension of validity when examining the complexities of the driving task that drivers perform every day. There appears to be no boundaries on technology advancements and with this, comes extra pressure for car manufacturers to produce similar interactive and connective devices to those that are already in use in homes. A successful in-car HMI system will be intuitive to use, aesthetically pleasing and possess an element of pleasure however, the design components that are needed for a highly usable HMI have to be considered within the context of the constraints of the manufacturing process and the risks associated with interacting with an in-car HMI whilst driving. The findings from the studies conducted in this research are discussed in relation to the usability and benefits of incorporating physiological measures that can assist in our understanding of driver interaction with different automotive HMIs