Proposal of an adaptive infotainment system depending on driving scenario complexity

Tesi en modalitat Doctorat industrialPla de Doctorat industrial de la Generalitat de CatalunyaThe PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios.Postprint (published version

Proposal of an adaptive infotainment system depending on driving scenario complexity

The PhD research project is framed within the plan of industrial doctorates of the “Generalitat de Catalunya”. During the investigation, most of the work was carried out at the facilities of the vehicle manufacturer SEAT, specifically at the information and entertainment (infotainment) department. In the same way, there was a continuous cooperation with the telematics department of the UPC. The main objective of the project consisted in the design and validation of an adaptive infotainment system dependent on the driving complexity. The system was created with the purpose of increasing driver’ experience while guaranteeing a proper level of road safety. Given the increasing number of application and services available in current infotainment systems, it becomes necessary to devise a system capable of balancing these two counterparts. The most relevant parameters that can be used for balancing these metrics while driving are: type of services offered, interfaces available for interacting with the services, the complexity of driving and the profile of the driver. The present study can be divided into two main development phases, each phase had as outcome a real physical block that came to be part of the final system. The final system was integrated in a vehicle and validated in real driving conditions. The first phase consisted in the creation of a model capable of estimating the driving complexity based on a set of variables related to driving. The model was built by employing machine learning methods and the dataset necessary to create it was collected from several driving routes carried out by different participants. This phase allowed to create a model capable of estimating, with a satisfactory accuracy, the complexity of the road using easily extractable variables in any modern vehicle. This approach simplify the implementation of this algorithm in current vehicles. The second phase consisted in the classification of a set of principles that allow the design of the adaptive infotainment system based on the complexity of the road. These principles are defined based on previous researches undertaken in the field of usability and user experience of graphical interfaces. According to these of principles, a real adaptive infotainment system with the most commonly used functionalities; navigation, radio and media was designed and integrated in a real vehicle. The developed system was able to adapt the presentation of the content according to the estimation of the driving complexity given by the block developed in phase one. The adaptive system was validated in real driving scenarios by several participants and results showed a high level of acceptance and satisfaction towards this adaptive infotainment. As a starting point for future research, a proof of concept was carried out to integrate new interfaces into a vehicle. The interface used as reference was a Head Mounted screen that offered redundant information in relation to the instrument cluster. Tests with participants served to understand how users perceive the introduction of new technologies and how objective benefits could be blurred by initial biases.El proyecto de investigación de doctorado se enmarca dentro del plan de doctorados industriales de la Generalitat de Catalunya. Durante la investigación, la mayor parte del trabajo se llevó a cabo en las instalaciones del fabricante de vehículos SEAT, específicamente en el departamento de información y entretenimiento (infotainment). Del mismo modo, hubo una cooperación continua con el departamento de telemática de la UPC. El objetivo principal del proyecto consistió en el diseño y la validación de un sistema de información y entretenimiento adaptativo que se ajustaba de acuerdo a la complejidad de la conducción. El sistema fue creado con el propósito de aumentar la experiencia del conductor y garantizar un nivel adecuado en la seguridad vial. El proyecto surge dado el número creciente de aplicaciones y servicios disponibles en los sistemas actuales de información y entretenimiento; es por ello que se hace necesario contar con un sistema capaz de equilibrar estas dos contrapartes. Los parámetros más relevantes que se pueden usar para equilibrar estas métricas durante la conducción son: el tipo de servicios ofrecidos, las interfaces disponibles para interactuar con los servicios, la complejidad de la conducción y el perfil del conductor. El presente estudio se puede dividir en dos fases principales de desarrollo, cada fase tuvo como resultado un componente que se convirtió en parte del sistema final. El sistema final fue integrado en un vehículo y validado en condiciones reales de conducción. La primera fase consistió en la creación de un modelo capaz de estimar la complejidad de la conducción en base a un conjunto de variables relacionadas con la conducción. El modelo se construyó empleando "Machine Learning Methods" y el conjunto de datos necesario para crearlo se recopiló a partir de varias rutas de conducción realizadas por diferentes participantes. Esta fase permitió crear un modelo capaz de estimar, con una precisión satisfactoria, la complejidad de la carretera utilizando variables fácilmente extraíbles en cualquier vehículo moderno. Este enfoque simplifica la implementación de este algoritmo en los vehículos actuales. La segunda fase consistió en la clasificación de un conjunto de principios que permiten el diseño del sistema de información y entretenimiento adaptativo basado en la complejidad de la carretera. Estos principios se definen en base a investigaciones anteriores realizadas en el campo de usabilidad y experiencia del usuario con interfaces gráficas. De acuerdo con estos principios, un sistema de entretenimiento y entretenimiento real integrando las funcionalidades más utilizadas; navegación, radio y audio fue diseñado e integrado en un vehículo real. El sistema desarrollado pudo adaptar la presentación del contenido según la estimación de la complejidad de conducción dada por el bloque desarrollado en la primera fase. El sistema adaptativo fue validado en escenarios de conducción reales por varios participantes y los resultados mostraron un alto nivel de aceptación y satisfacción hacia este entretenimiento informativo adaptativo. Como punto de partida para futuras investigaciones, se llevó a cabo una prueba de concepto para integrar nuevas interfaces en un vehículo. La interfaz utilizada como referencia era una pantalla a la altura de los ojos (Head Mounted Display) que ofrecía información redundante en relación con el grupo de instrumentos. Las pruebas con los participantes sirvieron para comprender cómo perciben los usuarios la introducción de nuevas tecnologías y cómo los sesgos iniciales podrían difuminar los beneficios

On combining Big Data and machine learning to support eco-driving behaviours

A conscious use of the battery is one of the key elements to consider while driving an electric vehicle. Hence, supporting the drivers, with information about it, can be strategic in letting them drive in a better way, with the purpose of optimizing the energy consumption. In the context of electric vehicles, equipped with regenerative brakes, the driver\u2019s braking style can make a significant difference. In this paper, we propose an approach which is based on the combination of big data and machine learning techniques, with the aim of enhancing the driver\u2019s braking style through visual elements (displayed in the vehicle dashboard, as a Human\u2013Machine Interface), actuating eco-driving behaviours. We have designed and developed a system prototype, by exploiting big data coming from an electric vehicle and a machine learning algorithm. Then, we have conducted a set of tests, with simulated and real data, and here we discuss the results we have obtained that can open interesting discussions about the use of big data, together with machine learning, so as to improve drivers\u2019 awareness of eco-behaviours

Sensor Technologies for Intelligent Transportation Systems

Modern society faces serious problems with transportation systems, including but not limited to traffic congestion, safety, and pollution. Information communication technologies have gained increasing attention and importance in modern transportation systems. Automotive manufacturers are developing in-vehicle sensors and their applications in different areas including safety, traffic management, and infotainment. Government institutions are implementing roadside infrastructures such as cameras and sensors to collect data about environmental and traffic conditions. By seamlessly integrating vehicles and sensing devices, their sensing and communication capabilities can be leveraged to achieve smart and intelligent transportation systems. We discuss how sensor technology can be integrated with the transportation infrastructure to achieve a sustainable Intelligent Transportation System (ITS) and how safety, traffic control and infotainment applications can benefit from multiple sensors deployed in different elements of an ITS. Finally, we discuss some of the challenges that need to be addressed to enable a fully operational and cooperative ITS environment

Platform-based design, test and fast verification flow for mixed-signal systems on chip

This research is providing methodologies to enhance the design phase from architectural space exploration and system study to verification of the whole mixed-signal system. At the beginning of the work, some innovative digital IPs have been designed to develop efficient signal conditioning for sensor systems on-chip that has been included in commercial products. After this phase, the main focus has been addressed to the creation of a re-usable and versatile test of the device after the tape-out which is close to become one of the major cost factor for ICs companies, strongly linking it to model’s test-benches to avoid re-design phases and multi-environment scenarios, producing a very effective approach to a single, fast and reliable multi-level verification environment. All these works generated different publications in scientific literature. The compound scenario concerning the development of sensor systems is presented in Chapter 1, together with an overview of the related market with a particular focus on the latest MEMS and MOEMS technology devices, and their applications in various segments. Chapter 2 introduces the state of the art for sensor interfaces: the generic sensor interface concept (based on sharing the same electronics among similar applications achieving cost saving at the expense of area and performance loss) versus the Platform Based Design methodology, which overcomes the drawbacks of the classic solution by keeping the generality at the highest design layers and customizing the platform for a target sensor achieving optimized performances. An evolution of Platform Based Design achieved by implementation into silicon of the ISIF (Intelligent Sensor InterFace) platform is therefore presented. ISIF is a highly configurable mixed-signal chip which allows designers to perform an effective design space exploration and to evaluate directly on silicon the system performances avoiding the critical and time consuming analysis required by standard platform based approach. In chapter 3 we describe the design of a smart sensor interface for conditioning next generation MOEMS. The adoption of a new, high performance and high integrated technology allow us to integrate not only a versatile platform but also a powerful ARM processor and various IPs providing the possibility to use the platform not only as a conditioning platform but also as a processing unit for the application. In this chapter a description of the various blocks is given, with a particular emphasis on the IP developed in order to grant the highest grade of flexibility with the minimum area occupation. The architectural space evaluation and the application prototyping with ISIF has enabled an effective, rapid and low risk development of a new high performance platform achieving a flexible sensor system for MEMS and MOEMS monitoring and conditioning. The platform has been design to cover very challenging test-benches, like a laser-based projector device. In this way the platform will not only be able to effectively handle the sensor but also all the system that can be built around it, reducing the needed for further electronics and resulting in an efficient test bench for the algorithm developed to drive the system. The high costs in ASIC development are mainly related to re-design phases because of missing complete top-level tests. Analog and digital parts design flows are separately verified. Starting from these considerations, in the last chapter a complete test environment for complex mixed-signal chips is presented. A semi-automatic VHDL-AMS flow to provide totally matching top-level is described and then, an evolution for fast self-checking test development for both model and real chip verification is proposed. By the introduction of a Python interface, the designer can easily perform interactive tests to cover all the features verification (e.g. calibration and trimming) into the design phase and check them all with the same environment on the real chip after the tape-out. This strategy has been tested on a consumer 3D-gyro for consumer application, in collaboration with SensorDynamics AG

Emotion on the Road—Necessity, Acceptance, and Feasibility of Affective Computing in the Car

Besides reduction of energy consumption, which implies alternate actuation and light construction, the main research domain in automobile development in the near future is dominated by driver assistance and natural driver-car communication. The ability of a car to understand natural speech and provide a human-like driver assistance system can be expected to be a factor decisive for market success on par with automatic driving systems. Emotional factors and affective states are thereby crucial for enhanced safety and comfort. This paper gives an extensive literature overview on work related to influence of emotions on driving safety and comfort, automatic recognition, control of emotions, and improvement of in-car interfaces by affect sensitive technology. Various use-case scenarios are outlined as possible applications for emotion-oriented technology in the vehicle. The possible acceptance of such future technology by drivers is assessed in a Wizard-Of-Oz user study, and feasibility of automatically recognising various driver states is demonstrated by an example system for monitoring driver attentiveness. Thereby an accuracy of 91.3% is reported for classifying in real-time whether the driver is attentive or distracted

User expectations of partial driving automation capabilities and their effect on information design preferences in the vehicle

Partially automated vehicles present interface design challenges in ensuring the driver remains alert should the vehicle need to hand back control at short notice, but without exposing the driver to cognitive overload. To date, little is known about driver expectations of partial driving automation and whether this affects the information they require inside the vehicle. Twenty-five participants were presented with five partially automated driving events in a driving simulator. After each event, a semi-structured interview was conducted. The interview data was coded and analysed using grounded theory. From the results, two groupings of driver expectations were identified: High Information Preference (HIP) and Low Information Preference (LIP) drivers; between these two groups the information preferences differed. LIP drivers did not want detailed information about the vehicle presented to them, but the definition of partial automation means that this kind of information is required for safe use. Hence, the results suggest careful thought as to how information is presented to them is required in order for LIP drivers to safely using partial driving automation. Conversely, HIP drivers wanted detailed information about the system's status and driving and were found to be more willing to work with the partial automation and its current limitations. It was evident that the drivers' expectations of the partial automation capability differed, and this affected their information preferences. Hence this study suggests that HMI designers must account for these differing expectations and preferences to create a safe, usable system that works for everyone. [Abstract copyright: Copyright © 2019 The Authors. Published by Elsevier Ltd.. All rights reserved.

Presentation adaptation for multimodal interface systems: Three essays on the effectiveness of user-centric content and modality adaptation

The use of devices is becoming increasingly ubiquitous and the contexts of their users more and more dynamic. This often leads to situations where one communication channel is rather impractical. Text-based communication is particularly inconvenient when the hands are already occupied with another task. Audio messages induce privacy risks and may disturb other people if used in public spaces. Multimodal interfaces thus offer users the flexibility to choose between multiple interaction modalities. While the choice of a suitable input modality lies in the hands of the users, they may also require output in a different modality depending on their situation. To adapt the output of a system to a particular context, rules are needed that specify how information should be presented given the users’ situation and state. Therefore, this thesis tests three adaptation rules that – based on observations from cognitive science – have the potential to improve the interaction with an application by adapting the presented content or its modality. Following modality alignment, the output (audio versus visual) of a smart home display is matched with the user’s input (spoken versus manual) to the system. Experimental evaluations reveal that preferences for an input modality are initially too unstable to infer a clear preference for either interaction modality. Thus, the data shows no clear relation between the users’ modality choice for the first interaction and their attitude towards output in different modalities. To apply multimodal redundancy, information is displayed in multiple modalities. An application of the rule in a video conference reveals that captions can significantly reduce confusion. However, the effect is limited to confusion resulting from language barriers, whereas contradictory auditory reports leave the participants in a state of confusion independent of whether captions are available or not. We therefore suggest to activate captions only when the facial expression of a user – captured by action units, expressions of positive or negative affect, and a reduced blink rate – implies that the captions effectively improve comprehension. Content filtering in movies puts the character into the spotlight that – according to the distribution of their gaze to elements in the previous scene – the users prefer. If preferences are predicted with machine learning classifiers, this has the potential to significantly improve the user’ involvement compared to scenes of elements that the user does not prefer. Focused attention is additionally higher compared to scenes in which multiple characters take a lead role

Big data analytics:Computational intelligence techniques and application areas

Big Data has significant impact in developing functional smart cities and supporting modern societies. In this paper, we investigate the importance of Big Data in modern life and economy, and discuss challenges arising from Big Data utilization. Different computational intelligence techniques have been considered as tools for Big Data analytics. We also explore the powerful combination of Big Data and Computational Intelligence (CI) and identify a number of areas, where novel applications in real world smart city problems can be developed by utilizing these powerful tools and techniques. We present a case study for intelligent transportation in the context of a smart city, and a novel data modelling methodology based on a biologically inspired universal generative modelling approach called Hierarchical Spatial-Temporal State Machine (HSTSM). We further discuss various implications of policy, protection, valuation and commercialization related to Big Data, its applications and deployment