DRIVERS’ HAZARD AVOIDANCE DURING VEHICLE AUTOMATION: IMPACT OF MENTAL MODELS AND IMPLICATIONS FOR TRAINING

Advanced Driver Assistance Systems (ADAS) are vehicle automation systems that have become more accessible and prevalent in vehicles in recent years. But the introduction of such technologies introduces new human factors challenges. Past literature suggests that users of vehicle automation lack the necessary and appropriate knowledge about their automation system. This may play a negative role in their hazard avoidance abilities when driving with automation features. Improving mental models and knowledge could generally lead to safer interactions with vehicle automation systems, but any effort to develop hazard avoidance skills when driving with vehicle automation is impeded by the lack of literature regarding the subject. Moreover, it is possible hazard avoidance for vehicle automation may actually differ from that for traditional driving. For vehicle automation, system-related changes occurring internally inside one’s vehicle also impact how the system responds and controls the vehicle. Failure to recognize certain critical system changes may have disastrous consequences. Hence, it is imperative that a new framework for hazard avoidance in the new context of vehicle automation, especially for ADAS features, is conceptualized. Initially, the research focused on realizing exactly this by proposing a conceptual framework for hazard avoidance in the context of vehicle automation by making use of past literary sources on hazard avoidance for traditional driving. Next, the relationship between mental models, training, and hazard avoidance was mapped and each new behavioral construct of hazard avoidance focusing on awareness, detection, and responses based on internal events was assigned potential outcome measure. Next, an observational study was conducted with ten experienced users of Adaptive Cruise Control (ACC). Among them, five were assigned to an eye movements group and five others to a verbal responses group. The eye movement observations gave us insights into how experienced users detect and respond to hazards and how these affect their interactions and responses using their ACC systems. The verbal group also provided insights about the participants’ awareness during the drive which featured several edge-case and normal events. These observations imply that hazard avoidance behaviors actually differ in the context of ADAS compared to traditional driving. The findings from the observational study were leveraged when designing and developing a new training program where drivers would receive an immersive and realistic training experience through a Virtual Reality (VR) headset. The main objective of the training program was to improve the user’s mental models about ACC and also equip them with the necessary skills to avoid hazard during edge case events of ACC. Finally, an evaluation study was conducted with 36 novice ACC users on a driving simulator capable of simulating ACC operations. The participants were equally and randomly assigned to one of three group – the VR group that received the newly designed VR training program; the SD group that received training material with state diagram visualization of ACC and other information derived from owner’s manuals; or the BI group that received basic textual information about ACC. The participants’ mental models before and after training were measured using a mental models survey, and the simulator drive was designed to collect valuable data about the participants interactions with ACC and their hazard avoidance behaviors. Findings revealed that although the VR training program had some impact on the participants\u27 mental models and hazard avoidance behaviors, the impact was not statistically significant. However, the VR training did show significantly positive influences on the participants’ internal glance activities that detect and assess system states, during edge case events. This finding is important since one of the modules of the VR training program was carefully curated to improve driver’s glance behavior when encountering edge case events of ACC. The results also establish the relationships between training and mental models although no significant correlations were found between the participants’ mental models and their hazard avoidance behaviors. However, this does fill a major gap in literature about our understanding about hazard avoidance in the context of vehicle automation and ADAS and could be extended for ADAS features other than ACC or even higher levels of automation. The VR training program can be built upon to include more ADAS features as well leading to better training practices in a rapidly developing world where vehicle automation has become a mainstay

Quantify resilience enhancement of UTS through exploiting connect community and internet of everything emerging technologies

This work aims at investigating and quantifying the Urban Transport System (UTS) resilience enhancement enabled by the adoption of emerging technology such as Internet of Everything (IoE) and the new trend of the Connected Community (CC). A conceptual extension of Functional Resonance Analysis Method (FRAM) and its formalization have been proposed and used to model UTS complexity. The scope is to identify the system functions and their interdependencies with a particular focus on those that have a relation and impact on people and communities. Network analysis techniques have been applied to the FRAM model to identify and estimate the most critical community-related functions. The notion of Variability Rate (VR) has been defined as the amount of output variability generated by an upstream function that can be tolerated/absorbed by a downstream function, without significantly increasing of its subsequent output variability. A fuzzy based quantification of the VR on expert judgment has been developed when quantitative data are not available. Our approach has been applied to a critical scenario (water bomb/flash flooding) considering two cases: when UTS has CC and IoE implemented or not. The results show a remarkable VR enhancement if CC and IoE are deploye

The Promise of VR Headsets: Validation of a Virtual Reality Headset-Based Driving Simulator for Measuring Drivers’ Hazard Anticipation Performance

The objective of the current study is to evaluate the use of virtual reality (VR) headsets to measure driving performance. This is desirable because they are several orders of magnitude less expensive and, if validated, could greatly extend the powers of simulation. Out of several possible measures of performance that could be considered for evaluating VR headsets, the current study specifically examines drivers’ latent hazard anticipation behavior both because it has been linked to crashes and because it has been shown to be significantly poorer in young drivers compared to their experienced counterparts in traditional driving simulators and in open road studies. The total time middle-aged drivers spend glancing at a latent hazard and the average duration of each glance was also compared to these same times for younger drivers using a VR headset and fixed-based driving simulator. In a between-subject design, forty-eight participants were equally and randomly assigned to one out of four experimental conditions – two young driver cohorts (18 – 21 years) and two middle-aged driver cohorts (30 – 55 years) navigating either a fixed-based driving simulator or a VR-headset-based simulator. All participants navigated six unique scenarios while their eyes were continually tracked. The proportion of latent hazards anticipated by participants which constituted the primary dependent measure was found to be greater for middle-aged drivers than young drivers across both platforms. Results also indicate that the middle-aged participants glanced longer than their younger counterparts on both platforms at latent hazards, as measured by the total glance duration but had no difference when measured by the average glance duration. Moreover, the difference in the magnitude of performance between middle-aged and younger drivers was the same across the two platforms. There were also no significant differences found for the severity of simulator sickness symptoms across the two platforms. The study provides some justification for the use of virtual reality headsets as a way of understanding drivers’ hazard anticipation behavior

Alternative avenues in the assessment of driving capacities in older drivers and implications for training

The population aging, combined with the overrepresentation of older drivers in car crashes, engendered a whole body of research destined at finding simple and efficient assessment methods of driving capacities. However, this quest is little more than a utopian dream, given that car crashes and unsafe driving behaviours can result from a plethora of interacting factors. This review highlights the main problems of the current assessment methods and training programs, and presents theoretical and empirical arguments justifying the need of reorienting the research focus. Our discussion is elaborated in light of the fundamental principle of specificity in learning and practice. We also identify overlooked variables that are deterministic when assessing, and training, a complex ability like driving. We especially focus on the role of the sensorimotor transformation process. Finally, we propose alternative methods that are in-line with the recent trends in educational programs that use virtual reality and simulation technologies

Beyond the Screen: Reshaping the Workplace with Virtual and Augmented Reality

Although extended reality technologies have enjoyed an explosion in popularity in recent years, few applications are effectively used outside the entertainment or academic contexts. This work consists of a literature review regarding the effective integration of such technologies in the workplace. It aims to provide an updated view of how they are being used in that context. First, we examine existing research concerning virtual, augmented, and mixed-reality applications. We also analyze which have made their way to the workflows of companies and institutions. Furthermore, we circumscribe the aspects of extended reality technologies that determined this applicability

Towards Everyday Virtual Reality through Eye Tracking

Durch Entwicklungen in den Bereichen Computergrafik, Hardwaretechnologie, Perception Engineering und Mensch-Computer Interaktion, werden Virtual Reality und virtuelle Umgebungen immer mehr in unser tägliches Leben integriert. Head-Mounted Displays werden jedoch im Vergleich zu anderen mobilen Geräten, wie Smartphones und Smartwatches, noch nicht so häufig genutzt. Mit zunehmender Nutzung dieser Technologie und der Gewöhnung von Menschen an virtuelle Anwendungsszenarien ist es wahrscheinlich, dass in naher Zukunft ein alltägliches Virtual-Reality-Paradigma realisiert wird. Im Hinblick auf die Kombination von alltäglicher Virtual Reality und Head-Mounted-Displays, ist Eye Tracking eine neue Technologie, die es ermöglicht, menschliches Verhalten in Echtzeit und nicht-intrusiv zu messen. Bevor diese Technologien in großem Umfang im Alltag eingesetzt werden können, müssen jedoch noch zahlreiche Aspekte genauer erforscht werden. Zunächst sollten Aufmerksamkeits- und Kognitionsmodelle in Alltagsszenarien genau verstanden werden. Des Weiteren sind Maßnahmen zur Wahrung der Privatsphäre notwendig, da die Augen mit visuellen biometrischen Indikatoren assoziiert sind. Zuletzt sollten anstelle von Studien oder Anwendungen, die sich auf eine begrenzte Anzahl menschlicher Teilnehmer mit relativ homogenen Merkmalen stützen, Protokolle und Anwendungsfälle für eine bessere Zugänglichkeit dieser Technologie von wesentlicher Bedeutung sein. In dieser Arbeit wurde unter Berücksichtigung der oben genannten Punkte ein bedeutender wissenschaftlicher Vorstoß mit drei zentralen Forschungsbeiträgen in Richtung alltäglicher Virtual Reality unternommen. Menschliche visuelle Aufmerksamkeit und Kognition innerhalb von Virtual Reality wurden in zwei unterschiedlichen Bereichen, Bildung und Autofahren, erforscht. Die Forschung im Bildungsbereich konzentrierte sich auf die Auswirkungen verschiedener Manipulationen im Klassenraum auf das menschliche Sehverhalten, während die Forschung im Bereich des Autofahrens auf sicherheitsrelevante Fragen und Blickführung abzielte. Die Nutzerstudien in beiden Bereichen zeigen, dass Blickbewegungen signifikante Implikationen für diese alltäglichen Situationen haben. Der zweite wesentliche Beitrag fokussiert sich auf Privatsphäre bewahrendes Eye Tracking für Blickbewegungsdaten von Head-Mounted Displays. Dies beinhaltet Differential Privacy, welche zeitliche Korrelationen von Blickbewegungssignalen berücksichtigt und Privatsphäre wahrende Blickschätzung durch Verwendung eines auf randomisiertem Encoding basierenden Frameworks, welches Augenreferenzunkte verwendet. Die Ergebnisse beider Arbeiten zeigen, dass die Wahrung der Privatsphäre möglich ist und gleichzeitig der Nutzen in einem akzeptablen Bereich bleibt. Wenngleich es bisher nur wenig Forschung zu diesem Aspekt von Eye Tracking gibt, ist weitere Forschung notwendig, um den alltäglichen Gebrauch von Virtual Reality zu ermöglichen. Als letzter signifikanter Beitrag, wurde ein Blockchain- und Smart Contract-basiertes Protokoll zur Eye Tracking Datenerhebung für Virtual Reality vorgeschlagen, um Virtual Reality besser zugänglich zu machen. Die Ergebnisse liefern wertvolle Erkenntnisse für alltägliche Nutzung von Virtual Reality und treiben den aktuellen Stand der Forschung in mehrere Richtungen voran.With developments in computer graphics, hardware technology, perception engineering, and human-computer interaction, virtual reality and virtual environments are becoming more integrated into our daily lives. Head-mounted displays, however, are still not used as frequently as other mobile devices such as smart phones and watches. With increased usage of this technology and the acclimation of humans to virtual application scenarios, it is possible that in the near future an everyday virtual reality paradigm will be realized. When considering the marriage of everyday virtual reality and head-mounted displays, eye tracking is an emerging technology that helps to assess human behaviors in a real time and non-intrusive way. Still, multiple aspects need to be researched before these technologies become widely available in daily life. Firstly, attention and cognition models in everyday scenarios should be thoroughly understood. Secondly, as eyes are related to visual biometrics, privacy preserving methodologies are necessary. Lastly, instead of studies or applications utilizing limited human participants with relatively homogeneous characteristics, protocols and use-cases for making such technology more accessible should be essential. In this work, taking the aforementioned points into account, a significant scientific push towards everyday virtual reality has been completed with three main research contributions. Human visual attention and cognition have been researched in virtual reality in two different domains, including education and driving. Research in the education domain has focused on the effects of different classroom manipulations on human visual behaviors, whereas research in the driving domain has targeted safety related issues and gaze-guidance. The user studies in both domains show that eye movements offer significant implications for these everyday setups. The second substantial contribution focuses on privacy preserving eye tracking for the eye movement data that is gathered from head-mounted displays. This includes differential privacy, taking temporal correlations of eye movement signals into account, and privacy preserving gaze estimation task by utilizing a randomized encoding-based framework that uses eye landmarks. The results of both works have indicated that privacy considerations are possible by keeping utility in a reasonable range. Even though few works have focused on this aspect of eye tracking until now, more research is necessary to support everyday virtual reality. As a final significant contribution, a blockchain- and smart contract-based eye tracking data collection protocol for virtual reality is proposed to make virtual reality more accessible. The findings present valuable insights for everyday virtual reality and advance the state-of-the-art in several directions

Holistic Approach for Authoring Immersive and Smart Environments for the Integration in Engineering Education

Die vierte industrielle Revolution und der rasante technologische Fortschritt stellen die etablierten Bildungsstrukturen und traditionellen Bildungspraktiken in Frage. Besonders in der Ingenieurausbildung erfordert das lebenslange Lernen, dass man sein Wissen und seine Fähigkeiten ständig verbessern muss, um auf dem Arbeitsmarkt wettbewerbsfähig zu sein. Es besteht die Notwendigkeit eines Paradigmenwechsels in der Bildung und Ausbildung hin zu neuen Technologien wie virtueller Realität und künstlicher Intelligenz. Die Einbeziehung dieser Technologien in ein Bildungsprogramm ist jedoch nicht so einfach wie die Investition in neue Geräte oder Software. Es müssen neue Bildungsprogramme geschaffen oder alte von Grund auf umgestaltet werden. Dabei handelt es sich um komplexe und umfangreiche Prozesse, die Entscheidungsfindung, Design und Entwicklung umfassen. Diese sind mit erheblichen Herausforderungen verbunden, die die Überwindung vieler Hindernisse erfordert. Diese Arbeit stellt eine Methodologie vor, die sich mit den Herausforderungen der Nutzung von Virtueller Realität und Künstlicher Intelligenz als Schlüsseltechnologien in der Ingenieurausbildung befasst. Die Methodologie hat zum Ziel, die Hauptakteure anzuleiten, um den Lernprozess zu verbessern, sowie neuartige und effiziente Lernerfahrungen zu ermöglichen. Da jedes Bildungsprogramm einzigartig ist, folgt die Methodik einem ganzheitlichen Ansatz, um die Erstellung maßgeschneiderter Kurse oder Ausbildungen zu unterstützen. Zu diesem Zweck werden die Wechselwirkung zwischen verschiedenen Aspekten berücksichtigt. Diese werden in den drei Ebenen - Bildung, Technologie und Management zusammengefasst. Die Methodik betont den Einfluss der Technologien auf die Unterrichtsgestaltung und die Managementprozesse. Sie liefert Methoden zur Entscheidungsfindung auf der Grundlage einer umfassenden pädagogischen, technologischen und wirtschaftlichen Analyse. Darüber hinaus unterstützt sie den Prozess der didaktischen Gestaltung durch eine umfassende Kategorisierung der Vor- und Nachteile immersiver Lernumgebungen und zeigt auf, welche ihrer Eigenschaften den Lernprozess verbessern können. Ein besonderer Schwerpunkt liegt auf der systematischen Gestaltung immersiver Systeme und der effizienten Erstellung immersiver Anwendungen unter Verwendung von Methoden aus dem Bereich der künstlichen Intelligenz. Es werden vier Anwendungsfälle mit verschiedenen Ausbildungsprogrammen vorgestellt, um die Methodik zu validieren. Jedes Bildungsprogramm hat seine eigenen Ziele und in Kombination decken sie die Validierung aller Ebenen der Methodik ab. Die Methodik wurde iterativ mit jedem Validierungsprojekt weiterentwickelt und verbessert. Die Ergebnisse zeigen, dass die Methodik zuverlässig und auf viele Szenarien sowie auf die meisten Bildungsstufen und Bereiche übertragbar ist. Durch die Anwendung der in dieser Arbeit vorgestellten Methoden können Interessengruppen immersiven Technologien effektiv und effizient in ihre Unterrichtspraxis integrieren. Darüber hinaus können sie auf der Grundlage der vorgeschlagenen Ansätze Aufwand, Zeit und Kosten für die Planung, Entwicklung und Wartung der immersiven Systeme sparen. Die Technologie verlagert die Rolle des Lehrenden in eine Moderatorrolle. Außerdem bekommen die Lehrkräfte die Möglichkeit die Lernenden individuell zu unterstützen und sich auf deren kognitive Fähigkeiten höherer Ordnung zu konzentrieren. Als Hauptergebnis erhalten die Lernenden eine angemessene, qualitativ hochwertige und zeitgemäße Ausbildung, die sie qualifizierter, erfolgreicher und zufriedener macht

Can adas distract driver’s attention? An rgb-d camera and deep learning-based analysis

Driver inattention is the primary cause of vehicle accidents; hence, manufacturers have introduced systems to support the driver and improve safety; nonetheless, advanced driver assistance systems (ADAS) must be properly designed not to become a potential source of distraction for the driver due to the provided feedback. In the present study, an experiment involving auditory and haptic ADAS has been conducted involving 11 participants, whose attention has been monitored during their driving experience. An RGB-D camera has been used to acquire the drivers’ face data. Subsequently, these images have been analyzed using a deep learning-based approach, i.e., a convolutional neural network (CNN) specifically trained to perform facial expression recognition (FER). Analyses to assess possible relationships between these results and both ADAS activations and event occurrences, i.e., accidents, have been carried out. A correlation between attention and accidents emerged, whilst facial expressions and ADAS activations resulted to be not correlated, thus no evidence that the designed ADAS are a possible source of distraction has been found. In addition to the experimental results, the proposed approach has proved to be an effective tool to monitor the driver through the usage of non-invasive techniques