Development of an augmented haptic control for in-vehicle law enforcement applications

Typical police vehicles are filled with as much technology as the department can afford. Everything from the newest digital camera system to a simple entertainment radio is within the reach of an officer. Currently, most of these systems are installed without any consideration toward the other systems or the interaction required between the driver and the car. The lack of a systems approach in the installation has spawned a project to look at the safety and overall system installation in the New Jersey State Police vehicles. The research presented in this thesis suggests consolidation of the various controls into one intuitive controller that employs haptic technologies. A prototype controller was developed using an Immersion rotary haptic knob and a tactile feedback indicator. The parameters of this interaction design were explored through experimentation with a group of subjects using a testing procedure. From this information a prototype controller was developed and optimized to provide better user/vehicle interaction and a cleaner, safer cockpit

Evaluating secondary input devices to support an automotive touchscreen HMI: a cross-cultural simulator study conducted in the UK and China

Touchscreen Human-Machine Interfaces (HMIs) are a well-established and popular choice to provide the primary control interface between driver and vehicle, yet inherently demand some visual attention. Employing a secondary device with the touchscreen may reduce the demand but there is some debate about which device is most suitable, with current manufacturers favouring different solutions and applying these internationally. We present an empirical driving simulator study, conducted in the UK and China, in which 48 participants undertook typical in-vehicle tasks utilising either a touchscreen, rotary-controller, steering-wheel-controls or touchpad. In both the UK and China, the touchscreen was the most preferred/least demanding to use, and the touchpad least preferred/most demanding, whereas the rotary-controller was generally favoured by UK drivers and steering-wheel-controls were more popular in China. Chinese drivers were more excited by the novelty of the technology, and spent more time attending to the devices while driving, leading to an increase in off-road glance time and a corresponding detriment to vehicle control. Even so, Chinese drivers rated devices as easier-to-use while driving, and felt that they interfered less with their driving performance, compared to their UK counterparts. Results suggest that the most effective solution (to maximise performance/acceptance, while minimising visual demand) is to maintain the touchscreen as the primary control interface (e.g. for top-level tasks), and supplement this with a secondary device that is only enabled for certain actions; moreover, different devices may be employed in different cultural markets. Further work is required to explore these recommendations in greater depth (e.g. during extended or real-world testing), and to validate the findings and approach in other cultural contexts

The cockpit for the 21st century

Interactive surfaces are a growing trend in many domains. As one possible manifestation of Mark Weiser’s vision of ubiquitous and disappearing computers in everywhere objects, we see touchsensitive screens in many kinds of devices, such as smartphones, tablet computers and interactive tabletops. More advanced concepts of these have been an active research topic for many years. This has also influenced automotive cockpit development: concept cars and recent market releases show integrated touchscreens, growing in size. To meet the increasing information and interaction needs, interactive surfaces offer context-dependent functionality in combination with a direct input paradigm. However, interfaces in the car need to be operable while driving. Distraction, especially visual distraction from the driving task, can lead to critical situations if the sum of attentional demand emerging from both primary and secondary task overextends the available resources. So far, a touchscreen requires a lot of visual attention since its flat surface does not provide any haptic feedback. There have been approaches to make direct touch interaction accessible while driving for simple tasks. Outside the automotive domain, for example in office environments, concepts for sophisticated handling of large displays have already been introduced. Moreover, technological advances lead to new characteristics for interactive surfaces by enabling arbitrary surface shapes. In cars, two main characteristics for upcoming interactive surfaces are largeness and shape. On the one hand, spatial extension is not only increasing through larger displays, but also by taking objects in the surrounding into account for interaction. On the other hand, the flatness inherent in current screens can be overcome by upcoming technologies, and interactive surfaces can therefore provide haptically distinguishable surfaces. This thesis describes the systematic exploration of large and shaped interactive surfaces and analyzes their potential for interaction while driving. Therefore, different prototypes for each characteristic have been developed and evaluated in test settings suitable for their maturity level. Those prototypes were used to obtain subjective user feedback and objective data, to investigate effects on driving and glance behavior as well as usability and user experience. As a contribution, this thesis provides an analysis of the development of interactive surfaces in the car. Two characteristics, largeness and shape, are identified that can improve the interaction compared to conventional touchscreens. The presented studies show that large interactive surfaces can provide new and improved ways of interaction both in driver-only and driver-passenger situations. Furthermore, studies indicate a positive effect on visual distraction when additional static haptic feedback is provided by shaped interactive surfaces. Overall, various, non-exclusively applicable, interaction concepts prove the potential of interactive surfaces for the use in automotive cockpits, which is expected to be beneficial also in further environments where visual attention needs to be focused on additional tasks.Der Einsatz von interaktiven Oberflächen weitet sich mehr und mehr auf die unterschiedlichsten Lebensbereiche aus. Damit sind sie eine mögliche Ausprägung von Mark Weisers Vision der allgegenwärtigen Computer, die aus unserer direkten Wahrnehmung verschwinden. Bei einer Vielzahl von technischen Geräten des täglichen Lebens, wie Smartphones, Tablets oder interaktiven Tischen, sind berührungsempfindliche Oberflächen bereits heute in Benutzung. Schon seit vielen Jahren arbeiten Forscher an einer Weiterentwicklung der Technik, um ihre Vorteile auch in anderen Bereichen, wie beispielsweise der Interaktion zwischen Mensch und Automobil, nutzbar zu machen. Und das mit Erfolg: Interaktive Benutzeroberflächen werden mittlerweile serienmäßig in vielen Fahrzeugen eingesetzt. Der Einbau von immer größeren, in das Cockpit integrierten Touchscreens in Konzeptfahrzeuge zeigt, dass sich diese Entwicklung weiter in vollem Gange befindet. Interaktive Oberflächen ermöglichen das flexible Anzeigen von kontextsensitiven Inhalten und machen eine direkte Interaktion mit den Bildschirminhalten möglich. Auf diese Weise erfüllen sie die sich wandelnden Informations- und Interaktionsbedürfnisse in besonderem Maße. Beim Einsatz von Bedienschnittstellen im Fahrzeug ist die gefahrlose Benutzbarkeit während der Fahrt von besonderer Bedeutung. Insbesondere visuelle Ablenkung von der Fahraufgabe kann zu kritischen Situationen führen, wenn Primär- und Sekundäraufgaben mehr als die insgesamt verfügbare Aufmerksamkeit des Fahrers beanspruchen. Herkömmliche Touchscreens stellen dem Fahrer bisher lediglich eine flache Oberfläche bereit, die keinerlei haptische Rückmeldung bietet, weshalb deren Bedienung besonders viel visuelle Aufmerksamkeit erfordert. Verschiedene Ansätze ermöglichen dem Fahrer, direkte Touchinteraktion für einfache Aufgaben während der Fahrt zu nutzen. Außerhalb der Automobilindustrie, zum Beispiel für Büroarbeitsplätze, wurden bereits verschiedene Konzepte für eine komplexere Bedienung großer Bildschirme vorgestellt. Darüber hinaus führt der technologische Fortschritt zu neuen möglichen Ausprägungen interaktiver Oberflächen und erlaubt, diese beliebig zu formen. Für die nächste Generation von interaktiven Oberflächen im Fahrzeug wird vor allem an der Modifikation der Kategorien Größe und Form gearbeitet. Die Bedienschnittstelle wird nicht nur durch größere Bildschirme erweitert, sondern auch dadurch, dass Objekte wie Dekorleisten in die Interaktion einbezogen werden können. Andererseits heben aktuelle Technologieentwicklungen die Restriktion auf flache Oberflächen auf, so dass Touchscreens künftig ertastbare Strukturen aufweisen können. Diese Dissertation beschreibt die systematische Untersuchung großer und nicht-flacher interaktiver Oberflächen und analysiert ihr Potential für die Interaktion während der Fahrt. Dazu wurden für jede Charakteristik verschiedene Prototypen entwickelt und in Testumgebungen entsprechend ihres Reifegrads evaluiert. Auf diese Weise konnten subjektives Nutzerfeedback und objektive Daten erhoben, und die Effekte auf Fahr- und Blickverhalten sowie Nutzbarkeit untersucht werden. Diese Dissertation leistet den Beitrag einer Analyse der Entwicklung von interaktiven Oberflächen im Automobilbereich. Weiterhin werden die Aspekte Größe und Form untersucht, um mit ihrer Hilfe die Interaktion im Vergleich zu herkömmlichen Touchscreens zu verbessern. Die durchgeführten Studien belegen, dass große Flächen neue und verbesserte Bedienmöglichkeiten bieten können. Außerdem zeigt sich ein positiver Effekt auf die visuelle Ablenkung, wenn zusätzliches statisches, haptisches Feedback durch nicht-flache Oberflächen bereitgestellt wird. Zusammenfassend zeigen verschiedene, untereinander kombinierbare Interaktionskonzepte das Potential interaktiver Oberflächen für den automotiven Einsatz. Zudem können die Ergebnisse auch in anderen Bereichen Anwendung finden, in denen visuelle Aufmerksamkeit für andere Aufgaben benötigt wird

The tactile sense as a mechanism for the reduction of visual load elicited by control interactions : an automotive case study approach to the development of generic design recommendations

This thesis examines the potential for using tactile feedback to reduce the visual load that can be associated with interacting with controls. Using the automotive context as a case study, the thesis describes the process followed in the design of a prototype tactile interface (PTI) for the control of in-car secondary functionality (navigation, entertainment and climate control). There have been many examples of the use of active and passive tactile feedback to provide information to visually impaired people. There is however a paucity of previous research into the field of tactile feedback in mainstream product design. A literature review was performed examining various issues that are associated with tactile design including cognitive processing of tactile inputs, the use oftactile feedback in products used by visually impaired people and standard control design recommendations. This was followed by the generation of initial concepts and the first study, which examined how visually impaired people interact with electronic products that are unfamiliar to them, and also examined how they used their own equipment. The results from this study, and the literature review findings were combined into a series of design recommendations for the production of tactile interfaces that aim to reduce the visual load on the driver. These design recommendations were the basis for an iterative design process that resulted in the first, non functioning PTI interface model. The first PTI was constructed using rapid prototype technologies. The first iteration PTI was examined in the second study, a user trial in a driving simulator. The study produced encouraging results with a >90% success rate for correct control selection without vision, whilst performing a driving task. The results from this study were used to refine the design of the PTI and a working, hi-fidelity prototype was constructed for use in the final study. This study involved 'on the road' user trials comparing the glance durations made to the PTI and to a baseline system using a 'repeated measures' structure. The data from these user trials were examined to determine if the PTI exhibited a reduced visual load when compared to the baseline system. The results showed the PTI fostered significantly reduced summed glance durations for 7 of the 11 tasks performed when compared to the baseline system. Three of the 11 tasks that were performed in the study produced a reduction of summed glance duration of >50%. The PTI was also shown to foster non-visual interaction, with all participants performing at least one control interaction without looking at the control arrays. The tactile coding and symbolic layout of the PTI have been shown to be beneficial in terms of reducing 'eyes off road time' and therefore reducing the risk of distraction related accidents. A review of the results from the three studies described in this thesis has enabled the development of generic design guidelines for the production of tactile interfaces where a reduction in visual load is required for the safety of the operator. The thesis has made a contribution to the understanding of the use of the tactile sense during product interactions, and highlighting the benefits as well as the limitations of the tactile sense as a feedback mechanism

The cockpit for the 21st century

Interactive surfaces are a growing trend in many domains. As one possible manifestation of Mark Weiser’s vision of ubiquitous and disappearing computers in everywhere objects, we see touchsensitive screens in many kinds of devices, such as smartphones, tablet computers and interactive tabletops. More advanced concepts of these have been an active research topic for many years. This has also influenced automotive cockpit development: concept cars and recent market releases show integrated touchscreens, growing in size. To meet the increasing information and interaction needs, interactive surfaces offer context-dependent functionality in combination with a direct input paradigm. However, interfaces in the car need to be operable while driving. Distraction, especially visual distraction from the driving task, can lead to critical situations if the sum of attentional demand emerging from both primary and secondary task overextends the available resources. So far, a touchscreen requires a lot of visual attention since its flat surface does not provide any haptic feedback. There have been approaches to make direct touch interaction accessible while driving for simple tasks. Outside the automotive domain, for example in office environments, concepts for sophisticated handling of large displays have already been introduced. Moreover, technological advances lead to new characteristics for interactive surfaces by enabling arbitrary surface shapes. In cars, two main characteristics for upcoming interactive surfaces are largeness and shape. On the one hand, spatial extension is not only increasing through larger displays, but also by taking objects in the surrounding into account for interaction. On the other hand, the flatness inherent in current screens can be overcome by upcoming technologies, and interactive surfaces can therefore provide haptically distinguishable surfaces. This thesis describes the systematic exploration of large and shaped interactive surfaces and analyzes their potential for interaction while driving. Therefore, different prototypes for each characteristic have been developed and evaluated in test settings suitable for their maturity level. Those prototypes were used to obtain subjective user feedback and objective data, to investigate effects on driving and glance behavior as well as usability and user experience. As a contribution, this thesis provides an analysis of the development of interactive surfaces in the car. Two characteristics, largeness and shape, are identified that can improve the interaction compared to conventional touchscreens. The presented studies show that large interactive surfaces can provide new and improved ways of interaction both in driver-only and driver-passenger situations. Furthermore, studies indicate a positive effect on visual distraction when additional static haptic feedback is provided by shaped interactive surfaces. Overall, various, non-exclusively applicable, interaction concepts prove the potential of interactive surfaces for the use in automotive cockpits, which is expected to be beneficial also in further environments where visual attention needs to be focused on additional tasks.Der Einsatz von interaktiven Oberflächen weitet sich mehr und mehr auf die unterschiedlichsten Lebensbereiche aus. Damit sind sie eine mögliche Ausprägung von Mark Weisers Vision der allgegenwärtigen Computer, die aus unserer direkten Wahrnehmung verschwinden. Bei einer Vielzahl von technischen Geräten des täglichen Lebens, wie Smartphones, Tablets oder interaktiven Tischen, sind berührungsempfindliche Oberflächen bereits heute in Benutzung. Schon seit vielen Jahren arbeiten Forscher an einer Weiterentwicklung der Technik, um ihre Vorteile auch in anderen Bereichen, wie beispielsweise der Interaktion zwischen Mensch und Automobil, nutzbar zu machen. Und das mit Erfolg: Interaktive Benutzeroberflächen werden mittlerweile serienmäßig in vielen Fahrzeugen eingesetzt. Der Einbau von immer größeren, in das Cockpit integrierten Touchscreens in Konzeptfahrzeuge zeigt, dass sich diese Entwicklung weiter in vollem Gange befindet. Interaktive Oberflächen ermöglichen das flexible Anzeigen von kontextsensitiven Inhalten und machen eine direkte Interaktion mit den Bildschirminhalten möglich. Auf diese Weise erfüllen sie die sich wandelnden Informations- und Interaktionsbedürfnisse in besonderem Maße. Beim Einsatz von Bedienschnittstellen im Fahrzeug ist die gefahrlose Benutzbarkeit während der Fahrt von besonderer Bedeutung. Insbesondere visuelle Ablenkung von der Fahraufgabe kann zu kritischen Situationen führen, wenn Primär- und Sekundäraufgaben mehr als die insgesamt verfügbare Aufmerksamkeit des Fahrers beanspruchen. Herkömmliche Touchscreens stellen dem Fahrer bisher lediglich eine flache Oberfläche bereit, die keinerlei haptische Rückmeldung bietet, weshalb deren Bedienung besonders viel visuelle Aufmerksamkeit erfordert. Verschiedene Ansätze ermöglichen dem Fahrer, direkte Touchinteraktion für einfache Aufgaben während der Fahrt zu nutzen. Außerhalb der Automobilindustrie, zum Beispiel für Büroarbeitsplätze, wurden bereits verschiedene Konzepte für eine komplexere Bedienung großer Bildschirme vorgestellt. Darüber hinaus führt der technologische Fortschritt zu neuen möglichen Ausprägungen interaktiver Oberflächen und erlaubt, diese beliebig zu formen. Für die nächste Generation von interaktiven Oberflächen im Fahrzeug wird vor allem an der Modifikation der Kategorien Größe und Form gearbeitet. Die Bedienschnittstelle wird nicht nur durch größere Bildschirme erweitert, sondern auch dadurch, dass Objekte wie Dekorleisten in die Interaktion einbezogen werden können. Andererseits heben aktuelle Technologieentwicklungen die Restriktion auf flache Oberflächen auf, so dass Touchscreens künftig ertastbare Strukturen aufweisen können. Diese Dissertation beschreibt die systematische Untersuchung großer und nicht-flacher interaktiver Oberflächen und analysiert ihr Potential für die Interaktion während der Fahrt. Dazu wurden für jede Charakteristik verschiedene Prototypen entwickelt und in Testumgebungen entsprechend ihres Reifegrads evaluiert. Auf diese Weise konnten subjektives Nutzerfeedback und objektive Daten erhoben, und die Effekte auf Fahr- und Blickverhalten sowie Nutzbarkeit untersucht werden. Diese Dissertation leistet den Beitrag einer Analyse der Entwicklung von interaktiven Oberflächen im Automobilbereich. Weiterhin werden die Aspekte Größe und Form untersucht, um mit ihrer Hilfe die Interaktion im Vergleich zu herkömmlichen Touchscreens zu verbessern. Die durchgeführten Studien belegen, dass große Flächen neue und verbesserte Bedienmöglichkeiten bieten können. Außerdem zeigt sich ein positiver Effekt auf die visuelle Ablenkung, wenn zusätzliches statisches, haptisches Feedback durch nicht-flache Oberflächen bereitgestellt wird. Zusammenfassend zeigen verschiedene, untereinander kombinierbare Interaktionskonzepte das Potential interaktiver Oberflächen für den automotiven Einsatz. Zudem können die Ergebnisse auch in anderen Bereichen Anwendung finden, in denen visuelle Aufmerksamkeit für andere Aufgaben benötigt wird

Modelling and evaluating drivers’ interactions with in-vehicle information systems (IVIS)

Evaluating the usability of In-Vehicle Information Systems (IVIS) guides engineers in understanding the interaction design limitations of current systems and assessing the potential of concept technologies. The complexity and diversity of the driving task presents a unique challenge in defining usability: user-IVIS interactions create a dual-task scenario, in which conflicts can arise between the primary driving tasks and secondary IVIS tasks. This, and the safety-critical nature of driving, must be specified in defining and evaluating IVIS usability.Work was carried out in the initial phases of this project to define usability for IVIS and to develop a framework for evaluation. One of the key findings of this work was the importance of context-of-use in defining usability, so that specific usability criteria and appropriate evaluation methods can be identified. The evaluation methods in the framework were categorised as either analytic, i.e. applicable at the earliest stages of product development to predict performance and usability; or empirical, i.e. to measure user performance under simulated or real-world conditions. Two case studies have shown that the evaluation framework is sensitive to differences between IVIS and can identify important usability issues, which can be used to inform design improvements.The later stages of the project have focussed on Multimodal Critical Path Analysis (CPA). Initially, CPA was used to predict IVIS task interaction times for a stationary vehicle. The CPA model was extended to produce fastperson and slowperson task time estimates, as well as average predictions. In order for the CPA to be of real use to designers of IVIS, it also needed to predict dual-task IVIS interaction times, i.e. time taken to perform IVIS tasks whilst driving. A hypothesis of shared glances was developed, proposing that drivers are able to monitor two visual information sources simultaneously. The CPA technique was extended for prediction of dual-task interaction times by modelling this shared glance pattern. The hypothesis has important implications for theories of visual behaviour and for the design of future IVIS

Haptic controls in cars for safer driving

With the spread of latest state of the art technologies geared towards utilization of the human senses, haptic technologies have been introduced as a way of utilising the sense of touch to either solve real world problems or to enhance present experiences. This thesis focuses on using haptic technology in cars to make the driving experience safer. Modern vehicles carry GPS, music systems, sunroofs and a number of other electronic gadgets. Interaction with these devices while driving often takes the driver???s eyes ???off the road??? and raises safety concerns. We are proposing a unique haptic design that uses the ???sense of touch??? as a mode of controlling or coordinating the various technologies and convenience devices found within a car. A pattern of distinguishable haptic feedback linked to a corresponding device allows the user to operate these devices through ???sense of touch??? and eliminates reliance on visual interaction. This design will help to reduce the driver???s distractions, as it will be installed in an easily accessible location such as on the steering wheel. A simulation has been done using a haptic interface ???i.e. desktop phantom to test the system??? and a prototype has been developed which can be installed in any vehicle. This prototype has been tested to work with a limited number of convenient devices. However, further development and enhancements can be made to incorporate more devices and other user preferences. The main objective of this research is to integrate various functionalities in a robust manner, which will focus on the driver???s safety by ensuring ???constant vision on the road???. Distinguishable distinct haptic responses will act as unique depictions for specific convenient devices within the car, allowing the driver to interact and manipulate the settings of the device based on the detection and identification of the various unique haptic depictions

A Human-Embodied Drone for Dexterous Aerial Manipulation

Current drones perform a wide variety of tasks in surveillance, photography, agriculture, package delivery, etc. However, these tasks are performed passively without the use of human interaction. Aerial manipulation shifts this paradigm and implements drones with robotic arms that allow interaction with the environment rather than simply sensing it. For example, in construction, aerial manipulation in conjunction with human interaction could allow operators to perform several tasks, such as hosing decks, drill into surfaces, and sealing cracks via a drone. This integration with drones will henceforth be known as dexterous aerial manipulation. Our recent work integrated the worker’s experience into aerial manipulation using haptic technology. The net effect was such a system could enable the worker to leverage drones and complete tasks while utilizing haptics on the task site remotely. However, the tasks were completed within the operator’s line-of-sight. Until now, immersive AR/VR frameworks has rarely been integrated in aerial manipulation. Yet, such a framework allows the drones to embody and transport the operator’s senses, actions, and presence to a remote location in real-time. As a result, the operator can both physically interact with the environment and socially interact with actual workers on the worksite. This dissertation presents a human-embodied drone interface for dexterous aerial manipulation. Using VR/AR technology, the interface allows the operator to leverage their intelligence to collaboratively perform desired tasks anytime, anywhere with a drone that possesses great dexterity

The tactile sense as a mechanism for the reduction of visual load elicited by control interactions : an automotive case study approach to the development of generic design recommendations

This thesis examines the potential for using tactile feedback to reduce the visual load that can be associated with interacting with controls. Using the automotive context as a case study, the thesis describes the process followed in the design of a prototype tactile interface (PTI) for the control of in-car secondary functionality (navigation, entertainment and climate control). There have been many examples of the use of active and passive tactile feedback to provide information to visually impaired people. There is however a paucity of previous research into the field of tactile feedback in mainstream product design. A literature review was performed examining various issues that are associated with tactile design including cognitive processing of tactile inputs, the use oftactile feedback in products used by visually impaired people and standard control design recommendations. This was followed by the generation of initial concepts and the first study, which examined how visually impaired people interact with electronic products that are unfamiliar to them, and also examined how they used their own equipment. The results from this study, and the literature review findings were combined into a series of design recommendations for the production of tactile interfaces that aim to reduce the visual load on the driver. These design recommendations were the basis for an iterative design process that resulted in the first, non functioning PTI interface model. The first PTI was constructed using rapid prototype technologies. The first iteration PTI was examined in the second study, a user trial in a driving simulator. The study produced encouraging results with a >90% success rate for correct control selection without vision, whilst performing a driving task. The results from this study were used to refine the design of the PTI and a working, hi-fidelity prototype was constructed for use in the final study. This study involved 'on the road' user trials comparing the glance durations made to the PTI and to a baseline system using a 'repeated measures' structure. The data from these user trials were examined to determine if the PTI exhibited a reduced visual load when compared to the baseline system. The results showed the PTI fostered significantly reduced summed glance durations for 7 of the 11 tasks performed when compared to the baseline system. Three of the 11 tasks that were performed in the study produced a reduction of summed glance duration of >50%. The PTI was also shown to foster non-visual interaction, with all participants performing at least one control interaction without looking at the control arrays. The tactile coding and symbolic layout of the PTI have been shown to be beneficial in terms of reducing 'eyes off road time' and therefore reducing the risk of distraction related accidents. A review of the results from the three studies described in this thesis has enabled the development of generic design guidelines for the production of tactile interfaces where a reduction in visual load is required for the safety of the operator. The thesis has made a contribution to the understanding of the use of the tactile sense during product interactions, and highlighting the benefits as well as the limitations of the tactile sense as a feedback mechanism.EThOS - Electronic Theses Online ServiceGBUnited Kingdo