The Emerging Professional Practice of Remote Sighted Assistance for People with Visual Impairments

People with visual impairments (PVI) must interact with a world they cannot see. Remote sighted assistance (RSA) has emerged as a conversational assistive technology. We interviewed RSA assistants ( agents ) who provide assistance to PVI via a conversational prosthetic called Aira (https://aira.io/) to understand their professional practice. We identified four types of support provided: scene description, navigation, task performance, and social engagement. We discovered that RSA provides an opportunity for PVI to appropriate the system as a richer conversational/social support tool. We studied and identified patterns in how agents provide assistance and how they interact with PVI as well as the challenges and strategies associated with each context. We found that conversational interaction is highly context-dependent. We also discuss implications for design

VANET Applications: Hot Use Cases

Current challenges of car manufacturers are to make roads safe, to achieve free flowing traffic with few congestions, and to reduce pollution by an effective fuel use. To reach these goals, many improvements are performed in-car, but more and more approaches rely on connected cars with communication capabilities between cars, with an infrastructure, or with IoT devices. Monitoring and coordinating vehicles allow then to compute intelligent ways of transportation. Connected cars have introduced a new way of thinking cars - not only as a mean for a driver to go from A to B, but as smart cars - a user extension like the smartphone today. In this report, we introduce concepts and specific vocabulary in order to classify current innovations or ideas on the emerging topic of smart car. We present a graphical categorization showing this evolution in function of the societal evolution. Different perspectives are adopted: a vehicle-centric view, a vehicle-network view, and a user-centric view; described by simple and complex use-cases and illustrated by a list of emerging and current projects from the academic and industrial worlds. We identified an empty space in innovation between the user and his car: paradoxically even if they are both in interaction, they are separated through different application uses. Future challenge is to interlace social concerns of the user within an intelligent and efficient driving

Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance

Remote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA

Interface design for a remote guidance system for the blind: Using dual-screen displays

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The mobility for the visually impaired people is one of the main challenges that researchers are still facing around the world. Although some projects have been conducted to improve the mobility of visually impaired people, further research is still needed. One of these projects is Brunel Remote Guidance System (BRGS). BRGS is aimed to assist visually impaired users in avoiding obstacles and reaching their destinations safely by providing online instructions via a remote sighted guide. This study comes as continuation of the development process of BRGS; the main aim that has been achieved of this research is the optimisation of the interface design for the system guide terminal. This helps the sighted guide to assist the VIUs to avoid obstacles safely and comfortably in the micro-navigation, as well as to keep them on the right track to reach their destination in the macro-navigation. After using the content analysis, the performance factors and their assessments method were identified in each BRGS‘ element, which concluded that there is a lack of research on the guide terminal setup and the assessment method for the sighted guide performance. Furthermore, there are no model to assist the sighted guide performance and two-screen displays used in the literature review and similar projects. A model was designed as a platform to conduct the evaluation on sighted guide performance. Based on this model, the computer-based simulation was established and tested, which made the simulation is ready for next task; the evaluation of the sighted guide performance. The conducted study determined the effects of the two-screen displays on the recognition performance of the 80 participants in the guide terminal. The performance was measured with the context of four different resolution conditions. The study was based on a simulation technique, which is consisted of two key performance elements in order to examine the sighted guide performance; the macro-navigation element and the micro-navigation element. The results show that the two-screen displays have an effect on the performance of the sighted guide. The optimum setup for the two-screen displays for the guide terminal consisted of a big digital map screen display (4CIF [704p x 576p]) and a small video image screen display (CIF [352p x 288p]), which one of the four different resolutions. This interface design has been recommended as a final setup in the guide terminal

Interface design for a remote guidance system for the blind : using dual-screen displays

The mobility for the visually impaired people is one of the main challenges that researchers are still facing around the world. Although some projects have been conducted to improve the mobility of visually impaired people, further research is still needed. One of these projects is Brunel Remote Guidance System (BRGS). BRGS is aimed to assist visually impaired users in avoiding obstacles and reaching their destinations safely by providing online instructions via a remote sighted guide. This study comes as continuation of the development process of BRGS; the main aim that has been achieved of this research is the optimisation of the interface design for the system guide terminal. This helps the sighted guide to assist the VIUs to avoid obstacles safely and comfortably in the micro-navigation, as well as to keep them on the right track to reach their destination in the macro-navigation. After using the content analysis, the performance factors and their assessments method were identified in each BRGS‘ element, which concluded that there is a lack of research on the guide terminal setup and the assessment method for the sighted guide performance. Furthermore, there are no model to assist the sighted guide performance and two-screen displays used in the literature review and similar projects. A model was designed as a platform to conduct the evaluation on sighted guide performance. Based on this model, the computer-based simulation was established and tested, which made the simulation is ready for next task; the evaluation of the sighted guide performance. The conducted study determined the effects of the two-screen displays on the recognition performance of the 80 participants in the guide terminal. The performance was measured with the context of four different resolution conditions. The study was based on a simulation technique, which is consisted of two key performance elements in order to examine the sighted guide performance; the macro-navigation element and the micro-navigation element. The results show that the two-screen displays have an effect on the performance of the sighted guide. The optimum setup for the two-screen displays for the guide terminal consisted of a big digital map screen display (4CIF [704p x 576p]) and a small video image screen display (CIF [352p x 288p]), which one of the four different resolutions. This interface design has been recommended as a final setup in the guide terminal.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Eyes-Free Vision-Based Scanning of Aligned Barcodes and Information Extraction from Aligned Nutrition Tables

Visually impaired (VI) individuals struggle with grocery shopping and have to rely on either friends, family or grocery store associates for shopping. ShopMobile 2 is a proof-of-concept system that allows VI shoppers to shop independently in a grocery store using only their smartphone. Unlike other assistive shopping systems that use dedicated hardware, this system is a software only solution that relies on fast computer vision algorithms. It consists of three modules - an eyes free barcode scanner, an optical character recognition (OCR) module, and a tele-assistance module. The eyes-free barcode scanner allows VI shoppers to locate and retrieve products by scanning barcodes on shelves and on products. The OCR module allows shoppers to read nutrition facts on products and the tele-assistance module allows them to obtain help from sighted individuals at remote locations. This dissertation discusses, provides implementations of, and presents laboratory and real-world experiments related to all three modules

A platform supporting the work of operators of tele-assistance centers for navigation of visually impaired people

Cílem této práce je navrhnout, vytvořit a otestovat prototypy platformy pro podporu práce operátorů tele-asistenčních center pro zrakově postižené, která jim má usnadnit práci a pomoci vyřešit problémy objevené při pozorování v Navigačním centru SONS. Zaměřili jsem se především na optimalizaci pracovního postupu operátora při vytváření detailních popisů tras a umístění většiny zdrojů informací do jednoho přístupového bodu. V průběhu práce na projektu jsme definovali případy užití, podle nichž jsem postupně vytvořili storyboardy, papírové návrhy a low-fidelity prototypy, které byly průběžně testovány a konzultovány s operátorem a dalšími odborníky. V závěrečné fázi projektu byl naimplementován high-fidelity prototyp v podobě webové aplikace demonstrující většinu funkcionalit platformy. Tato aplikace a výsledky z jejího testování i testování jejích předchůdců mohou posloužit jako zdroj dat v dalších fázích projektu.The aim of this thesis is to design, create and test a prototype of a platform to support the work of operators of tele-assistance centers for visually impaired people, which will simplify their work and help to solve the problems discovered during the observation in the SONS Navigation Center. We primarily focused on optimizing the operator's workflow when creating detailed route descriptions and aggregating most of the information sources to one access point. During the project, we have defined use cases, according to which we gradually created storyboards, paper designs and low-fidelity prototypes, which were continuously tested and consulted with the operator and other experts. In the final phase of the project, a high-fidelity prototype has been implemented in the form of a web application that demonstrates most of the functionalities of the platform. This application, its test result and test results of its predecessors can serve as a data source in the next phase of the project

Integrating Haptic Feedback into Mobile Location Based Services

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled device such as a mobile phone. Historically, human-computer interaction has been visual - text and images on the screen. Haptic feedback can be an important additional method especially in Mobile Location Based Services such as knowledge discovery, pedestrian navigation and notification systems. A knowledge discovery system called the Haptic GeoWand is a low interaction system that allows users to query geo-tagged data around them by using a point-and-scan technique with their mobile device. Haptic Pedestrian is a navigation system for walkers. Four prototypes have been developed classified according to the user’s guidance requirements, the user type (based on spatial skills), and overall system complexity. Haptic Transit is a notification system that provides spatial information to the users of public transport. In all these systems, haptic feedback is used to convey information about location, orientation, density and distance by use of the vibration alarm with varying frequencies and patterns to help understand the physical environment. Trials elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. Results from a memory recall test show that the users of haptic feedback for navigation had better memory recall of the region traversed than the users of landmark images. Haptics integrated into a multi-modal navigation system provides more usable, less distracting but more effective interaction than conventional systems. Enhancements to the current work could include integration of contextual information, detailed large-scale user trials and the exploration of using haptics within confined indoor spaces

차량용 헤드업 디스플레이 설계에 관한 인간공학 연구

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 산업공학과, 2020. 8. 박우진.Head-up display (HUD) systems were introduced into the automobile industry as a means for improving driving safety. They superimpose safety-critical information on top of the drivers forward field of view and thereby help drivers keep their eyes forward while driving. Since the first introduction about three decades ago, automotive HUDs have been available in various commercial vehicles. Despite the long history and potential benefits of automotive HUDs, however, the design of useful automotive HUDs remains a challenging problem. In an effort to contribute to the design of useful automotive HUDs, this doctoral dissertation research conducted four studies. In Study 1, the functional requirements of automotive HUDs were investigated by reviewing the major automakers' automotive HUD products, academic research studies that proposed various automotive HUD functions, and previous research studies that surveyed drivers HUD information needs. The review results indicated that: 1) the existing commercial HUDs perform largely the same functions as the conventional in-vehicle displays, 2) past research studies proposed various HUD functions for improving driver situation awareness and driving safety, 3) autonomous driving and other new technologies are giving rise to new HUD information, and 4) little research is currently available on HUD users perceived information needs. Based on the review results, this study provides insights into the functional requirements of automotive HUDs and also suggests some future research directions for automotive HUD design. In Study 2, the interface design of automotive HUDs for communicating safety-related information was examined by reviewing the existing commercial HUDs and display concepts proposed by academic research studies. Each display was analyzed in terms of its functions, behaviors and structure. Also, related human factors display design principles, and, empirical findings on the effects of interface design decisions were reviewed when information was available. The results indicated that: 1) information characteristics suitable for the contact-analog and unregistered display formats, respectively, are still largely unknown, 2) new types of displays could be developed by combining or mixing existing displays or display elements at both the information and interface element levels, and 3) the human factors display principles need to be used properly according to the situation and only to the extent that the resulting display respects the limitations of the human information processing, and achieving balance among the principles is important to an effective design. On the basis of the review results, this review suggests design possibilities and future research directions on the interface design of safety-related automotive HUD systems. In Study 3, automotive HUD-based take-over request (TOR) displays were developed and evaluated in terms of drivers take-over performance and visual scanning behavior in a highly automated driving situation. Four different types of TOR displays were comparatively evaluated through a driving simulator study - they were: Baseline (an auditory beeping alert), Mini-map, Arrow, and Mini-map-and-Arrow. Baseline simply alerts an imminent take-over, and was always included when the other three displays were provided. Mini-map provides situational information. Arrow presents the action direction information for the take-over. Mini-map-and-Arrow provides the action direction together with the relevant situational information. This study also investigated the relationship between drivers initial trust in the TOR displays and take-over and visual scanning behavior. The results indicated that providing a combination of machine-made decision and situational information, such as Mini-map-and-Arrow, yielded the best results overall in the take-over scenario. Also, drivers initial trust in the TOR displays was found to have significant associations with the take-over and visual behavior of drivers. The higher trust group primarily relied on the proposed TOR displays, while the lower trust group tended to more check the situational information through the traditional displays, such as side-view or rear-view mirrors. In Study 4, the effect of interactive HUD imagery location on driving and secondary task performance, driver distraction, preference, and workload associated with use of scrolling list while driving were investigated. A total of nine HUD imagery locations of full-windshield were examined through a driving simulator study. The results indicated the HUD imagery location affected all the dependent measures, that is, driving and task performance, drivers visual distraction, preference and workload. Considering both objective and subjective evaluations, interactive HUDs should be placed near the driver's line of sight, especially near the left-bottom on the windshield.자동차 헤드업 디스플레이는 차내 디스플레이 중 하나로 운전자에게 필요한 정보를 전방에 표시함으로써, 운전자가 운전을 하는 동안 전방으로 시선을 유지할 수 있게 도와준다. 이를 통해 운전자의 주의 분산을 줄이고, 안전을 향상시키는데 도움이 될 수 있다. 자동차 헤드업 디스플레이 시스템은 약 30년 전 운전자의 안전을 향상시키기 위한 수단으로 자동차 산업에 처음 도입된 이래로 현재까지 다양한 상용차에서 사용되고 있다. 안전과 편의 측면에서 자동차 헤드업 디스플레이의 사용은 점점 더 증가할 것으로 예상된다. 그러나 이러한 자동차 헤드업 디스플레이의 잠재적 이점과 발전 가능성에도 불구하고, 유용한 자동차 헤드업 디스플레이를 설계하는 것은 여전히 어려운 문제이다. 이에 본 연구는 이러한 문제를 해결하고, 궁극적으로 유용한 자동차 헤드업 디스플레이 설계에 기여하고자 총 4가지 연구를 수행하였다. 첫 번째 연구는 자동차 헤드업 디스플레이의 기능 요구 사항과 관련된 것으로서, 헤드업 디스플레이 시스템을 통해 어떤 정보를 제공할 것인가에 대한 답을 구하고자 하였다. 이에 주요 자동차 제조업체들의 헤드업 디스플레이 제품들과, 자동차 헤드업 디스플레이의 다양한 기능들을 제안한 학술 연구, 그리고 운전자의 정보 요구 사항들을 체계적 문헌 고찰 방법론을 통해 포괄적으로 조사하였다. 자동차 헤드업 디스플레이의 기능적 요구 사항에 대하여 개발자, 연구자, 사용자 측면을 모두 고려한 통합된 지식을 전달하고, 이를 통해 자동차 헤드업 디스플레이의 기능 요구 사항에 대한 향후 연구 방향을 제시하였다. 두 번째 연구는 안전 관련 정보를 제공하는 자동차 헤드업 디스플레이의 인터페이스 설계와 관련된 것으로, 헤드업 디스플레이 시스템을 통해 안전 관련 정보를 어떻게 제공할 것인가에 대한 답을 구하고자 하였다. 실제 자동차들의 헤드업 디스플레이 시스템에서는 어떤 디스플레이 컨셉들이 사용되었는지, 그리고 학계에서 제안된 디스플레이 컨셉들에는 어떤 것들이 있는지 체계적 문헌 고찰 방법론을 통해 검토하였다. 검토된 결과는 각 디스플레이의 기능과 구조, 그리고 작동 방식에 따라 정리되었고, 관련된 인간공학적 디스플레이 설계 원칙과 실험적 연구 결과들을 함께 검토하였다. 검토된 결과를 바탕으로 안전 관련 정보를 제공하는 자동차 헤드업 디스플레이의 인터페이스 설계에 대한 향후 연구 방향을 제시하였다. 세 번째 연구는 자동차 헤드업 디스플레이 기반의 제어권 전환 관련 인터페이스 설계와 평가에 관한 것이다. 제어권 전환이란, 자율주행 상태에서 운전자가 직접 운전을 하는 수동 운전 상태로 전환이 되는 것을 의미한다. 따라서 갑작스런 제어권 전환 요청이 발생하는 경우, 운전자가 안전하게 대처하기 위해서는 빠른 상황 파악과 의사 결정이 필요하게 되고, 이를 효과적으로 도와주기 위한 인터페이스 설계에 대해 연구할 필요성이 있다. 이에 본 연구에서는 자동차 헤드업 디스플레이 기반의 총 4개의 제어권 전환 관련 디스플레이(기준 디스플레이, 미니맵 디스플레이, 화살표 디스플레이, 미니맵과 화살표 디스플레이)를 제안하였고, 제안된 디스플레이 대안들은 주행 시뮬레이터 실험을 통해 제어권 전환 수행 능력과 안구의 움직임 패턴, 그리고 사용자의 주관적 평가 측면에서 평가되었다. 또한 제안된 디스플레이 대안들에 대해 운전자들의 초기 신뢰도 값을 측정하여 각 디스플레이에 따른 운전자들의 평균 신뢰도 점수에 따라 제어권 전환 수행 능력과 안구의 움직임 패턴, 그리고 주관적 평가가 어떻게 달라지는지 분석하였다. 실험 결과, 제어권 전환 상황에서 자동화된 시스템이 제안하는 정보와 그와 관련된 주변 상황 정보를 함께 제시해 주는 디스플레이가 가장 좋은 결과를 보여주었다. 또한 각 디스플레이에 대한 운전자의 초기 신뢰도 점수는 디스플레이의 실제 사용 행태와 밀접한 관련이 있음을 알 수 있었다. 신뢰도 점수에 따라 신뢰도가 높은 그룹과 낮은 그룹으로 분류되었고, 신뢰도가 높은 그룹은 제안된 디스플레이들이 보여주는 정보를 주로 믿고 따르는 경향이 있었던 반면, 신뢰도가 낮은 그룹은 룸 미러나 사이드 미러를 통해 주변 상황 정보를 더 확인 하는 경향을 보였다. 네 번째 연구는 전면 유리창에서의 인터랙티브 헤드업 디스플레이의 최적 위치를 결정하는 것으로서 주행 시뮬레이터 실험을 통해 디스플레이의 위치에 따라 운전자의 주행 수행 능력, 인터랙티브 디스플레이 조작 관련 과업 수행 능력, 시각적 주의 분산, 선호도, 그리고 작업 부하가 평가되었다. 헤드업 디스플레이의 위치는 전면 유리창에서 일정한 간격으로 총 9개의 위치가 고려되었다. 본 연구에서 활용된 인터랙티브 디스플레이는 음악 선택을 위한 스크롤 방식의 단일 디스플레이였고, 운전대에 장착된 버튼을 통해 디스플레이를 조작하였다. 실험 결과, 인터랙티브 헤드업 디스플레이의 위치가 모든 평가 척도, 즉 주행 수행 능력, 디스플레이 조작 과업 수행 능력, 시각적 주의 분산, 선호도, 그리고 작업 부하에 영향을 미침을 알 수 있었다. 모든 평가 지표를 고려했을 때, 인터랙티브 헤드업 디스플레이의 위치는 운전자가 똑바로 전방을 바라볼 때의 시야 구간, 즉 전면 유리창에서의 왼쪽 아래 부근이 가장 최적인 것으로 나타났다.Abstract i Contents v List of Tables ix List of Figures x Chapter 1 Introduction 1 1.1 Research Background 1 1.2 Research Objectives and Questions 8 1.3 Structure of the Thesis 11 Chapter 2 Functional Requirements of Automotive Head-Up Displays: A Systematic Review of Literature from 1994 to Present 13 2.1 Introduction 13 2.2 Method 15 2.3 Results 17 2.3.1 Information Types Displayed by Existing Commercial Automotive HUD Systems 17 2.3.2 Information Types Previously Suggested for Automotive HUDs by Research Studies 28 2.3.3 Information Types Required by Drivers (users) for Automotive HUDs and Their Relative Importance 35 2.4 Discussion 39 2.4.1 Information Types Displayed by Existing Commercial Automotive HUD Systems 39 2.4.2 Information Types Previously Suggested for Automotive HUDs by Research Studies 44 2.4.3 Information Types Required by Drivers (users) for Automotive HUDs and Their Relative Importance 48 Chapter 3 A Literature Review on Interface Design of Automotive Head-Up Displays for Communicating Safety-Related Information 50 3.1 Introduction 50 3.2 Method 52 3.3 Results 55 3.3.1 Commercial Automotive HUDs Presenting Safety-Related Information 55 3.3.2 Safety-Related HUDs Proposed by Academic Research 58 3.4 Discussion 74 Chapter 4 Development and Evaluation of Automotive Head-Up Displays for Take-Over Requests (TORs) in Highly Automated Vehicles 78 4.1 Introduction 78 4.2 Method 82 4.2.1 Participants 82 4.2.2 Apparatus 82 4.2.3 Automotive HUD-based TOR Displays 83 4.2.4 Driving Scenario 86 4.2.5 Experimental Design and Procedure 87 4.2.6 Experiment Variables 88 4.2.7 Statistical Analyses 91 4.3 Results 93 4.3.1 Comparison of the Proposed TOR Displays 93 4.3.2 Characteristics of Drivers Initial Trust in the four TOR Displays 102 4.3.3 Relationship between Drivers Initial Trust and Take-over and Visual Behavior 104 4.4 Discussion 113 4.4.1 Comparison of the Proposed TOR Displays 113 4.4.2 Characteristics of Drivers Initial Trust in the four TOR Displays 116 4.4.3 Relationship between Drivers Initial Trust and Take-over and Visual Behavior 117 4.5 Conclusion 119 Chapter 5 Human Factors Evaluation of Display Locations of an Interactive Scrolling List in a Full-windshield Automotive Head-Up Display System 121 5.1 Introduction 121 5.2 Method 122 5.2.1 Participants 122 5.2.2 Apparatus 123 5.2.3 Experimental Tasks and Driving Scenario 123 5.2.4 Experiment Variables 124 5.2.5 Experimental Design and Procedure 126 5.2.6 Statistical Analyses 126 5.3 Results 127 5.4 Discussion 133 5.5 Conclusion 135 Chapter 6 Conclusion 137 6.1 Summary and Implications 137 6.2 Future Research Directions 139 Bibliography 143 Apeendix A. Display Layouts of Some Commercial HUD Systems Appendix B. Safety-related Displays Provided by the Existing Commercial HUD Systems Appendix C. Safety-related HUD displays Proposed by Academic Research 국문초록 187Docto