Designing multimodal interaction for the visually impaired

Although multimodal computer input is believed to have advantages over unimodal input, little has been done to understand how to design a multimodal input mechanism to facilitate visually impaired users\u27 information access. This research investigates sighted and visually impaired users\u27 multimodal interaction choices when given an interaction grammar that supports speech and touch input modalities. It investigates whether task type, working memory load, or prevalence of errors in a given modality impact a user\u27s choice. Theories in human memory and attention are used to explain the users\u27 speech and touch input coordination. Among the abundant findings from this research, the following are the most important in guiding system design: (1) Multimodal input is likely to be used when it is available. (2) Users select input modalities based on the type of task undertaken. Users prefer touch input for navigation operations, but speech input for non-navigation operations. (3) When errors occur, users prefer to stay in the failing modality, instead of switching to another modality for error correction. (4) Despite the common multimodal usage patterns, there is still a high degree of individual differences in modality choices. Additional findings include: (I) Modality switching becomes more prevalent when lower working memory and attentional resources are required for the performance of other concurrent tasks. (2) Higher error rates increases modality switching but only under duress. (3) Training order affects modality usage. Teaching a modality first versus second increases the use of this modality in users\u27 task performance. In addition to discovering multimodal interaction patterns above, this research contributes to the field of human computer interaction design by: (1) presenting a design of an eyes-free multimodal information browser, (2) presenting a Wizard of Oz method for working with visually impaired users in order to observe their multimodal interaction. The overall contribution of this work is that of one of the early investigations into how speech and touch might be combined into a non-visual multimodal system that can effectively be used for eyes-free tasks

Human Factors in Automated and Robotic Space Systems: Proceedings of a symposium. Part 1

Human factors research likely to produce results applicable to the development of a NASA space station is discussed. The particular sessions covered in Part 1 include: (1) system productivity -- people and machines; (2) expert systems and their use; (3) language and displays for human-computer communication; and (4) computer aided monitoring and decision making. Papers from each subject area are reproduced and the discussions from each area are summarized

Human Factors Considerations in System Design

Human factors considerations in systems design was examined. Human factors in automated command and control, in the efficiency of the human computer interface and system effectiveness are outlined. The following topics are discussed: human factors aspects of control room design; design of interactive systems; human computer dialogue, interaction tasks and techniques; guidelines on ergonomic aspects of control rooms and highly automated environments; system engineering for control by humans; conceptual models of information processing; information display and interaction in real time environments

An integrated task manager for virtual command and control

The Task Manager is a desktop/tablet PC interface to the Battlespace research project that provides interactions and displays for supervisory control of unmanned aerial vehicles. Utilizing a north-up map display, the Task Manager provides a direct-manipulation interface to the units involved in an engagement. Used in two primary modes, the Task Manager can be used either in a planning/review mode that can be used to generate mission scenarios or a live-streaming mode that connects to a live Battlespace simulation via a network connection to edit and update path information on the fly. The goal of this research is to combine the precision of 2D mouse and pen-based interaction with the increased situational awareness provided by 3D battlefield visualizations like the Battlespace application. Combined use of these interfaces, either by a single operator or a small team of operators with task-specific roles, is proposed to produce a more favorable ratio of operators to units in field operations with superior decision-making capabilities due to the specific nature of the interfaces

Study to determine potential flight applications and human factors design guidelines for voice recognition and synthesis systems

A study was conducted to determine potential commercial aircraft flight deck applications and implementation guidelines for voice recognition and synthesis. At first, a survey of voice recognition and synthesis technology was undertaken to develop a working knowledge base. Then, numerous potential aircraft and simulator flight deck voice applications were identified and each proposed application was rated on a number of criteria in order to achieve an overall payoff rating. The potential voice recognition applications fell into five general categories: programming, interrogation, data entry, switch and mode selection, and continuous/time-critical action control. The ratings of the first three categories showed the most promise of being beneficial to flight deck operations. Possible applications of voice synthesis systems were categorized as automatic or pilot selectable and many were rated as being potentially beneficial. In addition, voice system implementation guidelines and pertinent performance criteria are proposed. Finally, the findings of this study are compared with those made in a recent NASA study of a 1995 transport concept

Human factors aspects of control room design: Guidelines and annotated bibliography

A human factors analysis of the workstation design for the Earth Radiation Budget Satellite mission operation room is discussed. The relevance of anthropometry, design rules, environmental design goals, and the social-psychological environment are discussed

'탑승자'의 관점의 시간, 위치 기반 차량 클러스터 UI 디자인 프레임 제안 연구

학위논문(석사)--서울대학교 대학원 :미술대학 디자인학부 디자인전공,2019. 8. 정의철.One important design issue is the examination of how the user interface (UI) supports the new user role in future mobility. However, there are few design studies on the passengers cognitive needs and behavior in Autonomous Vehicles (AVs) based on empirical data. There is no doubt that autonomous mobility technologies are growing. The technology is already aiding the driving experience, and it will change the mobility culture and the transition of driver into passenger. This study is based on the premise that future AV is capable of performing all driving tasks. It proposes a set of passenger-centered automotive cluster UI designs for future mobility employing two factors: time and path. A set of empirical data is provided to understand the passengers perspective. In this study, a solid set of empirical data on the cognitive needs of passengers is collected. Human cognitive characteristics and driving tasks are investigated from various viewpoints to understand the passengers iii perspective. The cognitive relationship in the driving environment is analyzed through a literature review on situation awareness (SA) and structuring of the data flow framework. The framework is further explored by connecting the technological role transformation to the passenger. To construct the empirical database on the passenger, three sets of user tests and in-depth interviews were undertaken. The user tests were designed employing the Wizard of Oz method, and the results were summarized using descriptive and exploratory analysis. Based on these insights, a set of UI designs from the perspective of the passenger was proposed, and usability tests were conducted to verify its effectiveness and usability. The results of the tests demonstrate that a major percentage of the information request was related to time (current time and duration) and path (vehicle location and surroundings). Based on the data, a UI framework was built. Two usage scenarios were designed, time-full and time-less, for better in-situation comprehension. Time- and path-based UI were proposed to flow with the scenarios. A usability test was conducted, and a passengers cognitive framework was defined. There are two aspects to this study: the data flow frameworks of the driver/passenger, and the UI design proposal. Situational precision from the perspective of the driver was analyzed to understand the relationship between the user, the vehicle and the road conditions. Further, the cognitive framework of the passenger was proposed based on the data. This study provides a solid understanding of drivers emerging needs when they are relieved of the cognitive burden of driving tasks. The UI features for AV are introduced based on the empirical data and research related to the provision of better situation awareness, focusing on time and location. This study contributes to the extant literature by observing the iv perspective of passengers in Autonomous vehicles based on a qualitative study. The proposed UI design will be further explored as a communication method between the system and the passive user in future mobility.사용자 인터페이스가 (UI) 미래 이동성에서 새로운 사용자 역할을 지지하는 디자인 도출은 미래 이동성 분야에서 중요한 디자인 이슈이다. 그러나 사용자 실험에 근거하여 자율주행차량 (AV) 의 탑승자인지 욕구와 행동에 대한 디자인 연구는 미미하다. 자율주행이 기술의 발전과 그 영역은 점차 넓어지고 있다. 해당 기술은 이미 운전 환경에 적용되고 있으며, 이로 인해 미래 이동문화에서 사용자의 역할은 '운전자'에서 '탑승자'로 변화한다. 본 연구는 미래 자율주행차량이 모든 운전 상황에 대처할 수 있다는 것을 전제로 한다. 사용자 실험을 통해 탑승자의 관점에 대한 분석을 진행하였고, 이를 기반으로 미래 모빌리티 환경에 적용될 사용자 인터페이스를 제안한다. 제안된 디자인은 운전자 중심의 상황인지에서 벗어나 탑승자 중심 인지 정보 요소를 분석하였고, 시간과 경로 두 가지 요소를 강조한 UI 를 제안한다. 본 연구에서 탑승자의 인지 정보 요구에 대한 실험적 데이터를 수집하였다. 탑승자의 관점을 이해하기 위해 다양한 관점에서 인간의 인지적 특성 및 운전 태스크를 관찰하였고, 상황인지 (SA) 에 관한 문헌 연구와 데이터 프레임워크 구조화를 통해 운전 환경에서 발생하는 인지적 요소 관계를 분석하였다. 제안된 프레임워크는 기술 변화에 따라 운전자가 탑승자로 변화되었을 때 운전 환경에서의 데이터 관계 변화를 시각적으로 구조화하여 심층적으로 탐구되었다. 탑승자의 인지 니즈 대한 실험적 데이터베이스를 수집하기 위해 총 3 세트의 유저 테스트와 심층 인터뷰가 수반되었다. 유저 테스트는 Wizard of Oz 방법을 사용하여 설계되었으며 실험 결과는 질적 연구방법론의 분석 방법을 통해 분석되었다. 실험을 통해 얻은 인사이트를 바탕으로 탑승자 관점에서 UI 디자인을 제안하고 사용성 테스트를 통해 효율성과 유용성을 5 점 리 커트 스케일로써 검증하였다. 실험 결과에 따르면 탑승자가 요청한 인지 정보는 시간 (현재 시각 및 기간)과 경로 (차량 위치 및 주변 환경)에 집중된 것을 관찰할 수 있었다. 이와 같은 데이터를 기반으로 UI 프레임워크를 구성하였다. 상황 속의 사용례를 제시하기 위하여도 가지 time-full 과 time-less 의 사용 시나리오를 구축하고, 제안된 시나리오에 따라 시간과 위치에 기반한 UI 를 제안하였다. 제안된 UI 에 대한 사용성 테스트를 진행하였고, 탑승자 관점에서의 운전상황 인지 워크 프레임을 완성하였다. 본 연구의 가치는 두 가지로 정리될 수 있다. 하나는 운전자 / 탑승자의 데이터 플로우 프레임워크를 제안하였다는 것과 두 번째는 탑승자의 관점을 지지하는 UI 디자인 제안에 있다. 운전자의 관점에서의 운전 상황을 분석하여 사용자, 차량, 그리고 도로 상태 간의 관계를 시각화하였고, 이는 탑승자인지 플로우 프레임워크를 제안하는데 기조적인 틀로써 사용되었다. 본 연구는 운전 태스크를 수행하는 데에 필요했던 인지 부담에서 벗어났을 때의 운전자가 필요로 하는 복합적인 니즈에 대해 관찰하고 미래 모빌리티 환경에 적합한 UI 의 디자인 요소에 대한 연구논문이다. 미래 자율주행차량 안의 사용자 인터페이스가 갖추어야 하는 요소를 실험적 데이터에 근거하여 제시하며, 시간과 루트를 강조하여 향상된 상황 인지를 제공하는 방법에 대한 심도있는 관찰을 기록한다. 본 연구는 질적 연구에 기초한 자율 차량의 탑승자 관점을 관찰함으로써 기존 자율주행이 디자인 연구에 기여할 것이다. 제안된 UI 디자인 미래 이동 성안에서 시스템과 탑승자 간의 커뮤니케이션 방법에 대한 연구로써 그 의의가 있다.ABSTRACT ...................................................................................................................... II CHAPTER 1. INTRODUCTION......................................................................................... １ 1.1. BACKGROUND ..............................................................................................................１ 1.2. PURPOSE .....................................................................................................................７ 1.3. RESEARCH QUESTION.....................................................................................................８ CHAPTER 2. LITERATURE REVIEW ..............................................................................１１ 2.1. SITATION AWARENESS (SA) ........................................................................................１１ 2.2. HUMAN INFORMATION PROCESSING MODEL..................................................................１５ 2.3. DRIVING SITUATION AWARENESS AND PERSPECTIVE.........................................................２０ 2.4. DRIVING TASK AND SENSORY INTERACTION ....................................................................２２ CHAPTER 3. COGNITIVE NEEDS IN AUTONOMOUS.....................................................２７ 3.1. DRIVING BEHAVIOR TRANSFORMATION AND CLUSTER UI..................................................２７ 3.2. COGNITIVE FRAMEWORK TRANSFORMATION ..................................................................３３ CHAPTER 4. USER TESTS ............................................................................................３６ 4.1. WIZARD OF OZ PROTOTYPING .....................................................................................３８ 4.2. PILOT TEST 1............................................................................................................４０ 4.2.1. Experiment Design & Laboratory Setting.................................................４０ 4.2.2. Persona Scenario & Task Design ..............................................................４２ 4.2.3. Preparation of Driving situation...............................................................４５ 4.2.4. Procedure.................................................................................................４７ 4.2.5. Data Analysis & Insight............................................................................４８ 4.3. PILOT TEST 2............................................................................................................５１ 4.3.1. Amendment: Experiment Design & Laboratory Setting ...........................５２ 4.3.2. Amendment: Task Scenario & Command Cue..........................................５４ 4.3.3. Amendment: Perform Role and preparation of driving situation ............５７ 4.3.4. Amendment: Procedure ...........................................................................５９ 4.3.5. Data Analysis & Insight............................................................................６２ 4.4. MAIN TEST ..............................................................................................................６５ 4.4.1. Experiment Design & Laboratory setting .................................................６６ 4.4.2. Task Design ..............................................................................................６９ 4.4.3. Procedure.................................................................................................７１ 4.4.4. Result Analysis & Insight..........................................................................７４ CHAPTER 5. UI CONCEPT DEVELOPMENT...................................................................８１ 5.1. UI DESIGN METHOD..................................................................................................８１ 5.2. DESIGN PROPOSAL ....................................................................................................８４ 5.3. USER SCENARIOS ......................................................................................................８６ 5.3.1 Scenario 1. Time-less: Late for a morning meeting..................................８６ 5.3.2 Scenario 2.Time-full: Leisure driving on weekends ..................................９３ CHAPTER 6. USABILITY TEST ......................................................................................９８ 6.1. USABILITY TEST GUIDE ...............................................................................................９８ 6.2. ASSESSMENT USABILITY TEST ..................................................................................１００ 6.2.1 Test planning........................................................................................１００ 6.2.2 Laboratory setting................................................................................１０２ 6.2.3 Test conduct and debriefing.................................................................１０６ 6.3. RESULT ANALYSIS ..................................................................................................１０６ CHAPTER 7. CONCLUSION......................................................................................１０７ APPENDIX 1...........................................................................................................１１０ APPENDIX 2...........................................................................................................１１１ APPENDIX 3...........................................................................................................１１３ APPENDIX 4...........................................................................................................１２１ APPENDIX 5...........................................................................................................１２４ APPENDIX 6...........................................................................................................１２８ APPENDIX 7...........................................................................................................１３３ BIBLIOGRAPHY ......................................................................................................１３６ 국문 초록 ............................................................................................................１４３Maste

A systematic approach for the analysis, design and implementation of Telecommunications-Supported Training (TST) systems

The objetive of the present thesis is to develop and test out a systematic approach to the analysis, design and implementation of Telecommunications-Supported Training (TST) systems. The Literature Review offers a set of approaches to similar problems in the field of Information Systems. Several other disciplines have also been considered: Psychology of Human Factors, Organizational and Innovation Theory, and applied research being currently carried out under the EU DELTA Programme. A global User-Centred Model of TST has been developed, based on the wellestablished principles of Systems Engineering and Soft Systems Methodology. The theoretical basis for this model is the concept of TST Architecture, conceived as a functional arrangement of technical components which are introduced in order to improve the performance of the actors involved in the system. The other key aspect is the design of a set of Adoption Strategies, aiming for the creation of the necessary conditions to achieve user acceptance of the technologies implemented. Following these principles, the ADAM (Architectural Design and Adoption Model) systematic approach is developed. It is structured in five stages: Context Analysis, System Analysis, System Design, System Implementation and System Maintenance. Each stage is structured into steps and activities, described in terms of key points, outcomes, deliverables, and roles involved. The ADAM approach has been tested out in its twofold dimension of analyzing already implemented TST systems and designing new ones. The first case discusses the application of ADAM to the EU Multimedia TeleSchool (MTS) TST system. The test is completed by discussing the design and implementation, performed by the author, of a TST system at the Universidad Politecnica de Madrid. The results confirm the usefulness of ADAM both for practitioners and researchers in the field. Also, the TST model is extended as regards the components of acceptance and adoption, and their impact on the introduction of technologies in organizations

Mediating disruption in human-computer interaction from implicit metrics of attention

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 143-150).Multitasking environments cause people to be interrupted constantly, often disrupting their ongoing activities and impeding reaching their goals. This thesis presents a disruption reducing approach designed to support the user's goals and optimize productivity that is based on a model of the user's receptivity to an interruption. The model uses knowledge of the interruption content, context and priority of the task(s) in progress, user actions and goal-related concepts to mediate interruptions. The disruption management model is distinct from previous work by the addition of implicit sensors that deduce the interruption content and user context to help determine when an interruption will disrupt an ongoing activity. Domain-independent implicit sensors include mouse and keyboard behaviors, and goal-related concepts extracted from the user documents. The model also identifies the contextual relationship between interruptions and user goals as an important factor in how interruptions are controlled. The degree to which interruptions are related to the user goal determines how those interruptions will be received. We tested and evolved the model in various cases and showed significant improvement in both productivity and satisfaction. A disruption manager application controls interruptions on common desktop computing activities, such as web browsing and instant messaging. The disruption manager demonstrates that mediating interruptions by supporting the user goals can improve performance and overall productivity. Our evaluation shows an improvement in success of over 25% across prioritization conditions for real life computing environments.(cont.) Goal priority and interruption relevance play an important role in the interruption decision process and several experiments these factors on people's reactions and availability to interruptions, and overall performance. These experiments demonstrate that people recognize the potential benefits of being interrupted and adjust their susceptibility to interruptions during highly prioritized tasks. The outcome of this research includes a usable model that can be extended to tasks as diverse as driving an automobile and performing computer tasks. This thesis supports mediating technologies that will recognize the value of communication and control interruptions so that people are able to maintain concentration amidst their increasingly busy lifestyles.by Ernesto Arroyo Acosta.Ph.D