Variable force and visual feedback effects on teleoperator man/machine performance

An experimental study was conducted to determine the effects of various forms of visual and force feedback on human performance for several telemanipulation tasks. Experiments were conducted with varying frame rates and subtended visual angles, with and without force feedback

Recollections on Presence Beginnings, and Some Challenges for Augmented and Virtual Reality

The idea for Presence began on the beach in Santa Barbara. Nat Durlach and I thought it would be fun to get some of the key programmers and engineers already doing VR and graphics software for computer gaming, film, and TV together with engineers from NASA, the military, and the aviation industry who had long been developing flight simulators and now were concerned about remote control and telepresence. We saw a close relation between VR and teleoperation, as depicted by Figure 1. To the extent that the computer-generated reality and the telepresence feedback from an actual tele- operator are both of sufficiently high quality, from the human operator’s viewpoint the mental model of the task and the interactions at the computer interface should be the same. In 1991 this sameness was clearly a stretch for many reasons, but it was clear that there was much we could gain from sharing ideas, and to the best of our knowledge there had been minimal interaction to date between those two communities

Constraint, Intelligence, and Control Hierarchy in Virtual Environments

This paper seeks to deal directly with the question of what makes virtual actors and objects that are experienced in virtual environments seem real. (The term virtual reality, while more common in public usage, is an oxymoron; therefore virtual environment is the preferred term in this paper). Reality is difficult topic, treated for centuries in those sub-fields of philosophy called ontology- "of or relating to being or existence" and epistemology- "the study of the method and grounds of knowledge, especially with reference to its limits and validity" (both from Webster s, 1965). Advances in recent decades in the technologies of computers, sensors and graphics software have permitted human users to feel present or experience immersion in computer-generated virtual environments. This has motivated a keen interest in probing this phenomenon of presence and immersion not only philosophically but also psychologically and physiologically in terms of the parameters of the senses and sensory stimulation that correlate with the experience (Ellis, 1991). The pages of the journal Presence: Teleoperators and Virtual Environments have seen much discussion of what makes virtual environments seem real (see, e.g., Slater, 1999; Slater et al. 1994; Sheridan, 1992, 2000). Stephen Ellis, when organizing the meeting that motivated this paper, suggested to invited authors that "We may adopt as an organizing principle for the meeting that the genesis of apparently intelligent interaction arises from an upwelling of constraints determined by a hierarchy of lower levels of behavioral interaction. "My first reaction was "huh?" and my second was "yeah, that seems to make sense." Accordingly the paper seeks to explain from the author s viewpoint, why Ellis s hypothesis makes sense. What is the connection of "presence" or "immersion" of an observer in a virtual environment, to "constraints" and what types of constraints. What of "intelligent interaction," and is it the intelligence of the observer or the intelligence of the environment (whatever the latter may mean) that is salient? And finally, what might be relevant about "upwelling" of constraints as determined by a hierarchy of levels of interaction

Human Factors in Automated and Robotic Space Systems: Proceedings of a symposium. Part 1

Human factors research likely to produce results applicable to the development of a NASA space station is discussed. The particular sessions covered in Part 1 include: (1) system productivity -- people and machines; (2) expert systems and their use; (3) language and displays for human-computer communication; and (4) computer aided monitoring and decision making. Papers from each subject area are reproduced and the discussions from each area are summarized

Apparatus for providing sensory substitution of force feedback

A feedback apparatus for an operator to control an effector that is remote from the operator to interact with a remote environment has a local input device to be manipulated by the operator. Sensors in the effector's environment are capable of sensing the amplitude of forces arising between the effector and its environment, the direction of application of such forces, or both amplitude and direction. A feedback signal corresponding to such a component of the force, is generated and transmitted to the environment of the operator. The signal is transduced into an auditory sensory substitution signal to which the operator is sensitive. Sound production apparatus present the auditory signal to the operator. The full range of the force amplitude may be represented by a single, audio speaker. Auditory display elements may be stereo headphones or free standing audio speakers, numbering from one to many more than two. The location of the application of the force may also be specified by the location of audio speakers that generate signals corresponding to specific forces. Alternatively, the location may be specified by the frequency of an audio signal, or by the apparent location of an audio signal, as simulated by a combination of signals originating at different locations

Use of graphics in decision aids for telerobotic control: (Parts 5-8 of an 8-part MIT progress report)

Four separate projects recently completed or in progress at the MIT Man-Machine Systems Laboratory are summarized. They are: a decision aid for retrieving a tumbling satellite in space; kinematic control and graphic display of redundant teleoperators; real time terrain/object generation: a quad-tree approach; and two dimensional control for three dimensional obstacle avoidance

Toward the definition and measurement of the mental workload of transport pilots

January 1979Includes bibliographical references (p. 69-72)This report describes work performed in the first year of a continuing research project aimed at developing useful methods for measuring the workload of pilots operating aircraft in the ATC system. Good methods of measuring mental workload of human operators are needed to evaluate the introduction of new technology and new procedures in the man-machine environment. The present research is concentrating on developing subjective assessment methods for any phase of an IFR (Instrument Flight Rules) flight and any crew station on the flight deck. One of the results achieved in the first year is an expanded conceptual structure which allows a more precise definition of terms and assumptions in defining pilot mental workload in a multi-task environment. A second area of research has concentrated on reviewing the alternative approaches to developing a measurement scheme for workload, with some emphasis on the subjective assessment approach. A tentative result in this area is the generation of a prototype subjective rating method for IFR pilot workload modeled closely on the Cooper-Harper rating developed in 1969 to evaluate aircraft handling qualities. This scheme and others will be tested in a transport aircraft simulation during the coming year. If successful, it will be used in a variety of cockpit simulators at NASA research centers (Ames and Langley) and FAA NAFEC as part of a joint research program to evaluate cockpit display of traffic information.Supported in part by the U.S. Dept. of Transportatio

Adjustable impedance, force feedback and command language aids for telerobotics (parts 1-4 of an 8-part MIT progress report)

Projects recently completed or in progress at MIT Man-Machine Systems Laboratory are summarized. (1) A 2-part impedance network model of a single degree of freedom remote manipulation system is presented in which a human operator at the master port interacts with a task object at the slave port in a remote location is presented. (2) The extension of the predictor concept to include force feedback and dynamic modeling of the manipulator and the environment is addressed. (3) A system was constructed to infer intent from the operator's commands and the teleoperation context, and generalize this information to interpret future commands. (4) A command language system is being designed that is robust, easy to learn, and has more natural man-machine communication. A general telerobot problem selected as an important command language context is finding a collision-free path for a robot

Individual Differences in Attributes of Trust in Automation: Measurement and Application to System Design

Computer-based automation of sensing, analysis, memory, decision-making, and control in industrial, business, medical, scientific, and military applications is becoming increasingly sophisticated, employing various techniques of artificial intelligence for learning, pattern recognition, and computation. Research has shown that proper use of automation is highly dependent on operator trust. As a result the topic of trust has become an active subject of research and discussion in the applied disciplines of human factors and human-systems integration. While various papers have pointed to the many factors that influence trust, there currently exists no consensual definition of trust. This paper reviews previous studies of trust in automation with emphasis on its meaning and factors determining subjective assessment of trust and automation trustworthiness (which sometimes but not always are regarded as an objectively measurable properties of the automation). The paper asserts that certain attributes normally associated with human morality can usefully be applied to computer-based automation as it becomes more intelligent and more responsive to its human user. The paper goes on to suggest that the automation, based on its own experience with the user, can develop reciprocal attributes that characterize its own trust of the user and adapt accordingly. This situation can be modeled as a formal game where each of the automation user and the automation (computer) engage one another according to a payoff matrix of utilities (benefits and costs). While this is a concept paper lacking empirical data, it offers hypotheses by which future researchers can test for individual differences in the detailed attributes of trust in automation, and determine criteria for adjusting automation design to best accommodate these user differences

Decision-Aiding and Optimization for Vertical Navigation of Long-Haul Aircraft

Most decisions made in the cockpit are related to safety, and have therefore been proceduralized in order to reduce risk. There are very few which are made on the basis of a value metric such as economic cost. One which can be shown to be value based, however, is the selection of a flight profile. Fuel consumption and flight time both have a substantial effect on aircraft operating cost, but they cannot be minimized simultaneously. In addition, winds, turbulence, and performance vary widely with altitude and time. These factors make it important and difficult for pilots to (a) evaluate the outcomes associated with a particular trajectory before it is flown and (b) decide among possible trajectories. The two elements of this problem considered here are: (1) determining what constitutes optimality, and (2) finding optimal trajectories. Pilots and dispatchers from major u.s. airlines were surveyed to determine which attributes of the outcome of a flight they considered the most important. Avoiding turbulence-for passenger comfort-topped the list of items which were not safety related. Pilots' decision making about the selection of flight profile on the basis of flight time, fuel burn, and exposure to turbulence was then observed. Of the several behavioral and prescriptive decision models invoked to explain the pilots' choices, utility maximization is shown to best reproduce the pilots' decisions. After considering more traditional methods for optimizing trajectories, a novel method is developed using a genetic algorithm (GA) operating on a discrete representation of the trajectory search space. The representation is a sequence of command altitudes, and was chosen to be compatible with the constraints imposed by Air Traffic Control, and with the training given to pilots. Since trajectory evaluation for the GA is performed holistically, a wide class of objective functions can be optimized easily. Also, using the GA it is possible to compare the costs associated with different airspace design and air traffic management policies. A decision aid is proposed which would combine the pilot's notion of optimality with the GA-based optimization, provide the pilot with a number of alternative pareto-optimal trajectories, and allow him to consider unmodelled attributes and constraints in choosing among them. A solution to the problem of displaying alternatives in a multi-attribute decision space is also presented