Modeling the Human Visuo-Motor System for Remote-Control Operation

University of Minnesota Ph.D. dissertation. 2018. Major: Computer Science. Advisors: Nikolaos Papanikolopoulos, Berenice Mettler. 1 computer file (PDF); 172 pages.Successful operation of a teleoperated miniature rotorcraft relies on capabilities including guidance, trajectory following, feedback control, and environmental perception. For many operating scenarios fragile automation systems are unable to provide adequate performance. In contrast, human-in-the-loop systems demonstrate an ability to adapt to changing and complex environments, stability in control response, high level goal selection and planning, and the ability to perceive and process large amounts of information. Modeling the perceptual processes of the human operator provides the foundation necessary for a systems based approach to the design of control and display systems used by remotely operated vehicles. In this work we consider flight tasks for remotely controlled miniature rotorcraft operating in indoor environments. Operation of agile robotic systems in three dimensional spaces requires a detailed understanding of the perceptual aspects of the problem as well as knowledge of the task and models of the operator response. When modeling the human-in-the-loop the dynamics of the vehicle, environment, and human perception-action are tightly coupled in space and time. The dynamic response of the overall system emerges from the interplay of perception and action. The main questions to be answered in this work are: i) what approach does the human operator implement when generating a control and guidance response? ii) how is information about the vehicle and environment extracted by the human? iii) can the gaze patterns of the pilot be decoded to provide information for estimation and control? In relation to existing research this work differs by focusing on fast acting dynamic systems in multiple dimensions and investigating how the gaze can be exploited to provide action-relevant information. To study human-in-the-loop systems the development and integration of the experimental infrastructure is described. Utilizing the infrastructure, a theoretical framework for computational modeling of the human pilot’s perception-action is proposed and verified experimentally. The benefits of the human visuo-motor model are demonstrated through application examples where the perceptual and control functions of a teleoperation system are augmented to reduce workload and provide a more natural human-machine interface

Mars Rotorcraft: Possibilities, Limitations, and Implications For Human/Robotic Exploration

Several research investigations have examined the challenges and opportunities in the use of small robotic rotorcraft for the exploration of Mars. To date, only vehicles smaller than 150 kg have been studied. This paper proposes to examine the question of maximum Mars rotorcraft size, range, and payload/cargo capacity. Implications for the issue of whether or not (from an extreme design standpoint) a manned Mars rotorcraft is viable are also discussed

North-Up, Track-Up, and Camera-Up Navigation of Unmanned Aircraft Systems

To optimize UAV reconnaissance operations, direction of viewing and direction of travel must be allowed to diverge. Our challenge was to design a control and display strategy to allow the operator to easily look where they’re going, go where they’re looking, and look and go in different directions. Two methods of control were devised to align traveling forward, viewing forward and commanding forward. The operator can command the UAS to turn to camera or command the camera to point in line with the direction of travel (eyes forward). We have also introduced a new camera-up map orientation. The operator can easily cycle through North-up, track-up, and camera-up to provide the best link between the exo-centric and ego-centric frames of reference. Ego-centric and exo-centric perspectives allow the operator to combine or separate the vehicle’s movement and the camera’s view to optimize the search task while maintaining situation awareness of flight hazards

Objective performance metrics for improved space telerobotics training

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 47-50).NASA astronauts undergo many hours of formal training and self-study to gain proficiency in space teleoperation tasks. After each lesson, instructors score an astronaut's performance in several broad skill categories, including 'General Situational Awareness', 'Maneuvers/Task Performance', and 'Hand- Controller Techniques'. A plus, check, or minus indicates that the student is ahead of, at, or behind the expected skill level. The scoring of the final evaluation for a robotics training course is also largely subjective, with the instructor designating an integer score for the student between 1 (Unsatisfactory) and 5 (Strong) in the same skill categories. This thesis research project was designed to: (1) consider the variety of quantitative metrics that could be embedded into a space robotics training simulation, and (2) investigate at what point and by what means it is most constructive for performance assessment to be revealed to an operator-in-training. We reviewed the current largely qualitative space robotics performance metrics, as well as new quantitative kinematic metrics of manual control skills-including those explored thus far only in laboratory experiments-and additional measures of executive function and supervisory control performance. Kinematic metrics include quantitative measures such as rate of change of linear and rotational acceleration. Potential measures of executive function and supervisory control include camera selection and clearance monitoring. To instantiate our ideas, we chose a specific "fly-to" space telerobotics task taught in the early phases of NASA Generic Robotics Training (GRT) and developed a pilot training experiment (n=16) using our virtual robotics training workstation. Our goal was to evaluate potential performance metrics designed to encourage use of multi-axis control, and to compare real-time ("live") performance feedback alternatives (live visual vs. live aural vs. none). Movement time decreased and multi-axis and bimanual control use gradually increased across trials. All subjects had the opportunity to view post-trial performance feedback including these metrics. Although our subjects overwhelmingly preferred the live, visual feedback condition, no reliable additional effects of live feedback condition were found, except perhaps among the more experienced subjects. However, the experiment demonstrated that embedded performance metrics potentially could quantify and improve some important aspects of GRT evaluations.Supported by the National Space Biomedical Research Institute through NASA NCC9-58by Rachel Emily Forman.S.M

Future Roles for Autonomous Vertical Lift in Disaster Relief and Emergency Response

System analysis concepts are applied to the assessment of potential collaborative contributions of autonomous system and vertical lift (a.k.a. rotorcraft, VTOL, powered-lift, etc.) technologies to the important, and perhaps underemphasized, application domain of disaster relief and emergency response. In particular, an analytic framework is outlined whereby system design functional requirements for an application domain can be derived from defined societal good goals and objectives

Tier-Scalable Reconnaissance Missions For The Autonomous Exploration Of Planetary Bodies

A fundamentally new (scientific) reconnaissance mission concept, termed tier-scalable reconnaissance, for remote planetary (including Earth) atmospheric, surface and subsurface exploration recently has been devised that soon will replace the engineering and safety constrained mission designs of the past, allowing for optimal acquisition of geologic, paleohydrologic, paleoclimatic, and possible astrobiologic information of Venus, Mars, Europa, Ganymede, Titan, Enceladus, Triton, and other extraterrestrial targets. This paradigm is equally applicable to potentially hazardous or inaccessible operational areas on Earth such as those related to military or terrorist activities, or areas that have been exposed to biochemical agents, radiation, or natural disasters. Traditional missions have performed local, ground-level reconnaissance through rovers and immobile landers, or global mapping performed by an orbiter. The former is safety and engineering constrained, affording limited detailed reconnaissance of a single site at the expense of a regional understanding, while the latter returns immense datasets, often overlooking detailed information of local and regional significance

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 315)

This bibliography lists 211 reports, articles and other documents introduced into the NASA scientific and technical information system in September, 1988

The use of modern tools for modelling and simulation of UAV with Haptic

Unmanned Aerial Vehicle (UAV) is a research field in robotics which is in high demand in recent years, although there still exist many unanswered questions. In contrast, to the human operated aerial vehicles, it is still far less used to the fact that people are dubious about flying in or flying an unmanned vehicle. It is all about giving the control right to the computer (which is the Artificial Intelligence) for making decisions based on the situation like human do but this has not been easy to make people understand that it’s safe and to continue the enhancement on it. These days there are many types of UAVs available in the market for consumer use, for applications like photography to play games, to map routes, to monitor buildings, for security purposes and much more. Plus, these UAVs are also being widely used by the military for surveillance and for security reasons. One of the most commonly used consumer product is a quadcopter or quadrotor. The research carried out used modern tools (i.e., SolidWorks, Java Net Beans and MATLAB/Simulink) to model controls system for Quadcopter UAV with haptic control system to control the quadcopter in a virtual simulation environment and in real time environment. A mathematical model for the controlling the quadcopter in simulations and real time environments were introduced. Where, the design methodology for the quadcopter was defined. This methodology was then enhanced to develop a virtual simulation and real time environments for simulations and experiments. Furthermore, the haptic control was then implemented with designed control system to control the quadcopter in virtual simulation and real time experiments. By using the mathematical model of quadcopter, PID & PD control techniques were used to model the control setup for the quadcopter altitude and motion controls as work progressed. Firstly, the dynamic model is developed using a simple set of equations which evolves further by using complex control & mathematical model with precise function of actuators and aerodynamic coefficients Figure5-7. The presented results are satisfying and shows that flight experiments and simulations of the quadcopter control using haptics is a novel area of research which helps perform operations more successfully and give more control to the operator when operating in difficult environments. By using haptic accidents can be minimised and the functional performance of the operator and the UAV will be significantly enhanced. This concept and area of research of haptic control can be further developed accordingly to the needs of specific applications

Aeronautical engineering: A continuing bibliography with indexes (supplement 268)

This bibliography lists 406 reports, articles, and other documents introduced into the NASA scientific and technical information system in July, 1991. Subject coverage includes: design, construction and testing of aircraft and aircraft engines; aircraft components, equipment and systems; ground support systems; and theoretical and applied aspects of aerodynamics and general fluid dynamics

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 391)

This bibliography lists 75 reports, articles, and other documents introduced into the NASA Scientific and Technical Information System during Aug. 1994. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance