The operator control station of a telerobot system has unique functional and human factors requirements. It has to satisfy the needs of a truly interactive and user-friendly complex system, a telerobot system being a hybrid between a teleoperated and an autonomous system. These functional, hardware and software requirements are discussed, with explicit reference to the design objectives and constraints of the JPL/NASA Telerobot Demonstrator System

Kan, Edwin P.

English

NASA Technical Reports Server

N89- 1987 6 
TELEROBOT OPERATOR CONTROL STATION REQUIREMENTS 
Edwin P. Kan 
Jet Propulsion Laboratory 
California Institute of Technology 
Pasadena, Ca. 91 109 
ABSTRACT 
The Operator Control Station of a Telerobot 
System has unique functional and human factors 
requirements. It has to satisfy the needs of a 
truly interactive and user-friendly complex 
system; a telerobot system being a hybrid 
between teleoperated and autonomous system. 
These functional, hardware and software 
requirements are discussed in this paper, with 
explicit reference to the design objectives and 
constraints of the JPL/NASA Telerobot 
Demonstrator System. 
INTRODUCTION 
A telerobot system is a hybrid system between a 
teleoperated system and an autonomous 
(robotic) system. It is capable of being 
teleoperated such as a master-slave 
manipulator system used widely in the nuclear 
industry. I t  is capable of being operated 
autonomously under preprogrammed control or 
under real-time intelligent, sensor-based and 
knowledge-based control, such as in various 
forms of automated factory systems. 
More importantly, a telerobot system is capable 
of being operated in a continuum mode, where 
teleoperated control and autonomous control can 
be traded or shared, as the circumstance 
requires. That is, when it is more natural and 
easier to do human teleoperated control, the 
telerobot system will be configured to do so. 
When it is more efficient to perform the task 
autonomously, the telerobot system will be 
configured to do so. And, when certain degrees 
of freedom are more readily achieved by 
autonomous control while other degrees of 
freedom are naturally achieved by teleoperated 
control, the overall control is shared between 
437 
PRECEDIKG PAGE BLA!dIK NOT FILMED 
the two control modes. In all cases, the human 
operator provides the guidance, direction, high- 
level decision, supervision and execution of the 
task. 
It is thus apparent that human factors issues 
are very demanding in the design of the operator 
control station of this telerobot system. Design 
requirements are’ very unique. Based on past 
experience of operational and research 
teleoperated and robotic systems, the Jet 
Propulsion Laboratory (JPL) is now developing a 
state-of-the-art Operator Control Station (OCS) 
as part of the NASA (National Aeronautics and 
Space Administration) Telerobot Demonstrator 
Project [l]. This paper will present the 
functional, hardware and software requirements 
of the OCS of this Telerobot Demonstrator 
System. 
TELEROBOT SYSTEM REQUIREMENTS 
The system requirements are driven by the tasks 
to be performed by the telerobot system: and 
also driven by the state of the technology. The 
telerobot system must be able to perform tasks 
equally as well, and hopefully better, than a 
straightforward teleoperated system and than a 
straightforward robotic system. For space 
applications, the telerobot system must offer 
advantages over expensive and hazardous EVA 
(extra-vehicular activities) and IVA (intra- 
vehicular activities) where astronauts actively 
participate in the performance of construction, 
servicing and repair. 
In the context of the JPL/NASA Telerobot 
Demonstrator System, the system is designed as 
a laboratory testbed for developing, testing and 
integrating multi-disciplinary technologies. The 
system is designed within certain overall 
https://ntrs.nasa.gov/search.jsp?R=19890010505 2020-03-20T03:49:54+00:00Z
constraints of the space environment, while not 
limited by near-term technology specifics 
regarding maturity in hardware and software 
designs. 
Overall telerobot system capabilities [2] are 
targeted as follows: 
1. Force reflection in teleoperation. 
2. Hybrid position/force control in robotic 
and advanced teleoperation. 
3. Dual arm coordinated control in robotic 
and teleoperation. 
4. Collision prediction, detection and 
avoidance in robotic operation; use of virtual 
force field in teleoperation. 
5. Supervision by human operator to perform 
robotic operation and to tradekhare between 
robotic operation and teleoperation. 
6. Automatic task planning and command 
generation: a must for robotic operation, and as 
a suggestive direction for teleoperation. 
7. On-line manipulator path planning: a must 
for robotic operation, and as an aide for 
teleoperation. 
8 .  Automatic tracking of labelled and 
unlabelled objects. 
9. Operator initiated object designation and 
subsequent automatic verification of modeled 
objects. 
10. Simulation of a delay of up to 500 
milliseconds (round trip) between the (local) 
operator 's stat ion and the (remote) 
manipulators. 
11. Integrated system operation; ease of 
o perator-mac h ine interaction. 
These capabilities are to be demonstrated by 
performing laboratory tasks simulating those 
encountered in servicing satellites in orbit. 
Typically, they include coordinated two-arm 
manipulation of a large module, an ORU (Orbit 
Replacement Unit), and include the grappling/ 
halting of a rotating satellite. Dexterous 
operations in terms of removal of panels, bolts, 
electrical connectors, tool exchange, object 
manipulation with precisely defined or loosely 
defined data bases, are in the list of 
demonstrations. 
I OCS AS PART OF THE TELEROBOT SYSTEM 
Figure 1 shows the current JPL Telerobot 
Demonstrator System. It is shown with a 
generic OCS which is to be replaced by a state- 
of-the-art OCS in mid-1989. 
The Telerobot System has the following 
subsystems: 
1. Operator Control Station (OCS) 
2. Artificial Intelligence Planner (AIP) 
3. Run-Time Control (RTC) 
4. Sensing and Perception (S&P) 
5. Manipulators and Control Mechanization 
(MCM), including Teleop components 
6. System Executive (SE) 
This above partitioning is driven by the 
identification and separation of system 
functions, specific technology, hardware and 
software. (For more specifics, refer to [2]). 
For a user-friendly and machine-interactive 
operation, the OCS discussed in this paper 
actually crosses some boundaries in the above 
system partition. The crossed boundaries 
include AIP, SE and MCM/Teleoperation 
components. It will be clear in the following 
sections that the user interface now placed in 
AIP, the system management in SE and the hand 
contro l ler  hardware in  Teleoperat ion 
subsystems are truly part of an overall Operator 
Control Station; but for reasons of subsystem 
maturity, component availability/compatibility, 
and for working group partitioning efficiency, 
these AIP, SE, and Teleop functions have been 
allocated outside the OCS. 
OCS's OPERATOR REQUIREMENTS 
The human Operator uses the OCS to interface 
with the system. The Operator manages system 
configuration, transmits system information and 
recieves feedback from the System. The OCS 
provides capability for the Operator to 
coordinate and monitor all other subsystems, 
permits the Operator to direct/supervise/ 
execute robotic and teleoperation control. In the 
JPL Telerobot Demonstrator System, OCS is 
designed for two operators, the Main Operator 
and the Auxiliary Operator (also known as the 
Test Conductor). The Main Operator has the 
capability to execute all functions regardless of 
the absencelpresence of the Auxiliary Operator. 
The Operator's functions include the following: 
- System management functions: system 
s t a r t u p / s h u t d o w n ,  s e t u p ,  s o f t w a r e  
configuration, other monitoring/diagnostics 
functions; 
- System operation functions: mode 
transitions, setting system parameters, system 
calibration, video switching, emergency halt 
(and other modes of halting), object data 
438 
manipulation; 
- Teleoperation functions: hand motion for 
input to hand controllers, setting subsystem 
parameters, establishing telepresence via 
visual, kinesthetic and proprioceptive feeback; 
- Robotic control functions: instantiate, 
monitor, supervise, direct, confirm and give 
permission to proceed all actions generated 
under autonomous planning; 
- Trading and sharing teleoperated and 
robotic controls; 
- Initiating and executing data logging 
functions for off-line analysis and system 
performance evaluation. 
OCS HARDWARE REQUIREMENT 
The OCS hardware is a station, in a "controlled" 
room environment where lighting, sound and 
sight are controllable, and houses the Operator 
and Test Conductor. The station is equipped 
with multiple monitors for video and graphics 
displays and mixing. Audio and voice 
input/output systems are provided for operator 
command inputs in addition to keyboard inputs. 
Mechanical input devices for teleoperation and 
shared robotic/teleoperation are provided. 
Multiple processors and computer networking is 
provided for OCS functions, planning functions, 
and system management functions. The OCS is 
des igned wi th  due considerat ion in  
anthropometric and ergonomic constraints. 
Specifically, the requirements are: 
1. Be configured with workstations for an 
Operator and a Test Conductor; 
2. The main workstation to have al l  
necessary control inputs and feedback; 
3. At the main workstation, to provide the 
following devices for display, record and 
transmit data: 
(a) mono, stereo video monitors for camera 
and buffer images 
(b) video switcher and mixer(s) 
(c) indicator lights, aural signals 
(d) synthesized voice 
(e) video recorder(s), audio mixer 
( f )  high resolution bit-mapped video display 
monitors; 
4. At the main workstation, to provide the 
following devices to accept Operator requests 
and commands: 
(a) keyboard(s), function switches 
(b) one left and one right bilateral force 
reflecting 6-dof hand controllers; the mechan- 
isms and electronics 
(c) voice in p u thecog n i zer 
(d) mouse(s) and joystick(s); 
5. At the auxiliary workstation (of the Test 
(a) all datahiews available at the main 
(b) peripheral data collection hardware for 
(c) keyboard function switches independent 
6. To provide the following physical and 
(a) regulated electrical power 
(b) VCR recorder 
(c) terminal/keyboard interface to the AIP 
(for autonomous and shared control  
instantiation, for object designation process) 
(d) terminal/keyboard interface to the SE (for 
executing system management functions); 
7. To provide display system to permit: 
(a) terminal emulation through windowing to 
allow the Operator to access each subsystem 
(b) switching of video images onto designated 
monitors; 
8. At both the main and auxi l iary 
workstations, to provide a "Panic Button" for 
emergency halt and other mode(s) of halt; 
9. To provide networking capability to all 
subsystems; 
10. To provide electronics, hardware, 
switches to permit teleoperation mode changes, 
indexing, triggering for robot end-effector 
opening and closing, force reflection, and other 
control processing for the hand controllers; 
11. In support of Operator-assisted object 
designation and verification, to provide: 
(a) mouse/keyboard/voice input for selection 
of objects 
(b) graphics generation, mixing capability to 
overlay wire frame models of objects onto video 
images; 
12. To provide the OCS computer(s) to 
perform OCS internal and interface functions. 
Conductor), to provide: 
workstation except the stereo view 
off-line data analysis 
of main workstation; 
electrical support: 
Figures 2 and 3 show the control station 
configuration of the JPL Telerobot Demonstrator 
System. Figure 4 show the functional block 
diagram of the OCS, illustrating the interface 
with all the subsystems. 
OCS SOFTWARE REQUIREMENTS 
Software required in OCS includes the 
processing of OCS input/output data; interface 
software with other subsystems; the system 
management software (currently allocated to 
SE); and the system mode switching/ 
439 
I 
I 
supervision, object-data-manipulation, man- 
machine-interface (currently allocated to AIP); 
and hand controller teleoperation software 
(currently allocated to MCM/Teleop). Specific 
software requirements are to provide: 
1. Command interpreter to process Operator 
generated commands via the keyboard, mouse, 
and voice; hence, to parse, translate and 
generate inter- and intra-OCS commands; 
2. Message processor for translating, 
generating and displaying messages on OCS 
monitor; for messages initiated from within or 
outside OCS; 
3. Status monitoring process, to monitor 
OCS health and parameter-setting status; in 
addition to the overall system and subsystem 
status monitoring process performed by the SE; 
4. RS-232 contro l lers  for graphics 
over lay/mixer control ler, voice display 
controller, and video switch controller; 
5. Gateway computer interface via an 
ethernet network; 
6. NIP (a custom Network Interface Package) 
gateway interface software for processing NIP 
transactions [3]. 
7. Object designation/definition software to 
create wire-frame models, overlay on camera 
images, manipulate using mouse cursors, 
perform best-fit, and update/augment data 
bases; to interface with the AIP for model/data; 
8. Interface software between the primary 
and auxiliary OCS workstations; 
9. Terminal emulation to all subsystems via 
multiple windows on OCS monitor; 
10. Pull-down menus for system and 
subsystem commands; 
1 1 .  Continuous speech voice recognition and 
mu It i-vo ice (g e nde r/pe rso n ) speech s y n t h es is 
for direct input and feedback; 
12. Graphics generation and overlay software 
for object designation/verification process; 
13. Graphics and simulation software for 
special displays including predictive displays, 
scenario simulation, sequence playback displays; 
14. Software to accept keyboard/mouse/ 
symbolic/graphics input, instantiate processes, 
specify goals and task sequences, as specified 
by Operator; 
15. Software to allow operator to display/ 
supervise/update/modify/cancel task sequences 
and parameters; 
16. Software and displays for smooth 
transition between system modes, including 
teleoperation mode to and from robotic mode; 
17. Data base maintenance, management, and 
creation of world models, object location, 
camera models, and calibration settings; 
18. Support software for interpreting and 
transmitting Operator generated (voice or 
keyboard) manipulator control commands; 
maintain command context and monitor status of 
commands; 
19. System configuration, startup/shutdown/ 
halt, statusing, health monitoring, data traffic 
monitoring of all subsystems; 
20. Control processing and data acquistion 
software to process Teleoperation hand 
controller joint data, special switches, triggers, 
force feedback, coordinate transformation and 
communciation; 
21. Graphics processing/generation of 
Teleoperation related displays including 
forcehorque data and predictive data. 
Requirements #1-13 are currently allocated to 
the OCS of the JPL Telerobot Demonstrator 
System, while #14-18 are allocated to the AIP, 
#19 to SE, and #20-21 to the MCMITeleop for 
reasons mentioned in an earlier Section of this 
paper. 
OCS in the NASREM ARCHITECTURE 
The NASA/NBS Standard Reference Model 
(NASREM) [4] for telerobot systems functionally 
partitions the telerobot process into six levels, 
where each level has its sensory processing, 
world modelling and task decomposition 
subprocesses. Levels interconnect with the 
level below and the level above it. All levels 
connect with a 'global data base' and an 'operator 
interface'. 
The OCS described in this paper, Le. the physical 
station and its functions, map right into the 
'operator interface' block in the NASREM 
architecture. A few of the functions, 
particularly those now allocated to AIP, SE, and 
MCM, but considered above to be OCS functions, 
overflow out of the NASREM 'operator interface' 
block. Figure 5 (excerpted from [5) ]  shows the 
mapping of the JPL Telerobot Demonstrator 
System into the NASREM. 
FUTURE RESEARCH and DEVELOPMENT 
Upon its completion, installation and integration 
in mid 1989 with the rest of the System, the 
OCS will serve as the focal point of the 
Telerobot Demonstration System. Real hands-on 
operational flow analysis and workload analysis 
could actually conducted, to evaluate the 
440 
effectiveness of the OCS design. 
More research and development items, 
improvements on point-deisgns, alterations of 
physical layout, addition of vocabulary, etc. will 
undoubtedly surface when more experience is 
gained from OCS experiments. Other already 
forseen technology development items include: 
interactive model/data base building; the use of 
CAD-type data base techniques for object 
trajectories planning/verification; faster and 
better algorithms in obect designation process, 
including hidden I i ne removal, incorporation of 
perspective, cues etc.; smoother and more 
unif ied oprator-machine interface; more 
powerful display/graphics systems. As more 
powerful computers become available, and as 
understanding of a telerobot system matures, 
the state-of-the-art OCS technology will evolve. 
ACKNOWLEDGEMENT 
This research was carried out at the Jet 
Propulsion Laboratory, California Institute of 
Technology, under contract with the National 
Aeronautics and Space Administration. 
REFERENCES 
P. Schenker, "Telerobot Program Objectives 
and Technology Outreach," Proceed' inas of 
jhe WorkshoD o n Soace Telerobotics, 
Pasadena, CA, Jan. 1987 (also JPL 
Publication 87-13, Vol. 1, pp. 3-17, July 
1987). 
J. Matijevic, et.al., "Functional Requirements 
for the Telerobotic Testbed," Jet ProDulsion 
m r v  D o c u W ,  #JPL D-3693, May 
1988. 
R. Cain, et.al., "Network Interface Package 
(NIP) User's Guide," SRI International, 
Contract #957908 Report to JPL, also SRI 
Project #3528, Nov. 1987. 
J. Albus, H. McCain, R. Lumia, "NASA/NBS 
Standard Reference Model for Telerobot 
Con t ro I S y s t e m Arch i tec t u re ( N AS R E M ) , " 
National Bureau of Standards, NBS Technical 
Note 1235, Jun. 1987. 
W. Keksz and J. Matijevic, "NASREM and JPL 
Te I e ro bo t i c Test bed, " Jet P ro p u Is i o n 
Laboratory Internal Publication 347-88- 
319, May 1988. 
-- 
AI 1 1 1 1  I I I I  
l u Q b  muxa / 
*ID., u
F M  ORIGINAL PAGE IS 
OF POOR QUALm 
TOP MEW 
442 
\ + Stop Switch + ETHERNET/TERMlNAL EMULATION (5) 
D ~ P ~ Y S  (2) 
PRIMARY OCS Monitor/ video 
WORKSTATION Keybd/Mouse Switches 
- 
2 
2 
---*E 
!F 
c- t S & P  SUBSYSTEM 
F r r  (2) 
4 > 
2 Wing (3) 4- 
SECONDARY 4 Stereo (2)  OCS PROCESSOR a 
TELEOP 
SUBSYSTEM 
OCS Monitor/ 
KeybdlMouse 
4 
SUBSYSTEM 
COMPUTER 
SUBSYSTEM 
Video Recorder 
Keyboard/Mouse SECONDARY 
WORKSTATION 
I Printer I COMM. NETWORK 
MICROVAX /VMS 1 7 - 1  SUBSYSTEM 
I I 
MCM 
SUBSYSTEM 
Power 4 Status 
Indicators Distribution 
Reby Control 
IIi'dgtmiPe 4, SFELEROBOT OPIEWASFOW CONTROL STATION 
FUNCTIONAL BLOCK DIAGRAM 
TASK 
I I S E R V O  I 
I WORLD 
WORLD 
MODELLINQ 
TASK 
DECOMPOSITION 
r 
c 
m . 
r 
< 
B 
Y 
r 
m 
c 
u 
r m c m r- 
r 
c 
B 
0 


Telerobot operator control station requirements

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server