The recent developments of techniques which assist an operator in the control of remote robotic systems are described. In particular, applications are aimed at two specific scenarios: The control of remote robot manipulators; and motion planning for remote transporter vehicles. Common to both applications is the use of realistic computer graphics images which provide the operator with pertinent information. The specific system developments for several recently completed and ongoing telepresence research projects are described

Chiang, Shih-Chien

Crane, Carl D., III

Duffy, Joseph

Vora, Rajul

English

NASA Technical Reports Server

N90_2 c :,.:,
Telepresence System Development for
Application to the Control of Remote Robotic Systems
Carl D. Crane III, Joseph Duffy
Rajul Vora, Shih-Chien Chiaug
Center for Intelligent Machines and Robotics
University of Florida
Gainesville, Florida 32611
I. ABSTRACT
This paper describes the recent developments of techniques which assist an operator in the control of
remote robotic systems. In particular, applications are aimed at two specific scenarios: (1) the control of remote
robot manipulators; and (2) motion planning for remote transporter vehicles. Common to both applications is the
use of realistic computer graphics images which provide the operator with pertinent information. This paper will
describe the specific system developments for several recently completed and ongoing telepresence research projects
in the Center for Intelligent Machines and Robotics (CIMAR) at the University of Florida.
II. INTRODUCTION
Significant advances have been made in a broad spectrum of component technologies such as kinematics
and dynamics, control, vision and pattern recognition, obstacle avoidance, computer graphics, and artificial
intelligence. Each of these technologies can individually impact on an operator's efficiency in the control of a
remote manipulator or transporter. An objective of telepresence systems is to combine these component
technologies in order to provide the operator with multiple sensory feedback data. In this manner an operator
can in effect directly experience the environment of the remote manipulator system. The abundance of sensory
feedback coupled with computer assisted operation (a low level of autonomy) is what distinguishes telepresence
from earlier teleoperated systems.
The operational scenario of remote systems may vary significantly from case to case. In some situations,
the operator may be required to control the remote system in real time for which it is necessary for the remote
system to respond directly to operator input. In many circumstances, however, a real time response is not possible
or desired. A significant communication time delay, for example, can make real time control awkward at best 1_3'.
A second classification can be made regarding the operational environment. Clearly, it is necessary to know whether
or not the operational environment of the remote system is known a priori. Knowledge of an accurate model of
the environment can significantly impact the type of telepresence system which is to be implemented. Operation
in a structured environment such as a nuclear power plant t21 is considerably different from controlling a remote
transporter in the field.
'Numbers in brackets refer to references at end of paper.
201
https://ntrs.nasa.gov/search.jsp?R=19900020483 2020-03-19T21:30:48+00:00Z
Thispaperwill describeseveralprojectsconductedat CIMARwhichalldealwiththecontrolof remote
manipulatoror transportersystems.Foreachcase,a scenarioisestablishedwhere:(1) theoperatoris either
directlycontrollingthesystemin realtimeor isperforminganoff-lineplanningtask;and(2) theenvironmental
descriptivemodelisknownapriorior thesystemisoperatinginatotallyunknownarea.
III. CONTROL OF REMOTE ROBOT MANIPULATORS
A. Initial TeleDresenc¢ System f3j
The first telepresence system developed at CIMAR was concerned with the real time control of a remote
manipulator in an unstructured environment. It was assumed that the operator could not directly see the
manipulator or the objects in its workspace as he was directly maneuvering the robot with a joystick device. The
task for the operator was to acquire and move several cubes in the workspace without coming into contact with a
sphere.
The primary sensory feedback required to perform this task is vision. In many instances, vision alone would
give the operator sufficient information in order to manipulate the cubes without contacting the sphere. Logically,
this vision could be provided by video cameras which surround the workspace of the manipulator or which are
attached to the manipulator itself. It should be noted, however, that the use of video cameras to provide vision
feedback does have certain disadvantages. Primarily, more than one camera must be used to provide sufficient
viewing positions. Each of these cameras will most likely have at least three degrees of freedom (zoom, tilt, and
pan) thereby giving the operator numerous additional parameters to control. Furthermore, environmental conditions
may cause the video image to be blurred or poor lighting and contrast may make the image confusing and unclear.
Because of these limitations, the goal of this project was to evaluate the operator's performance when the
vision feedback was provided via a computer graphics display. The use of computer graphics offers three distinct
advantages. First, images are clear and sharp since colors and contrast can be selected and programmed. Second,
the computer graphics system allows for the viewing of the image from any desired vantage point. This ability
removes the requirement for a multitude of video cameras and allows the operator to focus attention on only one
monitor screen. Third, the computer graphics system can provide additional feedback to the operator concerning
imminent collisions. When the manipulator was moved close to the spherical obstacle, the color of the sphere was
changed and an audio tone was emitted. This additional feedback regarding collisions significantly improved
operator confidence and performance. The collision warning feature was implemented by placing the obstacle in
an imaginary rectangular protective box. The graphics system in effect compares the image of the robot with this
protective box and determines in real time whether any part of the robot is in contact.
A photograph of the initial telepresence system is shown in Figure 1 while the system configuration is shown
in Figure 2. The system consisted of the following four component technologies:
(1) Robot manipulator.
(2) Six degree of freedom universal joystick.
(3) Obstacle detection vision system.
(4) Computer graphics system.
The manipulator was drawn on the graphics screen by communicating the joint angles from the manipulator to
the graphics workstation at every instant. The vision system was used to locate the objects in the manipulator
workspace so that they could be displayed on the graphics screen. Shown in Figure 3 is an image of the robot as
it comes close to the obstacle.
2O2
O!t!(I_NAL PAGE
BLACK AI'_D WHiTt.Z Fi*;,':L)i-CGRA, PN
Figure 1: Telepresence System
Figure 2: System Configuration
2O3
Figure 3: Warning of Imminent Collision
The initial telepresence system showed that it was feasible and practical to utilize a computer graphics
display to control a remote robot manipulator in real time in an unknown environment. The clarity of the image
coupled with the real time determination of imminent collisions with obstacles significantly improved operator
performance.
B. Robot Path Processor t41
As an extension to the previous project, an investigation of off-line task planning was conducted. For this
project it was assumed that the location of objects in the workspace of the robot were known (either from a priori
knowledge of a world model or from a sensor scan of the workspace). Off-line task planning offers an advantage
if communication delays prohibit real time operation, or if an operator can plan a task much quicker than the
manipulator can execute it. Further, off-line planning offers the operator the option to edit or modify the task
before it is sent to the manipulator for execution.
During this project, emphasis was placed on the development of the task editor. The objective was to be
able to allow the operator to rapidly enter a robot task and then be able to refine it. The resulting system consists
of the same universal joystick controller and computer graphics workstation used in the previous project together
with a PUMA 560 industrial robot. The operator moves the animated robot on the screen and then can go back
and perform such tasks as moving specific points on the path or modifying certain sections to be perfectly straight.
One problem that had to be overcome, however, was how to move the joystick in order to get a desircd
motion of the robot on the graphics screen. Confusion resulted in that the operator could look at the robot from
any viewing position and then had to mentally reference the joystick and manipulator coordinate systems (see
Figure 4). This problem was overcome by introducing an additional coordinate transformation which mapped the
joystick coordinates into the screen coordinates. The result was that a movement of the joystick to the right or left
would cause the end effector of the robot to also move to the right or left of the graphics screen, and so on. In this
2O4
Zf
Zl" Ys
Figure 4: Coordinate System Reference Frames
manner, the operator could always correlate the motion of the robot in response to any joystick move, no matter
how the scene was being viewed. This feature significantly improved the off-line planning process and was also later
applied to the real time control scenario.
C. Off-Line Programming with Autonomy 151
Off-line task planning can be significantly enhanced by the introduction of autonomy. In this project it was
again assumed that the location of objects in the workspace of the robot were known, cither from a priori knowlcdge
of a world model or from a sensor scan of the workspace. The objective of this effort was to develop algorithms
which would autonomously generate robot motions and tasks bascd on high level user inputs.
The development of motion planning algorithms which avoid obstacles was a major focus of this effort.
Shown in Figure 5 is a path which has been generated in order to move a manipulator between two user specified
positions. A significant feature of the algorithm is that resulting paths are generated rapidly (under five seconds
of computation time) [6]. The algorithms take advantage of the knowledge of the geometry of the manipulator and
its environment in order to avoid excessive heuristic searches.
Common to all off-line approaches, the operator is given the opportunity to preview the planned task on the
graphics screen prior to the manipulator beginning the task. In this manner, the operator can evaluate and critique
the performance of the autonomous algorithms.
This research program was conducted for the U.S. Army Belvoir R,D,&E Center, Fort Bclw)ir, Virginia.
The specific application for this project involved logistics operations and is shown in Figure 6. A camera on board
the fork attachment was able to locate a target mounted on a pallet and then calculate the position and orientation
of the pallet relative to the camera. With this information, the operator would be able to acquire thc pallet and
maneuver it off-line via the graphics animation system. Once a task was planned, prcvicwcd, and approved, the
manipulator would be commanded to perform the operation.
205
Figure 5: Autonomously Generated Path
BLACK A;,,D ',,:r-;iiL Fb;b i-OGRAPH
Figure 6: Logistics Application
206
IV. MOTION PLANNING FOR REMOTE TRANSPORTER VEHICLES
Many of the man/machine interface control issues associated with the operation of remote transporter
systems are directly analogous to those already discussed for manipulator systems. In particular, the transporter
operation can be categorized according to whether or not it moves in real time under operator control and as to
whether or not a prior knowledge of the environment is known.
The work in this area is sponsored by the U.S. Department of Energy. The University of Florida, in
cooperation with the Oak Ridge National Laboratory and the Universities of Michigan, Tennessee, and Texas is
developing an advanced robotic system under the University Program for Robotics for Advanced Reactors. As such,
the developments in this area are specifically aimed at the operation of a transporter vehicle in a nuclear power plant
environment where it is assumed that an accurate world descriptive model exists.
A. Development of an Articulated Transporter/Manipulator System (ATMS] [71
The complex obstacle strewn environment of a nuclear power plant requires that the robotic system be
capable of crossing or jumping over substantially sized obstacles. Transport mechanisms consisting of combinations
of wheels, tracks, and legs were considered as candidates, but the inherent mobility limitations of these mechanisms
led to the selection of a transporter comprised of multiple articulated segments (see Figure 7).
The ATMS is comprised of eighteen individual segments which provide both maneuverability and locomotion.
Each segment is twenty four inches in length, and has a pair of motor driven wheels to provide traction for forward
and reverse motion. Segments are connected in series by a pair of revolute joint axes. The significant feature of
the design is that the ATMS will be able to cross over horizontal gaps of up to twelve feet in length.
Figure 7: ATMS Model
207
,-u',,_, ',",'t; .. _-i_i_ i-O(_RAPH
Figure8:AnimatedRepresentation of ATMS
Initial control of the ATMS has been established by having an operator specify the direction and velocity
of the lead segment. Subsequent segments of the ATMS will follow the path of the segment directly in front of
it (see Figure 8). Work to date includes the development of follow-the-leader algorithms for vertical and horizontal
navigation. Further, a series of stability rules have been generated which govern the effective maneuverability of
the system. During a man-controlled real time operation of the system, the stability rules will be continuously
evaluated and monitored, and the operator will be warned of potentially unstable actions. This combination of direct
man-controlled operation with computer assistance is a fundamental principle of the telepresence concept. In
addition to the real time human control, off-line autonomous control algorithms axe currently being implemented.
B. HERMIES Implementation
A fully operational version of the ATMS has yet to be fabricated. In order to demonstrate the progress of
the research program, the HERMIES robotic transporter at the Oak Ridge National Laboratory is being used as
a system demonstrator (see Figure 9). HERMIES is limited to performing planar motion. However, many of the
man-machine control issues which will involve the ATMS can be resolved via HERMIES.
The system demonstration combines off-line and real time operation. During the planning phase, thc
operator can plan a motion path via (1) the computer graphics system alone (Univ. of Florida), (2) a three degrcc
of freedom force reflecting joystick (Univ. of Texas), or (3) an autonomous path planning algorithm (Univ. of
Michigan). Once a path is planned, the operator is able to review it and modify it if necessary prior to HERMIES
beginning its motion. Figure 10 shows the animated display provided to the user when a path is being planned.
Three interactive windows are provided which present a top view, the view from HERMIES, and the view from a
user controlled camera located anywhere in the environment.
An important aspect of the planning phase is the time acceleration concept lsJ. The operator can effectively
specify a time scale factor so that the planning phase can be accomplished in an accelerated time scale. For
example, the ATMS can achieve a maximum velocity of only two feet per second. By selecting an accelcrated time
2O8
Figure 9: HERMIES Mobile R_bot
Figure 10: Operator's Console lmagc
209
scale, the operator will be able to perform the motion planning as if the ATMS had an unlimited maximum velocity.
Upon the actual motion of the ATMS, the plan would be reconstructed in the real time domain.
V. CONCLUSION
Several recent and ongoing research programs have been described which all deal with some aspect of
telepresence system development. Varying techniques have been discussed which assist an operator in the control
of remote robotic systems either in real time or off-line, or when a prior knowledge of the environment is known
or not.
It appears evident that any resulting telepresence system must be able to perform in all of the operational
categories. For example, although a nuclear power plant may be a structured environment for navigation purposes,
the environment can become unstructured during a manipulation task. Because of this, the generic techniques that
were developed here can be combined together in order to offer the operator a highly sophisticated man-machine
interface.
VI. ACKNOWLEDGEMENTS
The authors wish to acknowledge the contributions made by other current and former members of the
Center for Intelligent Machines and Robotics, Dr. R. Harrell, Dr.J. Staudhammer, Mr. Y. Choi, and
Mr. V. Chesney. Significant contributions were also made by Mr. M. Locke of the U.S. Army Bclvoir R,D,&E
Center. Further the authors wish to acknowledge the financial support of the U.S. Department of Energy, U.S.
Army Bclvoir R,D,&E Center, McDonnell Douglas Astronautics Company, and of the Military Avionics Division
of Honeywell.
VII.
I11
121
I31
I41
I51
161
171
[81
REFERENCES
L. Conway, et al., "Tcle-Autonomous Systems: Methods and Architectures for Intermingling Autonomous
and Tclerobotic Technology," 1987 IEEE International Conference on Robotics and Automation, Raleigh,
N.C.
J.S. Tulenko, et al., "An Advanced Semi-Autonomous Robotic System for Hazardous Response Work at
Nuclear Power Stations," Proceedings of the Winter Meeting of the ANS, Los Angeles, Ca., January 1987.
C.D. Crane and J. Duffy, "An Interactive Animated Display of Man-Controlled and Autonomous Robots,"
Proceedings of 1986 ASME Computers in Engineering Conference, Chicago, July 1986.
C.D. Crane, Y. Choi, G. Matthew, "Interactive Off-Line Robot Path Processor," Proceedings of the 1988
ASME Computers in Engineering Conference, San Francisco, July 1988.
C.D. Crane, J. Duffy, and M. Locke, "Off-Line Programming and Path Generation for Robot Manipulators,"
Intelligent Systems for Army Logistics Support and Combat Engineering Symposium, Ft. Belvoir, Va., 25-27
March 1986.
C.S. Tseng, "Rapid Generation of Collision-Free Paths for Robot Manipulators with Computer Graphics
Animation," Ph.D. Dissertation, University of Florida 1987.
C.D. Crane, S. Chiang, et ai., "Algorithm Development and Computer Graphics Simulation of an Articulated
Transporter/Manipulator System," Proceedings of the 1988 ASME Computers in Engineering Conference,
San Francisco, July 1988.
C.D. Crane, J. Tulenko, et al., "Faster Than Real Time Robot Simulation for Plan Development and Robot
Safety," submitted for publication, 1989 IEEE International Conference on Robotics and Automation.
210
ROBOTIC VISION
211



Telepresence system development for application to the control of remote robotic systems

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server