A system for supervisory control is described, and preliminary results are presented. Supervisory control, where control is traded between man and computer, offers benefits in the control of a remote manipulator. The system has the potential to accomplish sophisticated tasks. It is indicated that supervisory control yields lower task completion times and is preferred over manual control

Starr, G. P.

English

NASA Technical Reports Server

N82-1967,
SUPERVISORYCONTROLOF REMOTEMANIPULATION:
A PRELIMINARYEVALUATION
GregoryP. Starr
b
Departmentof MechanicalEngineering
The Universityof New Mexico
Albuquerque,New Mexico87131 USA
ABSTRACT
Supervisorycontrol,where controlis tradedbetweenman and computer,
may offerbenefitsin the controlof a remotemanipulator.A systemfor the
studyof supervisorycontrolis described,and some preliminaryresultspre-
sented.
I. INTRODUCTION
Man's technologicalrisehas been accompaniedby an increasingneed for
him to work in hostileenvironments.Nuclearwaste disposalsites,radio-
activelaboratories,the depthsof the ocean,the vacuumof outer space,and
undergroundmines are examplesof suchenvironments.Teleoperatorsystems
projectman'smanipulatorycapabilitiesintothe remoteenvironment,allow-
ing his functionalpresencewithouthis physicalpresence.
The need for teleoperatorsimpliesoperatingconditionswhich exclude
or impairvisualor other humansensorycontactbetweenthe operatorand the
manipulator.The barrierimposedby the hazardousenvironmentcarrieswith
it limitson both sensorand control_ommunication.
It is possible,fora man and computerto cooperatein the controlof a
remotemanipulator,and achievea standardof performancebeyondthatpossi-
ble by eitheralone. This mode of controlhas been termedsupervisorycon-
trol [13.
The firstmanipulatorsweremaster-slavesused for radioactivemateri-
als handling. They providedbilateralforce feedback,and the operator
viewedthe site directlya few feet away. In this environmentthe operator
was capableof precisepositioningand finecontrolof appliedforces. How-
ever, as the linkagebetweenhumanoperatorand manipulatorwas made elec-
trical,and the distanceincreased,the impairmentof sensoryfeedbackmade
controlmore difficult.
c The firstsensorydegradationthatwas investigatedwas a transmission
-95-
1982005792-103
https://ntrs.nasa.gov/search.jsp?R=19820005802 2020-03-21T15:55:44+00:00Z
time-delay between the operator's commandand the manipulator's response. A
time-delay can arise from extreme separation (outer space manipulation), _r
sensor/displaylimitationssuch as limitedvideo frame rate.
Ferrjll[2], Black [3],Hill and Sword [4], and Starr [5],experimen-
tally verifiedthe negativeeffectsof time-delayon manipulation. Simple
• tasks becamefrustratingand laborious,and manipulationrequiringhigh -
accuracywas virtuallyimpossible.
Supervisorycontrolmay _llowmanipul_!on to be performedeffectively
even when a tlme-del_);_ pFesent. The philosophyof supervisorycontrolis
thatthe human operatorplansstrategy,monitorsperformance,and intervenes
when necessary,while the computeraccomplishesportionsof the task as
instructedby the human. Supervisorycontrolallocatescontrolresponsi-
bilitybetweenman and computers_chthat the inherentattributesoF each
are usedto best ddvantage.
Figure1 shows diagramsof manualcontroland supervisorycontrolof a
remotemanipulatorsystem. Undermanualcontrolthe operator'scommandsare
sent directlyto the manipulator,and feedbackfrom its sensorsis displayed I
directlyto him. Under supervisorycontrolthe operator'scommandsare
transmittedto a remotecomputer,which then commandsthe manipulator. The
remotecomputeralsoprocessesthe sensoryinformationfrom the manipulator
and _elaysit to the operator. Sincethe remotecomputeris "on site" it is
R_OI@
,¢wt r¢H_m4HW
f'_) I,' ,=_
kI&NUAL CONTROL ."
Recto
I_ I t
oem'llot
SUPERVISORY CONTROL]1
FigureI
)
t
-96-
1982005792-104
mnot subjectto the same feedbackdegradationas the human operator. TKe
portionof the systemcomprisedby the manipulatorand remotecomputercan
accomplishportionsof the taskon its own. The human functionsprincipally
as coordinatorand supervisor,but may still controlthe manipulator
directlyif necessary.
II. SYSTEMOVERVIEW
Figure2 shows a block diagramof the system. There are threeseoarate
digitalcomputers,eachwith its on responsibilities.The ManualCortrol
Computer(MCC)processesmanualcontrolsignalsoriginatingfrom the human
operator. The Arm ControlComputer(ACC) performsarm trajectorycalcula-
tion,monitorsthe jaw-mountedtactilesensors,and drivesthe arm through
Joystick ] Manual
[ _ Cant,o, ;
Humon Arm _
Comuter I
j Supervi|ory
, _ CRT Control ]k '
l(eyboerd _ -Computer -'-
Tactile
Servos
_]'_ ) Arm
BLOCK DIAGRAM OF SUPERVISORY
CONTROL SYSTEM
Figure2
desiredpositions. The SupervisoryControlComputer(SCC)monitorsthe
sensors,sendscommandsto the ACC, and performsquantitativeanalysisof
arm positionsand sensordata.
, The SCC and ACC are coupledby a 9600 band serialcommunicationslink.
The ACC executesthe VAL languagewhile the SCC uses the RT-11 operating
system,with programswrittenin FORTRANor assemblerlanguage.
-97-
#
1982005792-105
Example of Syst_ Operation. Although the individual elements in the
system win be described later, the following alignment routine (used in the
experiments) is an example of how they work together.
Consider the operation cf removing a nut from a st,d. when the plane on
which the stud is fastened is arbitrarily oriented and the manipulator hand
. is "far away" from the nut. Before a pre-programmed nut-removal routine can
be executed, the nut must be approached, and the hand must be "lined up"
with the axis of the nut, i.e., normal to the plate. The sequence of oper-
ations is as follows (see Figure 3):
1. The human operator brings the hand to the vicinity of the nut, man-
ually orients the hand to be roughly normal to the plate, then instructs the
SCC to execute the "ALIGN" routine. The SCC commands the ACC to execute its
stored alignment routine. The ACC advances the hand in the direction it is
pointing until it t_uches the plate at location I.
I. MAN MOVES ARM TO TOUCH AT "1" .
2. COMPUTER MOVES ARM TO TOUCH AT 12" ANDw3e
\
3. VECTOR ; • A x B COMPUTED
4. COMPUTER ORIENTS HANO ALONe C
m
5. CONTROL _ETURNIrO TO MAN
ALIGNING HANO WITH BOLT AXIS
a
Figure 3
-98-.
1982005792-106
2. The ACCdetects the touch, stores the position where the touch
occurred, and moves the hand to touch the plate at locations 2 and 3, stor-
ing these locations.
3. The three locations define vectors A and B, from which vector C is
computed by the ACC.
4. The ACCdrives the hand to point along C. The hand is now aligned
with the nut axis.
5. The SCC notifies the human operator that the hand is aligned, and
displays the options for continuation, which may include resumption of man-
ual control, or invocation of another computer routine.
The remainder of the paper will be devoted to describing the elements of time
system and discussing a preliminary experiment and results.
Ill. MANIPULATOR
The manipulator is a PUMA 600 manufactured by Uni_,lation,Inc. The arm,
shown in Figure 4, has six revolute joints, each powered by a DC servomotor.
The arm has a position repeatibility of 0.1 mm (0.004 in.). It can apply a
static force of 58.0 N (13.0 lb). Its length is approximately one meter.
Each arm joint is under the local control of an R6503 8-bit micropro-
cessor, which receives commands from the Arm Control Computer (ACC), a DEC
LSI-11. Each 6503 has its own PROM, RAM, and I/O logic, and servos its
joint via a D/A converter and shaft encoder.
Figure 4
-99-
1982005792-107
IV. MANUALCONTROLSYSTEM
Resolvedmotionrate control(RMRC)is the manualcontrolmode. With
RMRC, the operatoruses a six degree-of-freedomisometricjoystickto speci-
• fy velocitycomponentsof the hand along axesof a hand-mountedcartesian
coordinatesystem. The hand-mountedcoordinatesystemis shown in Figure5.
The six degree-of-freedomisometricjoystick,shown in Figure6, pro-
duces six outputsignalsproportionalto the six appliedforcesand torques.
The magnitudeof a force/torquec_nponentdeterminesthe valueof corres-
pondingtranslational/rotationalvelocitycomponent. Thus, if the operator
LIFT
W
PITCH_ ROL_
SWEEPjwr REACH
HAND - MOUNTED
COORDINATE SYSTEM
Figure 5
pushesforwardon the Joystick,the handmoves forward(in the directionit
is pointing);if he pushesdown, it movesdown, etc. The coordinatetrans-
formationsnecessaryto drive the hand alongthe cartesiandirectionsare
performedby the Arm _ntrol Computer(ACC),which computesarm positions
each 28 msec.
The six velocitycommandsfrom the Joystickdo not go directlyto the
" ACC, but are firstprocessedby the Motorola6800-basedManualControl
Computer(MCC). The MCC doesthe following:
-I00-
_ j
ii
1982005792-108
i
f
ORIGINALPAGE IS
OF POOR O;IALITY
\
6 DOF JOYSTICK
Figure6
I. Providesnonlinearjoystickforce/handvelocityrelationship.To
get good_anualcontrolat l'o'whand ve'locities,an'dstillbe able'_ocommand
high speedwithoutexcessivejoystickforce,a cubic force/velocityrela-
tionshipis implemented.Done in realtime with an AMD 9511 floatingpoint
processorat a _,p_,_g rate of 50 per second,the cubic relationis shown
in Figure7. We obtainlow sensitivityfor low joystickforce,and high
sensitivityfor high force.Thus, the arm can be easilydrivenat itsmini-
mum speed,and may stillbe drivenat maximumspeedwithoutrequiringexces-
sivelylargeor smalljoystickforce.
2. Implementa time-delay. To realisticallyevaluatethe supervisory
controlmanipulationsystem,a communicationbarriermustbe imposedbetween
human operatorand arm. The most studied,and easiestto systematically
vary, is a pure time-delay.A time-delaybetweenjoystickand arm is pro-
duced by the MCC, andmay be variedfrom zero to 10 sec,with 0.02 sec reso-
lution.
3. Joystickdeadbands. Sincevelocitycontrolis an integratingpro-
cess, any hystereticJoysticksignalafterthe operatorreleasesthe joy-
stickwill cause undesiredarm motion. Softwaredeadbandsin the MCC pre-
vent this.
4. Computercontroltakeover, k_en the oheratorwants to switchto
' comput_ controlmode, he depressesa button. On receiptof this signal,
the wICCimmediatelyswitchesthe ACC to computermode (frommanualmode),
-i01-
1982005792-109
i5' _,
J',
, I.C- '
speed
m/$ F"
O _ 2 3 4 5
force, Ib
HANDVELOCITYVERSUSJOYSTI(XFORCE
Figure7
then after0.05 sec the MCC issuesa commandto the SCC. This causesthe
SCC to eitherpromptthe operatorfor a command,or h_in a preprogrammed
activityin conjunctionwith the ACC. The 0.05 sec wait is necessaryto
a]lowfor mode-switchingtime in the ACC.
Note that even thoughthiscontrolmode is denotedmanualcontrol,it
is actuallya form of computerassistancewhere the MCC and ACC are in
serieswith the human operator.
V. SENSOR
A Jaw-mountedtactilesensoris used as the arm-basedenvironmental
sensingelement. The sensorconsistsof a numberof arraysof rectangular
plates,eachof which is sensitiveto force. The sensor/jawappearsin Fig-
ure 8. The sensorarraysare connectedto the SCC and the ACC via a multi-
, plexer,so that when a contactis sensed,both the faceof the Jaw and the
particular_lateresponsiblefor the contactare known.
-102-
1982005792-110
T
ORIGINAL PAGE IS ' ,
OF PoOR QUALITY
TACTILESENSORJAWS
Figure8
VI. COMPUTERCONTROLSYSTEM
The computerCu,Lt_; systemconsistsof two separatecomputers,the ACC
(ArmControlComputer)and the SCC (SupervisoryControlComputer). Both are
DEC LSI-11microcomputers.The ACC and the SCC work togetherin controlling
the manipul_tor. Somefunctionsperformedby eachmachineare:
Supervisor_ ControlComputerISCC):
I. Can cause executionof a given ACC arm controlprogram.
2. Read and store arm positions,numericallyanalyzethem,take
action.
3. Scan sensor,take appropriateactionon contact.
4. Displaypromptingor statusinformationto humanoperator.
Arm ControlComputerIACC):
1. Calculatetrajectories,drive arm at desiredspeed.
2. Modifystoredarm positionsto be relativeto new reference.
3. Read sensor,use data to governprogrambranching,causeJun_)to
subroutine.
4. Executestoredroutineuponrequestfrom SCC.
, While some functionsare commonto both, the SCC controlsthe arm at
a higher level,selectingACC programsto executed,analyzingpositions,
t
-103-
t
1982005792-111
?communicatingwith the operator. The ACC does the lower-level_r_ cnntrol, _,
calculatingtrajectoriesand issuingjoint commands. ;i_
One extremelyusefulcapabili_.yuf the ACC is thatof relativeposi- _j
tions. Preprogrammedroutinescan be "taught"in the laboratory,w;th all ,,
positionsbeingdefinedrelativeto somereferenceposition("position" :_
implieshere a six-dimensionalquantityincludingpositionand orientation), _
which is alsodefinedat this time. When the preprogrammedroutineis ;!
invokedin the field,the referencelocationwill in generalbe different _
tha_lthat in the laboratory.The first instructionin the ACC routinecan _
be a redefinitionof the referencepositionto that which existsat that
time. Upon executionof the routine,the ACC transformsall rela_si-
t--l-on-sto be relativeto the newlydefinedreferenceposition. Thisyield i_"_
the samerelativehandmotionregardlessof the hand orientationat the time
the preprogrammedroutineis executed.
The ACC executesthe VAL language,an interpretivelanguagewhich
residesin 161(EPROM. At the user level,VAL is basedon commonEnglish _
" "draw,"etc , simplifyingprogramdevelop- _:_i_languagewords,such as "move,
ment. :_
' i!i
The SCC uses the RT-II operatingsystem,and executesprogramswritten i
in FORFRANor assemblerlanguage. Communicationsbetweenthe ACC and SCC
scanningof the tactilesensor,and otherutilityfunctionsare done by _
assemblerlanguagesubprograms.Quantitativeprocessingof arm positionand _
sensorinformationcan be done in FORTRAN. It shoulabe noted that the SCC
and ACC work asynchronously,and whilethe SCC is processingdata the ACC is
stillcontrollingthe arm.
To concl_,dethis section,the nut removaltask discussedin SectionII )_
will be pursuedfurther. In SectionII, the hand was alignedwith the bolt
axis (Figure3). The secondphaseof the task is to actuallydeterminethe
locationof the a_is. lhismay be done using the tactilesensoras follows
(see Figure9):
1. After alignment,the humancan instructthe SCC to executethe .
"LOCATE"routine. The SCC thenpromptsthe operatorto switchto computer
mode.
2. The ACC moves the hand untilone faceof the jaw contactsthe nut,
storesthisposition,withdrawsthe jaw, thenmakes a contacton the oppo-
site side. This defineslineA.
3. The hand rotates90 degrees,and two more contactsare made, thus
defininglineB. The SCC determinesthe intersectionof A and B, thereby
findingthe axis.
If a time-delayis present,this taskwill likelybe muchmore quickly
done using supervisorycontrolthanmanualcontrol. In addition,a lower
,, workloadwill be imposedon the humanoperato,. _s ._¢eliminaryexperiment
confirmedthis.
-104-
1982005792-112
, A preltmtnery mmtpulatlon experhlmt ms Imrformd, prtmrtl¥ to tt_
i the worklngof all of the systm coep4hents. Theexpw-tmr_tconststM of
' the remval of a nut from a bolt, _ usedboth the "ALlGff' and"LOCATE"
routtnes describedearller. In addition, a nut remvel routtne ms devel-
opedto unscrewt;_ nut fromthe bolt.
The operator vtwed the task ustng a ftxed, black-and-Nhtte TV camera
and_nttor, andwasphysically tsolated frm the Nntpulator by a parti-
tion. The task beganw_.h the mntpulator handpostttOnfKIJUSt at the edge
of the camerafteld of vtston, aM randmly mtsallgmld (pttch of plus/minus
20 degrees, yaw of plus/minus20 degrees) relattve to tke table on _htch the
nut was tightened. TM task endedqhenthe operator hwJsuccessfully un-
_ screwedthe nut from the bolt andratsed tt way from the table surface a
_._, small mount.
_i The task was performedboth wtth I_l wtthout c_utor assistance, and
*_ at two time-delays, 0.0 aM 1.0 seconds. TWoreplications at eachof the
_ four conditions ytelded a total of etght trtals. 0nl_to_ subject parttc|-
'_ pared tn the experlmmt, i
!
-105- !
!
"J • m i
1982005792-113
Undern,anualcontrol,the subjectusedthe six-axisjoystickto control
the manipulatorthroughoutthe experiment. Using computerassistance,the
subjectbroughtthe hand nearthe table,then typed "ALIGN"to the SCC.
After completionof the allghmentroutine,the subjectbroughtthe hand
roughlyalongsidethe nut and typed "LOCATE"to the SCC. After"completion
of thisroutinethe hand was centeredover the nut and ready for grasping.
The subjectthen typed"NUT" and the manipulatorunscrewedthe nut under
computercontrol. The LOCATEand NUT routinescould be combined,but we
preferredto leaveLOCATEmore generalfor futureuses other than nut
I
removal.
The completiontimesfor this task are shown in Table I. Althoughthe
samplesizes are much too smallfor statisticalcomparison,it is evident
that supervisorycontrolyieldedlowercompletiontimes,especiallyat the
!.0 secondtime-delay. In addition,the subjectreporteda definiteprefer-
ence for the computer assistance.
VIII. CONCLUSION
This paperhas describeda systemfor the studyoF supervisorycontrol
of remotemanipulationwherecontrolis tradedbetweenman and computer. By
virtueof the excellentpositionalaccuracyof the manipulatorand the
effectivedistribtedprocessingarchitecturemade possibleby the VAL soft-
ware of the Arm ControlComputerand the FORTRAN/Assemblerlanguageof the
SupervisoryControlComputer,this systemhas the potentialto accomplish
sophisticatedtasks. The humanoperatorbecomesa scene analyzer,planner,
and superviso- Preliminaryexperimentsindicatethat supervisorycontrol
with thissystemyield lowertask completiontimes and is preferredby the
operatorover manualcontrol.
Table 1
TIME-IIEI.AyMANUALCOtlTROL SUPERVISORYCONTROl.
l,
163 1130,0
166 106
L
313 I_0
1,0
280 125
COMPLETION TIMES_
PRELIMINARY EXPERIMENT
-io6-
i
1982005792-114
REFERENCES
1. T. B. Sheridanand W. R. Ferret1,"SupervisoryControlof Remote
Manipulation,"IEEE Spectrum,Vol.4, No. 10, Oct. 1967.
2. W. R. Ferret1,"RemoteManipulationwith a TransmissionDelay,"NASA
TechnicalNote D-2665,1965.
3. J. H. Black,jr., "FactorialStudyof Remote Manipulationwith Trans-
missionTime Delay,"M.S. Thesis,MIT, 1970.
4. J. W. Hill and A. 5. Sword,"Studyto Designand Developa RemoteManip-
ulatorSystem,"SRI QuarterlyReport,1971.
5. G. P. Starr,"A Comparisonof ControlModes for Time-DelayedRemoteMan-
ipulation,"IEEETrans.on Systems,Man, and Cybernetics,Vol. SMC-9,
No. 4, April 1979.
z
s
-i0/-
1982005792-115


Supervisory control of remote manipulation: A preliminary evaluation

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server