The Software Technology Laboratory at the Johnson Space Center is testing a Space Time Neural Network (STNN) for observing tether oscillations present during retrieval of a tethered satellite. Proper identification of tether oscillations, known as 'skiprope' motion, is vital to safe retrieval of the tethered satellite. Our studies indicate that STNN has certain learning characteristics that must be understood properly to utilize this type of neural network for the tethered satellite problem. We present our findings on the learning characteristics including a learning rate versus momentum performance table

Copeland, Charles

Jani, Yashvant

Lea, Robert N.

Villarreal, James A.

English

NASA Technical Reports Server

/!
/'/
_3
Learning Characteristics of a Space-Time Neural Network
As a Tether "Skiprope Observer" -
N93-2217
Robert N. Lea and James A. Villarreal
NASA / Lyndon B. Johnson Space Center
Houston, Texas 77058
Yashvant Jani
Togai InfraLogic Inc.
Houston, Texas 77058
Charles Copeland
Loral Space Information Systems
Houston, Texas 77058
6
Abstract :
The Software Technology Laboratory at the Johnson Space Center is testing a Space Time Neural
Network (STNN) for observing tether oscillations present during retrieval of a tethered satellite.
Proper identification of tether oscillations, known as "skiprope" motion, is vital to safe retrieval of
the tethered satellite. Our studies indicate that STNN has certain learning characteristics that must
be understood properly to utilize this type of neural network for the tethered satellite problem. We
present our findings on the learning characteristics including a learning rate versus momentum
performance table.
1.0 Introduction
NASA and the Italian Space Agency plan to fly the Tethered Satellite System (TSS) aboard the
Space Shuttle in July, 1992. The mission, lasting approximately 40 hours, will deploy a 500 kg
satellite upward (away from the earth) [1, 2] to a length of 20 km, perform scientific experiments
while on-station, and retrieve the satellite safely. Throughout the deployment, experimentation, and
retrieval, the satellite will remain attached to the Orbiter by a thin tether through which current
passes, providing power to experiments on-board the satellite. In addition to the scientific
experiments on-board the satellite, the dynamics of the TSS itself will be studied. The TSS
dynamics are complex and non-linear due to the mass as well as spring-like characteristics of the
tether. When the tether is modeled as a massless spring, it typically exhibits longitudinal and
librational oscillations [2]. However, when the tether is modeled as beads connected via springs as
shown in fig. 1, the dynamics of TSS includes longitudinal, librational and transverse circular
oscillations or so-called "skiprope" phenomenon. These circular oscillations are generally induced
when current pulsing through the tether interacts with the Earth's magnetic field [3, 4]. The center
bead typically displaces the most from the center line. Thus, the "skiprope" can be viewed (fig. 2)
by plotting a trajectory of the mid-point of the tether as it is retrieved slowly from the Onstation-2
phase in a high fidelity simulation test case. Detection and control of the various tether modes,
including the 'skiprope' effect, is essential for a successful mission. Since there are no sensors that
can directly provide a measure of skiprope oscillations, indirect methods like Time Domain
Skiprope Observer [4] and Frequency Domain Skiprope Observer [3] are being developed for the
mission. We are investigating a Space Time Neural Network (STNN) based skiprope observer.
The STNN is basically an extension to a standard backpropagation network [5,6,7] in which the
single interconnection weight between two processing elements is replaced with a number of Finite
Impulse Response (FIR) filters [8]. The use of adaptable, adjustable filters as interconnection
weights provides a distributed temporal memory that facilitates the recognition of temporal
sequences inherent in a complex dynamic system such as the TSS. We have performed
experiments in detecting various parameters of skiprope motion using an STNN.
262
https://ntrs.nasa.gov/search.jsp?R=19930012987 2020-03-17T06:34:41+00:00Z
In BeadModel,theTethermassis
distributedin form of beads
connectedby springs.
Fig. 1Whentetheris modelledasbeads,thetransversecircular
oscillationsknownas"skiprope"areinducedduring retrieval.
Extensivestudiesusing high fidelity simulationshaveshownthat the tetheredsatelliteexhibits
characteristicrateoscillationsin thepresenceof skipropemotionasshownin figure 3. Sincethese
rateoscillationsaremeasuredby thesatellite'son-boardrategyros,themeasuredratescanbeused
asinputsto askipropedetectionsystemalongwith othermeasuredparameterssuchastensionand
length[9]. We havetrainedanSTNNusingdataloggedfrom a high fidelity Orbital Operations
Simulator (OOS)[10] which modelsthebehaviorof theTSS.The parametersusedin network
training includesatelliteroll, pitch,andyawrates,sensedtensionandlengthof thetether,andthe
positionof themid-point of thetetherduringskipropemotion.In this paper,we first describethe
STNN architecture in section 2. The STNN configuration used for skiprope observation is
describedin section3 alongwith training andtestdatageneratedby the simulation test cases.
Learningcharacteristicsarediscussedin section4, andconclusionsaresummarizedin section5.
14
10.5
3.5
TET 0
Z
(m) -3.5
-7
-10.5,
-14
-17.5
12 16
TET Y (m)
Figure 2 - Trajectory of tether mid-point during "skiprope".
263
(Figure 3 - Tether "slap rope" effect leads to highly characteristic satellite attitude oscillations
which can be used to detect the magnitude and phase of the skiprope
2.0 STNN Architecture
The STNN architecture [8] allows the dimension of time to be added to the strong spatial modelling
capabilities found in neural networks. The time dimension can be added to the standard processing
element used in conventional neural networks by replacing the synaptic weights between two
processing elements with an adaptable-adjustable filter as shown in figure 4.
,I,,,lllllJbI[11111111xl,o,
Figure 4 - A pictorial representation of the Space-Time processing element.
Instead of a single synaptic weight with which the standard backpropagation neural network
represented the association between two individual processing elements, there are now several
weights representing not only spatial association, but also temporal dependencies. In this case, the
synaptic weights are the coefficients to the adaptable digital filters:
N M
y(n)= '_ bkx(n-k)+ _ amy(n-m) (1)
k=0 m=l
Here the x and y time sampled sequences are the input and output respectively of the filter and am'S
and bk'S are the coefficients of the filter. Thus, if there are j parameters going into a neuron, the yj
264
areinput into theneuron,whereeachyj is afiltered valueof thexj usingn time seriessamplesas
shownin fig. 4. Thexj,s arethereal input from anexternalsource.Thus,the STNN is learninga
temporaldependencyof theinputparameters.
A space-timeneuralnetworkincludesatleasttwolayersof filter elements fully interconnected and
buffered by sigmoid transfer nodes at the intermediate and output layers as shown in figure 5. A
sigmoid transfer function is not used at the input. Forward propagation involves presenting a
separate sequence dependent vector to each input, propagating those signals throughout the
intermediate layers until the signal reaches the output processing elements. In adjusting the
weighting structure to minimize the error for static networks, such as the standard
backpropagation, the solution is straightforward. However, adjusting the weighting structure in a
space-time network is more complex because not only must present contributions be accounted for
but contributions from past history must also be considered. Therefore, the problem is that of
specifying the appropriate error signal at each time and thereby the appropriate weight adjustment
of each coefficient governing past histories to influence the present set of responses. A detailed
discussion of the algorithm can be found in the reference [8].
,llll ,,, ,,..
 lllll ',,' 'lj
..,,,b tllll,
 llllll' lllll'
Figure 5 - A depiction of a STNN architecture showing the
distribution of complex signals in the input space.
3.0 STNN Configuration and Test/Training Data
Several different simulation runs were used to gather data for STNN training. The simulation runs
are consistent with the requirement that the skiprope observer must be capable of performing
during various combinations of current flow through the tether and satellite spin. For example, one
simulation represents a case in which current flows through the tether continuously, and the
satellite is in yaw hold. Another simulation represents the case in which current flows through the
tether only during the on-station phase, and the satellite is in yaw hold. A third simulation
represents continuous current flow, and satellite spin at 4.2 degrees/second. These three scenarios
will form the basis for STNN skiprope observer training and testing, and are consistent with
simulations that are used for testing the Time-Domain Skiprope Observer (TDSO) [4] which will
be flown on TSS-1.
Ultimately, the network should utilize only roll rate, pitch rate, yaw rate, sensed tension and
sensed length since these are the only directly measurable parameters. However, we have
265
conductedexperimentsusingderivedparameterssuchasroll, pitch, andyaw positionin addition
to rateswith nosignificantimprovement.Thebiggestchallengeto networktrainingsofar hasbeen
to learnthephasemapping.Severaldifferentnetworkconfigurationshaveyieldedgoodresultsin
predicting skipropeamplitude,but we havenot beenaslucky with skipropephase.Sincethe
ultimategoalis to providethecrewwith accuratemeasurementsof skipropeamplitudeandphaseto
supportthe yaw maneuver,the skipropeobservershouldlearnto predict amplitudeandphase
basedon theavailableinputs.However,decisionsconcerningtheyawmaneuvercanbebasedon
the x and y coordinatesof the mid-point of the skipropemotion as well. Therefore,the basic
networkconfigurationconsistsof 6 inputs(roll rate,pitch rate,yawrate,sensedtension,x(t), and
y(t)) and2 outputs(x (t+l) andy(t+l)). Notice thatwearetrainingon thecurrentx andy position
andpredicting x andy position for the next time step.In previousexperimentswe focussedon
finding theoptimumnetworkconfigurationin termsof numbersof hiddenunitsandnumbersof
zerosfrom layer to layer.Throughexperimentation,we settledon 30 hiddenunitsand30 zeros
from the input layer to the hiddenlayer, and30 zerosfrom the hiddenlayer to theoutput layer,
althoughslight deviationsin theseparametershavelittle or noeffect in networkperformance.In
this paperweconcentrateprimarily on the effectsof learningrateandmomentumon theoverall
generalizationof theSpace-TimeNeuralNetwork.
4.0 Learning Characteristics
A well known characteristic of backpropagation networks, or networks derived from
backpropagation, is that in order to achieve reasonable generalization, the network must learn the
training data. Experiments have indicated that, like standard backpropagation, the learning
characteristics of STNN are such that if the training data is not learned, generalization will not
occur. These and other learning characteristics dictate that a particular sequence of steps be
followed in the training and testing of STNN. The following general steps were used as guidelines
throughout the STNN testing.Please note that the use of the word "momentum" in this report refers
to a term in the learning algorithm that represents a fraction of the previous weight change rather
than any physical properties of the TSS.
1. Train and test - evaluate learnability of training data
2. Adjust network as necessary (set learning rate and momentum in updating of interconnection
weights).
3. If network is unable to obtain sufficient convergence on training data, test individual
parameters one at a time. Eliminate un-mappable parameters and start over.
4. If reasonable convergence is realized on training data, divide the data set into a training set and
a separate test set.
5. When reasonable performance is achieved on the separate test data, then go for multi-test case
generalization.
Step 2 above generally involves trying different combinations of learning rate and momentum in
the interconnection weight update formulas. Table 1 illustrates the test case matrix we have
identified in order to test the effects of different combinations of learning rate and momentum.
The results that follow are from training and testing using data from the simulation which includes
current pulsing and satellite spin, which is considered the most difficult case. Following our
general training and testing steps listed above, we verified that the STNN was able to learn the
training data using a learning rate of 0.05, and momentum set to 0.9. We trained and tested on all
3500 Input/Output pairs and achieved a MAX error of 0.08 and RMS error of 0.02 at 140 cycles.
Since the network will be trained off-line before being placed in the operational environment, we
must determine how well the network will perform when presented with data that it has not
previously seen. Therefore, to test the generalization ability of STNN, we train on only the first
and last 400 input/output pairs from the full 3500, and test separately on the middle 2700
266
input/output pairs while trying variouscombinationsof learningrate and momentumwith the
following results.First, with a momentumof 0.9, we tried learning ratesof 0.05, 0.2, and0.7(testcases#4-6 in Table 1).Testcase#4 resultedin MAX error= 0.43,andRMSerror= 0.04at
cycle 100.Figure6 showstheerrorplot for testcase#4up to 500cycles.Figures7aand7bshow
aportionof thex andy predictionsfrom testcase#4.Testcase#5 resultedin MAX error= 0.43
andRMSerror = 0.04atcycle480.Figure8 showsthatthenetwork predictionof y in testcase5
is similar to that of testcase#4. Increasingthelearningrateto 0.7 in testcase#6 resultsin the
network neverreachingerrorsas low asin theprevious two test cases(at leastnot within 500
cycles)andoverallperformanceis similarly degradedasis seenin figures9aand9b.Next weset
momentumto 0.2 andtry learningratesof 0.05, 0.2, and0.7 (test cases#1-3 in Table 1). Test
case#1 yielded MAX error = 0.44, andRMS error = 0.05at 100cycles,as is shownin figure
10a.Figure 10bshowsthatthenetwork'spredictionof x in this testcaseis not quiteasaccurateas
test cases#4 and #5. As we increaselearning rate from 0.05 to 0.2, performancedegrades
significantly as is shownin figure lla. The error graphin figure 1lb showsthat no learning
occurredin test case#2, asRMS error neverdroppedsignificantly below 0.5, andMAX error
remainednear0.8. Similar resultsoccurredin testcase#3 aswe increasedthelearningratefrom
0.2to 0.7. Theoverall testerrorsaresummarizedin TableII.
Table 1 - Learning Rate Versus Momentum in
STNN Weight Update Formulas
Test Case Momentum in weight Learning
update
1 0.2 0.05
2 0.2 0.2
3 0.2 0.7
4 0.9 0.05
5 0.9 0.2
6 0.9 0.7
7 0.95 0.05
8 0.98 0.05
Rate
Table II - Number of Training Cycles to Reach Lowest Test Errors.
Test Case Max Error RMS Error Number of
1 0.44 0.05 100
2 0.78 0.49 280
3 0.8 0.5 480
4 0.43 0.04 100
5 0.43 0.04 480
6 0.5 0.09 400
7 0.41 0.04 480
8 0.41 0.04 480
Cycles
5.0 Conclusions
Through experimentation, we have gained insight into the learning characteristics of STNN in
terms of learning rate and momentum parameters. In particular, we find that the skiprope observer
problem requires high momentum and very low learning rate. In test case 4 we have seen that the
RMS error drops to 4 % within only 100 cycles of learning. We further verified this by performing
two test cases (#7 and #8) with high momentum and low learning rate. It should be noted that the
max error is reduced in both cases.
267
I Figure
0.50 I
0.i0
o.oo _ I '
o.oo lOO.OO 200.00
6 - Test Case 4, Max VS RMS Error I
300.00 400°00 500.00
-- Max i RMS ]
I Figure 7a - Test Case 4, Target X VS STNN X, at I00 cycles I
0"60 - _\
o_o_11I_
o_o_tI I /i
o'_o_t'¢ I_o:_o-1, , ,
\/ _
//
V
i I ! i
A
/1 t_
/ !
i I
! \\/
i | ! 1
'o
f\/ /_
,//
V
i ! | i
500.00 600.00 700.00 800.00 900.00 I000.00
Target
268
I Figure 7b - Test Case 4, Target Y VS STNN Y, at I00 cycles i
0.80
0.70 J
500.00 600.00 700.00 800.00 900.00 i000.00
I -- Target
-- STNN
8 - Test Case 5, Target Y VS STNN Y. IFigure
O. 80 "_
0.70 -J
" A
°°°i /_ / _
0.40 -j - tm
o.30 -j
0.20 -I ....
V/ V
i i i |
500.00
I
| i i *
II1'
! i
i i | ,
A
l,f v
* i , |
600.00 700.00 800.00 900.00 i000.00
I -- Target Y
-- STNN Y
269
r,' [ Figure 9a - Test Case 6, Max VS RMS Error I
_v_ _0.50
0.40
0.30
0.i0 --
0.00 ' ; '20.00 100.0 00.00 300.00 400.00 500.00
I Figure 9b - Test Case 6, Target X VS STNN X, 400 cycles [
0"901_ J
0.B0
o.7o _ A A A A ,, , , A A _ ^
r IOU I L_ t, l,f,I I
0.40U J I II ;/ _1_ ;J / :1 I I} \1
• V V v ' V v v0._o V y .... V v0.20 - , , , , , '. , .....
0.00 200.00 400.00 600.00
:FIF
! I I I
800.00 I000.00
I -- Target X -- STNN X I
270
IFigure 10a - Test Case i, Max VS KMS Error I
0.50 "
0.40 •
0.30 "
0.20 "
0.I0 "
0.00 "
A
| ! i |
0.00
| | |' i i i , i
I00.00 200.00 30(
v
, i i l , , l
_.00 400.00 500.00
--Max -- KMS
I Figure 10b - Test Case I, Target X VS STNN X, 100 cycles. I
!
I
0.80 ,
0.70 :
0.60 _ [
0.30 : £ ,0.20 "
500.0;'' '600.0;'' '7001. '80(o;'
l
O0 1000 O0
-- Target X
-- STNN X I
271
i Figure 11a - Test Case 2, Y VS STNN 280 i
i
Target Y, cycles.
0.80 J I0.70 A -- A _0.60 _ ^ ^ . A ^ . _ I_ A iI _ A I_
t/I /v\ _ Ih/k/I dl/'l 13 Ih i_, Jl /Illl I I I_
// hi _ V If I/ _1 _1 U I; II/ _! N/ _JI II) I/
040 • "" '" " v v V _'V ll
0.30
0.20
o._o!oL, I /o.oo ',' '',' 'l. ',.
0. 200.00 400.00 600.00 800.00
I Target Y STNN Y
i i
1000.00
I Figure llb - Test Case 2, Max VS RMS Error I
0
0
0
0
0
0
0
0
0.00
!!.....................................................................i_i .....
100.00 200.00 300.00 400.00 500.00
li Max -- RMS I
Based on our earlier results, we conclude that the STNN is slow in learning sharp discontinuities
like those encountered in phase behavior. The value of the phase goes from 180 to -180 abruptly
when the circle is complete. When we changed to the x- and y- component form (rather than
amplitude and phase), the STNN based skiprope observer performed much better in predicting x
and y coordinates of the mid-point of the tether.
We will have an opportunity to perform a side-by-side comparison of the STNN based skiprope
observer and the TDSO using simulation data. Next, we will test the STNN based skiprope
observer with the post mission data after the TSS-1 flight.
References :
.
2.
.
.
5.
6.
7.
8.
9.
10.
Coledan, S. : "Tether Satellite Advances", Space News, vol. 2, no. 15, p. 8, 1991.
Powers, C.B., Shea, C., and McMahan, T. • "The First Mission of the Tethered Satellite
System", A special brochure developed by the Tethered Satellite System Project Office,
NASA/Marshall Space Flight Center, Huntsville, Alabama, U.S. GPO 1992-324-999,
1992.
Ioup, G.E., Ioup, J.W., Rodrigue, S.M., Amini, A.M., Rayborn, G.H., and Carroll, S. :
"Frequency Domain Skiprope Observer", Skiprope Containment Status Meeting held at
Denver, Sep. 10-I I, 1991. (Research supported by NASA Contract NA8-38841)
Glowczwski, R. : " Time Domain Skiprope Observer Overview", Skiprope Containment
Status Meeting held at Martin Marietta, Denver, Sept. I0-I I, 1991.
Kosko, B. : "Neural Networks and Fuzzy Systems", Prentice-Hall, New Jersey, 1992.
J.A.Freeman and D.M.Skapura • "Neural Networks Algorithms, Applications and
Programming Techniques", Addison-Wesley Publishing Company, Reading, MA. 1991.
Wasserman, P.D. : "Neural Computing Theory and Practice" Van Nostrand Reinhold,
New York, 1989.
Villarreal, J.A., and Shelton, R.O. : "A Space-Time Neural Network", International
Journal of Approximate Reasoning, 6(2), 133-149, 1992.
Lea, R.N., Villarreal, J.A., Jani, Y., and Copeland, C. : "Application of Space Time
Neural Networks to detect Tether Skiprope Phenomenon in Space Operations",
Proceedings of the AIAA GN&C Conference held at Hilton Head, South Carolina, August
10-12, 1992.
Edwards, H. C., and Bailey, R. : "The Orbital Operations Simulator User's Guide",
LinCom corporation, ref. LM85-1001-01, June 87.
273



Learning characteristics of a space-time neural network as a tether skiprope observer

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server