This paper studies the interaction between two satellites after docking. In order to maintain the docked state with uncertainty in the motion of the target vehicle, a game theoretic controller with Stackelberg strategy to minimize the interaction between the satellites is considered. The small perturbation approximation leads to LQ differential game scheme, which is validated to address the docking interactions between a service vehicle and a target vehicle. The open-loop solution are compared with Nash strategy, and it is shown that less control efforts are obtained with Stackelberg strategy

Fitz-Coy, Norman G.

Hiramatsu, Takashi

NASA Technical Reports Server

GAME THEORETIC APPROACH TO POST-DOCKED
SATELLITE CONTROL
Takashi Hiramatsu ∗ and Norman G. Fitz-Coy †
University of Florida, Gainesville, FL 32611
Abstract
This paper studies the interaction between two satellites after docking. In order to maintain the docked state with
uncertainty in the motion of the target vehicle, a game theoretic controller with Stackelberg strategy to minimize
the interaction between the satellites is considered. The small perturbation approximation leads to LQ differential
game scheme, which is validated to address the docking interactions between a service vehicle and a target vehicle.
The open-loop solution are compared with Nash strategy, and it is shown that less control efforts are obtained with
Stackelberg strategy.
1 Background/Introduction
Autonomous on-orbit robots services have been studied recently with several architectures, such as Orbital Express
[1] (refueling), SUMO [2], and HEROS [3] (repair). These tasks include rendezvous and docking between a resident
space object (i.e., a disabled satellite) and the service vehicle, as shown in Fig. 1. During these operations, it is
possible that the disabled satellite is non-cooperative. This could be due either to a loss in communication and/or
control actuation. There are three possible non-cooperative scenarios between the target and the service vehicle, and
they can be considered in the framework of game theory: (i) the interaction is completely non-cooperative such that
each satellite acts independent of each other, (ii) they have an adversarial interaction, where one tries to evade the
other, while the other pursues, and (iii) it is partially non-cooperative where one’s action is non-cooperative (disabled
satellite) while the other is cooperative (service vehicle).
In game theory, the interactions mentioned above are classified as (i) Nash, (ii) Minimax, and (iii) Stackelberg
strategy [4]. In a Nash strategy, each player optimizes its objective without consideration of the other players; however,
it could result in the equilibrium solution which is not optimal to each individual player (e.g., Prisoner’s Dilemma).
In a Minimax strategy, by assuming the worst case and trying to minimize the damage, each player can obtain the
solution which is the safest. In a Stackelberg strategy, the leader can enforce their action and the resultant equilibrium
solution is always favorable to the leader. The Stackelberg leader always obtains the result which is better than or
equal to the result with a Nash strategy.
The situation considered in this study is as follows: the service vehicle (SV), after caught or docked with the
target (RSO: resident space object), tries to maintain its state without causing damages due to the interaction between
them. While the SV has a specific goal that it should track the motion of the RSO, the RSO has an arbitrary motion.
One way is to treat the RSO as the Stackelberg leader and the SV as the follower. Another way is to treat the SV
as the leader since the SV may be able to estimate the motion of the RSO by prior observation. How well the
∗Graduate Student, Department of Mechanical and Aerospace Engineering , takashi@ufl.edu.
†Associate Professor, Department of Mechanical and Aerospace Engineering , nfc@ufl.edu.
1
https://ntrs.nasa.gov/search.jsp?R=20080012704 2019-08-29T19:07:23+00:00Z
Figure 1: An image of two satellites docking.
docked state is maintained can be determined by investigating the interaction between them since ideally there is
zero force and torque when two satellites are docked to form a completely rigid body, and any collision or detach
will generate the force and torque between them. In order to study the interaction between the SV and the RSO, a
control problem based on the two-person linear quadratic differential game is designed and analyzed to minimize the
interactive forces between the docked satellites assuming non-cooperative behavior. The small variation in the vicinity
of the nominal operation condition is considered in order to model the system as a LQ (linear state equation with
quadratic cost) differential game. When simplified dynamics is allowed, LQ differential game approach works well,
providing analytical solutions. The Nash and the Stackelberg solutions are compared with classical LQR controller,
which assumes cooperation between the RSO and the SV (i.e., the RSO is not disabled). For simplicity only the
open-loop strategies are considered.
2 Method of Approach
The docking-maintenance problem is formulated as a two-person differential game. First two satellites moving in
coplanar circular orbits are modeled as rigid bodies with the RSO as the leader and the SV as the follower. In the
numerical simulation the case with reversed roles (the SV as leader and the RSO as the follower) is also studied. The
interaction between them is modeled as a spring and a damper. Thus, a change in distance and velocity between them
produces the interaction forces, which are to be minimized. Each body is controlled by a force input and a torque input.
It is further assumed that the thrusts and torques are decoupled. A linear state-space representation of the system is
x˙ (t) = Ax (t) + B
1
u1 (t) + B2u2 (t) , x (0) = x0 (1)
2
The SV and the RSO minimize their cost functionals:
Ji =
1
2
xT (tf )Kifx (tf )
+
1
2
∫ tf
0
⎧⎨
⎩xT (t)Qix (t) +
2∑
j=1
uTj (t)Rijuj (t)
⎫⎬
⎭ dt (2)
where i =SV, RSO and the final time tf may be finite (finite horizon) or infinite (infinite horizon), and in the latter
case, K
1f
= K
2f
= 0. Open-loop linear-feedback control inputs are
u1 = −R−111 B
T
1
K
1
x (3)
u2 = −R−122 B
T
2
K
2
x (4)
and therefore the states are found by solving
x˙ =
(
A−B
1
R−1
11
BT
1
K
1
−B
2
R−1
22
BT
2
K
2
)
x, x (0) = x0 (5)
Equations (3) - (5) hold for both Nash and Stackelberg strategies while the gain matrices K
1
and K
2
are different.
2.1 Nash strategy
In a game where each player plays Nash strategy, they try to minimize their own cost simultaneously. For a two-person
LQ differential game model like the one given by Eqs. ( 1) - (2), the optimality conditions are derived directly through
calculus of variations. Starr [5] has studied the Nash solution of a two-person open-loop nonzero-sum linear quadratic
differential game. Assuming full-state feedback, the game defined by the Eqs. ( 1) - (2) have unique open-loop Nash
strategies where K
1
and K
2
are obtained by solving the set of coupled Riccati differential equations
K˙
1
= −ATK
1
−K
1
A−Q
1
+ K
1
B
1
R−1
11
BT
1
K
1
+ K
1
B
2
R−1
22
BT
2
K
2
(6)
K˙
2
= −ATK
2
−K
2
A−Q
2
+ K
2
B
2
R−1
22
BT
2
K
2
+ K
2
B
1
R−1
11
BT
1
K
1
(7)
where K
1
(tf ) = K1f and K2 (tf ) = K2f for finitie-horizon case. For infinite-horizon, the corresponding algebraic
Riccati equations (Eqs. (6) - (7) with K˙
1
= K˙
2
= 0, K
1f
= K
1
, and K
2f
= K
2
) are solved. The algorithms by
Engwerda [6] are used to solve these Riccati equations.
2.2 Stackelberg strategy
In a two-person Stackelberg game, one player is assigned as the leader and the other as the follower. The leader has
advantages such as having faster computational power or having more information than the follower, and therefore has
power to enforce their action. Thus, the follower’s optimal strategy is found by solving an optimal control problem
treating the leader’s strategy as the prescribed function. Once the solution to this tracking problem is obtained, the fol-
lower’s control is substituted into the leader’s cost. Finally, the leader solves the optimal control problem knowing that
the follower will ”follow” the leader’s decision. One of the earliest complete solution methodologies for a Stackelberg
strategy in a two-person differential game was developed by Simaan [ 7]. Following [7] with assumption of full-state
feedback, the unique open-loop Stackelberg solution with the SV as leader, which is exactly of the same form as Eqs.
3
(3) - (4), is obtained by solving the coupled Riccati-type differential equations
K˙
1
= −ATK
1
−K
1
A−Q
1
+ K
1
B
1
R−1
11
BT
1
K
1
+ K
1
B
2
R−1
22
BT
2
K
2
(8)
K˙
2
= −ATK
2
−K
2
A−Q
2
+ Q
1
P + K
2
B
1
R−1
11
BT
1
K
1
+ K
2
B
2
R−1
22
BT
2
K
2
(9)
P˙ = AP − P A + P B
1
R−1
11
BT
1
K
1
+ P B
2
R−1
22
BT
2
K
2
−B
1
R−1
11
R
21
R−1
11
BT
1
K
1
+ B
1
R−1
11
BT
1
K
2
(10)
where K
1
(tf ) = K1f , K2 (tf ) = K2f , and P (0) = 0 for finite-horizon. Similarly to the Nash strategy, for infinite-
horizon Eqs. (8) - (10) are solved with K
1f
= K
1
, K
2f
= K
2
, P (0) = 0, and K˙
1
= K˙
2
= P˙ = 0. These
Riccati-type equations are solved based on the work by Freiling [ 8].
3 Mathematical Modeling and Problem Formulation
The following post-capture dynamics is considered. Two satellites previously shown in Fig. 1 are modeled as two
uniform rigid bodies, the target or the resident space object (RSO) and the service vehicle (SV), as shown in Fig. 2.
The interaction between the SV and the RSO is modeled by a spring and damper connecting them at the point P and Q
such that a change in distance or velocity between P and Q produces the interaction forces, which are not necessarily
radial and therefore cause torques, too. In order to maintain docking, those forces and torques need to be minimized.
Each body is controlled by a force input and a torque input. It is further assumed that the thrusts and torques are
decoupled.
Center (The Earth)
Resident Space Object (RSO)
Service Vehicle (SV)
k
c
P
QxRSO
zRSO
xSV
zSV
RRSO0
RSV0
ρ
RSO
ρ
SV
Figure 2: Two rigid bodies on circular orbits.
The translational motion of each body moving in a circular orbit is obtained from the two-body equation given by
[9] as
r¨i +
μ
‖ri‖3
ri =
1
mi
{F i + ui} , i = SV, RSO (11)
where ri denotes the position vector located in the center of mass of each body, F i denotes the external force due to
the spring and the damper, and ui is the control force input. The rotational motion is governed by Euler’s equation
I
i
ω˙i + ωi × Iiωi = M i + τ i, i = SV, RSO (12)
where I
i
is the inertia matrix, ωi is the angular velocity, M i is the moment due to the spring and the damper, and τ i is
4
the control torque input. The position vector of each body is
rSV = RSV0 + η (13)
rRSO = RRSO0 + ζ (14)
where the nominal radii of the orbit of the SV and the RSO, coordinatized in their respective frames, are
SV RSV0 =
[
0 0 −RSV0
]T
(15)
RSORRSO0 =
[
0 0 −RRSO0
]T
(16)
and the linear perturbation is coordinatized in the SV’s frame as
SV η =
[
η1 η2 η3
]T
(17)
SV ζ =
[
ζ1 ζ2 ζ3
]T
(18)
then Eqs. (11) - (12) are linearized to yield the linear dynamic model (Eq. ( 1)). With the assumption of small
perturbation and that two satellites are on the same orbit (i.e., RSV0  RRSO0 ) since they are docked and have small
body compared to the orbit, the coordinate frames are approximated to conincide. The relative distance and attitude
errors are defined as
δr =
[
δx δy δz
]T
 ζ − η (19)
δθ =
[
δθ1 δθ2 δθ3
]T
 β − α (20)
and the state vector x of the system is composed of the relative distance and attitude error and their rates such that
x =
[
δrT δθT δr˙T δθ˙
T
]T
(21)
In the following numerical analysis, the states are coordinatized in the body axis of the SV since the focus of this study
is to control the SV.
4 Simulation and Results
The simulation for a infinite-horizon case was run with the system parameters shown in Table 1 with the corresponding
state-space system
A =
[
0
6×6 I6×6
A
21
A
22
]
(22)
where
A
21
=
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
−1.852 −1.852 −1.852 −0.001 −1.482 0.001
−0.926 −0.926 −0.926 −0.001 −0.741 0.001
−0.926 −0.926 −0.926 −0.001 −0.741 0.001
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
(23)
5
A
22
=
⎡
⎢⎢⎢⎢⎢⎢⎢⎢⎣
−0.185 −0.275 −0.185 0 0 0
−0.188 −0.278 −0.185 0 0 0
−0.185 −0.185 −0.185 0 0 0
0 0 0 0 0.001 0
0 0 0 −0.001 0 0
0 0 0 0 0 0
⎤
⎥⎥⎥⎥⎥⎥⎥⎥⎦
(24)
and
B
1
=
[
0
6×6
diag{-0.01, -0.01, -0.01, -0.0167, -0.0167, -0.0167}
]
(25)
B
2
=
[
0
6×6
diag {0.067, 0.067, 0.067, 0.01, 0.01, 0.01}
]
(26)
The units represented in matrices A
ii
and B
i
are linear distance in meters, mass in kg, and angular displacement
in radians. The initial conditions are:
x (0) =
[
δrT δθT δr˙T δθ˙
T
]T ∣∣∣∣
0
=
[
0.2 0.2 0.2 0.05 0.05 0.05 −0.1 −0.1 −0.1 −0.1 −0.1 −0.1
]T
(27)
The cost functionals are chosen as
Q
1
= diag
{
1 1 1 0.1 0.1 0.1 0.5 0 0 0 0 0
}
(28)
Q
2
= −Q
1
(29)
R
11
= diag
{
1 0.1 0.1 1 1 1
}
(30)
R
12
= 1
6×6 (31)
R
21
= −R
11
(32)
R
22
= −R
12
(33)
Table 1: The system parameters used in the preliminary simulation.
c = 10 Ns/m RSV0 = 6600.00 km
k = 20 N/m RRSO0 = 6600.00 km
l0 = 0.3 m ISV = diag{60, 60, 60} kg·m2
mSV = 150 kg IRSO = diag{100, 100, 100} kg·m2
mRSO = 200 kg ρSV =
[
1 1 1
]T
m
ρ
RSO
=
[
1.5 1.5 1.5
]T
m
For the sake of comparison, the problem is also solved with the linear quadratic regulator (LQR) controller as-
suming the interaction between the SV and the RSO cooperative. If the objectives of both the SV and the RSO are to
6
minimize the interaction, a corresponding LQR problem can be constructed as
x˙ = Ax + B u (34)
and
J =
1
2
∫ ∞
0
{
xTQx + uTRu
}
dt (35)
where B =
[
B
1
B
2
]
, u =
[
uT1 u
T
2
]T
, Q = Q
1
−Q
2
, and R =diag
{
R
11
−R
22
}
. Note that in order to be able
to solve the LQR problem, R must be invertible and symmetric. For simplicity, the off-diagnal terms are chosen to be
zero instead of R
12
and R
21
. The resultant trajectories are plotted in Fig. 3, and the control force and torque inputs
are compared in Figs. 4 - 5.
0 10 20 30 40−0.5
0
0.5
time (s)
er
ro
r 
di
sta
nc
e 
(m
)
δx
δy
δz
0 10 20 30 40−0.2
0
0.2
time (s)
at
tit
ud
e 
er
ro
r (
rad
)
δθ1
δθ2
δθ3
(a) LQR
0 10 20 30 40−0.5
0
0.5
time (s)
er
ro
r 
di
sta
nc
e 
(m
)
δx
δy
δz
0 10 20 30 40−0.2
0
0.2
time (s)
at
tit
ud
e 
er
ro
r (
rad
)
δθ1
δθ2
δθ3
(b) Nash
0 10 20 30 40−0.5
0
0.5
time (s)
er
ro
r 
di
sta
nc
e 
(m
)
δx
δy
δz
0 10 20 30 40−0.2
0
0.2
time (s)
at
tit
ud
e 
er
ro
r (
rad
)
δθ1
δθ2
δθ3
(c) RSO as leader
0 10 20 30 40−0.5
0
0.5
time (s)
er
ro
r 
di
sta
nc
e 
(m
)
δx
δy
δz
0 10 20 30 40−0.5
0
0.5
time (s)
at
tit
ud
e 
er
ro
r (
rad
)
δθ1
δθ2
δθ3
(d) SV as leader
Figure 3: The resultant trajectory.
The resultant trajectory doesn’t have notable difference whichever player plays Stackelberg leader or Nash strategy,
or LQR. The control efforts, on the other hand, changes significantly. It is obvious that with LQR the control efforts are
the smallest because it is based on the ideal situation that RSO is not disabled. In cases where we can’t communicate
with the RSO and don’t know its motion, such a cooperation is impossible, and thus LQR is not applicable. Both the
SV and the RSO have much smaller control force and torque input by playing Stackelberg leader than Stackelberg
7
0 10 20 30 40−0.01
0
0.01
0.02
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
SV
u1
u2
u3
0 10 20 30 40−5
0
5 x 10
−4
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
RSO
u1
u2
u3
(a) LQR
0 10 20 30 40−5
0
5
10
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
SV
u1
u2
u3
0 10 20 30 40−10
−5
0
5
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
RSO
u1
u2
u3
(b) Nash
0 10 20 30 40−5
0
5
10
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
SV
u1
u2
u3
0 10 20 30 40−0.05
0
0.05
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
RSO
u1
u2
u3
(c) Stackalberg: RSO as leader
0 10 20 30 40−0.2
0
0.2
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
SV
u1
u2
u3
0 10 20 30 40−10
−5
0
5
time (s)
co
n
tr
ol
 fo
rc
e 
(N
)
RSO
u1
u2
u3
(d) Stackalberg: SV as leader
Figure 4: The control force inputs.
follower. Also, it is shown that the control efforts with Stackelberg follower is almost the same as with Nash strategy.
Note that the trajectories and the SV’s control history represent the potential upperbound, the control inputs of the
RSO in Figs. 4 - 5 are imaginary actuation to cause motion of the disabled RSO.
It is shown that the results using game-theoretic controllers are similar to LQR controller, which assumed cooper-
ation between the SV and the RSO. These preliminary results confirm feasibility of game theoretic controller as well
as the hypothesis that it is possible to obtain smaller actuation inputs for the leader of a Stackelberg approach while
obtaining relatively the same trajectory variables.
5 Conclusion
It was shown that the interaction between controlled service vehicle and the disabled target can be minimized with the
LQR and the game theoretic controllers for a post-docked maneuver where the distance and the attitude error are kept
small enough that linear approximation is valid. Although the LQR controller worked the best, it was based on an ideal
case where the RSO was not disabled. Since the RSO being disabled is the characteristic of the problem, the LQR
8
0 10 20 30 40−5
0
5
10 x 10
−3
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) SV
τ1
τ2
τ3
0 10 20 30 40−10
−5
0
5 x 10
−4
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) RSO
τ1
τ2
τ3
(a) LQR
0 10 20 30 40−2
0
2
4
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) SV
τ1
τ2
τ3
0 10 20 30 40−4
−2
0
2
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) RSO
τ1
τ2
τ3
(b) Nash
0 10 20 30 40−2
0
2
4
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) SV
τ1
τ2
τ3
0 10 20 30 40−0.2
0
0.2
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) RSO
τ1
τ2
τ3
(c) Stackalberg: RSO as leader
0 10 20 30 40−0.5
0
0.5
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) SV
τ1
τ2
τ3
0 10 20 30 40−4
−2
0
2
time (s)
co
n
tr
ol
 to
rq
ue
 (N
−m
) RSO
τ1
τ2
τ3
(d) Stackelberg: SV as leader
Figure 5: The control torque inputs.
controller is not practical. On the other hand, The game theoretic approach, with the Nash or the Stackelberg strategy,
still kept the distance and the attitude errors small even with noncooperative behavior of the target RSO. Therefore,
the feasibility of the game theoretic controllers in this noncooperative scenario was validated.
It was also shown that the control effort of the service vehicle can be lowered by utilizing Stackelberg strategy as
the leader; that is, if we are able to observe the motion of the RSO before docking in order to store information to
make an estimate of how the RSO will behave in the near future, we can utilize Stackelberg game-theoretic controller
to reduce the interaction with smaller control efforts.
References
[1] http://www.darpa.mil/tto/programs/oe.htm.
[2] A. Bosse, W. Barnds, M. Brown, N. Creamer, A. Feerst, C. Henshaw, A. Hope, B. Kelm, P. Klein, F. Pipitone
et al., “Sumo: spacecraft for the universal modification of orbits,” Proceedings of SPIE, vol. 5419, pp. 36–46,
9
2004.
[3] A. Tatsch, N. Fitz-Coy, and W. Edmonson, “Heterogeneous expert robots for on-orbit servicing: A new paradigm,”
Infotech@ Aerospoace, pp. 1–14, 2005.
[4] T. Basar and G. Olsder, Dynamic Noncooperative Game Theory. SIAM Classics in Applied Mathematics,
Philadelphia, 1999.
[5] A. W. Starr and Y. C. Ho, “Nonzero-sum differential games,” Journal of Optimization Theory and Applications,
vol. 3, no. 3, pp. 184–206, 1969.
[6] J. Engwerda, LQ Dynamic Optimization and Differential Games. John Wiley and Sons, 2005.
[7] M. Simaan and J. Cruz, “On the Stackelberg strategy in nonzero-sum games,” Journal of Optimization Theory and
Applications, vol. 11, no. 5, pp. 533–555, 1973.
[8] G. Freiling, G. Jank, and D. Kremer, “Solvability condition for a nonsymmetric riccati equation appearing in
stackelberg games,” in Proceedings of the European Control Conference ECC 2003, Cambridge, England, 2003.
[9] J. Prussing and B. Conway, Orbital Mechanics. Oxford University Press, 1993.
10


Game Theoretic Approach to Post-Docked Satellite Control

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server