275 research outputs found
Anytime Control using Input Sequences with Markovian Processor Availability
We study an anytime control algorithm for situations where the processing
resources available for control are time-varying in an a priori unknown
fashion. Thus, at times, processing resources are insufficient to calculate
control inputs. To address this issue, the algorithm calculates sequences of
tentative future control inputs whenever possible, which are then buffered for
possible future use. We assume that the processor availability is correlated so
that the number of control inputs calculated at any time step is described by a
Markov chain. Using a Lyapunov function based approach we derive sufficient
conditions for stochastic stability of the closed loop.Comment: IEEE Transactions on Automatic Control, to be publishe
Sequence-based Anytime Control
We present two related anytime algorithms for control of nonlinear systems
when the processing resources available are time-varying. The basic idea is to
calculate tentative control input sequences for as many time steps into the
future as allowed by the available processing resources at every time step.
This serves to compensate for the time steps when the processor is not
available to perform any control calculations. Using a stochastic Lyapunov
function based approach, we analyze the stability of the resulting closed loop
system for the cases when the processor availability can be modeled as an
independent and identically distributed sequence and via an underlying Markov
chain. Numerical simulations indicate that the increase in performance due to
the proposed algorithms can be significant.Comment: 14 page
Stochastic Stability of Event-triggered Anytime Control
We investigate control of a non-linear process when communication and
processing capabilities are limited. The sensor communicates with a controller
node through an erasure channel which introduces i.i.d. packet dropouts.
Processor availability for control is random and, at times, insufficient to
calculate plant inputs. To make efficient use of communication and processing
resources, the sensor only transmits when the plant state lies outside a
bounded target set. Control calculations are triggered by the received data. If
a plant state measurement is successfully received and while the processor is
available for control, the algorithm recursively calculates a sequence of
tentative plant inputs, which are stored in a buffer for potential future use.
This safeguards for time-steps when the processor is unavailable for control.
We derive sufficient conditions on system parameters for stochastic stability
of the closed loop and illustrate performance gains through numerical studies.Comment: IEEE Transactions on Automatic Control, under revie
Decision-theoretic planning with non-Markovian rewards
A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decision-theoretic planning, where many desirable behaviours are more naturally expressed a
Decision-Theoretic Planning with non-Markovian Rewards
A decision process in which rewards depend on history rather than merely on
the current state is called a decision process with non-Markovian rewards
(NMRDP). In decision-theoretic planning, where many desirable behaviours are
more naturally expressed as properties of execution sequences rather than as
properties of states, NMRDPs form a more natural model than the commonly
adopted fully Markovian decision process (MDP) model. While the more tractable
solution methods developed for MDPs do not directly apply in the presence of
non-Markovian rewards, a number of solution methods for NMRDPs have been
proposed in the literature. These all exploit a compact specification of the
non-Markovian reward function in temporal logic, to automatically translate the
NMRDP into an equivalent MDP which is solved using efficient MDP solution
methods. This paper presents NMRDPP (Non-Markovian Reward Decision Process
Planner), a software platform for the development and experimentation of
methods for decision-theoretic planning with non-Markovian rewards. The current
version of NMRDPP implements, under a single interface, a family of methods
based on existing as well as new approaches which we describe in detail. These
include dynamic programming, heuristic search, and structured methods. Using
NMRDPP, we compare the methods and identify certain problem features that
affect their performance. NMRDPPs treatment of non-Markovian rewards is
inspired by the treatment of domain-specific search control knowledge in the
TLPlan planner, which it incorporates as a special case. In the First
International Probabilistic Planning Competition, NMRDPP was able to compete
and perform well in both the domain-independent and hand-coded tracks, using
search control knowledge in the latter
Remote State Estimation with Smart Sensors over Markov Fading Channels
We consider a fundamental remote state estimation problem of discrete-time
linear time-invariant (LTI) systems. A smart sensor forwards its local state
estimate to a remote estimator over a time-correlated -state Markov fading
channel, where the packet drop probability is time-varying and depends on the
current fading channel state. We establish a necessary and sufficient condition
for mean-square stability of the remote estimation error covariance as
, where denotes the
spectral radius, is the state transition matrix of the LTI system,
is a diagonal matrix containing the packet drop probabilities in
different channel states, and is the transition probability matrix
of the Markov channel states. To derive this result, we propose a novel
estimation-cycle based approach, and provide new element-wise bounds of matrix
powers. The stability condition is verified by numerical results, and is shown
more effective than existing sufficient conditions in the literature. We
observe that the stability region in terms of the packet drop probabilities in
different channel states can either be convex or concave depending on the
transition probability matrix . Our numerical results suggest that
the stability conditions for remote estimation may coincide for setups with a
smart sensor and with a conventional one (which sends raw measurements to the
remote estimator), though the smart sensor setup achieves a better estimation
performance.Comment: The paper has been accepted by IEEE Transactions on Automatic
Control. Copyright may be transferred without notice, after which this
version may no longer be accessibl
- …