On gradual-impulse control of continuous-time Markov decision processes with multiplicative cost

In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very general conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies. Policies that we consider allow multiple simultaneous impulses, randomized selection of impulses with random effects, relaxed gradual controls, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the continuous-time gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions

NOTE ON DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH A LOWER BOUNDING FUNCTION

In this paper, we consider the discounted continuous-time Markov decision process (CTMDP) with a lower bounding function. In this model, the negative part of each cost rate is bounded by the drift function, say $w$, whereas the positive part is allowed to be arbitrarily unbounded. Our focus is on the existence of a stationary optimal policy for the discounted CTMDP problems out of the more general class. Both constrained and unconstrained problems are considered. Our investigations are based on a useful transformation for nonhomogeneous Markov pure jump processes that has not yet been widely applied to the study of CTMDPs. This technique was not employed in previous literature, but it clarifies the roles of the imposed conditions in a rather transparent way. As a consequence, we withdraw and weaken several conditions commonly imposed in the literature

A Novel Data Association Algorithm for Unequal Length Fluctuant Sequence

AbstractThere are quantities of such sensors as radar, ESM, navigator in aerospace areas and the sequence data is the most ordinary data in sensor domain. How to mine the information of these data has attracted a great interest in data mining. But sequence data is easily interfered and produces some fluctuant points. When dealing with these sequences, traditional sequence similarity measurement such as Euclidean distance arises large error, especially for unequal length fluctuant sequence. A novel average weight 1-norm unequal length fluctuant sequence similarity measurement algorithm based on dynamic time warping (DTW) is proposed to solve this problem. It constructs an absolute distance matrix based on DTW firstly, then weight average weight 1-norm and modify it with modifying factor to measure the distance of unequal length fluctuant sequence. It solves the fluctuation sensitivity of maximum distance measurement algorithm. Finally transform distance to similarity as the index of the association, associate the sequence data according to the maximum similarity association rule. Simulation results show the effectiveness of the proposed algorithm when associating unequal length fluctuant sequence, association rate is above 70% and simulate the effect of variation of the sequence length, fluctuant rate and processing time to the proposed algorithm