Partially observable queueing systems with controlled service rates under a discounted optimality criterion

summary:We are concerned with a class of $GI/GI/1$ queueing systems with controlled service rates, in which the waiting times are only observed when they take zero value. Applying a suitable filtering process, we show the existence of optimal control policies under a discounted optimality criterion

Solutions of semi-Markov control models with recursive discount rates and approximation by ϵ\epsilon-optimal policies

summary:This paper studies a class of discrete-time discounted semi-Markov control model on Borel spaces. We assume possibly unbounded costs and a non-stationary exponential form in the discount factor which depends of on a rate, called the discount rate. Given an initial discount rate the evolution in next steps depends on both the previous discount rate and the sojourn time of the system at the current state. The new results provided here are the existence and the approximation of optimal policies for this class of discounted Markov control model with non-stationary rates and the horizon is finite or infinite. Under regularity condition on sojourn time distributions and measurable selector conditions, we show the validity of the dynamic programming algorithm for the finite horizon case. By the convergence in finite steps to the value functions, we guarantee the existence of non-stationary optimal policies for the infinite horizon case and we approximate them using non-stationary $\epsilon$-optimal policies. We illustrated our results a discounted semi-Markov linear-quadratic model, when the evolution of the discount rate follows an appropriate type of stochastic differential equation

Discrete-time Markov control processes with recursive discount rates

summary:This work analyzes a discrete-time Markov Control Model (MCM) on Borel spaces when the performance index is the expected total discounted cost. This criterion admits unbounded costs. It is assumed that the discount rate in any period is obtained by using recursive functions and a known initial discount rate. The classic dynamic programming method for finite-horizon case is verified. Under slight conditions, the existence of deterministic non-stationary optimal policies for infinite-horizon case is proven. Also, to find deterministic non-stationary $\epsilon-$optimal policies, the value-iteration method is used. To illustrate an example of recursive functions that generate discount rates, we consider the expected values of stochastic processes, which are solutions of certain class of Stochastic Differential Equations (SDE) between consecutive periods, when the initial condition is the previous discount rate. Finally, the consumption-investment problem and the discount linear-quadratic problem are presented as examples; in both cases, the discount rates are obtained using a SDE, similar to the Vasicek short-rate model

On the strong convergence of the Faedo-Galerkin approximations to a strong T-periodic solution of the torso-coupled bidomain model

In this paper, we investigate the convergence of the Faedo-Galerkin approximations, in a strong sense, to a strong T-periodic solution of the torso-coupled bidomain model where T is the period of activation of the inner wall of the heart. First, we define the torso-coupled bidomain operator and prove some of its more important properties for our work. After, we define the abstract evolution system of the equations that are associated with torso-coupled bidomain model and give the definition of a strong solution. We prove that the Faedo-Galerkin’s approximations have the regularity of a strong solution, and we find that some restrictions can be imposed over the initial conditions, so that this sequence of Faedo-Galerkin fully converges to a strong solution of the Cauchy problem. Finally, these results are used for showing the existence a strong T-periodic solution