Investigation on energetic optimization problems of stochastic thermodynamics with iterative dynamic programming

The energetic optimization problem, e.g., searching for the optimal switch- ing protocol of certain system parameters to minimize the input work, has been extensively studied by stochastic thermodynamics. In current work, we study this problem numerically with iterative dynamic programming. The model systems under investigation are toy actuators consisting of spring-linked beads with loading force imposed on both ending beads. For the simplest case, i.e., a one-spring actuator driven by tuning the stiffness of the spring, we compare the optimal control protocol of the stiffness for both the overdamped and the underdamped situations, and discuss how inertial effects alter the irreversibility of the driven process and thus modify the optimal protocol. Then, we study the systems with multiple degrees of freedom by constructing oligomer actuators, in which the harmonic interaction between the two ending beads is tuned externally. With the same rated output work, actuators of different constructions demand different minimal input work, reflecting the influence of the internal degrees of freedom on the performance of the actuators.Comment: 14 pages, 7 figures, communications in computational physics, in pres

Optimal transport over a linear dynamical system

We consider the problem of steering an initial probability density for the state vector of a linear system to a final one, in finite time, using minimum energy control. In the case where the dynamics correspond to an integrator ($\dot x(t) = u(t)$) this amounts to a Monge-Kantorovich Optimal Mass Transport (OMT) problem. In general, we show that the problem can again be reduced to solving an OMT problem and that it has a unique solution. In parallel, we study the optimal steering of the state-density of a linear stochastic system with white noise disturbance; this is known to correspond to a Schroedinger bridge. As the white noise intensity tends to zero, the flow of densities converges to that of the deterministic dynamics and can serve as a way to compute the solution of its deterministic counterpart. The solution can be expressed in closed-form for Gaussian initial and final state densities in both cases

Inverse stochastic optimal controls

We study an inverse problem of the stochastic optimal control of general diffusions with performance index having the quadratic penalty term of the control process. Under mild conditions on the drift, the volatility, the cost functions of the state, and under the assumption that the optimal control belongs to the interior of the control set, we show that our inverse problem is well-posed using a stochastic maximum principle. Then, with the well-posedness, we reduce the inverse problem to some root finding problem of the expectation of a random variable involved with the value function, which has a unique solution. Based on this result, we propose a numerical method for our inverse problem by replacing the expectation above with arithmetic mean of observed optimal control processes and the corresponding state processes. The recent progress of numerical analyses of Hamilton-Jacobi-Bellman equations enables the proposed method to be implementable for multi-dimensional cases. In particular, with the help of the kernel-based collocation method for Hamilton-Jacobi-Bellman equations, our method for the inverse problems still works well even when an explicit form of the value function is unavailable. Several numerical experiments show that the numerical method recover the unknown weight parameter with high accuracy