2 research outputs found
Safe Offline Reinforcement Learning with Real-Time Budget Constraints
Aiming at promoting the safe real-world deployment of Reinforcement Learning
(RL), research on safe RL has made significant progress in recent years.
However, most existing works in the literature still focus on the online
setting where risky violations of the safety budget are likely to be incurred
during training. Besides, in many real-world applications, the learned policy
is required to respond to dynamically determined safety budgets (i.e.,
constraint threshold) in real time. In this paper, we target at the above
real-time budget constraint problem under the offline setting, and propose
Trajectory-based REal-time Budget Inference (TREBI) as a novel solution that
approaches this problem from the perspective of trajectory distribution.
Theoretically, we prove an error bound of the estimation on the episodic reward
and cost under the offline setting and thus provide a performance guarantee for
TREBI. Empirical results on a wide range of simulation tasks and a real-world
large-scale advertising application demonstrate the capability of TREBI in
solving real-time budget constraint problems under offline settings.Comment: We propose a method to handle the constraint problem with dynamically
determined safety budgets under the offline settin
Image Jacobian Matrix Estimation Based on Online Support Vector Regression
Research into robotics visual servoing is an important area in the field of robotics. It has proven difficult to achieve successful results for machine vision and robotics in unstructured environments without using any a priori camera or kinematic models. In uncalibrated visual servoing, image Jacobian matrix estimation methods can be divided into two groups: the online method and the offline method. The offline method is not appropriate for most natural environments. The online method is robust but rough. Moreover, if the images feature configuration changes, it needs to restart the approximating procedure. A novel approach based on an online support vector regression (OL-SVR) algorithm is proposed which overcomes the drawbacks and combines the virtues just mentioned