Autonomous robots in endovascular operations have the potential to navigate
circulatory systems safely and reliably while decreasing the susceptibility to
human errors. However, there are numerous challenges involved with the process
of training such robots such as long training duration due to sample
inefficiency of machine learning algorithms and safety issues arising from the
interaction between the catheter and the endovascular phantom. Physics
simulators have been used in the context of endovascular procedures, but they
are typically employed for staff training and generally do not conform to the
autonomous cannulation goal. Furthermore, most current simulators are
closed-source which hinders the collaborative development of safe and reliable
autonomous systems. In this work, we introduce CathSim, an open-source
simulation environment that accelerates the development of machine learning
algorithms for autonomous endovascular navigation. We first simulate the
high-fidelity catheter and aorta with the state-of-the-art endovascular robot.
We then provide the capability of real-time force sensing between the catheter
and the aorta in the simulation environment. We validate our simulator by
conducting two different catheterisation tasks within two primary arteries
using two popular reinforcement learning algorithms, Proximal Policy
Optimization (PPO) and Soft Actor-Critic (SAC). The experimental results show
that using our open-source simulator, we can successfully train the
reinforcement learning agents to perform different autonomous cannulation
tasks