Manipulation of tissue with surgical tools often results in large
deformations that current methods in tracking and reconstructing algorithms
have not effectively addressed. A major source of tracking errors during large
deformations stems from wrong data association between observed sensor
measurements with previously tracked scene. To mitigate this issue, we present
a surgical perception framework, SuPerPM, that leverages learning-based
non-rigid point cloud matching for data association, thus accommodating larger
deformations. The learning models typically require training data with ground
truth point cloud correspondences, which is challenging or even impractical to
collect in surgical environments. Thus, for tuning the learning model, we
gather endoscopic data of soft tissue being manipulated by a surgical robot and
then establish correspondences between point clouds at different time points to
serve as ground truth. This was achieved by employing a position-based dynamics
(PBD) simulation to ensure that the correspondences adhered to physical
constraints. The proposed framework is demonstrated on several challenging
surgical datasets that are characterized by large deformations, achieving
superior performance over state-of-the-art surgical scene tracking algorithms.Comment: Under review for ICRA202