A polynomial model of surgical gestures for real-time retrieval of surgery videos

International audienceThis paper introduces a novel retrieval frame- work for surgery videos. Given a query video, the goal is to retrieve videos in which similar surgical gestures ap- pear. In this framework, the motion content of short video subsequences is modeled, in real-time, using spatiotempo- ral polynomials. The retrieval engine needs to be trained: key spatiotemporal polynomials, characterizing semantically- relevant surgical gestures, are identified through multiple- instance learning. Then, videos are compared in a high-level space spanned by these key spatiotemporal polynomials. The framework was applied to a dataset of 900 manually-delimited clips from 100 cataract surgery videos. High classification performance (Az = 0.816 ± 0.118) and retrieval performance (MAP = 0.358) were observed