Fouille de séquences d'images médicales. Application en chirurgie mini-invasive augmentée

In this thesis, we are interested in computer-aided ophthalmic surgery. In this goal, we propose to use surgery videos already stored in database and associated with contextual information (data patients, diagnostics ... etc). During the surgery, the surgeon is focused on his task. We try to improve the surgical procedures by proposing a system able, at any time, to guide the surgery steps by generating surgical warnings or recommendations if the current surgery shares signs of complications with already stored videos. Our goal is to develop methods and a system to select in the databases videos similar to a video stream captured by a digital camera monitoring the surgery (query). Our work will therefore implement methods related to Content Based Video Retrieval (CBVR) and Case-Based Reasoning (CBR). The methods are evaluated on three databases. The first two databases are collected at Brest University Hospital (France): the epiretinal membrane surgery dataset and the cataract surgery dataset. Third, in order to assess its generality, the system is applied to a large dataset of movie clips (Holywood) with classified human actions. To caracterize our videos, we proposed three original indexing methods derived from the compressed ``MPEG-4 AVC/H.264'' video stream. 1) A global method is based on motion histogram created for every frame of a compressed video sequence to extract motion direction and intensity statistics. 2) A local method combine segmentation and tracking to extract region displacements between consecutive I-frames and therefore characterize region trajectories. 3) To reduce the loss of information caused by using only the I-frames, we constructed a summary of each video based on a selection of the Group Of Pictures (GOP defined in the standard of compression). An originality of these methods comes from the use of the compressed domain, they not rely on standard methods, such as the optical flow, to characterize motion in videos. Instead, motion is directly extracted from the compressed MPEG stream. The goal is to provide a fast video characterization. Once videos are characterized, search is made by computing, within the meaning of a given metric, the distance between the signature of the query video and the signature of videos in the database. This computing can select videos as answer to the query without any semantic meaning. For this we use three methods. DTW (Dynamic Time Warping) provides an effective distance between two sequences of images. This algorithm is at the origin of the fast algorithm (FDTW) that we use to compare signatures in the first method. To compare signatures resulting from approach based on region motion trajectories, we propose to use a combination of FDTW and EMD (Earth Mover's Distance). The proposed extension of FDTW is referred to as EFDTW. To improve the retrieval result, we introduce an optimization process for computing distances between signature, by using genetic algorithms. The results obtained on the two medical databases are satisfactory. Thus, the mean precision at five reaches 79% (4 videos similar to the query video) on the epiretinal membrane surgery dataset and 72,69% (3 to 4 videos similar to the query video) on the cataract surgery dataset.Dans cette thèse, nous nous intéressons à l'aide à la décision lors d'interventions chirurgicales. Dans ce but, nous proposons d'utiliser des enregistrements vidéos acquis lors d'interventions chirurgicales antérieures, vidéos numérisées et archivées dans des dossiers d'intervention, contenant toutes les informations relatives à leur déroulement. Au cours de l'opération, le chirurgien ne peut pas consulter lui même des dossiers et vidéos déjà archivées car il est totalement concentré sur l'acte; par contre des outils d'analyse automatique en temps réel des images acquises en cours d'opération pourraient permettre cette utilisation de séquences déjà archivées, avec comme applications directes : des alertes en cas de problème, des informations sur les suites de tel ou tel geste dans des situations opératoires voisines (opération, caractéristiques patient, etc ...), des conseils sur les décisions. Notre objectif est donc de développer des méthodes permettant de sélectionner dans des archives des vidéos similaires à la vidéo proposée en requête. Nous nous appuyons pour cela sur la recherche de vidéos par le contenu (CBVR : Content Based Video Retrieval) et le raisonnement à base de cas (CBR : Case Based Reasoning). Les méthodes sont évaluées sur trois bases de données. Les deux premières bases de données étudiées sont des bases réalisées en chirurgie ophtalmologique, en collaboration avec le service d'ophtalmologie du CHRU de Brest : une base de chirurgie de pelage de membrane de la rétine et une base de chirurgie de la cataracte. La troisième base est la base de clips vidéo Hollywood, utilisée pour montrer la généricité des méthodes proposées. Pour caractériser les vidéos, nous proposons trois méthodes originales d'indexation à partir du domaine compressé : 1) une première méthode consiste à caractériser globalement la vidéo en utilisant des histogrammes de directions de mouvement, 2) une deuxième méthode est basée sur une segmentation spatio-temporelle et sur le suivi des régions entre deux images I, pour construire une signature décrivant la trajectoire des régions identifiées comme les plus importantes visuellement, 3) la troisième méthode est une variante de la deuxième méthode : afin de réduire la perte d'information engendrée en utilisant uniquement les images I, nous avons construit un résumé de la vidéo basé sur une sélection des Group Of Pictures (groupes d'images définis dans la norme de compression). Une des originalités de ces trois méthodes est d'utiliser les données vidéos dans le domaine compressé. Ce choix nous permet d'accéder à des éléments caractérisant les vidéos d'une manière rapide et efficace, sans devoir passer par la reconstruction totale du flux vidéo à partir du flux compressé

Content-based medical video retrieval based on region motion trajectories

International audienceIn this paper, we address the problem of contentbased medical video retrieval. We propose the use of motion tracking to generate video features. First, we extract motion vectors derived from the 'MPEG-4 AVC/H.264' standard. Second, motion segmentation of the image sequence is performed by a combination of k-means clustering and motion consistency verification. Third, we used the well-known Kalman filter to track region motion between consecutive frames. This produced region's correspondences are concatenated to construct the region's motion trajectories. Finally, to compare videos, we adopted an extension of dynamic time warping (EDTW) to multidimensional time series. Results are promising: a retrieval precision at 5 of 62 % was achieved

Real-Time Retrieval of Similar Videos with Application to Computer-Aided Retinal Surgery

International audienceThis paper introduces ongoing research on computer-aided ophthalmic surgery. In particular, a novel Content-Based Video Retrieval (CBVR) system is presented. Its purpose is the following: given a video stream captured by a digital camera monitoring the surgery, the system should retrieve, in real-time, similar video subsequences in video archives. In order to retrieve semantically-relevant videos, most existing CBVR systems rely on temporally flexible distance measures such as Dynamic Time Warping. These distance measures are slow and therefore do not allow real-time retrieval. In the proposed system, temporal flexibility is introduced in the way video subsequences are characterized, which allows the use of simple and fast distance measures. As a consequence, realtime retrieval of similar video subsequences, among hundreds of thousands of examples, is now possible. Besides, the proposed system is adaptive: a fast training procedure is presented. The system has been successfully applied to automated recognition of retinal surgery steps on a 69-video dataset: areas under the Receiver Operating Characteristic curves range from Az=0.809 to Az=0.989

Motion-Based Video Retrieval with Application to Computer-Assisted Retinal Surgery

International audienceIn this paper, we address the problem of computer-aided ophthalmic surgery. In particular, a novel Content-Based Video Retrieval (CBVR) system is presented : given a video stream captured by a digital camera monitoring the current surgery, the system retrieves, within digital archives, videos that resemble the current surgery monitoring video. The search results may be used to guide surgeons' decisions, for example, let the surgeon know what a more experienced fellow worker would do in a similar situation. With this goal, we propose to use motion information contained in MPEG- 4 AVC/H.264 video standard to extract features from videos. We propose two approaches, one of which is based on motion histogram created for every frame of a compressed video sequence to extract motion direction and intensity statistics. The other combine segmentation and tracking to extract region displacements between consecutive frames and therefore char- acterize region trajectories. To compare videos, an extension of the fast dynamic time warping to multidimensional time series was adopted. The system is applied to a dataset of 69 video-recorded retinal surgery steps. Results are promising: the retrieval efficiency is higher than 69%

Studying Disagreements among Retinal Experts through Image Analysis

International audienceIn recent years, many image analysis algorithms have been presented to assist Diabetic Retinopathy (DR) screening. The goal was usually to detect healthy examination records automatically, in order to reduce the number of records that should be analyzed by retinal experts. In this paper, a novel application is presented: these algorithms are used to 1) discover image characteristics that sometimes cause an expert to disagree with his/her peers and 2) warn the expert whenever these characteristics are detected in an examination record. In a DR screening program, each examination record is only analyzed by one expert, therefore analyzing disagreements among experts is challenging. A statistical framework, based on Parzen- windowing and the Patrick-Fischer distance, is presented to solve this problem. Disagreements among eleven experts from the Ophdiat screening program were analyzed, using an archive of 25,702 examination records