TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks

Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopic cholecystectomy videos with and without the use of additional surgical tool information. Outperforming various state-of-the-art LSTM approaches, we verify the suitability of the proposed causal MS-TCN for surgical phase recognition.Comment: 10 pages, 2 figure

Why is the Winner the Best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work

Validation de la pertinence d'un simulateur d'arthroscopie en réalité virtuelle pour caractériser les chirurgiens expérimentés

National audienceIntroduction: La simulation par réalité virtuelle (VR) est adaptée à l'apprentissage de l'arthroscopie. Malgré de nombreuses études, il reste néanmoins difficile de distinguer des niveaux de compétence (Construct Validity) parmi les chirurgiens expérimentés. Il semble donc adéquat de rechercher de nouvelles méthodes de mesure de compétence en utilisant l'analyse des trajectoires instrumentales au lieu des mesures couramment utilisées. Hypothèse: Nous avons émis l'hypothèse qu'une plus grande expérience en arthroscopie d’épaule serait corrélée à une meilleure performance sur simulateur VR d'arthroscopie d’épaule et que les opérateurs expérimentés partageraient des trajectoires instrumentales similaires. Matériels andamp; méthodes: Après réponse à un questionnaire standardisé, 104 trajectoires de 52 chirurgiens répartis en 2 cohortes (26 intermédiaires et 26 experts) ont été enregistrées sur simulateur d'arthroscopie. La procédure analysée était le retrait de corps étranger dans une articulation d’épaule droite. Dix mesures ont été calculées sur les trajectoires, notamment la durée de procédure, la distance instrumentale, l’économie de mouvement et la fluidité. De plus, le Dynamic Time Warping (DTW) a été calculé sur les trajectoires pour une classification hiérarchique non supervisée des chirurgiens. Résultats: Les experts furent significativement plus rapides (Médiane 70,9s Écart Interquartile [56,4–86,3] vs. 116,1s [82,8–154,2], p andlt; 0,01), plus fluides (4,6,105 mm.s−3 [3,1,105–7,2,105] vs. 1,5,106 mm.s−3 [2,6,106–3,5,106], p = 0,05), et plus économes en mouvement (19,3 mm2 [9,1–25,9] vs. 33,8 mm2 [14,8–50,5], p andlt; 0,01), mais il n'y avait pas de différence significative de performance sur la distance (671,4 mm [503,8–846,1] vs. 694,6 mm [467,0–1090,1], p = 0,62). Le clustering par DTW a différencié deux groupes de trajectoires similaires liées à l'expertise avec respectivement 48 trajectoires d'experts pour le premier groupe et 52 trajectoires d'intermédiaires et 4 d'experts pour le second groupe (Sensibilité de 92 %, Spécificité de 100 %). Le clustering hiérarchique avec DTW a identifié significativement les chirurgiens experts des intermédiaires en trouvant des trajectoires similaires pour 24/26 experts. Conclusion: Cette étude a démontré la Construct Validity du simulateur VR d'arthroscopie d’épaule au sein d'un groupe de chirurgiens expérimentés. Grâce à de nouveaux types de mesures simplement basées sur les trajectoires, il a été possible de distinguer de manière significative les niveaux d'expertise. Nous avons démontré que l'analyse par clustering avec Dynamic Time Warping est capable de distinguer de manière fiable les opérateurs experts des opérateurs intermédiaires. Pertinence clinique: Les résultats ont des implications pour l'avenir de la formation en chirurgie arthroscopique ou des programmes d'accréditation postuniversitaire utilisant la simulation par réalité virtuelle. Niveau de preuve: III; étude comparative prospective. © 2021 Elsevier Masson SA

A systematic review of annotation for surgical process model analysis in minimally invasive surgery based on video

International audienceBackgroundAnnotated data are foundational to applications of supervised machine learning. However, there seems to be a lack of common language used in the field of surgical data science.The aim of this study is to review the process of annotation and semantics used in the creation of SPM for minimally invasive surgery videos.MethodsFor this systematic review, we reviewed articles indexed in the MEDLINE database from January 2000 until March 2022. We selected articles using surgical video annotations to describe a surgical process model in the field of minimally invasive surgery. We excluded studies focusing on instrument detection or recognition of anatomical areas only. The risk of bias was evaluated with the Newcastle Ottawa Quality assessment tool. Data from the studies were visually presented in table using the SPIDER tool.ResultsOf the 2806 articles identified, 34 were selected for review. Twenty-two were in the field of digestive surgery, six in ophthalmologic surgery only, one in neurosurgery, three in gynecologic surgery, and two in mixed fields. Thirty-one studies (88.2%) were dedicated to phase, step, or action recognition and mainly relied on a very simple formalization (29, 85.2%). Clinical information in the datasets was lacking for studies using available public datasets. The process of annotation for surgical process model was lacking and poorly described, and description of the surgical procedures was highly variable between studies.ConclusionSurgical video annotation lacks a rigorous and reproducible framework. This leads to difficulties in sharing videos between institutions and hospitals because of the different languages used. There is a need to develop and use common ontology to improve libraries of annotated surgical videos

Sequential surgical signatures in micro-suturing task

International audiencePurpose: Surgical processes are generally only studied by identifying differences in populations such as participants or level of expertise. But the similarity between this population is also important in understanding the process. We therefore proposed to study these two aspects. Methods: In this article, we show how similarities in process workflow within a population can be identified as sequential surgical signatures. To this purpose, we have proposed a pattern mining approach to identify these signatures.Validation: We validated our method with a data set composed of seventeen micro-surgical suturing tasks performed by four participants with two levels of expertise.Results: We identified sequential surgical signatures specific to each participant , shared between participants with and without the same level of expertise. These signatures are also able to perfectly define the level of expertise of the participant who performed a new micro-surgical suturing task. However, it is more complicated to determine who the participant is, and the method correctly determines this information in only 64% of cases.Conclusion: We show for the first time the concept of sequential surgical signature. This new concept has the potential to further help to understand surgical procedures and provide useful knowledge to define future CAS systems

Temporal Pattern Mining for E-commerce Dataset

International audienceOver the last few years, several data mining algorithms have been developed to understand customers' behaviors in e-commerce platforms. They aim to extract knowledge and predict future actions on the website. In this paper we present three algorithms: SEPM-, SEPM+ and SEPM++ (Sequential Event Pattern Mining), for mining sequential frequent patterns. Our goal is to mine clickstream data to extract and analyze useful sequential patterns of clicks. For this purpose, we augment the vertical representation of patterns with additional information about the items' duration. Then based on this representation, we propose the necessary algorithms to mine sequential frequent patterns with the average duration of each of their items. Also, the direction of durations' variation in the sequence is taken into account by the algorithms.This duration is used as a proxy of the interest of the user in the content of the page.Finally, we categorize the resulting patterns and we prove that they are more discriminating than the standard ones. Our approach is tested on real data, and patterns found are analyzed to extract users' discriminatory behaviors. The experimental results on both real and synthetic datasets indicate that our algorithms are efficient and scalable

Pairing of α‐Fused BODIPY: Towards Panchromatic n‐Type Semiconducting Materials

International audienceA chemical strategy to efficiently perform the dimerization of α-fused boron-dipyrromethene (BODIPY) is reported. The straightforward synthesis of one of these dimers is described and its properties have been investigated through UV/Vis spectroscopy, cyclic voltammetry, differential scanning calorimetry, and charge-carrier mobility measurements by using organic field-effect transistors and space-charge-limited current diodes. The results allow a chemical strategy to decrease the tendency of α-fused BODIPY to crystallize, to increase its light-harvesting properties, and to promote isotropic charge carriers transport. Moreover, the disclosed approach is also a way to maintain the deep LUMO level of α-fused BODIPY; thus making this class of materials highly desirable for optoelectronic applications