3D surface reconstruction using dense optical flow combined to feature matching: Application to endoscopy

International audienceIn structure from motion (SfM) algorithms, the surface reconstruction performance strongly depends on the quality of the determination of homologous points between images. Classical feature matching-based methods as integrated in the state-of the-art SfM-algorithms are often inoperative for scenes including weak structures and textures (e.g., as those in medical endoscopic videos). This contribution introduces an effective solution based on the combination of dense optical flow and feature matching. The accuracy and robustness of the proposed method were validated using results obtained for a phantom with known dimensions and with patient data, respectively. Apart from the high performance obtained for cystoscopy and gastroscopy, the proposed solution has a high potential in other medical and non-medical scenes.Dans les algorithmes de structures à partir du mouvement (SfM), la performance de la reconstruction des surfaces dépend fortement de la qualité de la détermination des points homologues entre images. Les méthodes SfM de référence sont souvent inopérantes pour les scènes avec peu de structures et textures faiblement contrastées car elles reposent uniquement sur l'appariement de caractéristiques. Cette contribution présente une solution associant un flot optique dense à la mise en correspondance de caractéristiques. La précision et la robustesse de la reconstruction ont été validées via des résultats obtenus pour un fantôme avec des dimensions connues et avec des données patient en cystoscopie et en gastroscopie, respectivement. Plus généralement, cette approche a un fort potentiel pour toute scène peu constrastée, médicales ou non

Hydrogen bond stabilization in Diels–Alder transition states: The cycloaddition of hydroxy-ortho-quinodimethane with fumaric acid and dimethylfumarate

DFT investigations on the mechanism of Diels–Alder reactions of a hydroxy-ortho-quinodimethane with fumaric acid derivatives were performed to understand the origin of the syn or anti configuration of the adducts. The diene hydroxyl group and the dieneophile carboxyl group show hydrogen bonding in the transition state, significantly favouring the syn product. This reaction is poorly diastereoselective for R = CO₂Me (ratio syn/anti = 57:43) and significantly improved for R = CO₂H (ratio syn/anti = 71:29). The stereoselectivities are properly predicted from transition structures calculated at the B3LYP/6-31G(d) level

Causal Scoring Medical Image Explanations: A Case Study On Ex-vivo Kidney Stone Images

On the promise that if human users know the cause of an output, it would enable them to grasp the process responsible for the output, and hence provide understanding, many explainable methods have been proposed to indicate the cause for the output of a model based on its input. Nonetheless, little has been reported on quantitative measurements of such causal relationships between the inputs, the explanations, and the outputs of a model, leaving the assessment to the user, independent of his level of expertise in the subject. To address this situation, we explore a technique for measuring the causal relationship between the features from the area of the object of interest in the images of a class and the output of a classifier. Our experiments indicate improvement in the causal relationships measured when the area of the object of interest per class is indicated by a mask from an explainable method than when it is indicated by human annotators. Hence the chosen name of Causal Explanation Score (CaES

Prospecting lighting applications with ligand field tools and density functional theory: a first-principles account of the 4f⁷–4f⁶5d¹ Luminescence of CsMgBr₃:Eu²⁺

The most efficient way to provide domestic lighting nowadays is by light-emitting diodes (LEDs) technology combined with phosphors shifting the blue and UV emission toward a desirable sunlight spectrum. A route in the quest for warm-white light goes toward the discovery and tuning of the lanthanide-based phosphors, a difficult task, in experimental and technical respects. A proper theoretical approach, which is also complicated at the conceptual level and in computing efforts, is however a profitable complement, offering valuable structure–property rationale as a guideline in the search of the best materials. The Eu²⁺-based systems are the prototypes for ideal phosphors, exhibiting a wide range of visible light emission. Using the ligand field concepts in conjunction with density functional theory (DFT), conducted in nonroutine manner, we develop a nonempirical procedure to investigate the 4f⁷–4f⁶5d¹ luminescence of Eu²⁺ in the environment of arbitrary ligands, applied here on the CsMgBr₃:Eu²⁺-doped material. Providing a salient methodology for the extraction of the relevant ligand field and related parameters from DFT calculations and encompassing the bottleneck of handling large matrices in a model Hamiltonian based on the whole set of 33 462 states, we obtained an excellent match with the experimental spectrum, from first-principles, without any fit or adjustment. This proves that the ligand field density functional theory methodology can be used in the assessment of new materials and rational property design

Deep Prototypical-Parts Ease Morphological Kidney Stone Identification and are Competitively Robust to Photometric Perturbations

Identifying the type of kidney stones can allow urologists to determine their cause of formation, improving the prescription of appropriate treatments to diminish future relapses. Currently, the associated ex-vivo diagnosis (known as Morpho-constitutional Analysis, MCA) is time-consuming, expensive and requires a great deal of experience, as it requires a visual analysis component that is highly operator dependant. Recently, machine learning methods have been developed for in-vivo endoscopic stone recognition. Deep Learning (DL) based methods outperform non-DL methods in terms of accuracy but lack explainability. Despite this trade-off, when it comes to making high-stakes decisions, it's important to prioritize understandable Computer-Aided Diagnosis (CADx) that suggests a course of action based on reasonable evidence, rather than a model prescribing a course of action. In this proposal, we learn Prototypical Parts (PPs) per kidney stone subtype, which are used by the DL model to generate an output classification. Using PPs in the classification task enables case-based reasoning explanations for such output, thus making the model interpretable. In addition, we modify global visual characteristics to describe their relevance to the PPs and the sensitivity of our model's performance. With this, we provide explanations with additional information at the sample, class and model levels in contrast to previous works. Although our implementation's average accuracy is lower than state-of-the-art (SOTA) non-interpretable DL models by 1.5 %, our models perform 2.8% better on perturbed images with a lower standard deviation, without adversarial training. Thus, Learning PPs has the potential to create more robust DL models.Comment: This paper has been accepted at the LatinX in Computer Vision Research Workshop at CVPR2023 as a full paper and it will appear on the CVPR proceeding

Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies

This contribution presents a deep learning method for the extraction and fusion of information relating to kidney stone fragments acquired from different viewpoints of the endoscope. Surface and section fragment images are jointly used during the training of the classifier to improve the discrimination power of the features by adding attention layers at the end of each convolutional block. This approach is specifically designed to mimic the morpho-constitutional analysis performed in ex-vivo by biologists to visually identify kidney stones by inspecting both views. The addition of attention mechanisms to the backbone improved the results of single view extraction backbones by 4% on average. Moreover, in comparison to the state-of-the-art, the fusion of the deep features improved the overall results up to 11% in terms of kidney stone classification accuracy.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Improving automatic endoscopic stone recognition using a multi-view fusion approach enhanced with two-step transfer learning

This contribution presents a deep-learning method for extracting and fusing image information acquired from different viewpoints, with the aim to produce more discriminant object features for the identification of the type of kidney stones seen in endoscopic images. The model was further improved with a two-step transfer learning approach and by attention blocks to refine the learned feature maps. Deep feature fusion strategies improved the results of single view extraction backbone models by more than 6% in terms of accuracy of the kidney stones classification.Comment: This paper has been accepted at the LatinX in Computer Vision (LXCV) Research workshop at ICCV 2023 (Paris, France