Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy

Objective: Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical structure classification and image tagging that features an intrinsic measure of confidence to estimate its own performance with high reliability and which can be applied to both RGB and multispectral imaging (MI) data. Methods: Organ recognition is performed using a superpixel classification strategy based on textural and reflectance information. Classification confidence is estimated by analyzing the dispersion of class probabilities. Assessment of the proposed technology is performed through a comprehensive in vivo study with seven pigs. Results: When applied to image tagging, mean accuracy in our experiments increased from 65% (RGB) and 80% (MI) to 90% (RGB) and 96% (MI) with the confidence measure. Conclusion: Results showed that the confidence measure had a significant influence on the classification accuracy, and MI data are better suited for anatomical structure labeling than RGB data. Significance: This work significantly enhances the state of art in automatic labeling of endoscopic videos by introducing the use of the confidence metric, and by being the first study to use MI data for in vivo laparoscopic tissue classification. The data of our experiments will be released as the first in vivo MI dataset upon publication of this paper.Comment: 7 pages, 6 images, 2 table

Preterm Infants' Pose Estimation with Spatio-Temporal Features

Objective: Preterm infants' limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants' health status and motor/cognitive development. Herein, we propose a new approach to preterm infants' limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability. Methods: Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset). Results: When applied to pose estimation, the median root mean square distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27 pixels). Conclusion: Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity). Significance: This article significantly enhances the state of art in automatic assessment of preterm infants' health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants' pose estimation

Preterm infants' limb-pose estimation from depth images using convolutional neural networks

Preterm infants' limb-pose estimation is a crucial but challenging task, which may improve patients' care and facilitate clinicians in infant's movements monitoring. Work in the literature either provides approaches to whole-body segmentation and tracking, which, however, has poor clinical value, or retrieve a posteriori limb pose from limb segmentation, increasing computational costs and introducing inaccuracy sources. In this paper, we address the problem of limb-pose estimation under a different point of view. We proposed a 2D fully-convolutional neural network for roughly detecting limb joints and joint connections, followed by a regression convolutional neural network for accurate joint and joint-connection position estimation. Joints from the same limb are then connected with a maximum bipartite matching approach. Our analysis does not require any prior modeling of infants' body structure, neither any manual interventions. For developing and testing the proposed approach, we built a dataset of four videos (video length = 90 s) recorded with a depth sensor in a neonatal intensive care unit (NICU) during the actual clinical practice, achieving median root mean square distance [pixels] of 10.790 (right arm), 10.542 (left arm), 8.294 (right leg), 11.270 (left leg) with respect to the ground-truth limb pose. The idea of estimating limb pose directly from depth images may represent a future paradigm for addressing the problem of preterm-infants' movement monitoring and offer all possible support to clinicians in NICUs

HIGH RESOLUTION SURVEY OF MOSAICS OF THE CRYPT OF THE ST. NICOLA’S BASILICA (BARI, ITALY) AND CHARACTERIZATION AND PROVENANCE STUDIES OF MARBLE TESSERAE

This paper focusses on the mosaics of the crypt of the St. Nicola’s Basilica in Bari, a valuable evidence of use and reuse of ancient white and coloured marbles from the Roman world, together with local and imitation stones. The study belongs to a wider research project (MARMORA), about ancient marbles employed in the Apulia Cultural Heritage, and aims to improve knowledge and preserve these artworks, in order to enhance their valorisation and enjoyment. Therefore, firstly a high definition survey of mosaic floors was performed and after, characterization and provenance studies of stone tesserae, recognition of geometrical motifs and stylistic influence were carried out. Preliminary results allowed to obtain a digital representation of mosaics, including all the contributions on material characterisation and provenance

A regression framework to head-circumference delineation from US fetal images

Background and Objectives: Measuring head-circumference (HC) length from ultrasound (US) images is a crucial clinical task to assess fetus growth. To lower intra- and inter-operator variability in HC length measuring, several computer-assisted solutions have been proposed in the years. Recently, a large number of deep-learning approaches is addressing the problem of HC delineation through the segmentation of the whole fetal head via convolutional neural networks (CNNs). Since the task is a edge-delineation problem, we propose a different strategy based on regression CNNs. Methods: The proposed framework consists of a region-proposal CNN for head localization and centering, and a regression CNN for accurately delineate the HC. The first CNN is trained exploiting transfer learning, while we propose a training strategy for the regression CNN based on distance fields. Results: The framework was tested on the HC18 Challenge dataset, which consists of 999 training and 335 testing images. A mean absolute difference of 1.90 ( ± 1.76) mm and a Dice similarity coefficient of 97.75 ( ± 1.32) % were achieved, overcoming approaches in the literature. Conclusions: The experimental results showed the effectiveness of the proposed framework, proving its potential in supporting clinicians during the clinical practice

Vocal Folds Disorders Detection and Classification in Endoscopic Narrow-Band Images

The diagnosis of vocal folds (VF) diseases is error- prone due to the large variety of diseases that can affect them. VF lesions can be divided in nodular, e.g. nodules, polyps and cysts, and diffuse, e.g. hyperplastic laryngitis and carcinoma. By endoscopic examination, the clinician traditionally evaluates the presence of macroscopic formations and mucosal vessels alteration. Endoscopic narrow-band imaging (NBI) has recently started to be employed since it provides enhanced vessels contrast as compared to classical white-light endoscopy. This work presents a preliminary study on the development of an automatic diagnostic tool based on the assessment of vocal cords symmetry in NBI images. The objective is to identify possible protruding mass lesions on which subsequent vessels analysis may be performed. The method proposed here is based on the segmentation of the glottal area (GA) from the endoscopic images, based on which the right and the left portions of the vocal folds are detected and analyzed for the detection of protruding areas. The obtained information is then used to classify the VF edges as healthy or pathological. Results from the analysis of 22 endoscopic NBI images demonstrated that the proposed algorithm is robust and effective, providing a 100% success rate in the classification of VF edges as healthy or pathological. Such results support the investment in further research to expand and improve the algorithm presented here, potentially with the addition of vessels analysis to determine the pathological classification of detected protruding areas

Heartbeat detection by laser doppler vibrometry and machine learning

Background: Heartbeat detection is a crucial step in several clinical fields. Laser Doppler Vibrometer (LDV) is a promising non-contact measurement for heartbeat detection. The aim of this work is to assess whether machine learning can be used for detecting heartbeat from the carotid LDV signal. Methods: The performances of Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF) and K-Nearest Neighbor (KNN) were compared using the leave-one-subject-out cross-validation as the testing protocol in an LDV dataset collected from 28 subjects. The classification was conducted on LDV signal windows, which were labeled as beat, if containing a beat, or no-beat, otherwise. The labeling procedure was performed using electrocardiography as the gold standard. Results: For the beat class, the f1-score (f 1) values were 0.93, 0.93, 0.95, 0.96 for RF, DT, KNN and SVM, respectively. No statistical differences were found between the classifiers. When testing the SVM on the full-length (10 min long) LDV signals, to simulate a real-world application, we achieved a median macro-f 1 of 0.76. Conclusions: Using machine learning for heartbeat detection from carotid LDV signals showed encouraging results, representing a promising step in the field of contactless cardiovascular signal analysis