Annotation-efficient learning of surgical instrument activity in neurosurgery

Machine learning-based solutions rely heavily on the quality and quantity of the training data. In the medical domain, the main challenge is to acquire rich and diverse annotated datasets for training. We propose to decrease the annotation efforts and further diversify the dataset by introducing an annotation-efficient learning workflow. Instead of costly pixel-level annotation, we require only image-level labels as the remainder is covered by simulation. Thus, we obtain a large-scale dataset with realistic images and accurateground truth annotations. We use this dataset for theinstrument localization activity task together with a student-teacher approach. We demonstrate the benefits of our workflow compared to state-of-the-art methods in instrument localization that are trained only on clinical datasets, which are fully annotated by human experts

Synthetic data generation for optical flow evaluation in the neurosurgical domain

Towards computer-assisted neurosurgery, scene understanding algorithms for microscope video data are required. Previous work utilizes optical flow to extract spatio-temporal context from neurosurgical video sequences. However, to select an appropriate optical flow method, we need to analyze which algorithm yields the highest accuracy for the neurosurgical domain. Currently, there are no benchmark datasets available for neurosurgery. In our work, we present an approach to generate synthetic data for optical flow evaluation on the neurosurgical domain. We simulate image sequences and thereby take into account domain-specific visual conditions such as surgical instrument motion. Then, we evaluate two optical flow algorithms, Farneback and PWC-Net, on our synthetic data. Qualitative and quantitative assessments confirm that our data can be used to evaluate optical flow for the neurosurgical domain. Future work will concentrate on extending the method by modeling additional effects in neurosurgery such as elastic background motion

Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks

Segmenting the boundary between tumor and healthy tissue during surgical cancer resection poses a significant challenge. In recent years, Hyperspectral Imaging (HSI) combined with Machine Learning (ML) has emerged as a promising solution. However, due to the extensive information contained within the spectral domain, most ML approaches primarily classify individual HSI (super-)pixels, or tiles, without taking into account their spatial context. In this paper, we propose an improved methodology that leverages the spatial context of tiles for more robust and smoother segmentation. To address the irregular shapes of tiles, we utilize Graph Neural Networks (GNNs) to propagate context information across neighboring regions. The features for each tile within the graph are extracted using a Convolutional Neural Network (CNN), which is trained simultaneously with the subsequent GNN. Moreover, we incorporate local image quality metrics into the loss function to enhance the training procedure's robustness against low-quality regions in the training images. We demonstrate the superiority of our proposed method using a clinical ex vivo dataset consisting of 51 HSI images from 30 patients. Despite the limited dataset, the GNN-based model significantly outperforms context-agnostic approaches, accurately distinguishing between healthy and tumor tissues, even in images from previously unseen patients. Furthermore, we show that our carefully designed loss function, accounting for local image quality, results in additional improvements. Our findings demonstrate that context-aware GNN algorithms can robustly find tumor demarcations on HSI images, ultimately contributing to better surgery success and patient outcome.Comment: 11 pages, 6 figure

Positron-emission tomography–based staging reduces the prognostic impact of early disease progression in patients with follicular lymphoma

Background: Previous studies reported that early progression of disease (POD) after initial therapy predicted poor overall survival (OS) in patients with follicular lymphoma (FL). Here, we investigated whether pre-treatment imaging modality had an impact on prognostic significance of POD. Methods: In this retrospective study, we identified 1088 patients with grade I–IIIA FL; of whom, 238 patients with stage II–IV disease were initially treated with rituximab, cyclophosphamide, doxorubicin, vincristine and prednisone (R-CHOP), and 346 patients were treated with rituximab-based chemotherapy. Patients (N = 484) from the FOLL05 study served as an independent validation cohort. We risk-stratified patients based on pre-treatment radiographic imaging (positron-emission tomography [PET] versus computed tomography [CT]) and early POD status using event-defining and landmark analyses. A competing risk analysis evaluated the association between early POD and histologic transformation. Results: In the discovery cohort, patients with POD within 24 months (PFS24) of initiating R-CHOP therapy had a 5-year OS of 57.6% for CT-staged patients compared with 70.6% for PET-staged patients. In the validation cohort, the 5-year OS for patients with early POD was 53.9% and 100% in CT- and PET-staged patients, respectively. The risk of histologic transformation in patients whose disease progressed within one year of initiating therapy was higher in CT-staged patients than in PET-staged patients (16.7% versus 6.3%, respectively), which was associated with a 9.7-fold higher risk of death. Conclusion: In FL, pre-treatment PET staging reduced the prognostic impact of early POD compared with CT staging. Patients with early POD and no histologic transformation have an extended OS with standard therapy

Variational and Deep Learning Approaches for Intrinsic Light Field Decomposition

Intrinsic image decomposition aims to separate an illumination invariant reflectance image from an input color image, which is still one of the fundamental problems in computer vision. This decomposition is widely used in photo and material editing, image segmentation and shape estimation tasks. According to the dichromatic reflection model, the light reflected from a scene point has two independent components: light reflected from the surface body and light at the interface. Body reflection is known as the diffuse component and it is independent of viewing direction, while interface reflection is known as the specular component and it is view-dependent. Most intrinsic image algorithms are designed for Lambertian scenes, with only diffuse reflection. However, their performance decreases if a scene contains specularity. In the real world, there are few scenes with only Lambertian objects. Instead, they have specular surfaces, which makes the decomposition problem harder due to the complicated nature of specular reflection. This thesis focuses on intrinsic light field decomposition, where we formulate and solve the problem with respect to three variables: albedo, shading, and specularity. Thus, we can deal with non-Lambertian scenes. We use a 4D light field, which is a collection of images sampled on a regular grid, instead of a single image. Rich information inherited from the light field allows us to distinguish between diffuse and specular reflection, and also allows us to robustly recover the intrinsic components. We tackle the problem with variational and deep learning approaches, compare their performance, and discuss the strengths and weaknesses of both techniques. In the variational method, we introduce priors for the intrinsic components and we solve an energy minimization problem with convex optimization. Because geometrical information plays an important role in the appearance and behavior of intrinsic components, we develop a disparity estimation method, where we not only optimize the disparity labels but also enforce piecewise smoothness of a normal map. Our deep learning approach is based on the assumption that if mathematical models allow us to compute a disparity and intrinsic components from a light field, then these models can be approximated with a deep convolutional neural network. Moreover, because disparity estimation and intrinsic light fields are closely related, a single network can be sufficient to perform all tasks together and they can benefit from each other. Thus, we establish a multi-task learning strategy for light fields, which is not only limited to the particular collection of tasks but (in theory) can also be used for various computer vision applications. We demonstrate the advantage of our approach on four state-of-the-art computer vision problems: disparity estimation, reflection separation, intrinsic images, and super-resolution. Extensive evaluations based on multiple, publicly-available, synthetic and real-world datasets prove our methodology and show the advantage of using light fields over other data structures. Our proposed algorithms outperform state-of-the-art methods for intrinsic images and disparity estimation, and achieve a competing quality for super-resolution and reflection separation.publishe

Ideas about difficult situations and other people among students with social frustrations

The article describes the problem of psychosocial personal characteristics as predictors of the coping behavior. Over the past few decades, the world has changed beyond recognition, the current generation of students has changed, and therefore the strategies that they use to cope with the difficulties that have appeared in their lives have also changed. The ways of their adaptation to modern reality have changed in comparison with the older generation. In this regard, the study of various predictors of coping behavior, which today prevail in the modern student environment, becomes relevant. The authors used content-analysis of metaphors, tests and methods of mathematical statistics (quartiles, regression analysis, Kruskal-Wallis H-test) and discovered the dependence between social frustration, metaphors of difficult life situations and coping-strategies. So, a high level of social frustration is connected with low rational and adaptive coping strategies in difficult life situations. Various coping strategies are associated with different metaphors of difficult life situations and of their participants: “friends” and “aliens”. The metaphors reflect personal perception of these situations based on scales “short distance-long distance”, “rational value-emotional value” and on parameters “undertaking the responsibility – placing the responsibility with others”, “stereotypical or differentiated image of the situation”

Light Field Intrinsics With a Deep Encoder-Decoder Network

We present a fully convolutional autoencoder for light fields, which jointly encodes stacks of horizontal and vertical epipolar plane images through a deep network of residual layers. The complex structure of the light field is thus reduced to a comparatively low-dimensional representation, which can be decoded in a variety of ways. The different pathways of upconvolution we currently support are for disparity estimation and separation of the lightfield into diffuse and specular intrinsic components. The key idea is that we can jointly perform unsupervised training for the autoencoder path of the network, and supervised training for the other decoders. This way, we find features which are both tailored to the respective tasks and generalize well to datasets for which only example light fields are available. We provide an extensive evaluation on synthetic light field data, and show that the network yields good results on previously unseen real world data captured by a Lytro Illum camera and various gantries.publishe

Digital core repository coupled with machine learning as a tool to classify and assess petrophysical rock properties

To make efficient use of image-based rock physics workflow, it is necessary to optimize different criteria, among which: quantity, representativeness, size and resolution. Advances in artificial intelligence give insights of databases potential. Deep learning methods not only enable to classify rock images, but could also help to estimate their petrophysical properties. In this study we prepare a set of thousands high-resolution 3D images captured in a set of four reservoir rock samples as a base for learning and training. The Voxilon software computes numerical petrophysical analysis. We identify different descriptors directly from 3D images used as inputs. We use convolutional neural network modelling with supervised training using TensorFlow framework. Using approximately fifteen thousand 2D images to drive the classification network, the test on thousand unseen images shows any error of rock type misclassification. The porosity trend provides good fit between digital benchmark datasets and machine learning tests. In a few minutes, database screening classifies carbonates and sandstones images and associates the porosity values and distribution. This work aims at conveying the potential of deep learning method in reservoir characterization to petroleum research, to illustrate how a smart image-based rock physics database at industrial scale can swiftly give access to rock properties

Synthetic data generation for optical flow evaluation in the neurosurgical domain

Towards computer-assisted neurosurgery, scene understanding algorithms for microscope video data are required. Previous work utilizes optical flow to extract spatio-temporal context from neurosurgical video sequences. However, to select an appropriate optical flow method, we need to analyze which algorithm yields the highest accuracy for the neurosurgical domain. Currently, there are no benchmark datasets available for neurosurgery. In our work, we present an approach to generate synthetic data for optical flow evaluation on the neurosurgical domain. We simulate image sequences and thereby take into account domain-specific visual conditions such as surgical instrument motion. Then, we evaluate two optical flow algorithms, Farneback and PWC-Net, on our synthetic data. Qualitative and quantitative assessments confirm that our data can be used to evaluate optical flow for the neurosurgical domain. Future work will concentrate on extending the method by modeling additional effects in neurosurgery such as elastic background motion