Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery

Semantic tool segmentation in surgical videos is important for surgical scene understanding and computer-assisted interventions as well as for the development of robotic automation. The problem is challenging because different illumination conditions, bleeding, smoke and occlusions can reduce algorithm robustness. At present labelled data for training deep learning models is still lacking for semantic surgical instrument segmentation and in this paper we show that it may be possible to use robot kinematic data coupled with laparoscopic images to alleviate the labelling problem. We propose a new deep learning based model for parallel processing of both laparoscopic and simulation images for robust segmentation of surgical tools. Due to the lack of laparoscopic frames annotated with both segmentation ground truth and kinematic information a new custom dataset was generated using the da Vinci Research Kit (dVRK) and is made available

Analytical modeling of micelle growth. 2. Molecular thermodynamics of mixed aggregates and scission energy in wormlike micelles

Hypotheses: Quantitative molecular-thermodynamic theory of the growth of giant wormlike micelles in mixed nonionic surfactant solutions can be developed on the basis of a generalized model, which includes the classical phase separation and mass action models as special cases. The generalized model describes spherocylindrical micelles, which are simultaneously multicomponent and polydisperse in size. Theory: The model is based on explicit analytical expressions for the four components of the free energy of mixed nonionic micelles: interfacial-tension, headgroup-steric, chain-conformation components and free energy of mixing. The radii of the cylindrical part and the spherical endcaps, as well as the chemical composition of the endcaps, are determined by minimization of the free energy. Findings: In the case of multicomponent micelles, an additional term appears in the expression for the micelle growth parameter (scission free energy), which takes into account the fact that the micelle endcaps and cylindrical part have different compositions. The model accurately predicts the mean mass aggregation number of wormlike micelles in mixed nonionic surfactant solutions without using any adjustable parameters. The endcaps are enriched in the surfactant with smaller packing parameter that is better accommodated in regions of higher mean surface curvature. The model can be further extended to mixed solutions of nonionic, ionic and zwitterionic surfactants used in personal-care and house-hold detergency

Vision-based and marker-less surgical tool detection and tracking: a review of the literature

In recent years, tremendous progress has been made in surgical practice for example with Minimally Invasive Surgery (MIS). To overcome challenges coming from deported eye-to-hand manipulation, robotic and computer-assisted systems have been developed. Having real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy is a key ingredient for such systems. In this paper, we present a review of the literature dealing with vision-based and marker-less surgical tool detection. This paper includes three primary contributions: (1) identification and analysis of data-sets used for developing and testing detection algorithms, (2) in-depth comparison of surgical tool detection methods from the feature extraction process to the model learning strategy and highlight existing shortcomings, and (3) analysis of validation techniques employed to obtain detection performance results and establish comparison between surgical tool detectors. The papers included in the review were selected through PubMed and Google Scholar searches using the keywords: “surgical tool detection”, “surgical tool tracking”, “surgical instrument detection” and “surgical instrument tracking” limiting results to the year range 2000 2015. Our study shows that despite significant progress over the years, the lack of established surgical tool data-sets, and reference format for performance assessment and method ranking is preventing faster improvement

Widening siamese architectures for stereo matching

Computational stereo is one of the classical problems in computer vision. Numerous algorithms and solutions have been reported in recent years focusing on developing methods for computing similarity, aggregating it to obtain spatial support and finally optimizing an energy function to find the final disparity. In this paper, we focus on the feature extraction component of stereo matching architecture and we show standard CNNs operation can be used to improve the quality of the features used to find point correspondences. Furthermore, we use a simple space aggregation that hugely simplifies the correlation learning problem, allowing us to better evaluate the quality of the features extracted. Our results on benchmark data are compelling and show promising potential even without refining the solution

The Challenge of Augmented Reality in Surgery

Imaging has revolutionized surgery over the last 50 years. Diagnostic imaging is a key tool for deciding to perform surgery during disease management; intraoperative imaging is one of the primary drivers for minimally invasive surgery (MIS), and postoperative imaging enables effective follow-up and patient monitoring. However, notably, there is still relatively little interchange of information or imaging modality fusion between these different clinical pathway stages. This book chapter provides a critique of existing augmented reality (AR) methods or application studies described in the literature using relevant examples. The aim is not to provide a comprehensive review, but rather to give an indication of the clinical areas in which AR has been proposed, to begin to explain the lack of clinical systems and to provide some clear guidelines to those intending pursue research in this area

Photometric and spectroscopic variability of the FUor star V582 Aurigae

We carried out BVRI CCD photometric observations in the field of V582 Aur from 2009 August to 2013 February. We acquired high-, medium-, and low-resolution spectroscopy of V582 Aur during this period. To study the pre-outburst variability of the target and construct its historical light curve, we searched for archival observations in photographic plate collections. Both CCD and photographic observations were analyzed using a sequence of 14 stars in the field of V582 Aur calibrated in BVRI. The pre-outburst photographic observations of V582 Aur show low-amplitude light variations typical of T Tauri stars. Archival photographic observations indicate that the increase in brightness began in late 1984 or early 1985 and the star reached the maximum level of brightness at 1986 January. The spectral type of V582 Aur can be defined as G0I with strong P Cyg profiles of H alpha and Na I D lines, which are typical of FU Orionis objects. Our BVRI photometric observations show large amplitude variations V~2.8 mag. during the 3.5 year period of observations. Most of the time, however, the star remains in a state close to the maximum brightness. The deepest drop in brightness was observed in the spring of 2012, when the brightness of the star fell to a level close to the pre-outburst. The multicolor photometric data show a color reversal during the minimum in brightness, which is typical of UX Ori variables. The corresponding spectral observations show strong variability in the profiles and intensities of the spectral lines (especially H alpha), which indicate significant changes in the accretion rate. On the basis of photometric monitoring performed over the past three years, the spectral properties of the maximal light, and the shape of the long-term light curve, we confirm the affiliation of V582 Aur to the group of FU Orionis objects.Comment: 9 pages, 8 figures, accepted for publication in A&

Orbital Complications of Functional Endoscopic Sinus Surgery

Objective: Functional endoscopic sinus surgery (FESS) is an operating procedure for surgical treatment of diseases of the paranasal sinuses. There are precised indications for this surgical intervention. Like any invasive intervention there is a risk of occurrence of complications. Some of the most common are the orbital complications.Methods: By analyzing the available literature, the authors compare and summarize the results of various studies on this subject.Results: Orbital complications are rare but can be with severe consequences, including the complete loss of vision, and even death. Most of them after adequate treatment have not substantial implications but the underestimation can have serious consequences.Conclusion: The first step to avoid complications is prevention. If we have doubts of complications they must be precised instantly and act responsibly towards them. In the presence of this complication, one should be immediately consulted with an ophthalmologist and periodic assessment of IOP and visual acuity

HAPNet: hierarchically aggregated pyramid network for real-time stereo matching

©Recovering the 3D shape of the surgical site is crucial for multiple computer-assisted interventions. Stereo endoscopes can be used to compute 3D depth but computational stereo is a challenging, non-convex and inherently discontinuous optimisation problem. In this paper, we propose a deep learning architecture which avoids the explicit construction of a cost volume of similarity which is one of the most computationally costly blocks of stereo algorithms. This makes training our network significantly more efficient and avoids the needs for large memory allocation. Our method performs well, especially around regions comprising multiple discontinuities around surgical instrumentation or around complex small structures and instruments. The method compares well to the state-of-the-art techniques while taking a different methodological angle to computational stereo problem in surgical video

Learning to Calibrate - Estimating the Hand-eye Transformation without Calibration Objects

Hand-eye calibration is a method to determine the transformation linking between the robot and camera coordinate systems. Conventional calibration algorithms use a calibration grid to determine camera poses, corresponding to the robot poses, both of which are used in the main calibration procedure. Although such methods yield good calibration accuracy and are suitable for offline applications, they are not applicable in a dynamic environment such as robotic-assisted minimally invasive surgery (RMIS) because changes in the setup can be disruptive and time-consuming to the workflow as it requires yet another calibration procedure. In this paper, we propose a neural network-based hand-eye calibration method that does not require camera poses from a calibration grid but only uses the motion from surgical instruments in a camera frame and their corresponding robot poses as input to recover the hand-eye matrix. The advantages of using neural network are that the method is not limited by a single rigid transformation alignment and can learn dynamic changes correlated with kinematics and tool motion/interactions. Its loss function is derived from the original hand-eye transformation, the re-projection error and also the pose error in comparison to the remote centre of motion. The proposed method is validated with data from da Vinci Si and the results indicate that the designed network architecture can extract the relevant information and estimate the hand-eye matrix. Unlike the conventional hand-eye approaches, it does not require camera pose estimations which significantly simplifies the hand-eye problem in RMIS context as updating the hand-eye relationship can be done with a trained network and sequence of images. This introduces a potential of creating a hand-eye calibratio

BIOCHEMICAL AND MORPHOLOGICAL INVESТIGAТIONS ON IRRADIATED WIТH ULTRAVIOLET RАYS AND INFECTED WIТH GRIPPE VIRUS СНIСК EMBRYOS

The study of biochemical and morphological changes occurring under the combined effect (interaction) of vira and macroorganisms is an important рrоblem in virology. Biochemical investigations have been reported in literature under various aspects. Thus Voluiskaya  investigates the sugar in the pulmonary tissue of infected with grippe virus (GV) mice and finds out 20-50 % increase of sugar as compared to control animals. Tovarnitzki studies the biochemical alterations in experimentally produced grippe infection of white mice and comes to the conclusion that the pathological process in grippe conditions is generalized and involves а number of visceral organs and the central nervous system. Knight investigates the aminoacid content of the allantoic fluid (AF) in chick embryos (СЕ), not contaminated and contaminated with grippe virus. Killborne and Horsfall established аn increased protein content in the AF of the СЕ, infected with GV. Lutikova finds out an increase of the total nitrogen аnd phosphorus in the chorioallantoic membranes of СЕ, infected with GV. Panayotov studies in СЕ substrates, injected with different vira, the following indices: Ph, aminoacid content, рrеsеnсе of RNA аnd DNA, phosphatese activity, aldolasc, pyrophosphatase etc. The effect of ultraviolet rays (UVR) оn the GV has bееn investigated bу numerous authors. Thus Wells and Brown carry out follow-up studies on the effect of UVR upon aerosol of GV.Salk and associates (cited bу Levin - 20) investigates the action of the UVR on the virulence of the GV.Vaskhov, Rosiisky аnd Smorodintzev study the influence of UVR оn the  pulmonary  suspension, containing GV. Ermeev and Chalkina study the effect of UVR on purified GV, type А. Zakastelskaia proves the infectious аnd toxic action of the allantoic fluid containing GV. Manolova studies the effect of UVR оn purified and nоn purified grippe vira В, А and А-1. Rappoport, Dyhno and assoc., Panayotov and assoc., аnd Sfoyanov follow the morphological changes in СЕ treated with microorganisms.In the pertinent literature surveyed nо informations were found concerning the effect of UVR оn the СЕ infected with GV; hеnсе the study of the biochemical indices аnd pathohistological alterations in СЕ irradiated with UVR and infected with GV is of utmost interest.The purpose of the present work is determination of the biochemical characteristics, the presence of hemagglutination activity for the GV аnd the morphological alterations in the AF of the СЕ infected with GV