Semantic segmentation of surgical hyperspectral images under geometric domain shifts

Robust semantic segmentation of intraoperative image data could pave the way for automatic surgical scene understanding and autonomous robotic surgery. Geometric domain shifts, however, although common in real-world open surgeries due to variations in surgical procedures or situs occlusions, remain a topic largely unaddressed in the field. To address this gap in the literature, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation networks in the presence of geometric out-of-distribution (OOD) data, and (2) address generalizability with a dedicated augmentation technique termed "Organ Transplantation" that we adapted from the general computer vision community. According to a comprehensive validation on six different OOD data sets comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs semantically annotated with 19 classes, we demonstrate a large performance drop of SOA organ segmentation networks applied to geometric OOD data. Surprisingly, this holds true not only for conventional RGB data (drop of Dice similarity coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the latter's rich information content per pixel. Using our augmentation scheme improves on the SOA DSC by up to 67 % (RGB) and 90 % (HSI) and renders performance on par with in-distribution performance on real OOD test data. The simplicity and effectiveness of our augmentation scheme makes it a valuable network-independent tool for addressing geometric domain shifts in semantic scene segmentation of intraoperative data. Our code and pre-trained models are available at https://github.com/IMSY-DKFZ/htc.Comment: The first two authors (Jan Sellner and Silvia Seidlitz) contributed equally to this pape

Geostatistical Analysis of White Matter Lesions in Multiple Sclerosis Identifies Gender Differences in Lesion Evolution

Multiple sclerosis (MS) is a chronic inflammatory demyelinating disease of the central nervous system with presumed autoimmune origin. The development of lesions within the gray matter and white matter, which are highly variable with respect to number, total volume, morphology and spatial evolution and which only show a limited correlation with clinical disability, is a hallmark of the disease. Population-based studies indicate a distinct outcome depending on gender. Here, we studied gender-related differences in the evolution of white matter MS-lesions (MS-WML) in early MS by using geostatistical methods. Within a 3 years observation period, a female and a male MS patient group received disease modifying drugs and underwent standardized annual brain magnetic resonance imaging, accompanied by neurological examination. MS-WML were automatically extracted and the derived binary lesion masks were subject to geostatistical analysis, yielding quantitative spatial-statistics metrics on MS-WML pattern morphology and total lesion volume (TLV). Through the MS-lesion pattern discrimination plot, the following differences were disclosed: corresponding to gender and MS-WML pattern morphology at baseline, two female subgroups (F1, F2) and two male subgroups (M1, M2) are discerned that follow a distinct MS-WML pattern evolution in space and time. F1 and M1 start with medium-level MS-WML pattern smoothness and TLV, both behave longitudinally quasi-static. By contrast, F2 and M2 start with high-level MS-WML pattern smoothness and medium-level TLV. F2 and M2 longitudinal development is characterized by strongly diminishing MS-WML pattern smoothness and TLV, i.e., continued shrinking and break-up of MS-WML. As compared to the male subgroup M2, the female subgroup F2 shows continued, increased MS-WML pattern smoothness and TLV. Data from neurological examination suggest a correlation of MS-WML pattern morphology metrics and EDSS. Our results justify detailed studies on gender-related differences

Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding

In clinical radiology reports, doctors capture important information about the patient's health status. They convey their observations from raw medical imaging data about the inner structures of a patient. As such, formulating reports requires medical experts to possess wide-ranging knowledge about anatomical regions with their normal, healthy appearance as well as the ability to recognize abnormalities. This explicit grasp on both the patient's anatomy and their appearance is missing in current medical image-processing systems as annotations are especially difficult to gather. This renders the models to be narrow experts e.g. for identifying specific diseases. In this work, we recover this missing link by adding human anatomy into the mix and enable the association of content in medical reports to their occurrence in associated imagery (medical phrase grounding). To exploit anatomical structures in this scenario, we present a sophisticated automatic pipeline to gather and integrate human bodily structures from computed tomography datasets, which we incorporate in our PAXRay: A Projected dataset for the segmentation of Anatomical structures in X-Ray data. Our evaluation shows that methods that take advantage of anatomical information benefit heavily in visually grounding radiologists' findings, as our anatomical segmentations allow for up to absolute 50% better grounding results on the OpenI dataset as compared to commonly used region proposals. The PAXRay dataset is available at https://constantinseibold.github.io/paxray/.Comment: 33rd British Machine Vision Conference (BMVC 2022

Unsupervised Domain Transfer with Conditional Invertible Neural Networks

Synthetic medical image generation has evolved as a key technique for neural network training and validation. A core challenge, however, remains in the domain gap between simulations and real data. While deep learning-based domain transfer using Cycle Generative Adversarial Networks and similar architectures has led to substantial progress in the field, there are use cases in which state-of-the-art approaches still fail to generate training images that produce convincing results on relevant downstream tasks. Here, we address this issue with a domain transfer approach based on conditional invertible neural networks (cINNs). As a particular advantage, our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood training. To showcase our method's generic applicability, we apply it to two spectral imaging modalities at different scales, namely hyperspectral imaging (pixel-level) and photoacoustic tomography (image-level). According to comprehensive experiments, our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks (binary and multi-class). cINN-based domain transfer could thus evolve as an important method for realistic synthetic data generation in the field of spectral imaging and beyond

Biological Effects of Antiprotons Are Antiprotons a Candidate for Cancer Therapy?

2009 Status Report of AD-4 Experimen

Protocol for German trial of Acyclovir and corticosteroids in Herpes-simplex-virus-encephalitis (GACHE): a multicenter, multinational, randomized, double-blind, placebo-controlled German, Austrian and Dutch trial [ISRCTN45122933]

Background The treatment of Herpes-simplex-virus-encephalitis (HSVE) remains a major unsolved problem in Neurology. Current gold standard for therapy is acyclovir, a drug that inhibits viral replication. Despite antiviral treatment, mortality remains up to 15%, less than 20% of patients are able to go back to work, and the majority of patients suffer from severe disability. This is a discouraging, unsatisfactory situation for treating physicians, the disabled patients and their families, and constitutes an enormous burden to the public health services. The information obtained from experimental animal research and from recent retrospective clinical observations, indicates that a substantial benefit in outcome can be expected in patients with HSVE who are treated with adjuvant dexamethasone. But currently there is no available evidence to support the routine use of adjuvant corticosteroid treatment in HSVE. A randomized multicenter trial is the only useful instrument to address this question. Design GACHE is a multicenter, randomized, double-blind, placebo-controlled, parallel group clinical trial of treatment with acyclovir and adjuvant dexamethasone, as compared with acyclovir and placebo in adults with HSVE. The statistical design will be that of a 3-stage-group sequential trial with potential sample size adaptation in the last stage. Conclusion 372 patients with proven HSVE (positive HSV-DNA-PCR), aged 18 up to 85 years; with focal neurological signs no longer than 5 days prior to admission, and who give informed consent will be recruited from Departments of Neurology of academic medical centers in Germany, Austria and The Netherlands. Sample size will potentially be extended after the second interim analysis up to a maximum of 450 patients. Trial Registration Current Controlled Trials ISRCTN4512293

Drug-perturbation-based stratification of blood cancer

As new generations of targeted therapies emerge and tumor genome sequencing discovers increasingly comprehensive mutation repertoires, the functional relationships of mutations to tumor phenotypes remain largely unknown. Here, we measured ex vivo sensitivity of 246 blood cancers to 63 drugs alongside genome, transcriptome, and DNA methylome analysis to understand determinants of drug response. We assembled a primary blood cancer cell encyclopedia data set that revealed disease-specific sensitivities for each cancer. Within chronic lymphocytic leukemia (CLL), responses to 62% of drugs were associated with 2 or more mutations, and linked the B cell receptor (BCR) pathway to trisomy 12, an important driver of CLL. Based on drug responses, the disease could be organized into phenotypic subgroups characterized by exploitable dependencies on BCR, mTOR, or MEK signaling and associated with mutations, gene expression, and DNA methylation. Fourteen percent of CLLs were driven by mTOR signaling in a non-BCR-dependent manner. Multivariate modeling revealed immunoglobulin heavy chain variable gene (IGHV) mutation status and trisomy 12 as the most important modulators of response to kinase inhibitors in CLL. Ex vivo drug responses were associated with outcome. This study overcomes the perception that most mutations do not influence drug response of cancer, and points to an updated approach to understanding tumor biology, with implications for biomarker discovery and cancer care.Peer reviewe

Plasma lipid profiles discriminate bacterial from viral infection in febrile children

Fever is the most common reason that children present to Emergency Departments. Clinical signs and symptoms suggestive of bacterial infection are often non-specific, and there is no definitive test for the accurate diagnosis of infection. The 'omics' approaches to identifying biomarkers from the host-response to bacterial infection are promising. In this study, lipidomic analysis was carried out with plasma samples obtained from febrile children with confirmed bacterial infection (n = 20) and confirmed viral infection (n = 20). We show for the first time that bacterial and viral infection produces distinct profile in the host lipidome. Some species of glycerophosphoinositol, sphingomyelin, lysophosphatidylcholine and cholesterol sulfate were higher in the confirmed virus infected group, while some species of fatty acids, glycerophosphocholine, glycerophosphoserine, lactosylceramide and bilirubin were lower in the confirmed virus infected group when compared with confirmed bacterial infected group. A combination of three lipids achieved an area under the receiver operating characteristic (ROC) curve of 0.911 (95% CI 0.81 to 0.98). This pilot study demonstrates the potential of metabolic biomarkers to assist clinicians in distinguishing bacterial from viral infection in febrile children, to facilitate effective clinical management and to the limit inappropriate use of antibiotics

Modelling human choices: MADeM and decision‑making

Research supported by FAPESP 2015/50122-0 and DFG-GRTK 1740/2. RP and AR are also part of the Research, Innovation and Dissemination Center for Neuromathematics FAPESP grant (2013/07699-0). RP is supported by a FAPESP scholarship (2013/25667-8). ACR is partially supported by a CNPq fellowship (grant 306251/2014-0)