Transformer Utilization in Medical Image Segmentation Networks

Owing to success in the data-rich domain of natural images, Transformers have recently become popular in medical image segmentation. However, the pairing of Transformers with convolutional blocks in varying architectural permutations leaves their relative effectiveness to open interpretation. We introduce Transformer Ablations that replace the Transformer blocks with plain linear operators to quantify this effectiveness. With experiments on 8 models on 2 medical image segmentation tasks, we explore -- 1) the replaceable nature of Transformer-learnt representations, 2) Transformer capacity alone cannot prevent representational replaceability and works in tandem with effective design, 3) The mere existence of explicit feature hierarchies in transformer blocks is more beneficial than accompanying self-attention modules, 4) Major spatial downsampling before Transformer modules should be used with caution.Comment: Accepted in NeurIPS 2022 workshop, Medical Imaging Meets NeurIPS (MedNeurIPS

Zurich Consensus: German Expert Opinion on the St. Gallen Votes on 15 March 2009 (11th International Conference at St. Gallen: Primary Therapy of Early Breast Cancer)

A German working group of 23 breast cancer experts discussed the results from the vote at this year's St. Gallen Consensus Conference on Primary Therapy for Early Breast Cancer ( March 11-14, 2009) and came up with some concrete recommendations for day-to-day therapeutic decisions in Germany. Due the fact that the concept of the St. Gallen Consensus Conference merely allows for a minimal consensus, the objective of the working group was to provide practice-related recommendations for day-to-day clinical decisions in Germany. One area of emphasis at St. Gallen was tumor biology as a starting point for reaching individual therapeutic decisions. Intensive discussion was necessary with respect to the clinical relevance of predictive and prognostic factors. A new addition to the area of systemic therapy was a first-ever discussion of the adjuvant administration of bisphosponates and the fact that therapy with trastuzumab in HER2 overexpressing breast cancer has been defined as the standard for neoadjuvant therapy. The value of taxanes as a component of (neo) adjuvant chemotherapy as well as the value of aromatase inhibitors for the endocrine adjuvant treatment of postmenopausal patients were affirmed

MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation

There has been exploding interest in embracing Transformer-based architectures for medical image segmentation. However, the lack of large-scale annotated medical datasets make achieving performances equivalent to those in natural images challenging. Convolutional networks, in contrast, have higher inductive biases and consequently, are easily trainable to high performance. Recently, the ConvNeXt architecture attempted to modernize the standard ConvNet by mirroring Transformer blocks. In this work, we improve upon this to design a modernized and scalable convolutional architecture customized to challenges of data-scarce medical settings. We introduce MedNeXt, a Transformer-inspired large kernel segmentation network which introduces - 1) A fully ConvNeXt 3D Encoder-Decoder Network for medical image segmentation, 2) Residual ConvNeXt up and downsampling blocks to preserve semantic richness across scales, 3) A novel technique to iteratively increase kernel sizes by upsampling small kernel networks, to prevent performance saturation on limited medical data, 4) Compound scaling at multiple levels (depth, width, kernel size) of MedNeXt. This leads to state-of-the-art performance on 4 tasks on CT and MRI modalities and varying dataset sizes, representing a modernized deep architecture for medical image segmentation. Our code is made publicly available at: https://github.com/MIC-DKFZ/MedNeXt.Comment: Accepted at MICCAI 202

Relating Biomass and Leaf Area Index to Non-destructive Measurements in Order to Monitor Changes in Arctic Vegetation

This paper reports an alternative method for seasonal and long-term monitoring of biomass and the leaf area index (LAI) at Arctic tundra sites. Information related to the historical and projected change in abundance and distribution of biomass and LAI is required to address numerous environmental and resource management issues. Observations of earth from satellites could potentially be used to derive seasonal and long-term changes in biomass and the LAI. To realize this potential, seasonal and long-term ground monitoring data for validation are essential; however, the conventional destructive sampling method for measuring biomass and the LAI does not allow repetitive measurements at the same plots and thus is not suitable for monitoring change over time. Alternative methods, such as sampling nearby similar plots, can be laborious and easily subject to large sampling errors, especially in Arctic tundra sites with low vegetation cover. In this study, we developed a practical method for relating non-destructive measurements (percent cover and mean height) to biomass and the LAI for 13 major Arctic plant groups, or seven plant functional types, on the basis of measurements at 196 plots across Canada’s Arctic tundra ecosystems. Using the method at the plant group level to estimate plot total vascular aboveground biomass, foliage biomass, and LAI, we had r2 = 0.91–0.95 and relative mean absolute error of 25–29%. By this method, one could monitor seasonal and long-term changes in biomass and the LAI through repeated, non-destructive observations of percent cover and mean height at the same permanent plots.Cette communication présente une méthode de rechange en vue de la surveillance saisonnière et à long terme de la biomasse et de l’indice de surface foliaire (LAI) de sites de toundra de l’Arctique. Afin de relever divers enjeux relatifs à la gestion de l’environnement et des ressources, il faut recueillir des données se rapportant au changement historique et projeté en matière d’abondance et de répartition de la biomasse et du LAI. On pourrait éventuellement recourir aux observations de la Terre à partir de satellites afin de déceler les changements saisonniers et à long terme caractérisant la biomasse et le LAI. Pour en arriver là, il est essentiel de disposer de données saisonnières et à long terme au sol à des fins de validation. Cependant, la méthode d’échantillonnage destructeur classique permettant de mesurer la biomasse et le LAI ne permettent pas la prise de mesures répétitives aux mêmes sites et par conséquent, elle ne convient pas à la surveillance du changement qui s’exerce au fil du temps. D’autres méthodes, telles que l’échantillonnage de sites semblables dans les environs, peuvent s’avérer laborieuses et facilement faire l’objet d’importantes erreurs d’échantillonnage, surtout aux sites de toundra de l’Arctique dont la couverture végétale est basse. Dans le cadre de cette étude, nous avons mis au point une méthode pratique pour établir un rapport entre les mesures non destructives (pourcentage de couverture et hauteur moyenne) et la biomasse et le LAI de 13 groupes végétaux importants de l’Arctique, ou sept types végétaux fonctionnels en fonction de la mesure de 196 sites à la grandeur des écosystèmes de toundra de l’Arctique canadien. En nous appuyant sur la méthode des groupes végétaux pour estimer la biomasse vasculaire totale à ciel ouvert des sites, la biomasse foliaire et le LAI, nous avions r2 = 0,91–0,95 et une erreur absolue relative moyenne de 25 à 29%. Au moyen de cette méthode, il serait possible de surveiller les changements saisonniers et à long terme en matière de biomasse et de LAI grâce à des observations répétées et non destructives du pourcentage de la couverture et de la hauteur moyenne aux mêmes sites permanents

RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement

Despite the remarkable success of deep learning systems over the last decade, a key difference still remains between neural network and human decision-making: As humans, we cannot only form a decision on the spot, but also ponder, revisiting an initial guess from different angles, distilling relevant information, arriving at a better decision. Here, we propose RecycleNet, a latent feature recycling method, instilling the pondering capability for neural networks to refine initial decisions over a number of recycling steps, where outputs are fed back into earlier network layers in an iterative fashion. This approach makes minimal assumptions about the neural network architecture and thus can be implemented in a wide variety of contexts. Using medical image segmentation as the evaluation environment, we show that latent feature recycling enables the network to iteratively refine initial predictions even beyond the iterations seen during training, converging towards an improved decision. We evaluate this across a variety of segmentation benchmarks and show consistent improvements even compared with top-performing segmentation methods. This allows trading increased computation time for improved performance, which can be beneficial, especially for safety-critical applications.Comment: Accepted at 2024 Winter Conference on Applications of Computer Vision (WACV

On-Surface Carbon Nitride Growth from Polymerization of 2,5,8-Triazido-s-heptazine

Carbon nitrides have recently come into focus for photo- and thermal catalysis, both as support materials for metal nanoparticles as well as photocatalysts themselves. While many approaches for the synthesis of three-dimensional carbon nitride materials are available, only top-down approaches by exfoliation of powders lead to thin film flakes of this inherently two-dimensional material. Here, we describe an in situ on-surface synthesis of monolayer 2D carbon nitride films, as a first step towards precise combination with other 2D materials. Starting with a single monomer precursor, we show that 2,5,8-triazido-s-heptazine (TAH) can be evaporated intact, deposited on a single crystalline Au(111) or graphite support, and activated via azide decomposition and subsequent coupling to form a covalent polyheptazine network. We demonstrate that the activation can occur in three pathways, via electrons (X-ray illumination), photons (UV illumination) and thermally. Our work paves the way to coat materials with extended carbon nitride networks which are, as we show, stable under ambient conditions