ProtoDiv: Prototype-guided Division of Consistent Pseudo-bags for Whole-slide Image Classification

Due to the limitations of inadequate Whole-Slide Image (WSI) samples with weak labels, pseudo-bag-based multiple instance learning (MIL) appears as a vibrant prospect in WSI classification. However, the pseudo-bag dividing scheme, often crucial for classification performance, is still an open topic worth exploring. Therefore, this paper proposes a novel scheme, ProtoDiv, using a bag prototype to guide the division of WSI pseudo-bags. Rather than designing complex network architecture, this scheme takes a plugin-and-play approach to safely augment WSI data for effective training while preserving sample consistency. Furthermore, we specially devise an attention-based prototype that could be optimized dynamically in training to adapt to a classification task. We apply our ProtoDiv scheme on seven baseline models, and then carry out a group of comparison experiments on two public WSI datasets. Experiments confirm our ProtoDiv could usually bring obvious performance improvements to WSI classification.Comment: 12 pages, 5 figures, and 3 table

AdvMIL: Adversarial Multiple Instance Learning for the Survival Analysis on Whole-Slide Images

The survival analysis on histological whole-slide images (WSIs) is one of the most important means to estimate patient prognosis. Although many weakly-supervised deep learning models have been developed for gigapixel WSIs, their potential is generally restricted by classical survival analysis rules and fully-supervision requirements. As a result, these models provide patients only with a completely-certain point estimation of time-to-event, and they could only learn from the well-annotated WSI data currently at a small scale. To tackle these problems, we propose a novel adversarial multiple instance learning (AdvMIL) framework. This framework is based on adversarial time-to-event modeling, and it integrates the multiple instance learning (MIL) that is much necessary for WSI representation learning. It is a plug-and-play one, so that most existing WSI-based models with embedding-level MIL networks can be easily upgraded by applying this framework, gaining the improved ability of survival distribution estimation and semi-supervised learning. Our extensive experiments show that AdvMIL could not only bring performance improvement to mainstream WSI models at a relatively low computational cost, but also enable these models to learn from unlabeled data with semi-supervised learning. Our AdvMIL framework could promote the research of time-to-event modeling in computational pathology with its novel paradigm of adversarial MIL.Comment: 13 pages, 10 figures, 8 table

Pseudo-Bag Mixup Augmentation for Multiple Instance Learning Based Whole Slide Image Classification

Given the special situation of modeling gigapixel images, multiple instance learning (MIL) has become one of the most important frameworks for Whole Slide Image (WSI) classification. In current practice, most MIL networks often face two unavoidable problems in training: i) insufficient WSI data, and ii) the data memorization nature inherent in neural networks. These problems may hinder MIL models from adequate and efficient training, suppressing the continuous performance promotion of classification models on WSIs. Inspired by the basic idea of Mixup, this paper proposes a Pseudo-bag Mixup (PseMix) data augmentation scheme to improve the training of MIL models. This scheme generalizes the Mixup strategy for general images to special WSIs via pseudo-bags so as to be applied in MIL-based WSI classification. Cooperated by pseudo-bags, our PseMix fulfills the critical size alignment and semantic alignment in Mixup strategy. Moreover, it is designed as an efficient and decoupled method adaptive to MIL, neither involving time-consuming operations nor relying on MIL model predictions. Comparative experiments and ablation studies are specially designed to evaluate the effectiveness and advantages of our PseMix. Test results show that PseMix could often improve the performance of MIL networks in WSI classification. Besides, it could also boost the generalization capacity of MIL models, and promote their robustness to patch occlusion and noisy labels. Our source code is available at https://github.com/liupei101/PseMix.Comment: 10 pages, 6 figures, 8 table

DSCA: A Dual-Stream Network with Cross-Attention on Whole-Slide Image Pyramids for Cancer Prognosis

The cancer prognosis on gigapixel Whole-Slide Images (WSIs) has always been a challenging task. To further enhance WSI visual representations, existing methods have explored image pyramids, instead of single-resolution images, in WSIs. In spite of this, they still face two major problems: high computational cost and the unnoticed semantical gap in multi-resolution feature fusion. To tackle these problems, this paper proposes to efficiently exploit WSI pyramids from a new perspective, the dual-stream network with cross-attention (DSCA). Our key idea is to utilize two sub-streams to process the WSI patches with two resolutions, where a square pooling is devised in a high-resolution stream to significantly reduce computational costs, and a cross-attention-based method is proposed to properly handle the fusion of dual-stream features. We validate our DSCA on three publicly-available datasets with a total number of 3,101 WSIs from 1,911 patients. Our experiments and ablation studies verify that (i) the proposed DSCA could outperform existing state-of-the-art methods in cancer prognosis, by an average C-Index improvement of around 4.6%; (ii) our DSCA network is more efficient in computation -- it has more learnable parameters (6.31M vs. 860.18K) but less computational costs (2.51G vs. 4.94G), compared to a typical existing multi-resolution network. (iii) the key components of DSCA, dual-stream and cross-attention, indeed contribute to our model's performance, gaining an average C-Index rise of around 2.0% while maintaining a relatively-small computational load. Our DSCA could serve as an alternative and effective tool for WSI-based cancer prognosis.Comment: 12 pages, 6 figures, 7 table

Ecoenzymatic stoichiometry reveals widespread soil phosphorus limitation to microbial metabolism across Chinese forests

8 páginas.- 4 figuras.- 57 referencias.- Supplementary information The online version contains supplementary material available at https://doi.org/10.1038/s43247-022-00523-5Forest soils contain a large amount of organic carbon and contribute to terrestrial carbon sequestration. However, we still have a poor understanding of what nutrients limit soil microbial metabolism that drives soil carbon release across the range of boreal to tropical forests. Here we used ecoenzymatic stoichiometry methods to investigate the patterns of microbial nutrient limitations within soil profiles (organic, eluvial and parent material horizons) across 181 forest sites throughout China. Results show that, in 80% of these forests, soil microbes were limited by phosphorus availability. Microbial phosphorus limitation increased with soil depth and from boreal to tropical forests as ecosystems become wetter, warmer, more productive, and is affected by anthropogenic nitrogen deposition. We also observed an unexpected shift in the latitudinal pattern of microbial phosphorus limitation with the lowest phosphorus limitation in the warm temperate zone (41-42 degrees N). Our study highlights the importance of soil phosphorus limitation to restoring forests and predicting their carbon sinks. Phosphorus limitation of soil microbial communities in forests is widespread, increases with soil depth, and is enhanced under wetter and warmer climates and elevated anthropogenic nitrogen deposition, according to ecoenzymatic stoichiometric analyses across 181 forests in China.This study was financially supported by the Strategic Priority Research Program of Chinese Academy of Sciences (XDB40000000), Funds for International Cooperation and Exchange of National Natural Science Foundation of China (32061123007), National Natural Science Foundation of China (41977031), Program of State Key Laboratory of Loess and Quaternary Geology CAS (SKLLQGZR1803). Contributions from Dr. Chen are funded by H2020 Marie Skłodowska-Curie Actions (No. 839806). M.D.-B. acknowledges support from the Spanish Ministry of Science and Innovation for the I+D+i project PID2020-115813RA-I00 funded by CIN/AEI/10.13039/501100011033. M.D.-B. is also supported by a project of the Fondo Europeo de Desarrollo Regional (FEDER) and the Consejería de Transformación Económica, Industria, Conocimiento y Universidades of the Junta de Andalucía (FEDER Andalucía 2014-2020 Objetivo temático “01–Refuerzo de la investigación, el desarrollo tecnológico y la innovación”) associated with the research project P20_00879 (ANDABIOMA).Peer reviewe

Upper ocean biogeochemistry of the oligotrophic North Pacific Subtropical Gyre : from nutrient sources to carbon export

Subtropical gyres cover 26–29% of the world’s surface ocean and are conventionally regarded as ocean deserts due to their permanent stratification, depleted surface nutrients, and low biological productivity. Despite tremendous advances over the past three decades, particularly through the Hawaii Ocean Time-series and the Bermuda Atlantic Time-series Study, which have revolutionized our understanding of the biogeochemistry in oligotrophic marine ecosystems, the gyres remain understudied. We review current understanding of upper ocean biogeochemistry in the North Pacific Subtropical Gyre, considering other subtropical gyres for comparison. We focus our synthesis on spatial variability, which shows larger than expected dynamic ranges of properties such as nutrient concentrations, rates of N2 fixation, and biological production. This review provides new insights into how nutrient sources drive community structure and export in upper subtropical gyres. We examine the euphotic zone in subtropical gyres as a two-layered vertically structured system: a nutrient-depleted layer above the top of the nutricline in the well-lit upper ocean and a nutrient-replete layer below in the dimly lit waters. These layers vary in nutrient supply and stoichiometries and physical forcing, promoting differences in community structure and food webs, with direct impacts on the magnitude and composition of export production. We evaluate long-term variations in key biogeochemical parameters in both of these euphotic zone layers. Finally, we identify major knowledge gaps and research challenges in these vast and unique systems that offer opportunities for future studies

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Pansharpening techniques fuse the complementary information from a high-resolution panchromatic (PAN) and low-resolution multispectral (LRMS) images to produce a high-resolution multispectral (HRMS) image. However, most of the existing pansharpening methods have been affected in the full-resolution domain due to both the absence of ground truths and unavoidable unknown noises. To address this problem, a new pansharpening method has been proposed that combines enhanced sparse models and gradient-domain guided image filtering. Specifically, a deep multiscale Laplacian pyramid super-resolution network improves the resolution of the original LRMS image instead of bicubic interpolation. Then, the accurate preservation of spatial-spectral characteristics is achieved in a variational framework with enhanced spatial-spectral fidelity in the image gradient domain. Meanwhile, the gradient-domain guided image filter is used to effectively improve the extraction accuracy of spatial characteristics from the PAN image. Finally, the enhanced sparse regularization on the latent HRMS image is designed to remove noise and artifacts while promoting piecewise-smooth solutions. The experimental results on public satellite datasets demonstrate the superiority of the proposed method against existing pansharpening methods in terms of both full-resolution performance indexes and visual quality

CIFAR10-DVS: An Event-Stream Dataset for Object Classification

Neuromorphic vision research requires high-quality and appropriately challenging event-stream datasets to support continuous improvement of algorithms and methods. However, creating event-stream datasets is a time-consuming task, which needs to be recorded using the neuromorphic cameras. Currently, there are limited event-stream datasets available. In this work, by utilizing the popular computer vision dataset CIFAR-10, we converted 10,000 frame-based images into 10,000 event streams using a dynamic vision sensor (DVS), providing an event-stream dataset of intermediate difficulty in 10 different classes, named as “CIFAR10-DVS.” The conversion of event-stream dataset was implemented by a repeated closed-loop smooth (RCLS) movement of frame-based images. Unlike the conversion of frame-based images by moving the camera, the image movement is more realistic in respect of its practical applications. The repeated closed-loop image movement generates rich local intensity changes in continuous time which are quantized by each pixel of the DVS camera to generate events. Furthermore, a performance benchmark in event-driven object classification is provided based on state-of-the-art classification algorithms. This work provides a large event-stream dataset and an initial benchmark for comparison, which may boost algorithm developments in even-driven pattern recognition and object classification