Privacy-Preserving Constrained Domain Generalization via Gradient Alignment

Deep neural networks (DNN) have demonstrated unprecedented success for medical imaging applications. However, due to the issue of limited dataset availability and the strict legal and ethical requirements for patient privacy protection, the broad applications of medical imaging classification driven by DNN with large-scale training data have been largely hindered. For example, when training the DNN from one domain (e.g., with data only from one hospital), the generalization capability to another domain (e.g., data from another hospital) could be largely lacking. In this paper, we aim to tackle this problem by developing the privacy-preserving constrained domain generalization method, aiming to improve the generalization capability under the privacy-preserving condition. In particular, We propose to improve the information aggregation process on the centralized server-side with a novel gradient alignment loss, expecting that the trained model can be better generalized to the "unseen" but related medical images. The rationale and effectiveness of our proposed method can be explained by connecting our proposed method with the Maximum Mean Discrepancy (MMD) which has been widely adopted as the distribution distance measurement. Experimental results on two challenging medical imaging classification tasks indicate that our method can achieve better cross-domain generalization capability compared to the state-of-the-art federated learning methods

Generalization Beyond Feature Alignment: Concept Activation-Guided Contrastive Learning

Learning invariant representations via contrastive learning has seen state-of-the-art performance in domain generalization (DG). Despite such success, in this paper, we find that its core learning strategy -- feature alignment -- could heavily hinder model generalization. Drawing insights in neuron interpretability, we characterize this problem from a neuron activation view. Specifically, by treating feature elements as neuron activation states, we show that conventional alignment methods tend to deteriorate the diversity of learned invariant features, as they indiscriminately minimize all neuron activation differences. This instead ignores rich relations among neurons -- many of them often identify the same visual concepts despite differing activation patterns. With this finding, we present a simple yet effective approach, Concept Contrast (CoCo), which relaxes element-wise feature alignments by contrasting high-level concepts encoded in neurons. Our CoCo performs in a plug-and-play fashion, thus it can be integrated into any contrastive method in DG. We evaluate CoCo over four canonical contrastive methods, showing that CoCo promotes the diversity of feature representations and consistently improves model generalization capability. By decoupling this success through neuron coverage analysis, we further find that CoCo potentially invokes more meaningful neurons during training, thereby improving model learning

Neuron Coverage-Guided Domain Generalization

This paper focuses on the domain generalization task where domain knowledge is unavailable, and even worse, only samples from a single domain can be utilized during training. Our motivation originates from the recent progresses in deep neural network (DNN) testing, which has shown that maximizing neuron coverage of DNN can help to explore possible defects of DNN (i.e., misclassification). More specifically, by treating the DNN as a program and each neuron as a functional point of the code, during the network training we aim to improve the generalization capability by maximizing the neuron coverage of DNN with the gradient similarity regularization between the original and augmented samples. As such, the decision behavior of the DNN is optimized, avoiding the arbitrary neurons that are deleterious for the unseen samples, and leading to the trained DNN that can be better generalized to out-of-distribution samples. Extensive studies on various domain generalization tasks based on both single and multiple domain(s) setting demonstrate the effectiveness of our proposed approach compared with state-of-the-art baseline methods. We also analyze our method by conducting visualization based on network dissection. The results further provide useful evidence on the rationality and effectiveness of our approach

Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization

The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i.e., in-distribution (InD). In this paper, we study the OOD problem from a neuron activation view. We first formulate neuron activation states by considering both the neuron output and its influence on model decisions. Then, to characterize the relationship between neurons and OOD issues, we introduce the \textit{neuron activation coverage} (NAC) -- a simple measure for neuron behaviors under InD data. Leveraging our NAC, we show that 1) InD and OOD inputs can be largely separated based on the neuron behavior, which significantly eases the OOD detection problem and beats the 21 previous methods over three benchmarks (CIFAR-10, CIFAR-100, and ImageNet-1K). 2) a positive correlation between NAC and model generalization ability consistently holds across architectures and datasets, which enables a NAC-based criterion for evaluating model robustness. Compared to prevalent InD validation criteria, we show that NAC not only can select more robust models, but also has a stronger correlation with OOD test performance.Comment: 28 pages, 9 figures, 20 table

A method review of the climate change impact on crop yield

Climate change significantly impacts global agricultural production, giving rise to considerable uncertainties. To explore these climate impacts, three independent methods have been employed: manipulated experiments, process-based crop models, and empirical statistical models. However, the uncertainty stemming from the use of different methods has received insufficient attention, and its implications remain unclear, necessitating a systematic review. In this study, we conducted a comprehensive review of numerous previous studies to summarize the historic development and current status of each method. Through a method comparison, we identified their respective strengths, limitations, and ideal areas of application. Additionally, we outlined potential prospects and suggested directions for future improvements, including clarifying the response mechanisms, updating simulation technologies, and developing multi-method ensembles. By addressing the knowledge gap regarding method differences, this review could contribute to a more accurate assessment of climate impacts on agriculture

Associative Nitrogen Fixation Linked With Three Perennial Bioenergy Grasses In Field and Greenhouse Experiments

© 2020 The Authors. Associative nitrogen (N2)‐fixation (ANF) by bacteria in the root‐zone of perennial bioenergy grasses has the potential to replace or supplement N fertilizer and support sustainable production of biomass, but its application in marginal ecosystems requires further evaluation. In this study, we first combined both greenhouse and field experiments, to explore the N2 fixation effects of three temperate feedstocks Miscanthus × giganteus (giant miscanthus, Freedom), Panicum virgatum (switchgrass, Alamo), and Saccharum sp. (energycane, Ho 02‐147). In field studies across three growing seasons, plant and soil pools of candidate feedstocks were partially composed of N derived from the atmosphere (Ndfa). Energycane, giant miscanthus, and switchgrass were estimated to derive \u3e30%, %Ndfa. Greenhouse studies were also performed to trace isotopically labeled 15N2 into plant biomass and soil pools. Evidence for Ndfa was detected in all three feedstock grasses (using reference 15N of soil, chicory, and sorghum, δ15N~+7.0). Isotopically labeled 15N2 was traced into biomass (during grass elongation stage) and soil pools. Extrapolation of rates during the 24 hr labeling period to 50 days estimated 30%–55% of plant Ndfa, with the greatest Ndfa for energycane. The findings of the field natural abundance and greenhouse 15N2 feeding experiments provided complementary evidence that perennial bioenergy grasses have the potential to support relatively high rates of ANF, and accumulate diazotroph‐derived N into biomass when grown on non‐fertilized soil

Terrestrial-derived soil protein in coastal water: metal sequestration mechanism and ecological function.

Terrestrial fungi, especially arbuscular mycorrhizal (AM) fungi, enhance heavy metal sequestration and promote ecosystem restoration. However, their ecological functions were historically overlooked in discussions regarding water quality. As an AM fungi-derived stable soil protein fraction, glomalin-related soil protein (GRSP) may provide insights into the ecological functions of AM fungi associated with water quality in coastal ecosystems. Here, we first assessed the metal-loading dynamics and ecological functions of GRSP transported into aquatic ecosystems, characterized the composition characteristics, and revealed the mechanisms underlying Cu and Cd sequestration. Combining in situ sampling and in vitro cultures, we found that the composition characteristics of GRSP were significantly affected by the element and mineral composition of sediments. In situ, GRSP-bound Cu and Cd contributed 18.91-22.03% of the total Cu and 2.27-6.37% of the total Cd. Functional group ligands and ion exchange were the principal mechanisms of Cu binding by GRSP, while Cd binding was dominated by functional group ligands. During the in vitro experiment, GRSP sequestered large amounts of Cu and Cd and formed stable complexes, while further dialysis only released 25.74 ± 3.85% and 33.53 ± 3.62% of GRSP-bound Cu and Cd, respectively

Terrestrial-derived soil protein in coastal water: Metal sequestration mechanism and ecological function

Abstract(#br)Terrestrial fungi, especially arbuscular mycorrhizal (AM) fungi, enhance heavy metal sequestration and promote ecosystem restoration. However, their ecological functions were historically overlooked in discussions regarding water quality. As an AM fungi-derived stable soil protein fraction, glomalin-related soil protein (GRSP) may provide insights into the ecological functions of AM fungi associated with water quality in coastal ecosystems. Here, we first assessed the metal-loading dynamics and ecological functions of GRSP transported into aquatic ecosystems, characterized the composition characteristics, and revealed the mechanisms underlying Cu and Cd sequestration. Combining in situ sampling and in vitro cultures, we found that the composition characteristics of GRSP were significantly affected by the element and mineral composition of sediments. In situ , GRSP-bound Cu and Cd contributed 18.91–22.03% of the total Cu and 2.27–6.37% of the total Cd. Functional group ligands and ion exchange were the principal mechanisms of Cu binding by GRSP, while Cd binding was dominated by functional group ligands. During the in vitro experiment, GRSP sequestered large amounts of Cu and Cd and formed stable complexes, while further dialysis only released 25.74 ± 3.85% and 33.53 ± 3.62% of GRSP-bound Cu and Cd, respectively

An Approach of Filtering Wrong-Type Entities for Entity Ranking

Entity is an important information carrier in Web pages. Users would like to directly get a list of relevant entities instead of a list of documents when they submit a query to the search engine. So the research of related entity finding (REF) is a meaningful work. In this paper we investigate the most important task of REF: Entity Ranking The wrong-type entities which don't belong to the target-entity type will pollute the ranking result. We propose a novel method to filter wrong-type entities. We focus on the acquisition of seed entities and automatically extracting the common Wikipedia categories of target-entity type. Also we demonstrate how to filter wrong-type entities using the proposed model. The experimental results show our method can filter wrong-type entities effectively and improve the results of entity ranking.Entity is an important information carrier in Web pages. Users would like to directly get a list of relevant entities instead of a list of documents when they submit a query to the search engine. So the research of related entity finding (REF) is a meaningful work. In this paper we investigate the most important task of REF: Entity Ranking The wrong-type entities which don't belong to the target-entity type will pollute the ranking result. We propose a novel method to filter wrong-type entities. We focus on the acquisition of seed entities and automatically extracting the common Wikipedia categories of target-entity type. Also we demonstrate how to filter wrong-type entities using the proposed model. The experimental results show our method can filter wrong-type entities effectively and improve the results of entity ranking