Fairness-aware Vision Transformer via Debiased Self-Attention

Vision Transformer (ViT) has recently gained significant interest in solving computer vision (CV) problems due to its capability of extracting informative features and modeling long-range dependencies through the self-attention mechanism. To fully realize the advantages of ViT in real-world applications, recent works have explored the trustworthiness of ViT, including its robustness and explainability. However, another desiderata, fairness has not yet been adequately addressed in the literature. We establish that the existing fairness-aware algorithms (primarily designed for CNNs) do not perform well on ViT. This necessitates the need for developing our novel framework via Debiased Self-Attention (DSA). DSA is a fairness-through-blindness approach that enforces ViT to eliminate spurious features correlated with the sensitive attributes for bias mitigation. Notably, adversarial examples are leveraged to locate and mask the spurious features in the input image patches. In addition, DSA utilizes an attention weights alignment regularizer in the training objective to encourage learning informative features for target prediction. Importantly, our DSA framework leads to improved fairness guarantees over prior works on multiple prediction tasks without compromising target prediction performance

Learning Compact Features via In-Training Representation Alignment

Deep neural networks (DNNs) for supervised learning can be viewed as a pipeline of the feature extractor (i.e., last hidden layer) and a linear classifier (i.e., output layer) that are trained jointly with stochastic gradient descent (SGD) on the loss function (e.g., cross-entropy). In each epoch, the true gradient of the loss function is estimated using a mini-batch sampled from the training set and model parameters are then updated with the mini-batch gradients. Although the latter provides an unbiased estimation of the former, they are subject to substantial variances derived from the size and number of sampled mini-batches, leading to noisy and jumpy updates. To stabilize such undesirable variance in estimating the true gradients, we propose In-Training Representation Alignment (ITRA) that explicitly aligns feature distributions of two different mini-batches with a matching loss in the SGD training process. We also provide a rigorous analysis of the desirable effects of the matching loss on feature representation learning: (1) extracting compact feature representation; (2) reducing over-adaption on mini-batches via an adaptive weighting mechanism; and (3) accommodating to multi-modalities. Finally, we conduct large-scale experiments on both image and text classifications to demonstrate its superior performance to the strong baselines.Comment: 11 pages, 4 figures, 6 tables. Accepted for publication by AAAI-23. arXiv admin note: text overlap with arXiv:2002.0991

A multi-decadal analysis of river discharge and suspended sediment load in three Texas coastal rivers in relation to hurricanes, seasonal rainfall, and ENSO

Coastal river discharge and sediment load exert major influence on the sustainability of coastal systems. Controlled by various hydroclimatic/hydrometeorological agents, they exhibit distinct trend/variability at different time scales. Coastal Texas, while being a major target for tropical cyclones over the past 6 decades, has been experiencing drought and flood cycles associated with ENSO in the long term. However, it is still unclear the temporal variability of river discharge and the associated sediment delivery over this area at different time scales, and the controlling factors behind it. In this study, a 58-years (1960–2017) dataset is compiled to analyze the influence of ENSO, seasonal rainfall distribution and hurricanes event on the river discharge and suspended sediment load of three Texas coastal rivers-the San Bernard River, the Brazos River, and the Trinity River, at annual, seasonal and event scales, respectively. In the short-term, all three rivers attained the highest average daily discharge and sediment load during Hurricane Harvey. On a seasonal scale, the precipitation regime exerts more influence on the Texas watersheds than tropical storms and hurricanes. Over a multi-decadal scale, amplified rainstorms during the El Niño phases likely play an important role in the overall discharge and sediment transport in large rivers along the northern Gulf coast. Overall, it is reasonable to conclude that the magnitude of hurricane impacts on the overall discharge and suspended sediment load is regulated by the duration and intensity of the rainfall, as well as the coupled drought-flood cycle in relation to the intensity of ENSO

Auto-Prompting SAM for Mobile Friendly 3D Medical Image Segmentation

The Segment Anything Model (SAM) has rapidly been adopted for segmenting a wide range of natural images. However, recent studies have indicated that SAM exhibits subpar performance on 3D medical image segmentation tasks. In addition to the domain gaps between natural and medical images, disparities in the spatial arrangement between 2D and 3D images, the substantial computational burden imposed by powerful GPU servers, and the time-consuming manual prompt generation impede the extension of SAM to a broader spectrum of medical image segmentation applications. To address these challenges, in this work, we introduce a novel method, AutoSAM Adapter, designed specifically for 3D multi-organ CT-based segmentation. We employ parameter-efficient adaptation techniques in developing an automatic prompt learning paradigm to facilitate the transformation of the SAM model's capabilities to 3D medical image segmentation, eliminating the need for manually generated prompts. Furthermore, we effectively transfer the acquired knowledge of the AutoSAM Adapter to other lightweight models specifically tailored for 3D medical image analysis, achieving state-of-the-art (SOTA) performance on medical image segmentation tasks. Through extensive experimental evaluation, we demonstrate the AutoSAM Adapter as a critical foundation for effectively leveraging the emerging ability of foundation models in 2D natural image segmentation for 3D medical image segmentation.Comment: 9 pages, 4 figures, 4 table

The relationship between Cho/NAA and glioma metabolism: implementation for margin delineation of cerebral gliomas

BACKGROUND: The marginal delineation of gliomas cannot be defined by conventional imaging due to their infiltrative growth pattern. Here we investigate the relationship between changes in glioma metabolism by proton magnetic resonance spectroscopic imaging ((1)H-MRSI) and histopathological findings in order to determine an optimal threshold value of choline/N-acetyl-aspartate (Cho/NAA) that can be used to define the extent of glioma spread. METHOD: Eighteen patients with different grades of glioma were examined using (1)H-MRSI. Needle biopsies were performed under the guidance of neuronavigation prior to craniotomy. Intraoperative magnetic resonance imaging (MRI) was performed to evaluate the accuracy of sampling. Haematoxylin and eosin, and immunohistochemical staining with IDH1, MIB-1, p53, CD34 and glial fibrillary acidic protein (GFAP) antibodies were performed on all samples. Logistic regression analysis was used to determine the relationship between Cho/NAA and MIB-1, p53, CD34, and the degree of tumour infiltration. The clinical threshold ratio distinguishing tumour tissue in high-grade (grades III and IV) glioma (HGG) and low-grade (grade II) glioma (LGG) was calculated. RESULTS: In HGG, higher Cho/NAA ratios were associated with a greater probability of higher MIB-1 counts, stronger CD34 expression, and tumour infiltration. Ratio threshold values of 0.5, 1.0, 1.5 and 2.0 appeared to predict the specimens containing the tumour with respective probabilities of 0.38, 0.60, 0.79, 0.90 in HGG and 0.16, 0.39, 0.67, 0.87 in LGG. CONCLUSIONS: HGG and LGG exhibit different spectroscopic patterns. Using (1)H-MRSI to guide the extent of resection has the potential to improve the clinical outcome of glioma surgery

Protein-metabolite association studies identify novel proteomic determinants of metabolite levels in human plasma

Although many novel gene-metabolite and gene-protein associations have been identified using high-throughput biochemical profiling, systematic studies that leverage human genetics to illuminate causal relationships between circulating proteins and metabolites are lacking. Here, we performed protein-metabolite association studies in 3,626 plasma samples from three human cohorts. We detected 171,800 significant protein-metabolite pairwise correlations between 1,265 proteins and 365 metabolites, including established relationships in metabolic and signaling pathways such as the protein thyroxine-binding globulin and the metabolite thyroxine, as well as thousands of new findings. In Mendelian randomization (MR) analyses, we identified putative causal protein-to-metabolite associations. We experimentally validated top MR associations in proof-of-concept plasma metabolomics studies in three murine knockout strains of key protein regulators. These analyses identified previously unrecognized associations between bioactive proteins and metabolites in human plasma. We provide publicly available data to be leveraged for studies in human metabolism and disease

Revealing the source of Jupiter’s x-ray auroral flares

Jupiter’s rapidly rotating, strong magnetic field provides a natural laboratory that is key to understanding the dynamics of high-energy plasmas. Spectacular auroral x-ray flares are diagnostic of the most energetic processes governing magnetospheres but seemingly unique to Jupiter. Since their discovery 40 years ago, the processes that produce Jupiter’s x-ray flares have remained unknown. Here, we report simultaneous in situ satellite and space-based telescope observations that reveal the processes that produce Jupiter’s x-ray flares, showing surprising similarities to terrestrial ion aurora. Planetary-scale electromagnetic waves are observed to modulate electromagnetic ion cyclotron waves, periodically causing heavy ions to precipitate and produce Jupiter’s x-ray pulses. Our findings show that ion aurorae share common mechanisms across planetary systems, despite temporal, spatial, and energetic scales varying by orders of magnitude

Comprehensive Benefit Evaluation of the Wind-PV-ES and Transmission Hybrid Power System Consideration of System Functionality and Proportionality

In the background of decreasing fossil fuels and increasing environmental pollution, the wind-photovoltaic energy storage and transmission hybrid power system (or called the wind-PV-ES and transmission hybrid system) has become a strategic choice to achieve energy sustainability. However, the comprehensive benefit evaluation of such a combined power system is in a relatively blank state in China, which will hinder the reasonable and orderly development of this station. Four parts, the technical performance, economic benefit, ecological impact and social benefit, are considered in this paper, and a multi-angle evaluation index system of the wind-PV-ES and transmission system is designed. The projection pursuit model is used to evaluated system functionality conventionally; relative entropy theory is used to evaluate the system functionality simultaneously; and a comprehensive benefit evaluation model of the technique for order preference by similar to ideal solution (TOPSIS) considering both system functionality and proportionality is constructed. Finally, the national demonstration station of the wind-PV-ES-transmission system is taken as an example to testify to the practicability and validity of the evaluation index system and model

Negative Flux Aggregation to Estimate Feature Attributions

There are increasing demands for understanding deep neural networks' (DNNs) behavior spurred by growing security and/or transparency concerns. Due to multi-layer nonlinearity of the deep neural network architectures, explaining DNN predictions still remains as an open problem, preventing us from gaining a deeper understanding of the mechanisms. To enhance the explainability of DNNs, we estimate the input feature's attributions to the prediction task using divergence and flux. Inspired by the divergence theorem in vector analysis, we develop a novel Negative Flux Aggregation (NeFLAG) formulation and an efficient approximation algorithm to estimate attribution map. Unlike the previous techniques, ours doesn't rely on fitting a surrogate model nor need any path integration of gradients. Both qualitative and quantitative experiments demonstrate a superior performance of NeFLAG in generating more faithful attribution maps than the competing methods. Our code is available at \url{https://github.com/xinli0928/NeFLAG}Comment: 14 pages, 4 figures, 2 table