Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.Comment: AAAI-202

Representation Learning Method of Graph Convolutional Network Based on Structure Enhancement

Network representation learning has attracted widespread attention as a pre-processing process for some machine learning and deep learning tasks. However, most existing methods only consider influence of nodes' low-order neighbors to represent them. Either nodes' high-order neighbor or the intrinsic characteristic attributes of nodes are ignored, leading to the effect of network representation learning that needs to be improved. This paper proposes a novel model named Structure Enhanced Graph Convolutional Network (SEGCN) to address these limitations. SEGCN consists of the following components, i.e., the network structure enhancement to transform weak relationship into strong relationship, the node feature aggregation to fuse high-order neighbor information. Hence, the SEGCN model can simultaneously integrate network structure information, attribute information, and high-order neighbor relationships together. Experimental results for node classification and node clustering on six datasets show that SEGCN achieves better effectiveness and efficiency than state-of-the-art baselines

Kosmos-2: Grounding Multimodal Large Language Models to the World

We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.g., bounding boxes) and grounding text to the visual world. Specifically, we represent refer expressions as links in Markdown, i.e., ``[text span](bounding boxes)'', where object descriptions are sequences of location tokens. Together with multimodal corpora, we construct large-scale data of grounded image-text pairs (called GrIT) to train the model. In addition to the existing capabilities of MLLMs (e.g., perceiving general modalities, following instructions, and performing in-context learning), Kosmos-2 integrates the grounding capability into downstream applications. We evaluate Kosmos-2 on a wide range of tasks, including (i) multimodal grounding, such as referring expression comprehension, and phrase grounding, (ii) multimodal referring, such as referring expression generation, (iii) perception-language tasks, and (iv) language understanding and generation. This work lays out the foundation for the development of Embodiment AI and sheds light on the big convergence of language, multimodal perception, action, and world modeling, which is a key step toward artificial general intelligence. Data, demo, and pretrained models are available at https://aka.ms/kosmos-2.Comment: 20 page

Educational degree differences in the association between work stress and depression among Chinese healthcare workers: Job satisfaction and sleep quality as the mediators

BackgroundDepressive status of medical personnel worldwide and especially in China is an important public health and social problem. There is a strong relationship between education and depression, but no studies have studied grouping healthcare workers (HCWs) with different educational degree to discuss whether there are differences in the factors that affect depression. This study aims to examine the role of job satisfaction and sleep quality in the relationship between work stress and depression among Chinese HCWs, and teste whether the mediation models are differed by the differences of educational degree.MethodsPatient Health Questionnaire-9 (PHQ-9) scale was used to test depression. Work stress was assessed using the Challenge-blocking stress scale (CBSS). Sleep quality was assessed using the Pittsburgh Sleep Quality Index (PSQI). HCWs’ satisfaction with their current work was assessed using the Job Satisfaction Index (JSI). The representative sample of HCWs was chosen using a multi-stage stratified cluster random sampling procedure and 844 HCWs were utilized to the statistical analysis of the study.ResultsIn the overall sample, sleep quality could mediate the relationship between work stress and depression in healthcare workers (p < 0.001, CMIN/DF = 3.816, GFI = 0.911, AGFI = 0.886, IFI = 0.943, TLI = 0.933, CFI = 0.942, RMSEA = 0.058, SRMR = 0.055, AIC = 1039.144), and the mediating effect accounted for 36.5%. After grouping educational qualifications, the model with sleep quality and job satisfaction as mediating variables reported a better fit in the group with low educational qualifications. The intermediary effect accounted for 50.6 and 4.43%, respectively. The highly educated group only has sleep quality as an intermediary variable in the structural model, and the mediating effect accounted for 75.4% (p < 0.001, CMIN/DF = 2.596, GFI = 0.887, AGFI = 0.857, IFI = 0.937, TLI = 0.926, CFI = 0.937, RMSEA = 0.044, SRMR = 0.056, AIC = 1481.322).ConclusionIn the overall sample, sleep quality could mediate the relationship between work stress and depression in HCWs. Among HCWs with technical secondary school education and below, job satisfaction can mediate the positive relationship between work stress and depression, while this mediating effect is not significant among HCWs with college degree and above

A Megavoltage CT Image Enhancement Method for Image-Guided and Adaptive Helical TomoTherapy

Purpose: To propose a novel method to improve the mega-voltage CT (MVCT) image quality for helical TomoTherapy while maintaining the stability on dose calculation.Materials and Methods: The Block-Matching 3D-transform (BM3D) and Discriminative Feature Representation (DFR) methods were combined into a novel BM3D + DFR method for their respective advantages. A phantom (Catphan504) and three serials of clinical (head & neck, chest, and pelvis) MVCT images from 30 patients were acquired using the helical TomoTherapy system. The contrast-to-noise ratio (CNR) and edge detection algorithm (canny) was employed for image quality comparisons between the original and BM3D + DFR enhanced MVCT. A simulated rectangular field of 6 MV X-ray beams were vertically delivered on the original and post-processed MVCT serials of the same CT density phantom, and the dose curves on both serials were compared to test the effects of image enhancement on dose calculation accuracy.Results: In total, 466 transversal MVCT slices were acquired and processed by both BM3D and the proposed BM3D + DFR methods. Compared to the original MVCT image, the BM3D + DFR method presented a remarkable improvement in terms of the soft tissue contrast and noise reduction. For the phantom image, the CNR of the region of interest (ROI) was improved from 1.70 to 4.03. The average CNR of ROIs for 10 patients from each anatomical group, were increased significantly from 1.45 ± 1.51 to 2.09 ± 1.68 for the head & neck (p < 0.001), from 0.92 ± 0.78 to 1.36 ± 0.85 for the chest (p < 0.001), and from 1.12 ± 1.22 to 1.76 ± 1.31 for the pelvis (p < 0.001), respectively. The canny edge detection operator showed that BM3D + DFR provided clearer organ boundaries with less chaos. The root-mean-square of the dosimetry difference on the iso-center passed horizontal dose profile curves and vertical percentage depth dose curves were only 0.09% and 0.06%, respectively.Conclusions: The proposed BM3D + DFR method is feasible to improve the soft tissue contrast for the original MVCT images with coincidence in dose calculation and without compromising resolution. After integration in clinical workflow, the post-processed MVCT may be better applied on image-guided and adaptive helical TomoTherapy

Headspace solid-phase microextraction coupled with gas chromatography-mass spectrometry (HS-SPME-GC-MS) and odor activity value (OAV) to reveal the flavor characteristics of ripened Pu-erh tea by co-fermentation

IntroductionPu-erh tea is a geographical indication product of China. The characteristic flavor compounds produced during the fermentation of ripened Pu-erh tea have an important impact on its quality.MethodsHeadspace solid-phase microextraction coupled with gas chromatography-mass spectrometry (HS-SPME-GC-MS) and odor activity value (OAV) is used for flavor analysis.ResultsA total of 135 volatile compounds were annotated, of which the highest content was alcohols (54.26%), followed by esters (16.73%), and methoxybenzenes (12.69%). Alcohols in ripened Pu-erh tea mainly contribute flower and fruit sweet flavors, while methoxybenzenes mainly contribute musty and stale flavors. The ripened Pu-erh tea fermented by Saccharomyces: Rhizopus: Aspergillus niger mixed in the ratio of 1:1:1 presented the remarkable flavor characteristics of flower and fruit sweet flavor, and having better coordination with musty and stale flavor.DiscussionThis study demonstrated the content changes of ripened Pu-erh tea’s flavor compounds in the fermentation process, and revealed the optimal fermentation time. This will be helpful to further understand the formation mechanism of the characteristic flavor of ripened Pu-erh tea and guide the optimization of the fermentation process of ripened Pu-erh tea